install NGsolve without root access

Hi,

I also had some troubles with MKL on our cluster. For me using statically linked libraries worked wonders...

have a try with -DMKL_STATIC=ON

Best,
Christoph

I added -DMKL_MULTI_THREADED=OFF, and finally I have a working ngsolve installed! Big thanks to both of you, Lukas and Matthias.

Here is what's going on:
I have a build directory, and a do-configure file that include all the CMake options.
To do a new compile, I just remove the old files in CMakeCache.txt and CMakeFiles/,
and run ./do-configure (is this not the correct way to configure ??!!)
Anyway, this seems to be wrong and some old files related to my MKL setup was not modified appropriately.
After creating a new folder and compile from scratch, the installation was successful.

Best,
Guosheng

I'm glad you got it finally running. Thinking about it, MKL_STATIC=ON might have helped too, actually.
I know about the issue that you sometimes have to delete the whole build directory when you change the configuration. That's also something we need to address.
I will go over the thread again and check how many of your issues we can work around automatically in the future. The remaining ones should at least be documented somewhere.

Thanks for your patience and have fun with NGSolve!

Best,
Matthias

Hi Matthias,

Here is the summary of two main issues:

Problem 1: MPI not working properly.
Reason: mpich in my cluster is only compatible with gcc version 4.9.2, all other versions do not work.

MKL related issue
Problem 2.1: with dynamic library -DMKL_STATIC=OFF (default version)
(a) blacs library not found
(b) libmkl_gnu_thread.so: undefined symbol: omp_get_num_procs

Fix (a): Manually change the blacs library to libmkl_blacs_lp64.a in ccmake .
(b): turn off multi threading by adding flag -DMKL_MULTI_THREADED=OFF in cmake
In conclusion, with the command
-DMKL_ROOT=... -DMKL_MULTI_THREADED=OFF -DMKL_SDL=OFF
mkl is installed correctly and mumps and pardiso are available in ngsolve.

Problem 2.2: with static library -DMKL_STATIC=ON (Christoph's suggestion)
This time the blacs library is found to be an openmpi version:
libmkl_blacs_openmpi_lp64.a
which is not compatible with my mpich, after manually changing it to libmkl_blacs_lp64.a in CMakeCache.txt
The code compiles with a multi threaded version of MKL.

Interestingly, I ran the same tests
mpirun -n 5 ngspy mpi_cmagnet.py, and observe a much faster speed with the dynamic MKL library than the static library. Don't understand why, though...

There is a final problem with building ngsolve with hyper, I got some error message, but I just gave up with hyper since pardiso and mumps are good enough for me right now


Here is the complete list of my build (python3 is installed in ~/local.bin):

export SRC_DIR=~/netgen/src
export INSTALL_DIR=~/local
export GCC_DIR=/panfs/roc/msisoft/gcc/4.9.2_2
export LD_LIBRARY_PATH=$GCC_DIR/lib/:$GCC_DIR/lib64:$GCC_DIR/lib/gcc/x86_64-unknown-linux-gnu/4.9.2:$LD_LIBRARY_PATH
export PATH=$INSTALL_DIR/bin:$GCC_DIR/bin/:$PATH

module load mpich
module load cmake
export LD_LIBRARY_PATH=$INSTALL_DIR/lib:$LD_LIBRARY_PATH

cmake \
-DCMAKE_C_COMPILER=$GCC_DIR/bin/gcc \
-DCMAKE_CXX_COMPILER=$GCC_DIR/bin/g++ \
-DCMAKE_PREFIX_PATH=$INSTALL_DIR \
-DCMAKE_BUILD_TYPE=Release \
-DINSTALL_DIR=$INSTALL_DIR \
-DUSE_GUI=OFF \
-DUSE_MPI=ON \
-DUSE_MUMPS=ON \
-DUSE_HYPRE=OFF \
-DUSE_MKL=ON \
-DMKL_ROOT=.../mkl \
-DMKL_SDL=OFF \
-DMKL_MULTI_THREADED=OFF \
$SRC_DIR/ngsolve

Best,
Guosheng