parallel build linker issue

1 month 6 days ago #3660 by ddrake
Hi, I'm trying a cmake configuration script like this on Ubuntu 20.04:
export BASEDIR=$HOME/ngsuite
rm -rf $BASEDIR/ngsolve-build
rm -rf $BASEDIR/ngsolve-install
mkdir $BASEDIR/ngsolve-build
mkdir $BASEDIR/ngsolve-install
cd $BASEDIR/ngsolve-build
cmake -DUSE_JPEG:BOOL=ON \
      -DUSE_MKL:BOOL=ON \
      -DMKL_ROOT:PATH=$HOME/intel/oneapi/mkl/2021.2.0 \
      -DUSE_UMFPACK:BOOL=ON \
      -DUSE_MPI:BOOL=ON \
      -DUSE_MUMPS:BOOL=ON \
      -DCMAKE_INSTALL_PREFIX:PATH=${BASEDIR}/ngsolve-install ${BASEDIR}/ngsolve-src

I really just wanted to try MUMPS as a sparse solver in a non-MPI context, but I don't mind having a parallel build. I noticed that if the MUMPS flag is set without the MPI flag, CMake fails to configure

CMake Error at cmake/external_projects/mumps.cmake:9 (find_package):
By not providing "FindPARMETIS.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "PARMETIS",
but CMake did not find one.

Could not find a package configuration file provided by "PARMETIS" with any
of the following names:

PARMETISConfig.cmake
parmetis-config.cmake

Add the installation prefix of "PARMETIS" to CMAKE_PREFIX_PATH or set
"PARMETIS_DIR" to a directory containing one of the above files. If
"PARMETIS" provides a separate development package or SDK, be sure it has
been installed.
Call Stack (most recent call first):
cmake/SuperBuild.cmake:212 (include)
CMakeLists.txt:57 (include)

If I remove both MUMPS and MPI, the build always works, and the Pardiso inverse from MKL is available. I have a local Pardiso installed from Intel OneAPI

With the configuration at the top of this post (both MUMPS and MKL set), I don't see any issues in the configuration step, Netgen and NGSolve build except for the final step when NGSolve gives a linker error about missing libblacs-related definitions.

[ 99%] Linking CXX executable ngs
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `blacs_gridinfo_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pzpotrf_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pdpotrf_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pdgetrs_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `blacs_gridexit_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `numroc_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pdgetrf_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pzgetrs_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `blacs_gridinit_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `descinit_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pzgetrf_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pdpotrs_'
/usr/bin/ld: ../linalg/libngla.so: undefined reference to `pzpotrs_'
collect2: error: ld returned 1 exit status

I have openmpi3 installed through apt. I tried installing scalapack through apt also, but that didn't seem to help with the linker issue. I see libmkl-blacs-openmpi packages available, but I haven't tried them. I'm not sure if my installation of the Intel OneAPI might also include a blacs implementation.

If anyone has ideas on how to get past this linker error, that would be great. I'm attaching a file with the full build output.

Thanks!
Dow
Attachments:

Please Log in or Create an account to join the conversation.

1 month 5 days ago #3671 by JuliusZ
Hi Dow,

I could solve this mumps-related linker issue by using this cmake flag:
-DSCALAPACK_LIBRARY=/usr/lib/x86_64-linux-gnu/libscalapack-openmpi.so

This worked for me on Ubuntu 18.04, but I guess it would work on any distro with any scalapack library.

Best,
Julius

Please Log in or Create an account to join the conversation.

1 month 5 days ago #3674 by ddrake
Thanks Julius! That makes perfect sense. I assumed CMake would have found it in such a standard location, and must not have checked the CMakeCache.txt carefully enough.

Best,
Dow

Please Log in or Create an account to join the conversation.

© 2019 Netgen/NGSolve