SIGSEGV fault for large problems

Dear NGSolve Developers;

I am having issues with the direct solver when dealing with problems when ndof ~ 2-10 million. My problem involves an inhomogeneous dirichlet bc and i use the technique given in Sec 1.3 of the documentation, namely:
The slurm script i use to launch ngsolve:

However the slurm script returns the following message for one of the nodes:

The exact same piece of code of course runs fine when the system is smaller. It's only when i use a refined mesh for the same geometry, i get in to the errors. I have tried refinements both outside (using gmsh and then reading the refined mesh in ngsolve) and inside (i.e. reading a corase mesh and then refining it with ngsolve's refine) but both end up in errors. Could you comment on likely cause of the problem? Thank you in advance for your help.

Trying with umfpack, i get out of memory error: UMFPACK V5.7.4 (Feb 1, 2016): ERROR: out of memory.
Also ngsolve writes the error message for line where i call umfpack:

Traceback (most recent call last):
File "script.py", line 107, in <module>
gfu.vec.data += a.mat.Inverse(freedofs=fes.FreeDofs(), inverse="umfpack") * res
netgen.libngpy._meshing.NgException: UmfpackInverse: Symbolic factorization failed.

The solver is running out of memory, 24GB is probably not enough for millions of unknowns using a direct solver. For larger problems you might want to switch to an iterative solver instead, have a look at the documentation about preconditioners:

ngsolve.org/docu/latest/i-tutorials/unit.../preconditioner.html

Best,
Matthias

Also, "sparsecholesky" and "umfpack" do not work with parallel matrices - they are for local matrices only!

For parallel matrices use "mumps" as a direct solver (if you have configured with it).

(You can also use "masterinverse" for small problems - then the master proc gathers the entire matrix and inverts it by itself).

I had the wrong information on how much RAM I can use (the reason for setting 24G in the earlier script). We have a 15 noded cluster with 256GB of RAM on each node. Setting with 6 nodes (each with 48 parallel threads) and RAM/node to be used as 150GB (leaving some for the OS), I still could not get the code running with MUMPS. With some 900GB of RAM among the 6 nodes requested, I can't understand why it still struggles to solve. I even reduced the problem size considerably: from previous 9m dofs, i now have 5.2m dofs.

Slurm output file has:
whereas the error file from slurm has:

matthiash wrote: For larger problems you might want to switch to an iterative solver instead, have a look at the documentation about preconditioners:

I am trying to solve high-frequency Helmholtz problems and I have not tried the iterative solvers yet as I don't know enough about the suitability of the preconditioners available in ngsolve.