- Thank you received: 0
Forum Message
We have moved the forum to https://forum.ngsolve.org . This is an archived version of the topics until 05/05/23. All the topics were moved to the new forum and conversations can be continued there. This forum is just kept as legacy to not invalidate old links. If you want to continue a conversation just look for the topic in the new forum.
Notice
The forum is in read only mode.
MultiVector class and MPI compatibility
2 years 1 month ago #4221
by aujxn
MultiVector class and MPI compatibility was created by aujxn
Hello,
I am converting an existing library to run in parallel with MPI. The library makes extensive use of the MultiVector class.
I have found that taking the InnerProduct of a MultiVector with parallel vectors computes the inner product of the local components for each rank. Is this the desired functionality or is MultiVector not compatible with MPI? Can I apply parallel operators, such as mumps inverse, to parallel MultiVectors?
Example:
Output:
Best regards,
Austen
I am converting an existing library to run in parallel with MPI. The library makes extensive use of the MultiVector class.
I have found that taking the InnerProduct of a MultiVector with parallel vectors computes the inner product of the local components for each rank. Is this the desired functionality or is MultiVector not compatible with MPI? Can I apply parallel operators, such as mumps inverse, to parallel MultiVectors?
Example:
Code:
from netgen.geom2d import unit_square
from ngsolve import grad, dx
import ngsolve as ng
import netgen
from mpi4py import MPI
import numpy as np
import logging
comm = MPI.COMM_WORLD
logging.basicConfig(
level=logging.DEBUG,
filename=f'rank{comm.rank}.txt',
format='%(asctime)s %(funcName)s:%(lineno)d\n%(message)s'
)
h = 0.1
if comm.rank == 0:
ngmesh = unit_square.GenerateMesh(maxh=h).Distribute(comm)
else:
ngmesh = netgen.meshing.Mesh.Receive(comm)
mesh = ng.Mesh(ngmesh)
fes = ng.H1(mesh, order=3, dirichlet=".*")
u,v = fes.TnT()
a = ng.BilinearForm(grad(u)*grad(v)*dx)
with ng.TaskManager():
a.Assemble()
n = fes.ndof
logging.debug(f'Degrees of freedom of H1 on current rank: {n}\n' +
f'FreeDofs bit array: {fes.FreeDofs()}\n')
# Create random value MultiVectors
gfu = ng.GridFunction(fes)
mv0 = ng.MultiVector(gfu.vec, 2)
mv1 = ng.MultiVector(gfu.vec, 2)
rng = np.random.default_rng()
random_vecs = rng.random((4, n))
mv0[0].FV().NumPy()[:] = random_vecs[0, :]
mv0[1].FV().NumPy()[:] = random_vecs[1, :]
mv1[0].FV().NumPy()[:] = random_vecs[2, :]
mv1[1].FV().NumPy()[:] = random_vecs[3, :]
# MultiVector inner product test
inner = ng.InnerProduct(mv0, mv1)
logging.debug(f'Inner product of 2 MultiVectors:\n{inner}\n')
# Inner product of MultiVector local components
local_inner = np.zeros((2, 2))
for i in range(2):
for j in range(2):
local_inner[i, j] = ng.InnerProduct(mv0[i].local_vec, mv1[j].local_vec)
logging.debug(f'Inner product of 2 MultiVectors local components:\n{local_inner}\n')
# Manual inner product of MultiVector
manual_inner = np.zeros((2,2))
for i in range(2):
for j in range(2):
manual_inner[i, j] = ng.InnerProduct(mv0[i], mv1[j])
logging.debug(f'Pairwise inner product of parallel vectors:\n{manual_inner}')[/i][/i]
Output:
Code:
2022-03-02 13:31:26,952 main:39
Degrees of freedom of H1 on current rank: 46
FreeDofs bit array: 0: 1111111111111111111111111111111111111111111111
2022-03-02 13:31:26,953 main:55
Inner product of 2 MultiVectors:
10.517 10.3929
10.6328 11.6122
2022-03-02 13:31:26,954 main:62
Inner product of 2 MultiVectors local components:
[[10.51703725 10.3928831 ]
[10.63276183 11.61222237]]
2022-03-02 13:31:26,955 main:69
Pairwise inner product of parallel vectors:
[[270.4623489 276.32998277]
[284.52705911 289.39745476]]
Best regards,
Austen
Time to create page: 0.148 seconds
