- Thank you received: 0
MultiVector class and MPI compatibility
2 years 8 months ago #4221
by aujxn
MultiVector class and MPI compatibility was created by aujxn
Hello,
I am converting an existing library to run in parallel with MPI. The library makes extensive use of the MultiVector class.
I have found that taking the InnerProduct of a MultiVector with parallel vectors computes the inner product of the local components for each rank. Is this the desired functionality or is MultiVector not compatible with MPI? Can I apply parallel operators, such as mumps inverse, to parallel MultiVectors?
Example:
Output:
Best regards,
Austen
I am converting an existing library to run in parallel with MPI. The library makes extensive use of the MultiVector class.
I have found that taking the InnerProduct of a MultiVector with parallel vectors computes the inner product of the local components for each rank. Is this the desired functionality or is MultiVector not compatible with MPI? Can I apply parallel operators, such as mumps inverse, to parallel MultiVectors?
Example:
Code:
from netgen.geom2d import unit_square
from ngsolve import grad, dx
import ngsolve as ng
import netgen
from mpi4py import MPI
import numpy as np
import logging
comm = MPI.COMM_WORLD
logging.basicConfig(
level=logging.DEBUG,
filename=f'rank{comm.rank}.txt',
format='%(asctime)s %(funcName)s:%(lineno)d\n%(message)s'
)
h = 0.1
if comm.rank == 0:
ngmesh = unit_square.GenerateMesh(maxh=h).Distribute(comm)
else:
ngmesh = netgen.meshing.Mesh.Receive(comm)
mesh = ng.Mesh(ngmesh)
fes = ng.H1(mesh, order=3, dirichlet=".*")
u,v = fes.TnT()
a = ng.BilinearForm(grad(u)*grad(v)*dx)
with ng.TaskManager():
a.Assemble()
n = fes.ndof
logging.debug(f'Degrees of freedom of H1 on current rank: {n}\n' +
f'FreeDofs bit array: {fes.FreeDofs()}\n')
# Create random value MultiVectors
gfu = ng.GridFunction(fes)
mv0 = ng.MultiVector(gfu.vec, 2)
mv1 = ng.MultiVector(gfu.vec, 2)
rng = np.random.default_rng()
random_vecs = rng.random((4, n))
mv0[0].FV().NumPy()[:] = random_vecs[0, :]
mv0[1].FV().NumPy()[:] = random_vecs[1, :]
mv1[0].FV().NumPy()[:] = random_vecs[2, :]
mv1[1].FV().NumPy()[:] = random_vecs[3, :]
# MultiVector inner product test
inner = ng.InnerProduct(mv0, mv1)
logging.debug(f'Inner product of 2 MultiVectors:\n{inner}\n')
# Inner product of MultiVector local components
local_inner = np.zeros((2, 2))
for i in range(2):
for j in range(2):
local_inner[i, j] = ng.InnerProduct(mv0[i].local_vec, mv1[j].local_vec)
logging.debug(f'Inner product of 2 MultiVectors local components:\n{local_inner}\n')
# Manual inner product of MultiVector
manual_inner = np.zeros((2,2))
for i in range(2):
for j in range(2):
manual_inner[i, j] = ng.InnerProduct(mv0[i], mv1[j])
logging.debug(f'Pairwise inner product of parallel vectors:\n{manual_inner}')[/i][/i]
Output:
Code:
2022-03-02 13:31:26,952 main:39
Degrees of freedom of H1 on current rank: 46
FreeDofs bit array: 0: 1111111111111111111111111111111111111111111111
2022-03-02 13:31:26,953 main:55
Inner product of 2 MultiVectors:
10.517 10.3929
10.6328 11.6122
2022-03-02 13:31:26,954 main:62
Inner product of 2 MultiVectors local components:
[[10.51703725 10.3928831 ]
[10.63276183 11.61222237]]
2022-03-02 13:31:26,955 main:69
Pairwise inner product of parallel vectors:
[[270.4623489 276.32998277]
[284.52705911 289.39745476]]
Best regards,
Austen
Time to create page: 0.096 seconds