MPI pre-requirements - D-Flow Flexible Mesh - Delft3D
intro story D-Flow FM
D-Flow Flexible MeshD-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:
An overview of the current developments can be found here. The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions!
======================================================= | Sub groups
|
Message Boards
MPI pre-requirements
PZ
Pavlo Zemskyy, modified 9 Years ago.
MPI pre-requirements
Youngling Posts: 2 Join Date: 9/13/11 Recent Posts 00
I am compiling Open Source Delft3D from trunk under 64-bit Linux (CentOS)
As it is mentioned in Open Source Pre-requirements, to compile Delft3D it is necessarily to have MPICH2 installed in Linux environment.
Do you plan to include support of other MPI implementations (such as MVAPICH/MVAPICH2)?
Currently at HPC cluster I have access to there is MVAPICH2 installed. I managed to pass through compilation by installing MPICH2 in my local folder, but it is not a good solution because:
1. such approach makes binaries available only for myself (while the initial idea was to share binaries with other team members) and
2. It is difficult to use benefits of parallel computations.
Best regards
Pavlo
As it is mentioned in Open Source Pre-requirements, to compile Delft3D it is necessarily to have MPICH2 installed in Linux environment.
Do you plan to include support of other MPI implementations (such as MVAPICH/MVAPICH2)?
Currently at HPC cluster I have access to there is MVAPICH2 installed. I managed to pass through compilation by installing MPICH2 in my local folder, but it is not a good solution because:
1. such approach makes binaries available only for myself (while the initial idea was to share binaries with other team members) and
2. It is difficult to use benefits of parallel computations.
Best regards
Pavlo
Bert Jagers, modified 9 Years ago.
RE: MPI pre-requirements
Jedi Knight Posts: 201 Join Date: 12/22/10 Recent Posts 10
Hi Pavlo,
Since the MPI interface is standardized, you should be able to use other MPI versions as well. I know that other users have been successful in compiling Delft3D against other MPI libraries e.g. OpenMPI. The only issue seems to be that Delft3D-FLOW recognizes MPI mode in the
src/engines_gpl/flow2d3d/packages/data/src/parallel_mpi/dfinitmpi.f90
routine. This routine contains the following lines:
call get_environment_variable('PMI_RANK', rankstr, len)
usempi = (len > 0)
It seems that the environment variable (here PMI_RANK) has not been standardized across all MPI libraries. The page:
http://dsc.sun.com/sunstudio/documentation/techart/mpi_apps/
contains a table "MPI Implementations Recognized for MPI API Tracing" that lists MPI implementations and associated MPI Rank Environment Variable.
So, besides changing the build environment to compile and link against another MPI implementation, you may want to adjust this part of the code. I know the OpenMPI user, just switched the code to enforce Delft3D to always run in MPI mode.
Bert
Since the MPI interface is standardized, you should be able to use other MPI versions as well. I know that other users have been successful in compiling Delft3D against other MPI libraries e.g. OpenMPI. The only issue seems to be that Delft3D-FLOW recognizes MPI mode in the
src/engines_gpl/flow2d3d/packages/data/src/parallel_mpi/dfinitmpi.f90
routine. This routine contains the following lines:
call get_environment_variable('PMI_RANK', rankstr, len)
usempi = (len > 0)
It seems that the environment variable (here PMI_RANK) has not been standardized across all MPI libraries. The page:
http://dsc.sun.com/sunstudio/documentation/techart/mpi_apps/
contains a table "MPI Implementations Recognized for MPI API Tracing" that lists MPI implementations and associated MPI Rank Environment Variable.
So, besides changing the build environment to compile and link against another MPI implementation, you may want to adjust this part of the code. I know the OpenMPI user, just switched the code to enforce Delft3D to always run in MPI mode.
Bert
JL
João Lencart e Silva, modified 8 Years ago.
RE: MPI pre-requirements
Padawan Posts: 71 Join Date: 3/30/11 Recent Posts 00
Hi Pavlo and Bert,
I manage to compile FLOW for an Intel cluster suite 2012 + Intel MPI.
The environment variable I included in src/engines_gpl/flow2d3d/packages/data/src/parallel_mpi/dfinitmpi.f90 was 'OMPI_MCA_ns_nds_vpid' but it works also with 'MPD_JRANK'.
It compiles and runs fine with 1 core. It won't work in my setup with more then one core.
Please see this forum message for details.
João.
I manage to compile FLOW for an Intel cluster suite 2012 + Intel MPI.
The environment variable I included in src/engines_gpl/flow2d3d/packages/data/src/parallel_mpi/dfinitmpi.f90 was 'OMPI_MCA_ns_nds_vpid' but it works also with 'MPD_JRANK'.
It compiles and runs fine with 1 core. It won't work in my setup with more then one core.
Please see this forum message for details.
João.