intro story D-Flow FM

 

D-Flow Flexible Mesh

D-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:

  • Easier 1D-2D-3D model coupling, intuitive setup of boundary conditions and meteorological forcings (amongst others).
  • More flexible 2D gridding in delta regions, river junctions, harbours, intertidal flats and more.
  • High performance by smart use of multicore architectures, and grid computing clusters.
An overview of the current developments can be found here.
 
The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions! 

 

=======================================================
We have launched a new website (still under construction so expect continuous improvements) and a new forum dedicated to Delft3D Flexible Mesh.

Please follow this link to the new forum: 
/web/delft3dfm/forum

Post your questions, issues, suggestions, difficulties related to our Delft3D Flexible Mesh Suite on the new forum.

=======================================================

** PLEASE TAG YOUR POST! **

 

 

Sub groups
D-Flow Flexible Mesh
DELWAQ
Cohesive sediments & muddy systems

 


Message Boards

how to run parallel on linux

VY
veli yumruktepe, modified 9 Years ago.

how to run parallel on linux

Padawan Posts: 42 Join Date: 2/7/12 Recent Posts
Hi,

I have compiled stable version tag:1252 with build.sh command with gfortran under 32bit ubuntu 11.10

So without any problem I managed to build the executables. I also ran my own model setup that I did with a licensed release. The results were pretty much the same.
The way I executed was:

./run_flow2d3d.sh  config_flow2d3d.ini


and everything worked fine.

However when I tried to invoke the parallel computing command, I get this error

~/Documents/delft3drelease/examples/01_standard $ ./run_flow2d3d_parallel.sh config_flow2d3d.ini
Contents of machinefile:
----------------------------------------------------------------------
./run_flow2d3d_parallel.sh: line 70: mpd: command not found
./run_flow2d3d_parallel.sh: line 71: mpdboot: command not found
Executing "/home/caglar/Documents/delft3drelease/bin/lnx/flow2d3d/bin/d_hydro.exe TMP_config_flow2d3d_10141.xml" ...
Executing "/home/caglar/Documents/delft3drelease/bin/lnx/flow2d3d/bin/d_hydro.exe TMP_config_flow2d3d_10142.xml" ...

ERROR: /home/caglar/Documents/delft3drelease/bin/lnx/flow2d3d/bin/d_hydro.exe: symbol lookup error: /usr/lib/libmpich.so.3: undefined symbol: MPL_env2str
       - Starting "d_hydro.exe" may give more information:
         - Run "deltares_hydro.exe <INI-inputfile> -keepXML".
         - Run "d_hydro.exe TMP_config_flow2d3d_<processId>.xml".

ERROR: /home/caglar/Documents/delft3drelease/bin/lnx/flow2d3d/bin/d_hydro.exe: symbol lookup error: /usr/lib/libmpich.so.3: undefined symbol: MPL_env2str
       - Starting "d_hydro.exe" may give more information:
         - Run "deltares_hydro.exe <INI-inputfile> -keepXML".
         - Run "d_hydro.exe TMP_config_flow2d3d_<processId>.xml".
./run_flow2d3d_parallel.sh: line 92: mpdallexit: command not found


I am using the default <run_flow2d3d_parallel> setting that shipped with the tagged version in examples
which is


#!/bin/bash
#$ -V
#$ -j yes

set NHOSTS manually here:
export NHOSTS=3
NPART=$NHOSTS
    #
    # This script starts a single-domain Delft3D-FLOW computation on Linux in parallel mode
    # asuming nodes are allocated manually
    #
    # Usage example (requesting 2 nodes/hosts):
    # qsub -pe distrib 2 run_flow2d3d_parallel.sh
    #


    #
    # Specify the config file to be used here
    #
argfile=config_flow2d3d.ini





    #
    # Set the directory containing delftflow.exe here
    #
export ARCH=intel
export D3D_HOME=../../bin/lnx
exedir=$D3D_HOME/flow2d3d/bin
libdir=$D3D_HOME/flow2d3d/lib

    #
    # No adaptions needed below
#

    # Set some (environment) parameters
export LD_LIBRARY_PATH=$exedir:$libdir:$LD_LIBRARY_PATH



    ### Specific setting for H3/H4 linuxclusters, needed for MPICH2
    ### commands (mpdboot, mpirun, mpiexed, mpdallexit etc.).
export PATH="/opt/mpich2/bin:${PATH}"

    ### The file "machinefile" is assumed to be created manually and should
    ### look like:
    ### machinename1:2
    ### machinename2:2
    ### machinename3:2

echo Contents of machinefile:
#cat $(pwd)/machinefile
echo $HOSTNAME:2 > $(pwd)/machinefile
echo ----------------------------------------------------------------------





    # Run

    ### command="d3d.run -nproc "$NHOSTS" -input "$inputfile" -back no
    ### eval $command

    ### General for MPICH2, startup your MPICH2 communication network (you
    ### can check if it is already there with mpdtrace).
mpd &
mpdboot -n $NHOSTS -f $(pwd)/machinefile --ncpus=2

# link mpich debug rubbish to /dev/null
node_number=$NPART
while test $node_number -ge 1
do
   node_number=`expr $node_number - 1`
   ln -s /dev/null log$node_number.irlog
done

    ### General, start delftflow in parallel by means of mpirun.
    ### The machines in the h4 cluster are dual core; start 2*NHOSTS parallel processes
mpirun -np $NHOSTS $exedir/deltares_hydro.exe $argfile
    ### alternatives:
    ### mpiexec -n $DELTAQ_NumNodes delftflow_91.exe -r $inputfile.mdf
    ### mpiexec -n `expr $DELTAQ_NumNodes \* 2` $exedir/deltares_hydro.exe $argfile


rm -f log*.irlog

    ### General for MPICH2, finish your MPICH2 communication network.
mpdallexit



Even though I managed to run the code with single processor, parallel computing is very important in my case, so I would appreciate all the effort in this issue

Thank a lot in advance
Adri Mourits, modified 9 Years ago.

RE: how to run parallel on linux

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Veli,

Some MPICH2 dynamic libraries can not be found. If they are all located in directory "/usr/lib", the easiest solution is adding that to your LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=$exedir:$libdir:/usr/lib:$LD_LIBRARY_PATH

Alternatively, you can use the commands "ldd" and "nm" to find out what library exactly is missing and copy that to libdir.

Regards,

Adri
JL
João Lencart e Silva, modified 8 Years ago.

RE: how to run parallel on linux

Padawan Posts: 71 Join Date: 3/30/11 Recent Posts
Hi Adri and Veli,

I run into exactly the same error and all of the suggested additions to LD_LIBRARY_PATH didn't work.
I ended up finding this post http://lists.mcs.anl.gov/pipermail/mpich-discuss/2011-March/009363.html that suggested to ask which libraries the mpi compiler needed.
For my system the result for
# mpif90 -show

was
gfortran -Wl,-Bsymbolic-functions -I/usr/include/mpich2 -I/usr/include/mpich2 -L/usr/lib -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lcr -lpthread

I add all of that to MPILIBS_ADDITIONAL and it worked:
MPILIBS_ADDITIONAL="-L/usr/lib/mpich2/lib -lmpichf90 -lmpichf90 -lmpich -lopa -lmpl -lrt -lcr -lpthread"

Hope it helps,

João.
VY
veli yumruktepe, modified 8 Years ago.

RE: how to run parallel on linux

Padawan Posts: 42 Join Date: 2/7/12 Recent Posts
Hi João,

Your solution worked for me as well. Thanks a lot for the input.