intro story D-Flow FM

 

D-Flow Flexible Mesh

D-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:

  • Easier 1D-2D-3D model coupling, intuitive setup of boundary conditions and meteorological forcings (amongst others).
  • More flexible 2D gridding in delta regions, river junctions, harbours, intertidal flats and more.
  • High performance by smart use of multicore architectures, and grid computing clusters.
An overview of the current developments can be found here.
 
The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions! 

 

=======================================================
We have launched a new website (still under construction so expect continuous improvements) and a new forum dedicated to Delft3D Flexible Mesh.

Please follow this link to the new forum: 
/web/delft3dfm/forum

Post your questions, issues, suggestions, difficulties related to our Delft3D Flexible Mesh Suite on the new forum.

=======================================================

** PLEASE TAG YOUR POST! **

 

 

Sub groups
D-Flow Flexible Mesh
DELWAQ
Cohesive sediments & muddy systems

 


Message Boards

Delft3d Parallel Running

AH
Alan Hills, modified 9 Years ago.

Delft3d Parallel Running

Youngling Posts: 4 Join Date: 3/23/11 Recent Posts
Hi. I've compiled and run Open Delft3d to run on a single processor/core. I understand that the parallel running version of Delft3d was being consolidated into the open distribution. I wanted to ask if there were any special compilation requirements for a parallel executable. I also wanted to ask if there were any particular instructions for running parallel models or any documentation?

Many Thanks Alan
Adri Mourits, modified 9 Years ago.

RE: Delft3d Parallel Running (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Dear Alan,

When compiling the code, all necessary MPI stuff is included by default. You don't have to build a special parallel executable.

We are preparing instructions/documentation to run a model parallel.

As a first trial:
Attached is an example run script "run_flow2d3d_parallel.sh"

Remarks:
- Don't forget to create a file named ".mpd.conf" in your home directory, see MPI manual
- The script depends on your cluster configuration. You may have to adapt it to use and optimise it.

Regards,

Adri
AH
Alan Hills, modified 9 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 4 Join Date: 3/23/11 Recent Posts
Hi Adri, Thanks very much for your reply. I was able to use your script to work out how to run a Delft3D model in parallel. I'm currently trying to gain speed improvements to a model on a dual quad core machine. I've managed to get a 40% reduction in running on 4 cores, but will need to work on getting a speed improvement from using the other 4.

Best Regards Alan
Chiranjeevi Ambati, modified 9 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 19 Join Date: 3/23/11 Recent Posts
Dear Adri,

Is it possible parallel running with WINDOWS OS? If so please send the example run script.

Thanks and regards
Chiru
Adri Mourits, modified 9 Years ago.

RE: Delft3d Parallel Running (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Chiru,

If you checkout the trunk, revision 459 or higher, you will get the example script "examples\01_standard\run_flow2d3d_parallel.bat". Unfortunately, there is no fully tested version yet with this functionality. We only ran a few testcases and they worked fine.

Regards,

Adri
Talal Etri, modified 9 Years ago.

Delft3d Parallel Running under windows

Youngling Posts: 5 Join Date: 3/28/11 Recent Posts
Hi,
I would like to ask about one problem that I faced when I tried to run flow model with source code version 4.00.02.0423. I did the compilation correctly under windows and I got the required files as I did with the previous version. But when I tried to run the compiled version 4.00.02.0423, I got the attached error mesage. Does anyone has any Idea about this error.

My best regards to you.

Talal Etri
Adri Mourits, modified 9 Years ago.

RE: Delft3d Parallel Running under windows (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Talal,

Are you able to run the example testcases in directory "https://svn.oss.deltares.nl/repos/delft3d/trunk/examples"?

Does the information in the forum post below help (just copy the line into your browser):
http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894#_19_message_42119

Regards,

Adri
Talal Etri, modified 9 Years ago.

RE: Delft3d Parallel Running under windows

Youngling Posts: 5 Join Date: 3/28/11 Recent Posts
Hi Adri,

Thank you it worked.

Best regards,

Talal
LQ
le quoc huy, modified 9 Years ago.

RE: Delft3d Parallel Running under fedora

Youngling Posts: 3 Join Date: 4/13/11 Recent Posts
Hi Adri
I downloaded your run_flow2d3d_parallel.sh file. But i don't known how to edit this file. I tried edit it but not run. Could you help me edit it?
I trying run delft3d parallel on duocore machine (i ran mpdboot).

exedir=../../src/bin
libdir=../../src/lib
# Set some (environment) parameters
export ARCH=intel
export D3D_HOME=$exedir
export LD_LIBRARY_PATH=$exedir:$libdir:$LD_LIBRARY_PATH
### Specific setting for H3/H4 linuxclusters, needed for MPICH2
### commands (mpdboot, mpirun, mpiexed, mpdallexit etc.).
export PATH=/opt/mpich2/bin:$PATH
### Some general information available via SGE. Note that NHOSTS can be
### smaller than NSLOTS (for instance on multicore nodes).
echo ----------------------------------------------------------------------
echo Parallel run of Delft3D-FLOW with MPICH2 on H4 linuxcluster.
echo SGE_O_WORKDIR: $SGE_O_WORKDIR ! I don't known
echo HOSTNAME : $HOSTNAME huy
echo NHOSTS : $NHOSTS 1
echo NQUEUES : $NQUEUES ! I don't known
echo NSLOTS : $NSLOTS ! I don't known
echo PE_HOSTFILE : $PE_HOSTFILE ! I don't known
echo MPI_VER : $MPI_VER mpich2-1.0.6p1
### General for MPICH2, create needed machinefile for mpdboot and
### mpdexec from $PE_HOSTFILE. The first column contains the node
### identifier, the second column the number of processes to be started
### on this node.
awk '{print $1":2"}' $PE_HOSTFILE > $(pwd)/machinefile

echo Contents of machinefile:
cat $(pwd)/machinefile
echo ----------------------------------------------------------------------



# Local run
# StageIn
# cd $DELTAQ_LocalTempDir


# Run

### command="d3d.run -nproc "$NHOSTS" -input "$inputfile" -back no
### eval $command

### General for MPICH2, startup your MPICH2 communication network (you
### can check if it is already there with mpdtrace).
mpdboot -n $NHOSTS -f $(pwd)/machinefile

# link mpich debug rubbish to /dev/null
node_number=2
while test $node_number -ge 1
do
node_number=`expr $node_number - 1`
ln -s /dev/null log$node_number.irlog
done

### General, start delftflow in parallel by means of mpirun.
mpirun -np `expr $NHOSTS \* 2` $exedir/deltares_hydro.exe $argfile
### alternatives:
### mpiexec -n $DELTAQ_NumNodes delftflow_91.exe -r $inputfile.mdf
### mpiexec -n `expr $DELTAQ_NumNodes \* 2` $exedir/deltares_hydro.exe $argfile


rm -f log*.irlog

### General for MPICH2, finish your MPICH2 communication network.
mpdallexit

# Local run: copy back
# StageOut
Regards!
Huy.
Adri Mourits, modified 9 Years ago.

RE: Delft3d Parallel Running under fedora (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Huy,

The example parallel script is tuned for our Linux cluster so I can imagine you're having problems.

If you want to start a parallel run on your local dual core machine, assuming you want to use 2 partitions, there is only one thing you have to change:
The writing of the machinefile
The example script contains the line "awk...machinefile" (line 58). Replace this by:
echo $HOSTNAME:2 > $(pwd)/machinefile
You can check the file yourself. It should look like:
huy:2

I think that's all you have to change. Don't bother about the wrong echo statements.

Regards,

Adri
LQ
le quoc huy, modified 9 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 3 Join Date: 4/13/11 Recent Posts
Thanks Adri for soon reply.
I did correct my file and it worked. I ran parallel on many core i7 machine successful.
TR
Tobias Rothhardt, modified 9 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 17 Join Date: 7/25/11 Recent Posts
Hi there,
I also try to get a simulation running parallel. For now I´m playing around on an Kubuntu 10.04 LTS 32bit Multicore Opteron System. Using the guidelines from another post in this section I compiled the most recent (checkout from 26.07.) version with intel fortran 11.0.084 and could run the example cases.
Next step would now be running a case parallel to use the full number of CPU available. First I got stuck because mpdboot was missing, I then recompiled mpich2 (1.3.2p1) with --with-pm=mpd and used the script run_flow2d3d_parallel.sh provided further up in this post and made the proposed adjustments. But I still get some errors.

Do you have any suggestions ?


---
tr@kubuntu32:~/delft3d/examples/01_standard$ ./run_flow2d3d_parallel.sh
ERROR: Unknown switch ''. Accepted values: ia32, intel64, ia64
----------------------------------------------------------------------
Parallel run of Delft3D-FLOW with MPICH2 on H4 linuxcluster.
SGE_O_WORKDIR:
HOSTNAME :
NHOSTS :
NQUEUES :
NSLOTS :
PE_HOSTFILE :
MPI_VER :
Contents of machinefile:
:2
----------------------------------------------------------------------
Traceback (most recent call last):
File "/opt/mpich2/bin/mpdboot", line 482, in <module>
mpdboot()
File "/opt/mpich2/bin/mpdboot", line 158, in mpdboot
totalnumToStart = int(argv[argidx+1])
ValueError: invalid literal for int() with base 10: '-f'
test: 89: Illegal number: numnode
expr: Syntaxfehler
** non-numeric arg to -n: /home/tr/delft3d/bin/intel/flow/bin/deltares_hydro.exe

usage:
mpiexec [-h or -help or --help] # get this message
mpiexec -file filename # (or -f) filename contains XML job description
mpiexec [global args] [local args] executable [args]
where global args may be
-l # line labels by MPI rank
-bnr # MPICH1 compatibility mode
-machinefile # file mapping procs to machines
-s <spec> # direct stdin to "all" or 1,2 or 2-4,6
-1 # override default of trying 1st proc locally
-ifhn # network interface to use locally
-tv # run procs under totalview (must be installed)
-tvsu # totalview startup only
-gdb # run procs under gdb
-m # merge output lines (default with gdb)
-a # means assign this alias to the job
-ecfn # output_xml_exit_codes_filename
-recvtimeout <integer_val> # timeout for recvs to fail (e.g. from mpd daemon)
-g<local arg name> # global version of local arg (below)
and local args may be
-n <n> or -np <n> # number of processes to start
-wdir <dirname> # working directory to start in
-umask <umask> # umask for remote process
-path <dirname> # place to look for executables
-host <hostname> # host to start on
-soft <spec> # modifier of -n value
-arch <arch> # arch type to start on (not implemented)
-envall # pass all env vars in current environment
-envnone # pass no env vars
-envlist <list of env var names> # pass current values of these vars
-env <name> <value> # pass this value of this env var
mpiexec [global args] [local args] executable args : [local args] executable...
mpiexec -gdba jobid # gdb-attach to existing jobid
mpiexec -configfile filename # filename contains cmd line segs as lines
(See User Guide for more details)

Examples:
mpiexec -l -n 10 cpi 100
mpiexec -genv QPL_LICENSE 4705 -n 3 a.out

mpiexec -n 1 -host foo master : -n 4 -host mysmp slave
---
Adri Mourits, modified 9 Years ago.

RE: Delft3d Parallel Running

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Tobias,

It seems that the expression totalnumToStart = int(argv[argidx+1]) is wrong. I don't know exactly how you want to define totalnumToStart. What happens when you begin with just a simple integer?

I try to collect all parallel information at http://oss.deltares.nl/web/opendelft3d/faq-running-a-calculation#parallel. Please let me know if you miss something.

Regards,

Adri
TR
Tobias Rothhardt, modified 9 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 17 Join Date: 7/25/11 Recent Posts
Hi,

I rewrote the shell script and then I got it running, maybe some variable was wrong before that.

As an addition to the parallel faq: For Windows the spreading over several systems of a single domain computation using MPI does not work. At least not for me and you told me you had the same problem and could not get it running either. I have an idea why this might happen and will research further on this topic.
Adri Mourits, modified 9 Years ago.

RE: Delft3d Parallel Running

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Tobias,

Yes, you're right; I added that to the parallel faq. Thanks.

Are your script changes generic enough to merge them back in the trunk?

Regards,

Adri
U
Anonymous, modified 8 Years ago.

RE: Delft3d Parallel Running

Posts: 1
Hi, I have Version 4.00.01.000000. I am currently trying to use run_flow2d3d_parallel.bat and running into some errors. Is it possible to use run_flow2d3d_parallel.bat with this version or do I have to upgrade?

Thanks!
MC
Janesh Devkota, modified 6 Years ago.

RE: Delft3d Parallel Running under windows

Youngling Posts: 5 Join Date: 11/11/11 Recent Posts
Adri Mourits:
Hi Talal,

Are you able to run the example testcases in directory "https://svn.oss.deltares.nl/repos/delft3d/trunk/examples"?

Does the information in the forum post below help (just copy the line into your browser):
http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894#_19_message_42119

Regards,

Adri


Hi Adri,

I couldn't load the link that you wrote above. http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894#_19_message_42119
The error or message I got is "You do not have permission to access the requested resource.

http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894 ".

I tried to run the parallel version in the window and it is stuck without doing anything. The configuration I used is as follows:

@ echo off
    rem
    rem This script is an example for running Delft3D-FLOW
    rem Adapt and use it for your own purpose
    rem
    rem edwin.spee@deltares.nl
    rem adri.mourits@deltares.nl
    rem 23 Oct 2013
    rem
    rem
    rem This script starts a single-domain Delft3D-FLOW computation on Windows
    rem parallel
    rem
    rem When using mpich2 for the first time on a machine:
    rem Execute "smpd -install" as administrator:
    rem     "Start" -> "All programs" -> "Accessories", right-click "Command Prompt", "Run as Administrator"
    rem     In this command box:
    rem         cd ..\..\bin\win32\flow2d3d\bin
    rem         (or when not compiled yet)
    rem         cd ..\..\src\third_party_open\mpich2\bin
    rem         (both paths above are relative to the location of this script "run_flow2d3d_parallel.bat")
    rem         smpd -install
    rem     When there is an smpd already running on the machine, it must be ended first, using the Microsoft Task Manager



    rem
    rem Set the config file here
    rem
set argfile=config_d_hydro.xml





    rem
    rem Set the directory containing ALL exes/dlls here (mpiexec.exe, delftflow.exe, flow2d3d.dll, mpich-dlls, DelftOnline dlls etc.)
    rem
set ARCH=win32
set D3D_HOME=..\..\..\..\bin
set exedir=%D3D_HOME%\%ARCH%\flow2d3d\bin

    rem
    rem No adaptions needed below
    rem

    rem Set some (environment) parameters
set PATH=%exedir%;%PATH%
    rem mpiexec is in %exedir%
    rem For some users, it is necessary to use the locally installed mpiexec:

    rem set MPIPATH=%exedir%
set MPIPATH="C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\mpirt"


    rem Run
    rem start computation on all your local cores (2 for dual core; 4 for quad core etc.)
    rem note the flag "-localonly" which may be needed to avoid "Aborting: unable to connect to machinename.local"
   
%MPIPATH%\mpiexec -n %NUMBER_OF_PROCESSORS% -localonly %exedir%\d_hydro.exe %argfile%


    rem To prevent the DOS box from disappearing immediately: remove the rem on the following line
pause


Do you have any suggestions ?
Adri Mourits, modified 6 Years ago.

RE: Delft3d Parallel Running under windows (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Janesh,

The directory "opendelft3d" on the server was renamed into "delft3d". All references to http://oss.deltares.nl/web/opendelft3d/..... are broken. You can repair them by removing "open". So for the post you mentioned:

http://oss.deltares.nl/web/delft3d/general/-/message_boards/view_message/28894

You will need to provide some more information. Are you trying to run one of the example testcases? Does it run without using parallelMPI? Did you (exactly!) follow the instructions about starting "smpd -install" as stated in the comments in the run script? What error message do you get? Is there additional information in the tri-diag files?

Regards,

Adri
GK
Garrett Kannon, modified 4 Months ago.

RE: Delft3d Parallel Running

Youngling Posts: 5 Join Date: 5/23/20 Recent Posts
I hope that your compilation will work very well, and there will be no problems. I will write about your work in my dragon resumes paper if it works flawlessly. Let us know what happens with this project. Cheers!