Message Boards

RE: Delft3d Parallel Running under windows

Alan Hills, modified 13 Years ago.

Delft3d Parallel Running

Youngling Posts: 2 Join Date: 3/23/11 Recent Posts
Hi. I've compiled and run Open Delft3d to run on a single processor/core. I understand that the parallel running version of Delft3d was being consolidated into the open distribution. I wanted to ask if there were any special compilation requirements for a parallel executable. I also wanted to ask if there were any particular instructions for running parallel models or any documentation?

Many Thanks Alan
thumbnail
Adri Mourits, modified 13 Years ago.

RE: Delft3d Parallel Running (Answer)

Jedi Master Posts: 303 Join Date: 1/3/11 Recent Posts
Dear Alan,

When compiling the code, all necessary MPI stuff is included by default. You don't have to build a special parallel executable.

We are preparing instructions/documentation to run a model parallel.

As a first trial:
Attached is an example run script "run_flow2d3d_parallel.sh"

Remarks:
- Don't forget to create a file named ".mpd.conf" in your home directory, see MPI manual
- The script depends on your cluster configuration. You may have to adapt it to use and optimise it.

Regards,

Adri
Alan Hills, modified 13 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 2 Join Date: 3/23/11 Recent Posts
Hi Adri, Thanks very much for your reply. I was able to use your script to work out how to run a Delft3D model in parallel. I'm currently trying to gain speed improvements to a model on a dual quad core machine. I've managed to get a 40% reduction in running on 4 cores, but will need to work on getting a speed improvement from using the other 4.

Best Regards Alan
thumbnail
Chiranjeevi Ambati, modified 12 Years ago.

RE: Delft3d Parallel Running

Youngling Post: 1 Join Date: 3/23/11 Recent Posts
Dear Adri,

Is it possible parallel running with WINDOWS OS? If so please send the example run script.

Thanks and regards
Chiru
thumbnail
Adri Mourits, modified 12 Years ago.

RE: Delft3d Parallel Running (Answer)

Jedi Master Posts: 303 Join Date: 1/3/11 Recent Posts
Hi Chiru,

If you checkout the trunk, revision 459 or higher, you will get the example script "examples\01_standard\run_flow2d3d_parallel.bat". Unfortunately, there is no fully tested version yet with this functionality. We only ran a few testcases and they worked fine.

Regards,

Adri
Talal Etri, modified 12 Years ago.

Delft3d Parallel Running under windows

Youngling Posts: 2 Join Date: 3/28/11 Recent Posts
Hi,
I would like to ask about one problem that I faced when I tried to run flow model with source code version 4.00.02.0423. I did the compilation correctly under windows and I got the required files as I did with the previous version. But when I tried to run the compiled version 4.00.02.0423, I got the attached error mesage. Does anyone has any Idea about this error.

My best regards to you.

Talal Etri
thumbnail
Adri Mourits, modified 12 Years ago.

RE: Delft3d Parallel Running under windows (Answer)

Jedi Master Posts: 303 Join Date: 1/3/11 Recent Posts
Hi Talal,

Are you able to run the example testcases in directory "https://svn.oss.deltares.nl/repos/delft3d/trunk/examples"?

Does the information in the forum post below help (just copy the line into your browser):
http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894#_19_message_42119

Regards,

Adri
Talal Etri, modified 12 Years ago.

RE: Delft3d Parallel Running under windows

Youngling Posts: 2 Join Date: 3/28/11 Recent Posts
Hi Adri,

Thank you it worked.

Best regards,

Talal
le quoc huy, modified 12 Years ago.

RE: Delft3d Parallel Running under fedora

Youngling Posts: 2 Join Date: 4/13/11 Recent Posts
Hi Adri
I downloaded your run_flow2d3d_parallel.sh file. But i don't known how to edit this file. I tried edit it but not run. Could you help me edit it?
I trying run delft3d parallel on duocore machine (i ran mpdboot).

exedir=../../src/bin
libdir=../../src/lib
# Set some (environment) parameters
export ARCH=intel
export D3D_HOME=$exedir
export LD_LIBRARY_PATH=$exedir:$libdir:$LD_LIBRARY_PATH
### Specific setting for H3/H4 linuxclusters, needed for MPICH2
### commands (mpdboot, mpirun, mpiexed, mpdallexit etc.).
export PATH=/opt/mpich2/bin:$PATH
### Some general information available via SGE. Note that NHOSTS can be
### smaller than NSLOTS (for instance on multicore nodes).
echo ----------------------------------------------------------------------
echo Parallel run of Delft3D-FLOW with MPICH2 on H4 linuxcluster.
echo SGE_O_WORKDIR: $SGE_O_WORKDIR ! I don't known
echo HOSTNAME : $HOSTNAME huy
echo NHOSTS : $NHOSTS 1
echo NQUEUES : $NQUEUES ! I don't known
echo NSLOTS : $NSLOTS ! I don't known
echo PE_HOSTFILE : $PE_HOSTFILE ! I don't known
echo MPI_VER : $MPI_VER mpich2-1.0.6p1
### General for MPICH2, create needed machinefile for mpdboot and
### mpdexec from $PE_HOSTFILE. The first column contains the node
### identifier, the second column the number of processes to be started
### on this node.
awk '{print $1":2"}' $PE_HOSTFILE > $(pwd)/machinefile

echo Contents of machinefile:
cat $(pwd)/machinefile
echo ----------------------------------------------------------------------



# Local run
# StageIn
# cd $DELTAQ_LocalTempDir


# Run

### command="d3d.run -nproc "$NHOSTS" -input "$inputfile" -back no
### eval $command

### General for MPICH2, startup your MPICH2 communication network (you
### can check if it is already there with mpdtrace).
mpdboot -n $NHOSTS -f $(pwd)/machinefile

# link mpich debug rubbish to /dev/null
node_number=2
while test $node_number -ge 1
do
node_number=`expr $node_number - 1`
ln -s /dev/null log$node_number.irlog
done

### General, start delftflow in parallel by means of mpirun.
mpirun -np `expr $NHOSTS \* 2` $exedir/deltares_hydro.exe $argfile
### alternatives:
### mpiexec -n $DELTAQ_NumNodes delftflow_91.exe -r $inputfile.mdf
### mpiexec -n `expr $DELTAQ_NumNodes \* 2` $exedir/deltares_hydro.exe $argfile


rm -f log*.irlog

### General for MPICH2, finish your MPICH2 communication network.
mpdallexit

# Local run: copy back
# StageOut
Regards!
Huy.
thumbnail
Adri Mourits, modified 12 Years ago.

RE: Delft3d Parallel Running under fedora (Answer)

Jedi Master Posts: 303 Join Date: 1/3/11 Recent Posts
Hi Huy,

The example parallel script is tuned for our Linux cluster so I can imagine you're having problems.

If you want to start a parallel run on your local dual core machine, assuming you want to use 2 partitions, there is only one thing you have to change:
The writing of the machinefile
The example script contains the line "awk...machinefile" (line 58). Replace this by:
echo $HOSTNAME:2 > $(pwd)/machinefile
You can check the file yourself. It should look like:
huy:2

I think that's all you have to change. Don't bother about the wrong echo statements.

Regards,

Adri
le quoc huy, modified 12 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 2 Join Date: 4/13/11 Recent Posts
Thanks Adri for soon reply.
I did correct my file and it worked. I ran parallel on many core i7 machine successful.
Tobias Rothhardt, modified 12 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 2 Join Date: 7/25/11 Recent Posts
Hi there,
I also try to get a simulation running parallel. For now I´m playing around on an Kubuntu 10.04 LTS 32bit Multicore Opteron System. Using the guidelines from another post in this section I compiled the most recent (checkout from 26.07.) version with intel fortran 11.0.084 and could run the example cases.
Next step would now be running a case parallel to use the full number of CPU available. First I got stuck because mpdboot was missing, I then recompiled mpich2 (1.3.2p1) with --with-pm=mpd and used the script run_flow2d3d_parallel.sh provided further up in this post and made the proposed adjustments. But I still get some errors.

Do you have any suggestions ?


---
tr@kubuntu32:~/delft3d/examples/01_standard$ ./run_flow2d3d_parallel.sh
ERROR: Unknown switch ''. Accepted values: ia32, intel64, ia64
----------------------------------------------------------------------
Parallel run of Delft3D-FLOW with MPICH2 on H4 linuxcluster.
SGE_O_WORKDIR:
HOSTNAME :
NHOSTS :
NQUEUES :
NSLOTS :
PE_HOSTFILE :
MPI_VER :
Contents of machinefile:
:2
----------------------------------------------------------------------
Traceback (most recent call last):
File "/opt/mpich2/bin/mpdboot", line 482, in <module>
mpdboot()
File "/opt/mpich2/bin/mpdboot", line 158, in mpdboot
totalnumToStart = int(argv[argidx+1])
ValueError: invalid literal for int() with base 10: '-f'
test: 89: Illegal number: numnode
expr: Syntaxfehler
** non-numeric arg to -n: /home/tr/delft3d/bin/intel/flow/bin/deltares_hydro.exe

usage:
mpiexec [-h or -help or --help] # get this message
mpiexec -file filename # (or -f) filename contains XML job description
mpiexec [global args] [local args] executable [args]
where global args may be
-l # line labels by MPI rank
-bnr # MPICH1 compatibility mode
-machinefile # file mapping procs to machines
-s <spec> # direct stdin to "all" or 1,2 or 2-4,6
-1 # override default of trying 1st proc locally
-ifhn # network interface to use locally
-tv # run procs under totalview (must be installed)
-tvsu # totalview startup only
-gdb # run procs under gdb
-m # merge output lines (default with gdb)
-a # means assign this alias to the job
-ecfn # output_xml_exit_codes_filename
-recvtimeout <integer_val> # timeout for recvs to fail (e.g. from mpd daemon)
-g<local arg name> # global version of local arg (below)
and local args may be
-n <n> or -np <n> # number of processes to start
-wdir <dirname> # working directory to start in
-umask <umask> # umask for remote process
-path <dirname> # place to look for executables
-host <hostname> # host to start on
-soft <spec> # modifier of -n value
-arch <arch> # arch type to start on (not implemented)
-envall # pass all env vars in current environment
-envnone # pass no env vars
-envlist <list of env var names> # pass current values of these vars
-env <name> <value> # pass this value of this env var
mpiexec [global args] [local args] executable args : [local args] executable...
mpiexec -gdba jobid # gdb-attach to existing jobid
mpiexec -configfile filename # filename contains cmd line segs as lines
(See User Guide for more details)

Examples:
mpiexec -l -n 10 cpi 100
mpiexec -genv QPL_LICENSE 4705 -n 3 a.out

mpiexec -n 1 -host foo master : -n 4 -host mysmp slave
---
thumbnail
Adri Mourits, modified 12 Years ago.

RE: Delft3d Parallel Running

Jedi Master Posts: 303 Join Date: 1/3/11 Recent Posts
Hi Tobias,

It seems that the expression totalnumToStart = int(argv[argidx+1]) is wrong. I don't know exactly how you want to define totalnumToStart. What happens when you begin with just a simple integer?

I try to collect all parallel information at http://oss.deltares.nl/web/opendelft3d/faq-running-a-calculation#parallel. Please let me know if you miss something.

Regards,

Adri
Tobias Rothhardt, modified 12 Years ago.

RE: Delft3d Parallel Running

Youngling Posts: 2 Join Date: 7/25/11 Recent Posts
Hi,

I rewrote the shell script and then I got it running, maybe some variable was wrong before that.

As an addition to the parallel faq: For Windows the spreading over several systems of a single domain computation using MPI does not work. At least not for me and you told me you had the same problem and could not get it running either. I have an idea why this might happen and will research further on this topic.
thumbnail
Adri Mourits, modified 12 Years ago.

RE: Delft3d Parallel Running

Jedi Master Posts: 303 Join Date: 1/3/11 Recent Posts
Hi Tobias,

Yes, you're right; I added that to the parallel faq. Thanks.

Are your script changes generic enough to merge them back in the trunk?

Regards,

Adri
Janesh Devkota, modified 9 Years ago.

RE: Delft3d Parallel Running under windows

Youngling Post: 1 Join Date: 11/11/11 Recent Posts
Adri Mourits:
Hi Talal,

Are you able to run the example testcases in directory "https://svn.oss.deltares.nl/repos/delft3d/trunk/examples"?

Does the information in the forum post below help (just copy the line into your browser):
http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894#_19_message_42119

Regards,

Adri


Hi Adri,

I couldn't load the link that you wrote above. http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894#_19_message_42119
The error or message I got is "You do not have permission to access the requested resource.

http://oss.deltares.nl/web/opendelft3d/general/-/message_boards/view_message/28894 ".

I tried to run the parallel version in the window and it is stuck without doing anything. The configuration I used is as follows:

1
@ echo off
2
    rem
3
    rem This script is an example for running Delft3D-FLOW
4
    rem Adapt and use it for your own purpose
5
    rem
6
    rem edwin.spee@deltares.nl
7
    rem adri.mourits@deltares.nl
8
    rem 23 Oct 2013
9
    rem
10
    rem
11
    rem This script starts a single-domain Delft3D-FLOW computation on Windows
12
    rem parallel
13
    rem
14
    rem When using mpich2 for the first time on a machine:
15
    rem Execute "smpd -install" as administrator:
16
    rem     "Start" -> "All programs" -> "Accessories", right-click "Command Prompt", "Run as Administrator"
17
    rem     In this command box:
18
    rem         cd ..\..\bin\win32\flow2d3d\bin
19
    rem         (or when not compiled yet)
20
    rem         cd ..\..\src\third_party_open\mpich2\bin
21
    rem         (both paths above are relative to the location of this script "run_flow2d3d_parallel.bat")
22
    rem         smpd -install
23
    rem     When there is an smpd already running on the machine, it must be ended first, using the Microsoft Task Manager
24

25

26

27
    rem
28
    rem Set the config file here
29
    rem
30
set argfile=config_d_hydro.xml
31

32

33

34

35

36
    rem
37
    rem Set the directory containing ALL exes/dlls here (mpiexec.exe, delftflow.exe, flow2d3d.dll, mpich-dlls, DelftOnline dlls etc.)
38
    rem
39
set ARCH=win32
40
set D3D_HOME=..\..\..\..\bin
41
set exedir=%D3D_HOME%\%ARCH%\flow2d3d\bin
42

43
    rem
44
    rem No adaptions needed below
45
    rem
46

47
    rem Set some (environment) parameters
48
set PATH=%exedir%;%PATH%
49
    rem mpiexec is in %exedir%
50
    rem For some users, it is necessary to use the locally installed mpiexec:
51

52
    rem set MPIPATH=%exedir%
53
set MPIPATH="C:\Program Files (x86)\Common Files\Intel\Shared Libraries\redist\intel64\mpirt"
54

55

56
    rem Run
57
    rem start computation on all your local cores (2 for dual core; 4 for quad core etc.)
58
    rem note the flag "-localonly" which may be needed to avoid "Aborting: unable to connect to machinename.local"
59
   
60
%MPIPATH%\mpiexec -n %NUMBER_OF_PROCESSORS% -localonly %exedir%\d_hydro.exe %argfile%
61

62

63
    rem To prevent the DOS box from disappearing immediately: remove the rem on the following line
64
pause


Do you have any suggestions ?
thumbnail
Adri Mourits, modified 9 Years ago.

RE: Delft3d Parallel Running under windows (Answer)

Jedi Master Posts: 303 Join Date: 1/3/11 Recent Posts
Hi Janesh,

The directory "opendelft3d" on the server was renamed into "delft3d". All references to http://oss.deltares.nl/web/opendelft3d/..... are broken. You can repair them by removing "open". So for the post you mentioned:

http://oss.deltares.nl/web/delft3d/general/-/message_boards/view_message/28894

You will need to provide some more information. Are you trying to run one of the example testcases? Does it run without using parallelMPI? Did you (exactly!) follow the instructions about starting "smpd -install" as stated in the comments in the run script? What error message do you get? Is there additional information in the tri-diag files?

Regards,

Adri