intro story D-Flow FM

 

D-Flow Flexible Mesh

D-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:

  • Easier 1D-2D-3D model coupling, intuitive setup of boundary conditions and meteorological forcings (amongst others).
  • More flexible 2D gridding in delta regions, river junctions, harbours, intertidal flats and more.
  • High performance by smart use of multicore architectures, and grid computing clusters.
An overview of the current developments can be found here.
 
The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions! 

 

=======================================================
We have launched a new website (still under construction so expect continuous improvements) and a new forum dedicated to Delft3D Flexible Mesh.

Please follow this link to the new forum: 
/web/delft3dfm/forum

Post your questions, issues, suggestions, difficulties related to our Delft3D Flexible Mesh Suite on the new forum.

=======================================================

** PLEASE TAG YOUR POST! **

 

 

Sub groups
D-Flow Flexible Mesh
DELWAQ
Cohesive sediments & muddy systems

 


Message Boards

Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

OL
Oliver Lojek, modified 6 Years ago.

Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Jedi Knight Posts: 111 Join Date: 4/30/13 Recent Posts
Hi everybody,


I have a question regarding the parallel calculation under Windows.
Currently I am working on a Quad-Core machine. However, initializing the parallel.bat gives me a notification (see attachment).
I installed MPICH2 MPI for Windows 7 / 64-bit architecture.
From my understanding this should enable me to access the four cores of the machine separately? Or not?

Furthermore, I re-ran a calculation I previously did on the Linux cluster to see the performance difference. The calculation finished in exactly the same time as on the Linux server.
The Linux version ran on 64-bit with 8 and 4 cores, while the Windows 7 ran under 64-bit (32-bit for Delft) and supposedly single core. Since I could not start the parallel.bat

Input regarding the above questions would help me quite a bit. It seems as if the single core modus makes use of the Quad-Core CPU completely? Is this assumption correct?


Kind regards,


Oliver
OL
Oliver Lojek, modified 6 Years ago.

Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Jedi Knight Posts: 111 Join Date: 4/30/13 Recent Posts
Seems it is not making use of the full power as only CPU 0 and CPU 2 are working.

Attachments:

OL
Oliver Lojek, modified 6 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Jedi Knight Posts: 111 Join Date: 4/30/13 Recent Posts
What is more, is that the machine now shows that screen where Delft preprocesses the domain for nearly 30 minutes.
I copied the files onto the Linux Server and tried the model there, it started on the jump and is at 20% completion now.

Anybody has a suggestion why the Windows machine is not starting?

Update:

I left the Windows machine running over night. To no avail. I just set up a new test model featuring a similar extend as the one I tried yesterday and it does not start either. It simply stalls in the start screen.

The machine has an Intel(R) i5-3570 CPU with 3.4 GHz and 8 Gb of DDR 333. The model features no additional processes, merely 23.000 nodes and 14 days simulation time.



Cheers,


Oliver
Adri Mourits, modified 6 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
Hi Oliver,

Have a look at "https://svn.oss.deltares.nl/repos/delft3d/trunk/examples/01_standard/run_flow2d3d_parallel.bat".
Did you execute "smpd -install" with the correct smpd executable?
What happens if you remove the flag "-localanly"?

When starting a calculation via the GUI, it does NOT run in parallel. You do need to start your calculation using a script like mentioned above.

The initialization may take long when having large input files. When looking to the working directory you then will see the creation of some temp files that are equally big. When dealing with large files, the speed depends on the system where you have your input files. If you run on a network drive with slow access, that may explain why the Windows run is much slower than the Linux run. If this is the case, the solution is to copy all files to a local drive.

Another possibility is that you are on the edge of the system capabilities. Do you see the usage of a lot of memory in the Microsoft TaskManager? Depending on your settings, your machine might start swapping to a scratch file.

Regards,

Adri
OL
Oliver Lojek, modified 6 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Jedi Knight Posts: 111 Join Date: 4/30/13 Recent Posts
Hi Adri,

I had a look at the file.

The MPI executables are not in the same directory as stated by the file. I tried copying them, which did not succeed.
If I execute the smpd.exe or the mpiexec.exe the cmd flashes up shortly, closes again and that is all.
I removed the -localonly flag, which did not work and rendered the run_parallel..bat inoperable at the second attempt.

I removed the MPICH2 from the system.

The Mpich2 is installed under programms while Delft_3D resides on the desktop due to access issues with different directories.
Could that result in the current issue? Do I need to install the Mpich2 into the same directory as Delft_3D?

Kind regards,


Oliver
OL
Oliver Lojek, modified 6 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Jedi Knight Posts: 111 Join Date: 4/30/13 Recent Posts
Or could it be due to plattform issues? I am running on 64 bit and Delft_3D currently is 32 bit under Windows?
Adri Mourits, modified 6 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit (Answer)

Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
Hi Oliver,

Delft3D-FLOW (Windows) is 32-bit, but runs on a 64-bit machine, also in parallel. It's important that the 32-bit version of mpich2 is used.

When compiling the release version, all binaries are copied to "<yoursourcecodedir>\bin\win32". This includes the copying of "<yoursourcecodedir>\src\third_party_open\mpich2\bin\smpd.exe" to "<yoursourcecodedir>\bin\win32\flow2d3d\bin\smpd.exe".

To run smpd you need administrator rights on your machine:
"Start" -> "All programs" -> "Accessories", right-click "Command Prompt", "Run as Administrator". In this command box: cd "<yoursourcecodedir>\bin\win32\flow2d3d\bin" and then:
smpd -install
When an smpd is already running, you need to end that process first, using the Task Manager.

Regards,

Adri
OL
Oliver Lojek, modified 6 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Jedi Knight Posts: 111 Join Date: 4/30/13 Recent Posts
Hi Adri,

I have admin rights on the machine I am using by now.
I followed your instructions and executed the smpd -install, which worked.

However, if I go to the examples folder and execute the 01_standard_parallel batch file it won't start.
I commented out the -localonly flag as you suggested but it did not help.

The cmd says that either mpiexec is wrongly written or was not found.
Adri Mourits, modified 6 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
Hi Oliver,

Can you try again with everything in a directory without a space in the path? May be something has problems with "Program Files". What happens when you use something like "C:\temp"?

Regards,

Adri
JF
João Paulo Ferreira, modified 2 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Padawan Posts: 32 Join Date: 10/16/15 Recent Posts
Hi Adri Mourits.

Is it possible to run a parallel simulation (7 cores) using the file "run_flow2d3d.bat" for a model with DDB (ddboundaries)?

Thank you very much!
Adri Mourits, modified 2 Years ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
Hi João,

When DomainDecomposition is used to create the model, the subdomains will be spread automatically over the available cores in the machine by the operating system, see the FAQ. You can check this (on Windows) by having a look at the Microsoft Task Manager while your computation is running.

If you want to use 7 cores for a DomainDecomposition computation, you have to divide your model in 7 subdomains.

When using parallelMPI, your model must contain one (sub)domain only. By specifying the number of partitions, the domain will be split automatically.

Regards,

Adri
MN
Magaly Rocabado, modified 1 Year ago.

RE: Delft 3D | Parallel.bat on a Quad-Core Windows 7 64-bit

Youngling Posts: 2 Join Date: 2/20/18 Recent Posts

Dear Adri,

I am using Delft3D on a workstation with Dual Intel (R) Xeon (R) 2.2 GHz processors with 24 cores.

I am trying to run Delft3D on windows in parallel mode. I followed the instructions above and executed the smpd, which worked. 

However, if I go to the examples: "01_standard_paralell" folder and execute the "run_flow2d3d_parallel.bat" file it doesnt run properly.

I'm sending the results of the run and my .bat file 


Best regards,
Magaly Navia