intro story D-Flow FM

 

D-Flow Flexible Mesh

D-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:

  • Easier 1D-2D-3D model coupling, intuitive setup of boundary conditions and meteorological forcings (amongst others).
  • More flexible 2D gridding in delta regions, river junctions, harbours, intertidal flats and more.
  • High performance by smart use of multicore architectures, and grid computing clusters.
An overview of the current developments can be found here.
 
The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions! 

 

=======================================================
We have launched a new website (still under construction so expect continuous improvements) and a new forum dedicated to Delft3D Flexible Mesh.

Please follow this link to the new forum: 
/web/delft3dfm/forum

Post your questions, issues, suggestions, difficulties related to our Delft3D Flexible Mesh Suite on the new forum.

=======================================================

** PLEASE TAG YOUR POST! **

 

 

Sub groups
D-Flow Flexible Mesh
DELWAQ
Cohesive sediments & muddy systems

 


Message Boards

Size limitation for simple domain grid?

EM
Ernesto Molinas, modified 7 Years ago.

Size limitation for simple domain grid?

Youngling Posts: 8 Join Date: 1/27/12 Recent Posts
I'm trying to figure if there is a size limitation for simple domain grids. The model i tried to run was quite large (about 61.000 cells) and i found the following errors:

*** MESSAGE Executable for 32-bits platform.
*** MESSAGE Double precision computation using reals of kind 8
*** ERROR in double array allocation of wrkc3
Contact Deltares
*** ERROR Flow exited abnormally

After this,I derefined the grid and it worked. does it make sense? Is there any limitation for grid size?
Could Domain Decomposition be a good solution for this issue?

thanks,
Qinghua Ye, modified 7 Years ago.

RE: Size limitation for simple domain grid? (Answer)

Jedi Council Member Posts: 612 Join Date: 3/2/11 Recent Posts
Hi Ernesto,

It is possible because the model is too big and windows cannot handle such big heap size. You can check how much memory is used by Delft3D-FLOW, using the task manager (Windows) or the "top"-command (Linux).

There are 3 options:

1) Simplify your model; decrease the number of cells in a dimension, or the number of processes/constituents
2) Use the single precision executable of Delft3D-FLOW (see information below)
3) Using Linux 64-bit version.

2. Use the single precision Delft3D-FLOW executable
===================================================
With the following steps, the single precision binary is used instead of the double precision binary:
a) Go to the directory where Delft3D is installed. Go to subdirectory "delft3d/w32/flow/bin"
b) Disable the double precision binary:
Rename "flow2d3d.dll" into "flow2d3d_dp.dll"
c) Enable the single precision binary:
Rename "flow2d3d_sp.dll" into "flow2d3d.dll"
d) Check what version is used when running your model:
A file is created each time a calculation is started named "tri-diag.<runid>", containing one of the following lines:
*** MESSAGE Double precision computation using reals of kind 8
*** MESSAGE Single precision computation using reals of kind 4

Regards,

Qinghua
EW
Edwin Wang, modified 5 Years ago.

RE: Size limitation for simple domain grid?

Youngling Posts: 8 Join Date: 12/17/13 Recent Posts
Hi Qinghua,

I recently set up a model with over 150,000 grid points and plus 20 layers. I ran into the memory issue. I switched to single-precision following the instruction you provided in this thread. The model works fine. However, when I activated "Salinity" and set initial salinity to 31, I am getting vertical wiggle warning "“*** WARNING Vertical wiggle for l, m ,n, nst: 1 81 10 737" within a few time steps. I have already turned on Forrester Filter. Any thought on why this happened?

Edwin
Adri Mourits, modified 5 Years ago.

RE: Size limitation for simple domain grid? (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Edwin,

I don't expect a relation between "single precision" and "salinity induced verticle wiggles". Did you switch on salinity in all subdomains? Is your initial salinity condition stable enough? Are your open boundary conditions correct? Can you analyse the ponts/cells where the instability occur? Are they close to an open boundary, or DD boundary?

Regards,

Adri
EW
Edwin Wang, modified 5 Years ago.

RE: Size limitation for simple domain grid?

Youngling Posts: 8 Join Date: 12/17/13 Recent Posts
Hi Adri,

Thanks for the response.

As part of the debugging process, I set up a simple model instead of DD models.

Here is what I did:
- model domain: a rectangular basin with no open boundaries.
- number of layers: 15
- initial condition: uniform value of 31.0
- Forrester filter = YES for horizontal and vertical.

When I ran the model using the single precision dll file (flow2d3d_sp.dll) and received the vertical wiggle warning. I looked at the output and salinity profile is wiggled as one would expect... but the salinity difference between layers is really small, like in the order of 1e-4 or so.

When I ran the model using the original double precision dll file, no warning message and the salinity profile looks uniform. This was why I think single precision is the cause to the problem.

Edwin
Adri Mourits, modified 5 Years ago.

RE: Size limitation for simple domain grid? (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Edwin,

It seems that the vertical Forester filter is doing its work. The maximum number of iterations is 1000. The warning occurs when there are still wiggles after 1000 iterations. It's a notification that if instabilities occur in that area it will probably have a numerical reason. (You should decrease the timestep size if that happens.) But if the wiggles are acceptably small, you shouldn't be worried.

If the warning only occurs in single precision, the values of the related parameters are probably in the range of "single precision" machine precision (1e-6, 1e-8). The resulting wiggles in the order of 1e-4 also point in that direction. The additional precision in "double precision" mode might just be enough to need less Forester filter iterations and avoid the generation of the related warning.

To summarize: If the results are fine, don't bother about this warning.

Regards,

Adri
EW
Edwin Wang, modified 5 Years ago.

RE: Size limitation for simple domain grid?

Youngling Posts: 8 Join Date: 12/17/13 Recent Posts
Hi Adri,

Thanks. One last question with respect to this subject - Is there a keyword to modify the number of iterations or is this hard coded and would require recompiling the program?

Edwin
Adri Mourits, modified 5 Years ago.

RE: Size limitation for simple domain grid? (Answer)

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Edwin,

The number of Forester filter iterations is hard coded. Search for parameter "maxfil" in ".../src/engines_gpl/flow2d3d/packages/kernel/src/compute/forfil.f90" and/or "z_forfil.f90". When changing it, recompilation is needed.

Regards,

Adri
EW
Edwin Wang, modified 5 Years ago.

RE: Size limitation for simple domain grid?

Youngling Posts: 8 Join Date: 12/17/13 Recent Posts
Thanks.
KO
Kyssyanne Oliveira, modified 3 Years ago.

RE: Size limitation for simple domain grid?

Youngling Posts: 13 Join Date: 9/4/13 Recent Posts
Hi Qinghua,

I recently set up a model with over 132,864 grid points (grid resolution 300 meters) and plus 14 layers. My time step is 0.4 minute.
I had a memory issue when the sediment process was activated. The error below appeared:
*** MESSAGE Executable for 32-bits platform.
*** MESSAGE Double precision computation using reals of kind 8
*** ERROR in double array allocation of wrkc3
Contact Deltares
*** ERROR Flow exited abnormally

I switched to single-precision following the instruction you provided in the forum.
After this, I am getting vertical wiggle warning "“*** WARNING Vertical wiggle for l, m, n, nst:", since the first time step. The warning appears near the "continental shelf break", where the bottom has a steep gradient.
In my model, I have waves (stationary model + wind), tidal, salinity and now sediment process (two sediment fractions - 1 non-cohesive and 1 cohesive).
Qinghua Ye, modified 3 Years ago.

RE: Size limitation for simple domain grid?

Jedi Council Member Posts: 612 Join Date: 3/2/11 Recent Posts
Hi Kyssyanne,

If you used the latest release in the service package or compiled a 64-bits version binary by yourself, the problem should be solved.

Greetings,

Qinghua