intro story D-Flow FM

 

D-Flow Flexible Mesh

D-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:

  • Easier 1D-2D-3D model coupling, intuitive setup of boundary conditions and meteorological forcings (amongst others).
  • More flexible 2D gridding in delta regions, river junctions, harbours, intertidal flats and more.
  • High performance by smart use of multicore architectures, and grid computing clusters.
An overview of the current developments can be found here.
 
The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions! 

 

=======================================================
We have launched a new website (still under construction so expect continuous improvements) and a new forum dedicated to Delft3D Flexible Mesh.

Please follow this link to the new forum: 
/web/delft3dfm/forum

Post your questions, issues, suggestions, difficulties related to our Delft3D Flexible Mesh Suite on the new forum.

=======================================================

** PLEASE TAG YOUR POST! **

 

 

Sub groups
D-Flow Flexible Mesh
DELWAQ
Cohesive sediments & muddy systems

 


Message Boards

build error....mpi issue?

SJ
Steven Jachec, modified 7 Years ago.

build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
I recently installed unbuntu 12.04 linux and the latest Delft3D.

Once downloaded Delft3d source code (and I install autoconf, flex, mpich2, gfrotran, etc.....like in the webinar), I type in the src directory:

$ ./build.sh -gnu -64bit

It fails during make stage. Looking at the end of the make.log file I get .................


Making install in parallel_mpi
make[6]: Entering directory `/home/sjachec/research/delft3d_repository/src/engines_gpl/flow2d3d/packages/data/src/parallel_mpi'
/bin/bash ../../../../../../libtool --tag=FC --mode=compile /opt/mpich2-1.4.1-gcc-4.6.2/bin/mpif90 -DHAVE_CONFIG_H -I. -I../../../../../.. -I/opt/mpich2-1.4.1-gcc-4.6.2/include -I../../../../../../utils_lgpl/parallel_mpi/include -I../../../../../../utils_lgpl/precision/include -I../../../../../../engines_gpl/flow2d3d/packages/data/include -I../../../../../../utils_lgpl/precision/packages/precision/src -I../../../../../../engines_gpl/flow2d3d/packages/data/src/parallel_mpi -I../../../../../../engines_gpl/flow2d3d/packages/data/src/basics -I../../../../../../utils_lgpl/ec_module/packages/ec_module/src -I../../../../../../utils_lgpl/inifiles/packages/inifiles/src -I../../../../../../utils_lgpl/utils_flow/packages/utils_flow/src -DDFMPI -O2 -fPIC -m64 -fopenmp -ffree-line-length-none -cpp -c -o dfparall.lo dfparall.F90
libtool: compile: /opt/mpich2-1.4.1-gcc-4.6.2/bin/mpif90 -DHAVE_CONFIG_H -I. -I../../../../../.. -I/opt/mpich2-1.4.1-gcc-4.6.2/include -I../../../../../../utils_lgpl/parallel_mpi/include -I../../../../../../utils_lgpl/precision/include -I../../../../../../engines_gpl/flow2d3d/packages/data/include -I../../../../../../utils_lgpl/precision/packages/precision/src -I../../../../../../engines_gpl/flow2d3d/packages/data/src/parallel_mpi -I../../../../../../engines_gpl/flow2d3d/packages/data/src/basics -I../../../../../../utils_lgpl/ec_module/packages/ec_module/src -I../../../../../../utils_lgpl/inifiles/packages/inifiles/src -I../../../../../../utils_lgpl/utils_flow/packages/utils_flow/src -DDFMPI -O2 -fPIC -m64 -fopenmp -ffree-line-length-none -cpp -c dfparall.F90 -fPIC -o .libs/dfparall.o
../../../../../../libtool: line 1125: /opt/mpich2-1.4.1-gcc-4.6.2/bin/mpif90: No such file or directory
make[6]: *** [dfparall.lo] Error 1
make[6]: Leaving directory `/home/sjachec/research/delft3d_repository/src/engines_gpl/flow2d3d/packages/data/src/parallel_mpi'
make[5]: *** [install-recursive] Error 1
make[5]: Leaving directory `/home/sjachec/research/delft3d_repository/src/engines_gpl/flow2d3d/packages/data/src'
make[4]: *** [install-recursive] Error 1
make[4]: Leaving directory `/home/sjachec/research/delft3d_repository/src/engines_gpl/flow2d3d/packages/data'
make[3]: *** [install-recursive] Error 1
make[3]: Leaving directory `/home/sjachec/research/delft3d_repository/src/engines_gpl/flow2d3d/packages'
make[2]: *** [install-recursive] Error 1
make[2]: Leaving directory `/home/sjachec/research/delft3d_repository/src/engines_gpl/flow2d3d'
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory `/home/sjachec/research/delft3d_repository/src/engines_gpl'
make: *** [install-recursive] Error 1



It seems like it cannot find mpif90....it's looking in /opt/mpich2-1.4.1-gcc-4.6.2/bin ...... but I don't have mpich2 installed there; I just did the apt-get install mpich2 command (as in the webinar). If I ask "which mpif90" I get \usr\bin

As a fix, I edit the build.sh file (attached) and look for the problem line (/opt/mpich2-1.4.1-gcc-4.6.2/bin); I change to export MPIFC=mpif90 (and lose the /opt/.... path).

....but still no luck. Although it completes the build without error, it seems no executbles are produced.

Next steps?

Thanks,
Steven

Attachments:

SJ
Steven Jachec, modified 7 Years ago.

RE: build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
I found a potential fix on Forum Group >> General called "Any joy with Ubuntu 12.04" on 6/24/2012 thread.

The fix was related to mpich2 as I thought.

I clean: \clean.sh

Next, I invoke: ./build.sh -gnu -64bit [[[[[This build.sh script includes the fixes]]]]]

Based on the screen comments, it concludes with "Build finished" So, it seems all went well. When I look at the end of the make.log file, i get......


oss-install...
Target directory: ../bin
Source : all engines
installing all open source projects . . .
installing d_hydro . . .
installing flow2d3d . . .
cp: cannot stat `lib/libflow2d3d_sp.so': No such file or directory
installing flow2d3d_openda . . .
cp: cannot stat `lib/libflow2d3d_openda_sp.so': No such file or directory
installing wave . . .
installing plugin_culvert . . .
installing plugin_delftflow_traform . . .
installing datsel . . .
installing kubint . . .
installing lint . . .
installing mormerge . . .
installing vs . . .



Is that how the make.log file should conclude???
Adri Mourits, modified 7 Years ago.

RE: build error....mpi issue?

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Steven,

Yes.

Now you should be able to run the examples without any copying of binaries.

Regards,

Adri
SJ
Steven Jachec, modified 7 Years ago.

RE: build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
Dear A,

When I go the examples directory: ~\research\delft3d_repository\examples\01_standard, at the command line I type

$ ./run_flow2d3d.sh


After I do this, I get nothing....just the next terminal prompt
$

No text flies by on the screen, nor do I see any log files generated to try to assess the problem. No errors are displayed either.

What am I doing wrong? How to identify the problem?

Thanks,
Steven
run
Adri Mourits, modified 7 Years ago.

RE: build error....mpi issue?

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Steven,

That is not possible: The scripts executes "../../bin/lnx/flow2d3d/bin/deltares_hydro.exe". If this binary is missing it will report that. If it is not missing it will do something.

The Bash script interpreter is used to execute "run_flow2d3d.sh". Is that installed?

Does directory "../../bin/lnx/flow2d3d/bin" exist? What does it contain?

Does file "src/logs/make.log" contain an error?

Regards,

Adri
SJ
Steven Jachec, modified 7 Years ago.

RE: build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
Dear A,

1) Yes the directory does exists and here is the contents of the directory:

sjachec:~/research/delft3d_repository/bin/lnx/flow2d3d/bin$ ls -l
total 7588
-rwxr-xr-x 1 sjachec sjachec 158652 Feb 17 15:46 datsel
-rwxrwxr-x 1 sjachec sjachec 1555764 Feb 16 19:04 deltares_hydro.exe
-rwxr-xr-x 1 sjachec sjachec 31692 Feb 17 15:46 d_hydro.exe
-rwxr-xr-x 1 sjachec sjachec 38086 Feb 17 15:41 esm_create
-rwxr-xr-x 1 sjachec sjachec 38056 Feb 17 15:41 esm_delete
-rwxr-xr-x 1 sjachec sjachec 38010 Feb 17 15:41 esm_info
-rwxr-xr-x 1 sjachec sjachec 48994 Feb 17 15:46 kubint
-rwxr-xr-x 1 sjachec sjachec 80184 Feb 17 15:45 libflow2d3d_openda.so
-rwxr-xr-x 1 sjachec sjachec 5569531 Feb 17 15:45 libflow2d3d.so
-rwxr-xr-x 1 sjachec sjachec 48970 Feb 17 15:46 lint
-rwxr-xr-x 1 sjachec sjachec 95069 Feb 17 15:46 mormerge.exe
-rwxr-xr-x 1 sjachec sjachec 21351 Feb 17 15:46 plugin_culvert.so
-rwxr-xr-x 1 sjachec sjachec 21847 Feb 17 15:46 plugin_delftflow_traform.so
sjachec:~/research/delft3d_repository/bin/lnx/flow2d3d/bin$


2) I am also attaching the make.log file for your review.


3) Finally, here are my contents in the directory 01_standard:

-rw-rw-r-- 1 sjachec sjachec 249 Feb 16 19:09 config_flow2d3d.ini
-rw-rw-r-- 1 sjachec sjachec 3164 Feb 16 19:09 f34.bca
-rw-rw-r-- 1 sjachec sjachec 80 Feb 16 19:09 f34.bch
-rw-rw-r-- 1 sjachec sjachec 60 Feb 16 19:09 f34.bnd
-rw-rw-r-- 1 sjachec sjachec 192 Feb 16 19:09 f34.crs
-rw-rw-r-- 1 sjachec sjachec 4004 Feb 16 19:09 f34.dep
-rw-rw-r-- 1 sjachec sjachec 464 Feb 16 19:09 f34.dis
-rw-rw-r-- 1 sjachec sjachec 28 Feb 16 19:09 f34.dry
-rw-rw-r-- 1 sjachec sjachec 276 Feb 16 19:09 f34.enc
-rw-rw-r-- 1 sjachec sjachec 17074 Feb 16 19:09 f34.grd
-rw-rw-r-- 1 sjachec sjachec 1299 Feb 16 19:09 f34.ldb
-rw-rw-r-- 1 sjachec sjachec 3372 Feb 16 19:09 f34.mdf
-rw-rw-r-- 1 sjachec sjachec 175 Feb 16 19:09 f34.obs
-rw-rw-r-- 1 sjachec sjachec 53 Feb 16 19:09 f34.par
-rw-rw-r-- 1 sjachec sjachec 50 Feb 16 19:09 f34.src
-rw-rw-r-- 1 sjachec sjachec 30 Feb 16 19:09 f34.thd
-rw-rw-r-- 1 sjachec sjachec 51 Feb 16 19:09 f34.wnd
-rw-rw-r-- 1 sjachec sjachec 0 Feb 17 15:52 machinefile
-rwxrwxr-x 1 sjachec sjachec 877 Feb 16 19:09 run_flow2d3d.bat
-rw-rw-r-- 1 sjachec sjachec 1358 Feb 16 19:09 run_flow2d3d_parallel.bat
-rwxrwxr-x 1 sjachec sjachec 2831 Feb 16 19:09 run_flow2d3d_parallel_sge.sh
-rwxrwxr-x 1 sjachec sjachec 2181 Feb 16 19:09 run_flow2d3d_parallel.sh
-rwxrwxr-x 1 sjachec sjachec 712 Feb 16 19:09 run_flow2d3d.sh

I just double-checked by running:

$ ./run_flow2d3d.sh

The result is just a carriege return in the terminal.

4) I'm not sure about the Bash interpreter. How can I check it exists and is being applied here?

Next steps to try?????

Thanks,
Steven

Attachments:

Adri Mourits, modified 7 Years ago.

RE: build error....mpi issue?

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Steven,

Everything looks fine.

If you execute:
type bash
then Linux will report the location of the bash executable, for example:
bash is /bin/bash
If bash isn't installed, you'll get some error message.

What happens if you put some write statements in the run script before and after the execution line, like:
echo "Hello world."

Regards,

Adri
SJ
Steven Jachec, modified 7 Years ago.

RE: build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
Dear A,

When I:

$ type bash

I get:

/bin/bash



I added some echo statements to run_floe2d3d.sh in 01_examples directory (attached for you). I run the code and get.....

$ ./run_flow2d3d.sh
Starting the solve.....
Done running the code.

This shows it's getting through the script but really not executing anything.

Next steps?

Steven
Adri Mourits, modified 7 Years ago.

RE: build error....mpi issue?

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Steven,

Go to directory "/research/delft3d_repository/bin/lnx/flow2d3d/bin" and execute:
./deltares_hydro.exe
This should result in:
ERROR: Start "deltares_hydro.exe" with 1 or 2 arguments.
Usage:
deltares_hydro.exe <config.ini> [-keepXML]
<config.ini> : Name of configuration file in ini format
Example configuration file:
[Component]
Name = flow2d3d
MdfFile = f34
-keepXML : Do not remove temporary file "TMP_config_flow2d3d_<processId>.xml"

And if you execute:
./d_hydro.exe
the result should be:
d_hydro ABORT: Improper usage. Execute "d_hydro.exe -?" for command-line syntax

Another suggestion:
Does "run_flow2d3d_xml.sh" in directory "...\examples\01_standard" work?

Regards,

Adri
SJ
Steven Jachec, modified 7 Years ago.

RE: build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
Dear A,

I looked at the three items below you suggested.

1) When I execute "./deltares_hydro.exe" in the 01_standard example directory....I just get a carriage return.

2) When I execute "./d_hydro.exe " I do get the error below you said I should get.

3) I do not have the file "run_flow2d3d_xml.sh"

Thanks,
Steven
Adri Mourits, modified 7 Years ago.

RE: build error....mpi issue?

Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts
Hi Steven,

I think the problem is related to "deltares_hydro.exe" being a precompiled, 32-bit executable. The problem may be solved if you can enable execution of 32-bit binaries on your machine.

I expect that you use an "old" version of the Delft3D source code. The latest tagged version does contain "run_flow2d3d_xml.sh" in example 1. Can you use svn to download this latest tagged version? If you compile it, and if my analysis is correct, I expect that "run_flow2d3d_xml.sh" will not show this problem.

We are working on removing "deltares_hydro.exe".

Regards,

Adri
SJ
Steven Jachec, modified 7 Years ago.

RE: build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
Dear A,

I created a new directory on my machine and downloaded the latest tagged version: 5.01.2163.

I modified the build.sh file as before to account for the MPI issue (trying it as-is without that modification resulted in a failed build). With the modified build.sh script I got a successful build using ./build.sh -gnu -64bit

When I go into the delft3d/bin/lnx/flow2d3d/bin directory, deltares_hydro.exe and d_hydro.exe are present. When I execute ./deltares_hydro.exe I just get a carriage return; no error messages are produced. When I execute ./d_hydro.exe I get the error message (like you said I should get). Which is the "real" executable?....deltares_hydro.exe or d_hydro.exe???

When I go into the examples/01_standard directory, I DO see the run_flow2d3d_xml.sh file. When I execute, it looks like it runs and I get output (another scientist confirmed that indeed Delft3D output files were produced). This is good news! When I execute ./run_flow2d3d.sh....just a carriage return. What's the difference between these two similar scripts?


I also have tried ./build.sh -gnu (no -64bit flag). The above results do not change.

I think we're getting there....next steps?

Thanks for your continued help,
Steven
DN
Dan Nowacki, modified 7 Years ago.

RE: build error....mpi issue? (Answer)

Youngling Posts: 6 Join Date: 2/29/12 Recent Posts
Steven,

As Adri mentioned above the non-xml script calls the 32-bit deltares_hydro.exe, while the xml script calls d_hydro.exe. This is the primary difference between the two shell scripts. You want to use the XML script that calls d_hydro, which is the "real" Delft3D and the only binary that will run on your 64-bit machine. You can check out my Ubuntu compilation notes, and this other thread, which might be helpful to you as well.

Dan
SJ
Steven Jachec, modified 7 Years ago.

RE: build error....mpi issue?

Youngling Posts: 14 Join Date: 7/9/12 Recent Posts
Dear Adri and Dan,

Thanks a bunch for your help.

All is up and running.

Steven