intro story D-Flow FM

 

D-Flow Flexible Mesh

D-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:

  • Easier 1D-2D-3D model coupling, intuitive setup of boundary conditions and meteorological forcings (amongst others).
  • More flexible 2D gridding in delta regions, river junctions, harbours, intertidal flats and more.
  • High performance by smart use of multicore architectures, and grid computing clusters.
An overview of the current developments can be found here.
 
The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions! 

 

=======================================================
We have launched a new website (still under construction so expect continuous improvements) and a new forum dedicated to Delft3D Flexible Mesh.

Please follow this link to the new forum: 
/web/delft3dfm/forum

Post your questions, issues, suggestions, difficulties related to our Delft3D Flexible Mesh Suite on the new forum.

=======================================================

** PLEASE TAG YOUR POST! **

 

 

Sub groups
D-Flow Flexible Mesh
DELWAQ
Cohesive sediments & muddy systems

 


Message Boards

GPU - CUDA computation in Delft3D ?

Bogdan Hlevca, modified 6 Years ago.

GPU - CUDA computation in Delft3D ?

Youngling Posts: 23 Join Date: 3/28/12 Recent Posts
I posted this question in the Linked in group, but the group is moderated and is seems that the massage has been in the Pending zone for the last two day with no perspective to move in the public zone.

So here we go:

I would like to know if Delft3D Flow has API that would allow computation within a multimedia card. This is called GPU computing and Computer Unified Device Architecture (CUDA) is used by NVIDIA cards ATI cards have a different API.

I could not find any information about this, therefore, I suspect that it is not possible
However, even if I know that I can use parallel cluster of computers, a GPU solution would make sense and it would give a great boost in processing speed in a single computer setup.

Thanks,
Bogdan
Adri Mourits, modified 6 Years ago.

RE: GPU - CUDA computation in Delft3D ?

Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
Hi Bogdan,

That's a very interesting topic.

To get Delft3D-FLOW running on GPUs, you will need to recompile the full set of source code; there is no API you can use.

As far as I know, nobody tried this before.

I must admit that I don't expect miracles. Delft3D-FLOW has a rather implicit algorithm, needing a lot of data exchange between the partitions. GPUs don't like that.

Please post your findings here.

Thanks.

Regards,

Adri
Bogdan Hlevca, modified 6 Years ago.

RE: GPU - CUDA computation in Delft3D ?

Youngling Posts: 23 Join Date: 3/28/12 Recent Posts
Hi Adri,


Some documentation can be found here : http://www.nvidia.ca/object/cuda_home_new.html

It appears that NVIDIA has a toolkit https://developer.nvidia.com/cuda-toolkit
that is comprised of a compiler and API that allows you to customize parallelization with GPU and CPU programming.

They support all operating systems that are supported by Delft3D https://developer.nvidia.com/cuda-downloads

I installed the toolkit and I will see what can be done to make Delft3D compile with CUDA bindings. I will have to look at the NVIDIA examples first and compile them.


Regards,
Bogdan
Bert Jagers, modified 6 Years ago.

RE: GPU - CUDA computation in Delft3D ?

Jedi Knight Posts: 201 Join Date: 12/22/10 Recent Posts
Hi Bogdan,

There are a couple of ways in which you can make code ready for (NVIDIA) GPUs.
  • Direct coding for NVIDIA GPU in CUDA. This is gives you most flexibility and most options to fine-tune the data flow. However, as far as I know, this would imply rewriting significant parts of the Delft3D code.
  • Replace regular BLAS libraries, by GPU optimized BLAS libraries. Unfortunately, Delft3D uses it's own internal solvers and no standard external solver. So, there is nothing to replace.
  • Use OpenACC compiler directives and let the compiler generate GPU specific code. This seems currently the most easiest and therefore attractive option for Delft3D. See this page of NVIDIA and the OpenACC Home for more information about OpenACC.


Best regards,

Bert
Bogdan Hlevca, modified 6 Years ago.

RE: GPU - CUDA computation in Delft3D ?

Youngling Posts: 23 Join Date: 3/28/12 Recent Posts
Bert Jagers:
  • Use OpenACC compiler directives and let the compiler generate GPU specific code. This seems currently the most easiest and therefore attractive option for Delft3D. See this page of NVIDIA and the OpenACC Home for more information about OpenACC.


  • Hi Bert

    Obviously the third option is the easiest. Thanks for pointing me to it.

    Since I am not familiar with the code I would probably need some help from Delft3D programmers to identify the files that are best to be provided with compiler directives. Probably looking for code that targets parallel programming would be a good start.

    This could be a big effort and it should be shared with the community so that others would benefit.

    Who should I contact to ask for help and if these compiler directives are inserted for code that can run in parallel?
    I believe that if this task is done it should go in the main code after tested on a branch.

    Regards,
    Bogdan
    Adri Mourits, modified 6 Years ago.

    RE: GPU - CUDA computation in Delft3D ?

    Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
    Hi Bogdan,

    I'm the first contact point.

    Do you want me to create a branch for you?

    Regards,

    Adri
    Bogdan Hlevca, modified 6 Years ago.

    RE: GPU - CUDA computation in Delft3D ?

    Youngling Posts: 23 Join Date: 3/28/12 Recent Posts
    Adri Mourits:

    Do you want me to create a branch for you?


    Hi Adri,

    I'd love to do this. However, I don't have time to do it until late January early February.
    Would you be able to point me in the right direction, i.e. the files/code sections that can benefit parallelized computation?

    I did some tests with examples provided by Nvidia and the results are stunning.

    Please let me know if anyone can give me the information about the files that host the heavy computation code and I may be able to help with code instrumentation.
    I think we should do the branch closer to the time I can work on it so it is closer to the trunk.


    Regards,
    Bogdan

    PS. are you using SVN only or is GIT or mercurial an option?
    Adri Mourits, modified 6 Years ago.

    RE: GPU - CUDA computation in Delft3D ? (Answer)

    Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
    Hi Bogdan,

    Yes, I can help you with that. You can already have a look at the key calculation routines in https://svn.oss.deltares.nl/repos/delft3d/trunk/src/engines_gpl/flow2d3d/packages/kernel/src/compute:
    cucnp.f90
    difu.f90 (difuvl.f90, z_difu.f90)
    sud.f90
    tratur.f90
    uzd.f90 (z_uzd.f90)

    We are using SVN only.

    Regards,

    Adri
    Bogdan Hlevca, modified 6 Years ago.

    RE: GPU - CUDA computation in Delft3D ?

    Youngling Posts: 23 Join Date: 3/28/12 Recent Posts
    Hi Adri,

    I will contact you at the end of January. I am a little busy until then and I think is better to take branches as close as possible to the main trunk.

    Meanwhile, I am going to have a look at those files although I am a little rusty with Fortran and I do much better with C++.

    Lately I am in love with Python and I see the Deltares is moving slowly in that direction too. It may be of interest to you to see this video at this page:

    https://developer.nvidia.com/how-to-cuda-python

    Regards,
    Bogdan
    Adri Mourits, modified 6 Years ago.

    RE: GPU - CUDA computation in Delft3D ?

    Yoda Posts: 1224 Join Date: 1/3/11 Recent Posts
    Hi Bogdan,

    That's fine.

    Yes, we're using Python more and more.

    Thanks for the cuda-python link.

    Regards,

    Adri