Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit - D-Flow Flexible Mesh - Delft3D
intro story D-Flow FM
D-Flow Flexible MeshD-Flow Flexible Mesh (D-Flow FM) is the new software engine for hydrodynamical simulations on unstructured grids in 1D-2D-3D. Together with the familiar curvilinear meshes from Delft3D 4, the unstructured grid can consist of triangles, pentagons (etc.) and 1D channel networks, all in one single mesh. It combines proven technology from the hydrodynamic engines of Delft3D 4 and SOBEK 2 and adds flexible administration, resulting in:
An overview of the current developments can be found here. The D-Flow FM - team would be delighted if you would participate in discussions on the generation of meshes, the specification of boundary conditions, the running of computations, and all kinds of other relevant topics. Feel free to share your smart questions and/or brilliant solutions!
======================================================= | Sub groups
|
Message Boards
Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
JL
João Lencart e Silva, modified 9 Years ago.
Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
Padawan Posts: 71 Join Date: 3/30/11 Recent Posts 00
Hi all,
I managed to compile D3D-Flow 32bit in a core i7 machine running 64bit Ubuntu 10.10.
My first tests were amazing: Running in a single core the simulation was 2 times faster than in a XP virtual machine (VMware) and in parallel using 7 virtual cores was 5x faster.
The problem is that I can't read the output.
I tried GPP, VSI and VS (for trih, trim and com files) and WAQ (com files) in windows and qpread, vs_use and the like in matlab Linux with no luck.
In the GUI apps all the files caused the respective program either to crash or file not found was declared (in WAQ the error log asked if I was sure i was feeding in a COM file).
Using the matlab functions (which work OK for the outputs D3D runs in XP) vs_use almost froze the machine giving out access errors.
From the 01_standards example:
This problem was common to either the single core or the multicore runs. Is this a big endiean, little endian problem?
I tried using the vs binary which resulted from the compilation of the svn source code but the program returned :
Delft3D-Flow was compiled with mpich2 1.0.2p1 and ifort 11.1 with ia32 option.
Any clues?
Regards,
João.
(Please see attached result files (trih, trim and tri-diag) for the 01_standard run.)
I managed to compile D3D-Flow 32bit in a core i7 machine running 64bit Ubuntu 10.10.
My first tests were amazing: Running in a single core the simulation was 2 times faster than in a XP virtual machine (VMware) and in parallel using 7 virtual cores was 5x faster.
The problem is that I can't read the output.
I tried GPP, VSI and VS (for trih, trim and com files) and WAQ (com files) in windows and qpread, vs_use and the like in matlab Linux with no luck.
In the GUI apps all the files caused the respective program either to crash or file not found was declared (in WAQ the error log asked if I was sure i was feeding in a COM file).
Using the matlab functions (which work OK for the outputs D3D runs in XP) vs_use almost froze the machine giving out access errors.
From the 01_standards example:
>>vs_use('trih-f34.dat') | |
??? Error using ==> vs_use at 984 | |
Error using ==> vs_use at 358 | |
Unknown data format 'L' or invalid filesize: 260792 (real size: 260784) | |
>> vs_use('trim-f34.dat') | |
Data inaccessible at location : 1953260868. | |
Data inaccessible at location : 1953260868. | |
Data inaccessible at location : 1953260868. | |
??? Error using ==> vs_use at 984 | |
Index exceeds matrix dimensions. |
This problem was common to either the single core or the multicore runs. Is this a big endiean, little endian problem?
I tried using the vs binary which resulted from the compilation of the svn source code but the program returned :
Environment Variable PAGER not set |
Delft3D-Flow was compiled with mpich2 1.0.2p1 and ifort 11.1 with ia32 option.
Any clues?
Regards,
João.
(Please see attached result files (trih, trim and tri-diag) for the 01_standard run.)
Attachments:
Bert Jagers, modified 9 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit (Answer)
Jedi Knight Posts: 201 Join Date: 12/22/10 Recent Posts 00
Hi João,
I don't think that we have compiled a 32 bit executable on a 64 bit Linux system. So, I don't know whether all settings are correct yet. Have you considered compiling a 64 bit executable?
Both the c library used by Delft3D, VS and VSI and the MATLAB version should work independent of reading/writing platform. Little and big endian switches are resolved automatically. I have looked at the trim/trih files that you provided using both vs_use and VSI. Both seem to work just fine for me on those files. Did you provide the "correct" files?
The vs_use command includes a debug option to get a full listing of all information that it finds in the file: vs_use(FILENAME,'debug'). It writes a vs_use.dbg file to your temp folder. The beginning of the log for the trih file that you provided reads:
Your new trih-file seems to be a bit shorter than is indicated in the file. I have seen this kind of behavior in the past on UNIX systems, but I hadn't encountered it recently: the file system didn't allocate space for unwritten bytes (so, the file looks smaller than it actually should be). Upon zipping the files, those bytes may be included; explaining why the files now look OK. In the past, the MATLAB code accepted a mismatch but I dropped that feature when 64bit version of NEFIS was introduced a couple of years ago since this phenomenon didn't occur anymore. Lines 1196-1213 of vs_use.m still include this type of checks but should be checked for validity in the recent code context. It would help to see what vs_use debug option gives for your files on your system.
Best regards,
Bert
I don't think that we have compiled a 32 bit executable on a 64 bit Linux system. So, I don't know whether all settings are correct yet. Have you considered compiling a 64 bit executable?
Both the c library used by Delft3D, VS and VSI and the MATLAB version should work independent of reading/writing platform. Little and big endian switches are resolved automatically. I have looked at the trim/trih files that you provided using both vs_use and VSI. Both seem to work just fine for me on those files. Did you provide the "correct" files?
The vs_use command includes a debug option to get a full listing of all information that it finds in the file: vs_use(FILENAME,'debug'). It writes a vs_use.dbg file to your temp folder. The beginning of the log for the trih file that you provided reads:
Data file : c:\trih-f34.dat | |
Definition file: c:\trih-f34.def | |
Data file opened using filehandle: 4. | |
File header: | |
Deltares, NEFIS Data File; 5.00.00 | |
File format indicator : L. | |
Real file length is : 260784 bytes | |
File length big-endian (8 bytes): 1.332897e+019 bytes | |
File length little-endian (8 bytes): 260792 bytes | |
File length big-endian (4 bytes): 3103392512 bytes | |
File length little-endian (4 bytes): 260792 bytes | |
Hash table suggests formatting as little-endian (8 bytes). | |
Header agrees with address size: 8 bytes. | |
Format flag 'L' agrees with little-endian ordering. | |
... |
Your new trih-file seems to be a bit shorter than is indicated in the file. I have seen this kind of behavior in the past on UNIX systems, but I hadn't encountered it recently: the file system didn't allocate space for unwritten bytes (so, the file looks smaller than it actually should be). Upon zipping the files, those bytes may be included; explaining why the files now look OK. In the past, the MATLAB code accepted a mismatch but I dropped that feature when 64bit version of NEFIS was introduced a couple of years ago since this phenomenon didn't occur anymore. Lines 1196-1213 of vs_use.m still include this type of checks but should be checked for validity in the recent code context. It would help to see what vs_use debug option gives for your files on your system.
Best regards,
Bert
JL
João Lencart e Silva, modified 9 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
Padawan Posts: 71 Join Date: 3/30/11 Recent Posts 00
Hi Bert,
Thanks for your answer.
I'll try zipping and unzipping the files to see if they work.
I'll also post the debug from vs_use later today.
What is the best stable release for 64bit compilation?
I had the impression that there were limitations to the model's functionality under 64bit. If so, what are they?
Cheers,
João.
Thanks for your answer.
I'll try zipping and unzipping the files to see if they work.
I'll also post the debug from vs_use later today.
What is the best stable release for 64bit compilation?
I had the impression that there were limitations to the model's functionality under 64bit. If so, what are they?
Cheers,
João.
Adri Mourits, modified 9 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit (Answer)
Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts 00
Hi Joao,
The best version to use for 64bit compilation is the current head version of the trunk:
https://svn.oss.deltares.nl/repos/delft3d/trunk, revision 892.
This version is not completely tested (the testbench is currently running with exactly this revision), but it does contain the latest 64bit modifications (revision 882).
There is no restriction with respect to functionality. The disadvantage of 64bit is that some manual adaptions are needed (hard coded paths) to get it compiled. I have planned to work on that in week 44.
The best version to use for 64bit compilation is the current head version of the trunk:
https://svn.oss.deltares.nl/repos/delft3d/trunk, revision 892.
This version is not completely tested (the testbench is currently running with exactly this revision), but it does contain the latest 64bit modifications (revision 882).
There is no restriction with respect to functionality. The disadvantage of 64bit is that some manual adaptions are needed (hard coded paths) to get it compiled. I have planned to work on that in week 44.
BW
Brian White, modified 9 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
Youngling Posts: 2 Join Date: 4/27/11 Recent Posts 00
Hi Adri,
I would like to compile on 64 bit Linux. Is the current version the best place to start? Also, are the manual adaptations you referred to in your post still necessary or have new modifications been made? If manual, could you advise whether there is any guide available for this?
Many thanks,
Brian
I would like to compile on 64 bit Linux. Is the current version the best place to start? Also, are the manual adaptations you referred to in your post still necessary or have new modifications been made? If manual, could you advise whether there is any guide available for this?
Many thanks,
Brian
Adri Mourits, modified 9 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts 00
Hi Brian,
My remarks from my previous post still holds: The current trunk is the best candidate to use. I started our testbench Today with revision 1080. Keep your fingers crossed...
Regards,
Adri
My remarks from my previous post still holds: The current trunk is the best candidate to use. I started our testbench Today with revision 1080. Keep your fingers crossed...
Regards,
Adri
JL
João Lencart e Silva, modified 9 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
Padawan Posts: 71 Join Date: 3/30/11 Recent Posts 00
Bert,
It turns out that I was using a very old version of vs_use... (hence the Unknown L file format)
I'm now up date and able to read outputs.
Thanks for your support!
João.
It turns out that I was using a very old version of vs_use... (hence the Unknown L file format)
I'm now up date and able to read outputs.
Thanks for your support!
João.
U
lei guo, modified 8 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
00
Hi, I have a question on using vs_use and vs_get in accessing the velocity components from the trih- files.
I found that the output data by vs_get(F, 'his-series', 'ZCURU' or 'ZCURV' ) is the m and n components as that from the Quickplot, but not the x and y components.
It seems I need to do some conversion based on the local grid alignment which is site specific...
so I am wondering is there a way or already made matlab script to do that??
Candle Guo
I found that the output data by vs_get(F, 'his-series', 'ZCURU' or 'ZCURV' ) is the m and n components as that from the Quickplot, but not the x and y components.
It seems I need to do some conversion based on the local grid alignment which is site specific...
so I am wondering is there a way or already made matlab script to do that??
Candle Guo
Adri Mourits, modified 8 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
Yoda Posts: 1221 Join Date: 1/3/11 Recent Posts 00
Hi Candle Guo,
Yes, the conversion is:
zcurx(ii,k) = zcuru(ii,k)*cos(alfas(ii)) - zcurv(ii,k)*sin(alfas(ii))
zcury(ii,k) = zcuru(ii,k)*sin(alfas(ii)) + zcurv(ii,k)*cos(alfas(ii))
with alfas from 'his-const','ALFAS' in degrees.
See "src\tools_lgpl\matlab\quickplot\progsrc\private\cur2ca.m".
Regards,
Adri
Yes, the conversion is:
zcurx(ii,k) = zcuru(ii,k)*cos(alfas(ii)) - zcurv(ii,k)*sin(alfas(ii))
zcury(ii,k) = zcuru(ii,k)*sin(alfas(ii)) + zcurv(ii,k)*cos(alfas(ii))
with alfas from 'his-const','ALFAS' in degrees.
See "src\tools_lgpl\matlab\quickplot\progsrc\private\cur2ca.m".
Regards,
Adri
U
lei guo, modified 8 Years ago.
RE: Can't read D3D-Flow output: Ubuntu 10.10 64bit Compile at 32bit
00
Hi, Adri,
Got it, thanks very much.
Candle Guo
Got it, thanks very much.
Candle Guo