Message Boards

Back

Long computation times while writting to netCDF.

JL
João Lencart e Silva, modified 5 Months ago.

Long computation times while writting to netCDF.

Padawan Posts: 70 Join Date: 3/30/11 Recent Posts

Hi All,

 

I've been compiling in Ubuntu the brand new (for me at least) r7545 and when I test writing to netCDF with the provide examples I get ~12x longer computation run times than with the standard NEFIS files.
Is this what is expected or do I need to change my netCDF compiling options?

I'm still on Ubuntu 14.04 and using the netCDF library out of the main repo ( 1:4.1.3-7ubuntu2).

Cheers,

 

João.

netcdf nefis run time ubuntu 14.04
JL
João Lencart e Silva, modified 2 Months ago.

RE: Long computation times while writting to netCDF.

Padawan Posts: 70 Join Date: 3/30/11 Recent Posts

Still no answer on this. Has anyone tested the speed of outputting to netCDF ( FlNcdf = #map his#)?

 

João.

Adri Mourits, modified 2 Months ago.

RE: Long computation times while writting to netCDF.

Yoda Posts: 1214 Join Date: 1/3/11 Recent Posts

Hi João,

example_01(Nefis), CentOS: 2.09 sec
example_01(Nefis), Windows10: 2.27 sec

example_11(NetCDF), CentOS: 1.94 sec
example_11(NetCDF), Windows10: 1.47 sec

I used revision 63032 and 63014.

So without compression, producing NetCDF files seems to be faster than producing Nefis files.

Regards,

Adri

 

JL
João Lencart e Silva, modified 2 Months ago.

RE: Long computation times while writting to netCDF.

Padawan Posts: 70 Join Date: 3/30/11 Recent Posts

Thanks Adri,

I'll try your revisions. With r62538 I get:

example_01 (Nefis): 1.27s

example_01 (neCDF): 13.44s

example_11 (netCDF): 29.14 s

and it prints an extra line outputting to the screen for both examples using netCDF:

Time to finish   0s,   97.7% completed, time steps left  7
           4           9          12           9

instead of the usual

Time to finish   0s,   97.0% completed, time steps left  9

It may be my build of libnetcdf. I'm using it from the Ubuntu 14.04 repo. Do you build you're own? What version and switches?

Adri Mourits, modified 2 Months ago.

RE: Long computation times while writting to netCDF.

Yoda Posts: 1214 Join Date: 1/3/11 Recent Posts

Hi João,

That additional line does not ring a bell.

We compiled the NetCDF library ourself. Below is our build script, including version numbers.

#! /bin/bash

module load intel/16.0.3
module load hdf5/1.10.1_intel_16.0.3

module list
echo "$LD_LIBRARY_PATH"

export FC=ifort
export F77=$FC
export CC=icc
export CXX=icpc
export H5DIR="/opt/hdf5/1.10.1_intel_16.0.3"
export NETCDFROOT="/opt/netcdf/v4.6.1_v4.4.0_intel_16.0.3"

CPPFLAGS=-I$NETCDFROOT/include LDFLAGS=-L$NETCDFROOT/lib LD_LIBRARY_PATH=$NETCDFROOT/lib:$LD_LIBRARY_PATH ./configure --prefix=/opt/netcdf/v4.6.1_v4.4.0_intel_16.0.3 --disable-dap

make
make install

Regards,

Adri