Skip to content

NetCDF

NetCDF (Network Common Data Form) is a set of software libraries and machine-independent data formats that support the creation, access, and sharing of array-oriented scientific data. This includes the NetCDF library as well as the NetCDF Operators (NCO), Climate Data Operators (CDO), NCCMP, and NCVIEW packages.

Using NetCDF at NERSC

NetCDF libraries on the Cori are provided by Cray, for each of the PrgEnvs. To see the available Cray installations and versions use the following command:

module avail cray-netcdf

No compile or link options are required as long as you use the Cray compiler wrappers ftn, cc, and CC. Below is an example:

module load cray-netcdf
ftn my_netcdf_code.f90
cc  my_netcdf_code.c
CC  my_netcdf_code.cpp

Note

The netCDF and HDF libraries provided by recent versions of the cray-netcdf, cray-netcdf-hdf5parallel, cray-hdf5 and cray-hdf5-parallel modules use a file locking feature. This feature is supported in the CSCRATCH file system, but it is not supported in the NGF file systems (CFS, HOME, ...). Before you run a program built with the Cray libraries in such a file system, you need to disable file locking by running the command:

export HDF5_USE_FILE_LOCKING=FALSE

Other NetCDF tools at NERSC

NERSC provides several tools which can be useful for analyzing and manipulating NetCDF data.

NetCDF Operators (NCO)

The NetCDF Operators (NCO) are a suite of file operators that facilitate manipulation and analysis of self-describing data stored in the NetCDF or HDF4/5 formats.

To access the NetCDF operators, load the nco module file.

NCCMP

The NCCMP tool compares two NetCDF files bitwise or with a user-defined tolerance (absolute or relative percentage). To use NCCMP, just load the nccmp module file with the module load nccmp command. This command will automatically load the NetCDF module file.

CDO

Climate Data Operators (CDO) is a large set of tools for working on climate data. NetCDF 3/4, GRIB (including SZIP compression), EXTRA, SERVICE, and IEG are supported as I/O-formats. CDO can also be used to analyse any kind of gridded data not related to climate science. CDO has very small memory requirements and can process files larger than physical memory.

CDO is available as the cdo module at NERSC:

module load cdo
cdo [options] Operators ...

NCView

NCView is a visual browser for NetCDF format files. To use NCView, first set up the necessary X11 environment (NERSC strongly recommends using NoMachine), then load the module:

module load ncview

To get usage instructions for NCView, type simply:

ncview

NetCDF4-python

netCDF4-python is an object-oriented python interface to the NetCDF library. It is available via the python module.

netCDF4-python supports various classic netcdf versions

netCDF4-python also supports netcdf3 and netcdf3-classic. Please make sure the format is consistent when you read and write the data.