File Systems¶
It is the responsibility of every user to organize and maintain software and data critical to their science. Managing your data requires understanding NERSC's storage systems.
File Systems¶
NERSC provides several file systems. Making efficient use of NERSC computing facilities requires an understanding of the strengths and limitations of each file system.
Note
JAMO is JGI's in-house-built hierarchical file system, which has functional ties to NERSC's file systems and tape archive. JAMO is not maintained by NERSC.
All NERSC file systems have per-user quotas on total storage and number of files and directories (known as inodes). To check your file system usage, use the showquota command from any login node or check the Data Dashboard on my.nersc.gov.
DnA (Data n' Archive)¶
DnA is a 2.4PB GPFS file system for the JGI's archive, shared databases, and project directories.
| DnA Projects | DnA Shared | DnA DM Archive | |
|---|---|---|---|
| Location | /global/dna/projectdirs/ | /global/dna/shared | /global/dna/dm_archive |
| Quota | 5TB default | Defined by agreement with the JGI Management | Defined by agreement with the JGI Management |
| Backups | Daily, only for projectdirs with quota <= 5TB | Backed up by JAMO | Backed up by JAMO |
| File purging | Files are not automatically purged | Purge policy set by users of the JAMO system | Files are not automatically purged |
The intention of the DnA "Project" and "Shared" space is to be a place for data that is needed by multiple users collaborating on a project which allows for easy reading of shared data. The "Project" space is owned and managed by the JGI. The "Shared" space is a collaborative effort between the JGI and NERSC. Write access to DnA is restricted to protect high performance; data can only be written to DnA from Data Transfer Nodes or by loading the esslurm module and then using the --qos=dna_xfer QOS.
If you would like a project directory, please contact JGI management to discuss the use case and requirements for the proposed directory.
The "DM Archive" is a data repository maintained by the JAMO system. Files are stored here during migration using the JAMO system. The files can remain in this space for as long as the user specifies. Any file that is in the "DM Archive" has also been placed in the HPSS tape archive. This section of the file system is owned by the JGI data management team.
Other file systems¶
All JGI users also have access to the other file systems NERSC provides. For more details about each of them visit the section about NERSC File systems.