Cori SCRATCH¶
Cori scratch is a Lustre file system designed for high performance temporary storage of large files. It is intended to support large I/O for jobs that are being actively computed on the Cori system. We recommend that you run your jobs, especially data intensive ones, from the Cori scratch file system.
Warning
The scratch file system is subject to purging. Please make sure to back up your important files e.g. to HPSS, or CFS. We strongly advise storing a copy of your important data at multiple sites for disaster recovery: please consult the NERSC data policy.
Usage¶
The /global/cscratch1 file system should always be referenced using the environment variable $SCRATCH (which expands to /global/cscratch1/sd/YourUserName). The scratch file system is available from all nodes and is tuned for high performance.
Quotas¶
If your $SCRATCH usage exceeds your quota, you will not be able to submit batch jobs until you reduce your usage. The batch job submit filter checks the usage of /global/cscratch1.
Note
See quotas for detailed information about inode, space quotas and file system purge policies.
Performance¶
The Cori scratch file system has 30 PB of disk space and an aggregate bandwidth of >700 GB/sec I/O bandwidth. Cori scratch is a Lustre file system designed for high performance temporary storage of large files. It contains 10000+ disks and 248 I/O servers called OSTs.
Warning
For large files you should stripe your files across multiple OSTs. Please see our Lustre striping page for details.
Backup¶
All NERSC users should backup important files on a regular basis. Ultimately, it is the user's responsibility to prevent data loss.
Lifetime¶
Cori scratch directories are purged. Cori scratch directories may be deleted after a user is no longer active.