Skip to content

File System Quotas and Purging

NERSC sets quotas on file systems shown in the table below. Purged file system are purged of files that have not been accessed in the time period shown below.

Overview

File system Space Inodes Purge time Consequence for Exceeding Quota
Community 20 TB 20 M - No new data can be written
Global HOME 40 GB 1 M - No new data can be written
Global common 10 GB 1 M - No new data can be written
Cori SCRATCH 20 TB 10 M 12 weeks Can't submit batch jobs
Perlmutter SCRATCH 20 TB 10 M 8 weeks Jobs can be submitted but no new data can be written

Policy

NERSC data management policy.

Quotas

Warning

Writes to an over-quota file system may fail.

Note that the quota on the Community File System and on Global Common is shared among all members of the project, so showquota/cfsquota will report the aggregate project usage and quota.

You may briefly exceed your quota on Perlmutter scratch by as much as 10 TB for up to 24 hours before no new data can be written. This is known as a 'grace period'. When you are above your 20 TB quota, showquota --grace will show how much time you have left in your grace period before the file system becomes read-only.

Current usage

NERSC provides a showquota command which displays applicable quotas and current usage.

To see current usage for home and available scratch file systems:

showquota

For Community you can use

showquota PROJECT [PROJECT]

For global common software you can use

showquota --cmn PROJECT [PROJECT]

Note

Cori users can continue to use the old myquota, cfsquota and cmnquota, which are also available as backwards-compatible scripts on Perlmutter, producing the same output.

Different output formats and unit of measurements are available, see showquota --help.

For scripting purposes we suggest to use the json formatting option -J/--json, which can be quite powerful when paired with e.g. the jq utility:

$ showquota -J dasrepo nstaff |jq '.[] | "\(.fs) \(.space_perc)"'
"home 0.2%"
"pscratch 0.7%"
"dasrepo 0.9%"
"nstaff 0.4%"

The -L/--limit/--grace option shows how much more storage and inodes you can temporarily write above your quota and how much time you have left before no new data can be written.

Increases

If you or your project needs additional space for your scratch file system or HPSS you may request it via the Disk Quota Increase Form.

Quotas on the Community File System are determined by DOE Program Managers based on information PIs supply in their yearly ERCAP requests. If you need a mid-year quota increase on the Community File System, please use the Disk Quota Increase Form and we will pass the information along to the appropriate DOE Program Manager for approval.

Purging

Some NERSC file systems are purged. This means the files not read (i.e. atime is updated) within a certain time period are automatically deleted. You can see the time period for the purged file systems at NERSC in the overview table. When a purge is done, a file named .purged_<date> is left behind. This is a text file that holds the names of the files that have been removed. These .purged files will not be deleted by our purges to make sure a record of purging activities is retained. Touching files or other actions intended to circumvent the purge are forbidden by NERSC policy.