Delta Resource Users:

This is a reminder of the scheduled maintenance event on Delta and DeltaAI on Tuesday October 22nd from 6AM to 8PM.  

There are some IMPORTANT updates to bring to your attention.

1. To address recent GPU XID errors, the Delta GPU nodes will move to a newer NVIDIA driver: moving from the 535 driver family to the 550 driver family. Initial testing did not indicate any need for recompilation or linking. Please check your GPU jobs as they run tomorrow for any unexpected behavior. 

2. The maintenance will require the /scratch and /work/hdd file systems to be offline and unavailable during the maintenance.

3. The login nodes will need to be rebooted to unmount /scratch and /ime. 

4. Please note that the /ime cache front-end to /scratch is currently being flushed and jobs that use the ime constraint will not be allowed to run before the maintenance. 

5. The home and projects file system will be available, and jobs not using /scratch  or /work/hdd will be allowed to run if submitted to one of the following reservations:

On Delta, to run jobs that DO NOT require /scratch, please submit jobs using the no_scratch_requirements reservation by adding:
--reservation=no_scratch_requirements
to your sbatch, srun and Open OnDemand scripts.

On DeltaAI, to run jobs that DO NOT require /work/hdd, please submit jobs using the no_workhdd_requirements reservation by adding:
--reservation=no_workhdd_requirements
to your sbatch, srun and Open OnDemand scripts.

A final sync'ing of data on /scratch to /work/hdd will be done during the maintenance period.

Upon maintenance completion, the Delta /scratch file system will be replaced by /work/hdd, which will also be accessible from DeltaAI, and the new NVME portion of /work called /work/nvme (currently available on DeltaAI) will also be accessible from Delta.

On Delta a symbolic link will be set to link /scratch/hdd/abcd to /work/hdd/abcd to maintain functionality of existing scripts.