HPC Bulletin 4/14/2021COLL+IT :COLL+IT

HPC Bulletin 4/14/2021

By:chemweb

April 14, 2021

The monthly maintenance was completed. IU’s high performance computing and storage systems are fully available to you.

Sunday, May 16, 2021, 7am – 7pm

TORQUE and Moab job scheduling was retired April 12, 2021 and are no longer available on Carbonate. Job scheduling is now managed through Slurm.
- For more about SLURM, see Use Slurm to submit and manage jobs on high performance computing systems.
There have been several changes to deeplearning, python, and tensorflow modules. The current status of each is:
- deeplearning/1.13.1 (retired)
- deeplearning/2.0.0 (retired)
- deeplearning/2.2.0 (retired)
- deeplearning/2.3.0 (default)
- python/2.7.13 (retired)
- python/3.6.1 (retired)
- python/3.6.8 (default)
- All tensorflow modules (retired)
- The CPU version of TensorFlow is now included in the Python modules.
- For the GPU version of TensorFlow use the Deep Learning modules.

All data on /N/slate has been migrated to a faster set of newer hardware.
Slate users now have a quota of 6.4 million files.
- Note: directories and symbolic links are also considered files in Linux.
The new Slate system does not support native on disk compression. Some of your files may appear larger as they are now uncompressed and occupying more space.

Slate-Project will also be migrating to a faster set of newer hardware. The sheer size of Slate-Project makes it prohibitive to attempt to finalize migration for every project during a single monthly maintenance period.
- The staggered migration will begin after May maintenance.
- Data will be migrated on a per project basis.
- All project owners and participants will receive a generic email with further details before the overall migration begins.
- A per project communication will follow to individual project owners and participants to inform them of their upcoming migration period.
- Project paths will remain the same. However, to allow continuous migration, symbolic links will be used to point projects to old Slate-Project until migration is complete for all projects.
The new Slate-Project system, like Slate, does not support native on disk compression. Any quota gains you received from using compression will be gone.

A new file system, Slate-Scratch, has passed performance metrics and has been mounted on IU’s HPC systems for further testing.
Performance has exceeded 500,000 file creates and removes per second and has surpassed an aggregate transfer rate of 60 GB/s.
Detailed information regarding the new Slate-Scratch service will be available via the IU Knowledge Base in the coming weeks. The goal is to make the service available by June.