Source repo: ciml-summer-institute-2025 | Branch: main | Last synced: 2026-04-24 10:27:17.425 UTC

Session 2.4 Data Management and File Systems

Date: Tuesday, June 24, 2025

Summary: Managing data efficiently on a supercomputer is important from both users' and system's perspectives. We will cover a few basic data management techniques and I/O best practices in the context of the Expanse system at SDSC.

Presented by: Marty Kandes (mkandes @ucsd.edu)

Reading and Presentations:

• Lecture material:

  • COMPLECS: Data Management 1: Data Storage and File Systems
  • COMPLECS: Data Management 2: Data Transfer

• Source Code/Examples:

  • Data Management: Or how (not) to handle your data in an HPC environment