Proactive Data Containers (PDC)

Suren Byna, Houjun Tang
Nov 7, 2022

Proactive Data Containers (PDC) are containers within a locus of storage (memory, NVRAM, disk, etc.) that store science data in an object-oriented manner. Managing data as objects enables powerful optimization opportunities for data movement and transformations, and storage mechanisms that take advantage of the deep storage hierarchy and enable automated performance tuning

Python interface to an object-centric data management system

  • Topics: Python, object-centric data management, PDC
  • Skills: Python, C, PDC
  • Difficulty: Medium
  • Size: Large (350 hours)
  • Mentor: Suren Byna, Houjun Tang

Proactive Data Containers (PDC) is an object-centric data management system for scientific data on high performance computing systems. It manages objects and their associated metadata within a locus of storage (memory, NVRAM, disk, etc.). Managing data as objects enables powerful optimization opportunities for data movement and transformations, and storage mechanisms that take advantage of the deep storage hierarchy and enable automated performance tuning. Currently PDC has a C interface. Providing a python interface would make it easier for more Python applications to utilize it.

osre22 uc data management