Technology Integration

In the following projects, we explore non-volatile memory devices from several perspectives including fault tolerance, memory extension, and out-of-core data analytics.

SSD Lifetime Analysis under Checkpointing and Memory Extension Workload

Future extreme-scale compute systems will take advantage of non-volatile memory technology for both memory extension and checkpointing. While SSD devices offer desirable properties such as lower cost ($/GB) and power consumption than DRAM, they suffer from poor random write performance and write endurance problems. Specifically the SSD lifetime can be highly influenced by the access pattern, the level of over-provisioning, and the choices of garbage collection reclaiming policy and wear-leveling techniques on the SSDs. In this project, we explore the SSD lifetime and performance issues for various scientific applications on extreme-scale systems that employ SSDs for both memory extension as well as checkpointing. The work involves several areas.

Contributors: Chao Wang, and Sudharshan Vazhkudai

  Top

NVMalloc

DRAM is a precious resource in extreme-scale machines and is increasingly becoming scarce, mainly due to the growing number of cores per node. On future multi-petaflop and exaflop machines, the memory pressure is likely to be so severe that we need to rethink our memory usage models. Fortunately, the advent of non-volatile memory (NVM) offers a unique opportunity in this space. Current NVM offerings possess several desirable properties, such as low cost and power efficiency, but also suffer from high latency and lifetime issues. We need rich techniques to be able to use them alongside DRAM. NVMalloc is an approach for exploiting NVM as a secondary memory partition so that applications can explicitly allocate and manipulate memory regions therein. More specifically, it is a middleware library with a suite of services that enables applications to access node-local or distributed NVM storage systems in a seamless fashion. The work inovolves several areas.

Contributors: Chao Wang, and Sudharshan Vazhkudai

Status: On-going

Selected publications

  Top

Active Processing

Active Flash

Modern scientific discovery is increasingly driven by large-scale supercomputing simulations, followed by data analysis tasks. These data analyses are either performed offline, on smaller-scale clusters, or on the supercomputer itself. Unfortunately, these techniques suffer from performance and energy inefficiencies due to increased data movement between the compute and storage subsystems. We propose Active Flash, an in-situ scientific data analysis approach, wherein data analysis is conducted on the solid-state device (SSD), where the data already resides. Active Flash has the potential to enable true out-of-core data analytics by freeing up both the compute core and the associated main memory. The work involves several areas.

Contributors: Sudharshan Vazhkudai

Status: On-going

Selected publications

AnalyzeThis

The need for novel data analysis is urgent in the face of a data deluge from modern applications. Traditional approaches to data analysis incur significant data movement costs, moving data back and forth between the storage system and the processor. Emerging Active Flash devices enable processing on the flash, where the data already resides. An array of such Active Flash devices allows us to revisit how analysis workflows interact with storage systems. By seamlessly blending together the flash storage and data analysis, we create an analysis workflow-aware storage system, AnalyzeThis. Our guiding principle is that analysis-awareness be deeply ingrained in each and every layer of the storage, elevating data analyses as first-class citizens, and transforming AnalyzeThis into a potent analytics-aware appliance. We implement the AnalyzeThis storage system atop an emulation platform of the Active Flash array. Our results indicate that AnalyzeThis is viable, expediting workflow execution and minimizing data movement.

TechInt contributors: Sudharshan Vazhkudai, Hyogi Sim

Selected publications:

AnalyzeThat

AnalyzeThat is a programmable shared-memory system for parallel data processing with PIM (processing-in-memory) devices.

Processing In Memory (PIM), the concept of integrating processing directly with memory, has been attracting a lot of attention since PIM can assist in overcoming the throughput limitation caused by data movement between CPU and memory. The challenge, however, is that it requires the programmers to have a deep understanding of the PIM architecture to maximize the benefits such as data locality and parallel thread execution on multiple PIM devices. In this study, we present AnalyzeThat, a programmable shared-memory system for parallel data processing with PIM devices. Thematic to AnalyzeThat is a rich PIM-Aware Data Structure (PADS), which is an encapsulation that integrally ties together the data, the analysis tasks and the runtime needed to interface with the PIM device array. The PADS abstraction provides (i) a key-value data container that allows programmers to easily store data on multiple PIMs, (ii) a suite of parallel operations with which users can easily implement data analysis applications, and (iii) a runtime, hidden to programmers, which provides the mechanisms needed to overlay both the data and the tasks on the PIM device array in an intelligent fashion, based on PIM-specific information collected from the hardware. We have developed a PIM emulation framework called AnalyzeThat. Our experimental evaluation with representative data analytics applications suggests that the proposed system can significantly reduce the PIM programming effort without losing its technology benefits.

Contributors: Hyogi Sim, Sudharshan Vazhkudai

Selected publications:

  Top

Emerging Storage Technologies

Hard disk drives (HDDs) have been the preferred media for data storage in high-performance computing. A center-wide file system at ORNL has deployed 13,440 HDDs for back-end storage systems, running the Lustre parallel file system. However, there are shortcomings inherent to HDDs. Alongside improvements in HDD technology, significant advances have also been made in various forms of solid-state memory such as NAND flash memory, phase-change memories (PRAM), and spin-transfer torque memories (STT-RAM). These emerging storage technologies close a huge gap between DRAM and disk based storage systems. In this project, we evaluate these emerging memory technologies, study how they are to coexist with magnetic disks, and address technology challenges towards employing these memory devices in HPC storage systems. More specifically we measure the technology's impact on I/O intensive scientific applications. The work includes several areas.

Contributors: Sarp Oral, and Sudharshan Vazhkudai

Status: On-going

Selected publications:

  Top