Memphis: Finding and fixing NUMA-related performance problems on multi-core platforms

Collin McCurdy, Jeffrey Vetter

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

97 Scopus citations

Abstract

Until recently, most high-end scientific applications have been immune to performance problems caused by Non-Uniform Memory Access (NUMA). However, current trends in micro-processor design are pushing NUMA to smaller and smaller scales. This paper examines the current state of NUMA and makes several contributions. First, we summarize the performance problems that NUMA can present for multi-threaded applications and describe methods of addressing them. Second, we demonstrate that NUMA can indeed be a significant problem for scientific applications, showing that it can mean the difference between an application scaling perfectly and failing to scale at all. Third, we describe, in increasing order of usefulness, three methods of using hardware performance counters to aid in finding NUMA-related problems. Finally, we introduce Memphis, a data-centric toolset that uses Instruction Based Sampling to help pinpoint problematic memory accesses, and demonstrate how we used it to improve the performance of several production-level codes - HYCOM, XGC1 and CAM - by 13%, 23% and 24% respectively.

Original languageEnglish
Title of host publicationISPASS 2010 - IEEE International Symposium on Performance Analysis of Systems and Software
Pages87-96
Number of pages10
DOIs
StatePublished - 2010
Event2010 IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS 2010 - White Plains, NY, United States
Duration: Mar 28 2010Mar 30 2010

Publication series

NameISPASS 2010 - IEEE International Symposium on Performance Analysis of Systems and Software

Conference

Conference2010 IEEE International Symposium on Performance Analysis of Systems and Software, ISPASS 2010
Country/TerritoryUnited States
CityWhite Plains, NY
Period03/28/1003/30/10

Fingerprint

Dive into the research topics of 'Memphis: Finding and fixing NUMA-related performance problems on multi-core platforms'. Together they form a unique fingerprint.

Cite this