Performance evaluation of high-speed interconnects using dense communication patterns

Rod Fatoohi; Ken Kardys; Sumy Koshy; Soundarya Sivaramakrishnan; Jeffrey S. Vetter

doi:10.1016/j.parco.2006.09.007

Performance evaluation of high-speed interconnects using dense communication patterns

Rod Fatoohi, Ken Kardys, Sumy Koshy, Soundarya Sivaramakrishnan, Jeffrey S. Vetter

Computer Science and Mathematics Div

Research output: Contribution to journal › Article › peer-review

4 Scopus citations

Abstract

We study the performance of high-speed interconnects using a set of communication micro-benchmarks. The goal is to identify certain limiting factors and bottlenecks with these interconnects. Our micro-benchmarks are based on dense communication patterns with different communicating partners and varying degrees of these partners. We tested our micro-benchmarks on five platforms: an IBM system of 68-node 16-way Power3, interconnected by a SP switch2; another IBM system of 264-node 4-way Power PC 604e, interconnected by an SP switch; a Compaq cluster of 128-node 4-way ES40/EV67 processor, interconnected by an Quadrics interconnect; an Intel cluster of 16-node dual-CPU Xeon, interconnected by an Quadrics interconnect; and a cluster of 22-node Sun Ultra Sparc, interconnected by an Ethernet network. Our results show many limitations of these networks including the memory contention within a node as the number of communicating processors increased and the limitations of the network interface for communication between multiple processors of different nodes.

Original language	English
Pages (from-to)	794-807
Number of pages	14
Journal	Parallel Computing
Volume	32
Issue number	11-12
DOIs	https://doi.org/10.1016/j.parco.2006.09.007
State	Published - Dec 2006

Funding

Part of this work was performed under the auspices of the US Dept. of Energy by University of California LLNL under contract W-7405-Eng-48.

Keywords

Interconnect architecture
Micro-benchmarks
Pair-wise communication
Performance evaluation

Access to Document

10.1016/j.parco.2006.09.007

Cite this

@article{f531143a514d4b8388affbee6dad0e77,

title = "Performance evaluation of high-speed interconnects using dense communication patterns",

abstract = "We study the performance of high-speed interconnects using a set of communication micro-benchmarks. The goal is to identify certain limiting factors and bottlenecks with these interconnects. Our micro-benchmarks are based on dense communication patterns with different communicating partners and varying degrees of these partners. We tested our micro-benchmarks on five platforms: an IBM system of 68-node 16-way Power3, interconnected by a SP switch2; another IBM system of 264-node 4-way Power PC 604e, interconnected by an SP switch; a Compaq cluster of 128-node 4-way ES40/EV67 processor, interconnected by an Quadrics interconnect; an Intel cluster of 16-node dual-CPU Xeon, interconnected by an Quadrics interconnect; and a cluster of 22-node Sun Ultra Sparc, interconnected by an Ethernet network. Our results show many limitations of these networks including the memory contention within a node as the number of communicating processors increased and the limitations of the network interface for communication between multiple processors of different nodes.",

keywords = "Interconnect architecture, Micro-benchmarks, Pair-wise communication, Performance evaluation",

author = "Rod Fatoohi and Ken Kardys and Sumy Koshy and Soundarya Sivaramakrishnan and Vetter, {Jeffrey S.}",

year = "2006",

month = dec,

doi = "10.1016/j.parco.2006.09.007",

language = "English",

volume = "32",

pages = "794--807",

journal = "Parallel Computing",

issn = "0167-8191",

publisher = "Elsevier",

number = "11-12",

}

TY - JOUR

T1 - Performance evaluation of high-speed interconnects using dense communication patterns

AU - Fatoohi, Rod

AU - Kardys, Ken

AU - Koshy, Sumy

AU - Sivaramakrishnan, Soundarya

AU - Vetter, Jeffrey S.

PY - 2006/12

Y1 - 2006/12

N2 - We study the performance of high-speed interconnects using a set of communication micro-benchmarks. The goal is to identify certain limiting factors and bottlenecks with these interconnects. Our micro-benchmarks are based on dense communication patterns with different communicating partners and varying degrees of these partners. We tested our micro-benchmarks on five platforms: an IBM system of 68-node 16-way Power3, interconnected by a SP switch2; another IBM system of 264-node 4-way Power PC 604e, interconnected by an SP switch; a Compaq cluster of 128-node 4-way ES40/EV67 processor, interconnected by an Quadrics interconnect; an Intel cluster of 16-node dual-CPU Xeon, interconnected by an Quadrics interconnect; and a cluster of 22-node Sun Ultra Sparc, interconnected by an Ethernet network. Our results show many limitations of these networks including the memory contention within a node as the number of communicating processors increased and the limitations of the network interface for communication between multiple processors of different nodes.

AB - We study the performance of high-speed interconnects using a set of communication micro-benchmarks. The goal is to identify certain limiting factors and bottlenecks with these interconnects. Our micro-benchmarks are based on dense communication patterns with different communicating partners and varying degrees of these partners. We tested our micro-benchmarks on five platforms: an IBM system of 68-node 16-way Power3, interconnected by a SP switch2; another IBM system of 264-node 4-way Power PC 604e, interconnected by an SP switch; a Compaq cluster of 128-node 4-way ES40/EV67 processor, interconnected by an Quadrics interconnect; an Intel cluster of 16-node dual-CPU Xeon, interconnected by an Quadrics interconnect; and a cluster of 22-node Sun Ultra Sparc, interconnected by an Ethernet network. Our results show many limitations of these networks including the memory contention within a node as the number of communicating processors increased and the limitations of the network interface for communication between multiple processors of different nodes.

KW - Interconnect architecture

KW - Micro-benchmarks

KW - Pair-wise communication

KW - Performance evaluation

UR - http://www.scopus.com/inward/record.url?scp=33751250203&partnerID=8YFLogxK

U2 - 10.1016/j.parco.2006.09.007

DO - 10.1016/j.parco.2006.09.007

M3 - Article

AN - SCOPUS:33751250203

SN - 0167-8191

VL - 32

SP - 794

EP - 807

JO - Parallel Computing

JF - Parallel Computing

IS - 11-12

ER -

Performance evaluation of high-speed interconnects using dense communication patterns

Abstract

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this