Heterogeneous Network-Based Concurrent Computing Systems

This chapter discusses that parallel processing, the method of having many small tasks solve one large problem, has emerged as a key enabling technology in modern computing. It has been witnessed an ever-increasing acceptance and adoption of parallel processing, both for high-performance scientific computing and for more general-purpose applications, was a result of the demand for higher performance, lower cost, and sustained productivity. The acceptance has been facilitated by two major developments: massively parallel processors (MPPs) and the widespread use of distributed computing. MPPs are now the most powerful computers in the world. These machines combine a few hundred to a few thousand CPUs in a single large cabinet connected to hundreds of gigabytes of memory. MPPs offer enormous computational power and are used to solve computational Grand Challenge problems such as global climate modeling and drug design. As simulations become more realistic, the computational power required to produce them grows rapidly. Thus, researchers on the cutting edge turn to MPPs and parallel processing to get the most computational power possible. The chapter also discusses the second major development that affects scientific problem solving, which is distributed computing. Distributed computing is a process whereby a set of computers connected by a network are used collectively to solve a single large problem. The Parallel Virtual Machine (PVM) system described in the chapter uses the message-passing model to allow programmers to exploit distributed computing across a wide variety of computer types, including MPPs. A key concept in PVM is that it makes a collection of computers appear as one large virtual machine, hence the name.