ScaLAPACK tutorial

ScaLAPACK is a library of high performance linear algebra routines for distributed memory MIMD computers. It is a continuation of the LAPACK project, which designed and produced analogous software for workstations, vector supercomputers, and shared memory parallel computers. The goals of the project are efficiency (to run as fast as possible), scalability (as the problem size and number of processors grow), reliability (including error bounds), portability (across all important parallel machines), flexibility (so users can construct new routines from well-designed parts), and ease-of-use (by making LAPACK and ScaLAPACK look as similar as possible). Many of these goals, particularly portability, are aided by developing and promoting standards, especially for low-level communication and computation routines. We have been successful in attaining these goals, limiting most machine dependencies to two standard libraries called the BLAb, or Basic Linear Algebra Subroutines, and BLACS, or Basic Linear Algebra Communication Subroutines. ScaLAPACK will run on any machine where both the BLAb and the BLACS are available. This tutorial will begin by reviewing the fundamental design principles of the BLAb and LAPACK and their influence on the development of ScaLAPACK. The two dimensional block cyclic data decomposition will be presented, followed by a discussion of the underlying building blocks of ScaLAPACK, the BLACS and the PBLAS. The contents of the ScaLAPACK library will then be enumerated, followed by example programs and performance results. And finally, future directions and related projects will be described.