The Distributed ASCI Supercomputer 3

DAS-3 Overview


DAS-3 (The Distributed ASCI Supercomputer 3) is a five-cluster wide-area distributed system designed by the Advanced School for Computing and Imaging (ASCI). DAS-3 is funded by NWO/NCF (the Netherlands Organization for Scientific Research), the VL-e project, and the participating universities and organizations (see below). As one of its distinguishing features, DAS-3 employs a novel internal wide-area interconnect based on light paths.

The goal of DAS-3 is to provide a common computational infrastructure for researchers within ASCI, who work on various aspects of parallel, distributed, and grid computing, and large-scale multimedia content analysis. The following institutes and organisations are directly involved in the realization and running of DAS-3:

DAS-3: five clusters, one system


DAS-3 consists of 272 dual AMD Opteron compute nodes, spread out over five clusters, located at the four universities. The system has been built by ClusterVision. Unlike its predecessor, DAS-2, DAS-3 is rather heterogeneous in design:

Cluster
Nodes
Type
Speed
Memory
Storage
Node HDDs
Network
VU
85 dual
dual-core
2.4 GHz
4 GB
10 TB
85 x 250 GB
Myri-10G and GbE
LU
32 dual
single-core
2.6 GHz
4 GB
10 TB
32 x 400 GB
Myri-10G and GbE
UvA
41 dual
dual-core
2.2 GHz
4 GB
5 TB
41 x 250 GB
Myri-10G and GbE
TUD
68 dual
single-core
2.4 GHz
4 GB
5 TB
68 x 250 GB
GbE (no Myri-10G)
UvA-MN
46 dual
single-core
2.4 GHz
4 GB
3 TB
46 x 1.5 TB
Myri-10G and GbE

The operating system the DAS-3 runs is Scientific Linux. Cluster management is done using ClusterVision's ClusterVisionOS. In addition, software from many sources is available to support research on DAS-3: the Grid Engine resource management system, various MPI implementations (e.g., MPICH and OpenMPI), the Globus Grid toolkit, optimizing compilers, visualization packages, performance analysis tools, etc. See also: DAS-3 Usage Policy and DAS-3 Job Execution.

Besides using the ubiquitous 1 and 10 Gigabit/s Ethernet, DAS-3 is exceptional in that it employs the novel high speed Myri-10G interconnect technology. Myri-10G is used both as an internal high-speed interconnect as well as to directly communicate with extremely low latency and jitter at up to 10 Gbit/s (or even multiples of that) to remote DAS-3 computing resources via a fully optical (DWDM) backbone, SURFnet6. See also: DAS-3 Connectivity.

The research specifically planned for DAS-3 includes topics like Grid computing, performance analysis, Virtual Laboratory design, and distributed image and video content analysis and visualization. Apart from this, DAS-3 will be made available as a general Grid resource for the larger academic High Performance Computing community in the Netherlands. We also cooperate with the French Grid'5000 project. The dynamic performance aspects of light path management in DAS-3 will be investigated within the StarPlane project. See also: DAS-3 Research.



RELATED LINKS