The Distributed ASCI Supercomputer (DAS)

A 200-node wide-area distributed system built out of four Myrinet-based Pentium Pro clusters

DAS (Distributed ASCI Supercomputer) is a wide-area distributed cluster designed by the Advanced School for Computing and Imaging (ASCI). DAS will be used for research on parallel and distributed computing by five Dutch universities:

DAS consists of four clusters, located at the first four universities. The first cluster (at the VU) contains 128 nodes, the other three clusters have 24 nodes (200 nodes in total). The system was built by Parsytec and runs the BSD/OS (version 3.0) operating system from BSDI.

The DAS system is funded by NWO (the Netherlands organization for scientific research), SION (the foundation for computer science research in the Netherlands) and the participating universities.

Technical description:

Each node contains:

The nodes within a local cluster are connected by a Myrinet SAN network, which is used as high-speed interconnect, mapped in user-space. In addition, Fast Ethernet is used as OS network (file transport). The four local clusters are connected by wide-area ATM networks, so the entire system can be used as a 200-node wide-area distributed cluster. More information about the wide-area networks is provided by the University of Utrecht .

The system was built by Parsytec (Achen, Germany), which has many years of experience in building large-scale parallel systems. Parsytec used the same packaging techniques for DAS as for their CC series.

The four clusters were delivered in May 1997. Below are several pictures of the original 64-node cluster of the VU (which was installed 22 May 1997). The pictures illustrate the integration of the different components, as designed by Parsytec. The orignal 64-node cluster consisted of five cabinets. The leftmost and rightmost cabinet each contain 14 nodes, stored in five 19-inch racks; the three middle cabinets each contain 12 nodes, stored in four racks. Each 19-inch rack has (at most) three nodes.

Each cluster has a separate file-server and gateway-server. These machines are regular tower-PCs. The file server contains a 9 GByte SCSI (Ultra-wide, ultra-fast) disk. The gateways will be used to interconnect the four clusters over a wide-area network.

All connectors (of the motherboard and PCI cards) are at the front of the machine, so they are easy to access. Each node has a Myrinet SAN cable (the flat cable) and a Fast Ethernet cable connected to it. Each node also has keyboard and VGA connectors, so it's easy to attach a keyboard and monitor to any node.

Each cabinet contains a Fast Ethernet hub (the black boxes in the space between the second and third rack from the bottom). The hubs are connected through a single switch. Each cabinet also contains one or two dual 8-port Myrinet switches (the small white boxes). In total, there are 16 (i.e., eight dual) 8-port switches. Each switch is connected to four nodes and four other switches. The connection between the switches has a torus topology (a 2D-grid with wrap-around).

Each computing node is packaged as a metal cassette, containing the motherboard (LPX form factor) and a riser card with three PCI slots (two of which are occupied by the Myrinet and Fast Ethernet adaptors). The motherboard has RS232, parallel, and USB interfaces and a graphics adapter (Cirrus 5446). Specially-designed foam is used by Parsytec for integrating cooling channels. Below is a picture of one node (without the Myrinet card):

The 64-node cluster at the VU was upgraded to 128 nodes in May 1998. Below are pictures of the 128-node cluster.

In September 1999, the chemistry department added a 16-node Pentium III cluster, shown below

Larger pictures are here:

Slides of a talk about DAS by Andy Tanenbaum can be found here.

Documentation for users of DAS can be found here.

Research projects using DAS

Vrije Universiteit Amsterdam

University of Amsterdam

Delft University of Technology

University of Leiden

University of Utrecht

Advanced School for Computing and Imaging

This page is maintained by Henri Bal.

Last modified: 24 August 1998.