Distributed ASCI Supercomputer (DAS)

A 200-node wide-area distributed system built out of four Myrinet-based Pentium Pro clusters

DAS (Distributed ASCI Supercomputer) is a wide-area distributed cluster designed by the Advanced School for Computing and Imaging (ASCI). The DAS machine is used for research on parallel and distributed computing by five Dutch universities:

DAS consists of four clusters, located at the first four universities. The first cluster (at the VU) contains 128 nodes, the other three clusters have 24 nodes (200 nodes in total). The system was built by Parsytec (Achen, Germany), which has many years of experience in building large-scale parallel systems. Parsytec used the same packaging techniques for DAS as for their CC series. The operating system the DAS runs is RedHat Linux.

The DAS system is funded by NWO (the Netherlands organization for scientific research), SION (the foundation for computer science research in the Netherlands) and the participating universities.

Technical description

Each node contains: The nodes within a local cluster are connected by a Myrinet SAN network, which is used as high-speed interconnect, mapped in user-space. In addition, Fast Ethernet is used as OS network (file transport). The four local clusters are connected by wide-area ATM links, so the entire system can be used as a 200-node wide-area distributed cluster. More information about the wide-area networks is provided by the University of Utrecht .

Here is the Myrinet interconnection layout of the VU cluster. The topology is a 3D-mesh, in which dimension order routing is used.

The DAS topology

Each DAS cluster internally uses two networking technologies: 100 Mbit/sec FastEthernet for regular OS related traffic (NFS, telnet, rsh) and 1.2 Gbit/sec (full duplex) Myrinet SAN network for low-latency, high-bandwidth user-level communication.

The four DAS clusters are interconnected over the SurfNet4 ATM network, which offers lower latency and higher bandwidth than the standard Internet connections between the sites. Every site has a special gateway host, which routes inter-cluster packets from the FastEthernet network over the wide area ATM links. It is important to mention that the links are actually Constant Bitrate (CBR) virtual circuits, offering guaranteed and reliable performance. This makes the behaviour and run time of applications run on the wide-area DAS much more predictable than would otherwise be the case. An overview of the planning for this part of the project is available.

The ATM links between the four clusters are fully operational since October 1998. At the moment they have a capacity of 6 Mbit/sec. The roundtrip latency between the sites is between 1.5 and 3.7 msec, mainly depending on the distance (fotons and electrons do have a speed limit unfortunately). An overview of the current network status is maintained at the Utrecht site.

Currently we only run IP over ATM; we might additionally be using native ATM at the application level later on.

Advanced School for
Computing and Imaging

Back to the DAS home page
This page is maintained by Kees Verstoep. Last modified: Fri Sep 17 12:19:05 MET DST 1999