Announcements

23 February 2021

The next generation DAS, DAS-6, will be available shortly! DAS-6 is currently being installed and tested. It is expected to become available for regular use in March/April 2021.

May, 2016

IEEE Computer publishes paper about 20 years of Distributed ASCI Supercomputer. See the DAS Achievements page.

Network setup

Management network

Each DAS-6 cluster has both an internal 1 Gbit/s Ethernet and a 100 Gbit/s network. The 1 Gbit/s Ethernet network (with IP addresses 10.141.0.<node-id>) is mainly used for management of the cluster.

High speed networks

The best performance is obtained using the 100 Gbit/s network, which offers very low latency and throughputs close to peak (depending on the networking API used). The DAS-6 clusters use the 100 Gbit/s both for HPC interconnect and to mount file systems on the headnode via NFS.

100 Gbit/s Ethernet network

Sites DAS-6/VU (fs0.das6.cs.vu.nl), DAS-6-UvA/SNE (fs2.das6.science.uva.nl) and DAS-6/ASTRON (fs5.das6.astron.nl) emplot a 100 Gb/s "RoCE" Ethernet implementation. Via the RoCE API (which is user level, rdma based), it has performance close to 100 Gb/s InfiniBand. In this mode, it is most often programmed via MPI. Being Ethernet, it can also be accessed via the regular in-kernel IP layer, using IP addresses 10.149.<site-id>.<node-id>.

100 Gbit/s InfiniBand network

Sites DAS-6/Leiden (fs1.das6.liacs.nl), DAS-6/Delft (fs3.das6.tudelft.nl) and DAS-6/UvA-AI (fs4.das6.science.uva.nl) employ a 100 Gb/s InfiniBand network. It can be accessed via a fast native "verbs/rdma" interface, but is typically used through MPI. It can also be accessed via IP using the IPoIB implementation with IP addresses 10.149.<site-id>.<node-id>.

External connectivity

For external connectivity, each cluster has a 1 or 10 Gbit/s ethernet connection to the local university backbone; the head node acts as a router for this purpose.