Announcements

6 Jul 2015

DAS-5 is now fully operational! To make room for DAS-5, DAS-4/UvA and DAS-4/ASTRON have been decomissioned, only their headnodes remain available.

28 Oct 2014

The Hadoop setup on DAS-4/VU has been updated to version 2.5.0.

30 Jan 2014

The Intel OpenCL package for Intel CPU's and Xeon Phi has been updated to version 3.2.1.

3 Sep 2013

The Nvidia CUDA development kit has been updated to version 5.5.

25 April 2013

Slides of the DAS-4 workshop presentations are now available.

14 Jan 2013

DAS-4/VU now has 8 new nodes with latest Nvidia K20 GPU.


Accelerators and special compute nodes

The standard compute node type of DAS-4 sites has a dual-quad-core 2.4 GHz CPU configuration and 24GB memory (48 GB in Leiden). In addition, several DAS-4 sites include non-standard node types for specific research purposes. To allocate a special resource in SGE or prun, this resource should be specified as follows:

  • -l gpu=GTX480
    nodes with an Nvidia GTX480 (with 1.5 GB onboard memory)
  • -l gpu=GTX470
    nodes with an Nvidia GTX470 (with 1.28 GB onboard memory)
  • -l gpu=GTX680
    node with an Nvidia "Kepler" GTX680 (with 2 GB onboard memory)
  • -l gpu=C2050
    nodes with an Nvidia Tesla C2050 (with 2.625 GB onboard memory)
  • -l fat,gpu=HD7970
    fat nodes with an AMD Radeon HD7970 (with 3 GB onboard memory)
  • -l fat,gpu=K20
    fat nodes with an Nvidia "Kepler" K20 (with 6 GB onboard memory)
  • -l fat,gpu=GTX-Titan
    fat node with an Nvidia "Kepler" GTX-Titan (with 6 GB onboard memory)
  • -l ngpus=1
    nodes with an (arbitrary) additional GPU
  • -l fat,accel=XeonPhi
    fat nodes with an Intel Xeon Phi accelerator (with 6 GB onboard memory)
  • -l fat,m_type=bigmem
    regular node with extra memory (NOTE: node073 is currently allocated to VMs);
  • -l fat,m_type=sixcore
    dual six-core-based nodes instead of dual quad-core
  • -l fat,m_type=magnycours
    node with AMD Magny Cours CPUs
  • -l fat,m_type=sandybridge
    node with Sandybridge architecture
  • -l ssd=1
    node with an additional SSD device

This resource selector should be added as

#$ -l resource
in an SGE job script, or passed as
-native '-l resource'
option to prun/preserve.

VU University

At fs0.das4.cs.vu.nl the following special-purpose equipment is available for various experiments:

  • 16 (out of 72) of the regular nodes in addition have an NVidia GTX480 GPU and a 250 GB SSD drive mounted as /mnt/ssd;
  • 7 additional regular nodes also have an NVidia GTX480 GPU (but no SSD)
  • 2 of the regular nodes (node061 and node062) have an Nvidia C2050 Tesla GPU and a 250 GB SSD drive mounted as /mnt/ssd;
  • regular node node068 has an NVidia GTX680 GPU
  • regular nodes node001 and node002 in addition have a 500 GB SSD drive mounted as /mnt/ssd;
  • node073: a node with 96GB memory and 12 TB (6*2TB RAID0) local storage (this node is currently used to run VMs);
  • node074: a node with X5650 CPU (dual 6-cores, 2.67 GHz), 96GB memory and 12 TB (6*2TB RAID0) local storage;
  • node075: a 48-core (quad socket "Magny Cours") AMD system with 128 GB memory, 10 TB (5*2TB RAID0) local storage and 256 GB SSD;
  • node076: an X5650 node with 24 GB memory, AMD Radeon HD7970, and 10 TB (5x2TB RAID0) local storage and 256 GB SSD.
  • node079 to node085: Intel "Sandy Bridge" E5-2620 (2.0 GHz) nodes, with 64 GB memory and a K20m "Kepler" GPU. In addition, node079 is equipped with an Intel Xeon Phi accelerator.
  • node078, node086-node091, node093: Intel "Sandy Bridge" E5-2620 (2.0 GHz) nodes, with 64 GB memory, reserved for running Hadoop and Cloud VMs.

Leiden University

At fs1.das4.liacs.nl all 16 compute nodes are "fat" in that they have more memory and local storage than default on the other sites:

  • the nodes have 48 GB instead of 24 GB memory;
  • the nodes have have 10 TB of local storage (5*2 TB RAID);
  • each compute node also has a fast 512 GB SSD (OCZ Z-Drive p88).

University of Amsterdam

At fs2.das4.science.uva.nl the following special node type exist:

  • 4 router nodes (router01-router04) with additional network interfaces.

Delft University of Technology

At fs3.das4.tudelft.nl the following special-purpose equipment is available:
  • 8 (out of 28) of the regular nodes have an NVidia GTX480 GPU;
  • 2 of the regular nodes have an Nvidia C2050 Tesla GPU;
  • 4 "fat" nodes are available that have 48GB memory and 2*2TB RAID0 local storage.

University of Amsterdam - MultiMediaN

At fs4.das4.science.uva.nl the following special-purpose equipment is available:

  • 8 (out of 34) of the regular nodes in addition have an NVidia GTX470 GPU;
  • 7 of the regular nodes have an NVidia C2050 Tesla GPU;
  • 2 of the nodes (1 of the regular ones and 1 of the fat ones) have an Nvidia GTX480 GPU;
  • 2 "fat" nodes are available that have 96GB memory and 6*2TB local storage.

ASTRON

At fs5.das4.astron.nl the following special-purpose equipment is available:

  • 1 special node (b7015) has 8 GTX 580 3GB GPUs, that can be used for GPU scaling experiments and "green computing" research. Note that the 8 GPUs occupy all PCIe slots on the b7015, leaving no space for an InfiniBand NIC. When running with OpenMPI on this node, explicitly specify "--mca btl tcp,self" to avoid a warning about failing InfiniBand initialization.
  • 1 special node (r815) has 48 cores (quad Opteron 6172), 128 GB memory, 4 * 240 GB PCIe SSD (RAID0), and a fast 8 * 2 TB external RAID (up to 1000 MB/s)
  • 1 special node (node521) has 4 InfiniBand HCAs and a HotLava hex-port 10-GbE interface, for I/O experiments.
  • 1 special node (node522) has another hex-port 10-GbE interface (connected to node521).
  • 1 regular compute node (node501) has an ATI HD 6970 GPU (2GB)
  • 1 regular compute node (node502) has an Nvidia C2050 GPU (3GB)
  • 1 regular compute node (node503) has an Nvidia GTX 580 GPU (3GB)

Note that these configurations change frequently.

Some of these systems (b7015, node521) have faster (dual X5650) CPUs and 48 GB RAM.

The GTX 580 3GB GPUs are faster than the original GTX 480 GPUs within DAS-4, and have twice the regular amount of memory. The HD 6970 is a GPU from ATI, unlike most of the other GPUs on DAS-4, which are from Nvidia.

  • The HD 6970 can be programmed in OpenCL (http://www.khronos.org/#tab-opencl). For use of OpenCL on DAS-4, see the DAS-4 GPU page.
  • Important: always set the DISPLAY environment variable to ":0.0", or your program will not execute.
  • Tip: use the C++ bindings in combination with exceptions in your host CPU code (#define __CL_ENABLE_EXCEPTIONS and #include ); the C++ interface is much easier to use and roughly eight times less verbose than the C interface.