Announcements

17 February 2011

The Nvidia CUDA development kit has been updated to version 4.1.

24 Jan 2012

DAS-4/ASTRON has succesfully correlated three hours of data coming from 288 LOFAR antennas, possibly setting a world record.

13 Jan 2012

The GPU page documents how the OpenCL implementations on DAS-4 can be used.

15 November 2011

DAS-4/VU achieved the 16th place on the Graph500 list. A very good result, as all higher-ranked systems are significantly bigger.

11 May 2011

DAS-4 supports the OpenNebula package for Cloud computing.

Accelerators and special compute nodes

The standard compute node type of DAS-4 sites has a dual-quad-core 2.4 GHz CPU configuration and 24GB memory (48 GB in Leiden). In addition, several DAS-4 sites include non-standard node types for specific research purposes. To allocate a special resource in SGE or prun, this resource should be specified as follows:

  • -l gpu=GTX480
    nodes with an Nvidia GTX480 (with 1.5 GB onboard memory)
  • -l gpu=GTX470
    nodes with an Nvidia GTX470 (with 1.28 GB onboard memory)
  • -l gpu=C2050
    nodes with an Nvidia Tesla C2050 (with 2.625 GB onboard memory)
  • -l ngpus=1
    nodes with an (arbitrary) additional GPU
  • -l fat,m_type=bigmem
    node with extra memory
  • -l fat,m_type=sixcore
    dual six-core-based nodes instead of dual quad-core
  • -l fat,m_type=magnycours
    node with AMD Magny Cours CPUs
  • -l das3
    node from previous-generation DAS-3

This resource selector should be added as 

#$ -l resource
in an SGE job script, or passed as 
-native '-l resource'
option to prun/preserve.

VU University

At fs0.das4.cs.vu.nl the following special-purpose equipment is available for various experiments:

  • 16 (out of 72) of the regular nodes in addition have an NVidia GTX480 GPU;
  • 2 of the regular nodes (node061 and node062) have an Nvidia C2050 Tesla GPU;
  • node073: a node with 96GB memory and 12 TB (6*2TB RAID0) local storage;
  • node074: a node with X5650 CPU (dual 6-cores, 2.67 GHz), 96GB memory and 12 TB (6*2TB RAID0) local storage;
  • node075: a 48-core (quad socket "Magny Cours") AMD system with 128 GB memory, 10 TB (5*2TB RAID0) local storage and 256 GB SSD;
  • node076: an X5650 node with 24 GB memory, two GTX480 GPUs, and 10 TB (5x2TB RAID0) local storage and 256 GB SSD.
  • das3001-das3008: nodes migrated from the previous-generation DAS-3 cluster, with dual-cpu, dual-core, 2.4 GHz AMD Opteron CPUs

Leiden University

At fs1.das4.liacs.nl all 16 compute nodes are "fat" in that they have more memory and local storage than default on the other sites:

  • the nodes have 48 GB instead of 24 GB memory;
  • the nodes have have 10 TB of local storage (5*2 TB RAID);
  • each compute node also has a fast 512 GB SSD (OCZ Z-Drive p88).

University of Amsterdam

At fs2.das4.science.uva.nl the following special node type exist:

  • 4 router nodes (router01-router04) with additional network interfaces.

Delft University of Technology

At fs3.das4.tudelft.nl the following special-purpose equipment is available:
  • 8 (out of 28) of the regular nodes have an NVidia GTX480 GPU;
  • 2 of the regular nodes have an Nvidia C2050 Tesla GPU;
  • 4 "fat" nodes are available that have 48GB memory and 2*2TB RAID0 local storage.

University of Amsterdam - MultiMediaN

At fs4.das4.science.uva.nl the following special-purpose equipment is available:

  • 8 (out of 34) of the regular nodes in addition have an NVidia GTX470 GPU;
  • 7 of the regular nodes have an NVidia C2050 Tesla GPU;
  • 2 of the nodes (1 of the regular ones and 1 of the fat ones) have an Nvidia GTX480 GPU;
  • 2 "fat" nodes are available that have 96GB memory and 6*2TB local storage.

ASTRON

At fs5.das4.astron.nl the following special-purpose equipment is available:

  • 1 special node (b7015) has 8 GTX 580 3GB GPUs, that can be used for GPU scaling experiments and "green computing" research.
  • 1 special node (r815) has 48 cores (quad Opteron 6172), 128 GB memory, 4 * 240 GB PCIe SSD (RAID0), and a fast 8 * 2 TB external RAID (up to 1000 MB/s)
  • 1 special node (node521) has 4 InfiniBand HCAs and a HotLava hex-port 10-GbE interface, for I/O experiments.
  • 1 special node (node522) has another hex-port 10-GbE interface (connected to node521).
  • 1 regular compute node (node501) has an ATI HD 6970 GPU (2GB)
  • 1 regular compute node (node502) has an Nvidia C2050 GPU (3GB)
  • 1 regular compute node (node503) has an Nvidia GTX 580 GPU (3GB)

Note that these configurations change frequently.

Some of these systems (b7015, node521) have faster (dual X5650) CPUs and 48 GB RAM.

The GTX 580 3GB GPUs are currently the newest/fastest GeForce GPUs within DAS-4, and have twice the regular amount of memory. DAS-4 users are encouraged to use these GPUs for (the final versions of) their papers.

The HD 6970 is currently the only ATI GPU available within DAS-4; we encourage all users that have OpenCL applications to compare the performance of the applications on both the HD 6970 and GTX 580.

  • The HD 6970 can be programmed in OpenCL (http://www.khronos.org/#tab-opencl), not in CUDA. The SDK and example programs can be found in:
    /cm/shared/package/amd-app-sdk/2.5
  • Important: always set the DISPLAY environment variable to ":0.0", or your program will not execute.
  • Tip: use the C++ bindings in combination with exceptions in your host CPU code (#define __CL_ENABLE_EXCEPTIONS and #include ); the C++ interface is much easier to use and roughly eight times less verbose than the C interface.