Announcements

6 Jul 2015

DAS-5 is now fully operational! To make room for DAS-5, DAS-4/UvA and DAS-4/ASTRON have been decomissioned, only their headnodes remain available.

28 Oct 2014

The Hadoop setup on DAS-4/VU has been updated to version 2.5.0.

30 Jan 2014

The Intel OpenCL package for Intel CPU's and Xeon Phi has been updated to version 3.2.1.

3 Sep 2013

The Nvidia CUDA development kit has been updated to version 5.5.

25 April 2013

Slides of the DAS-4 workshop presentations are now available.

14 Jan 2013

DAS-4/VU now has 8 new nodes with latest Nvidia K20 GPU.


GPU programming

GPUs on DAS-4 (see Special Nodes) can be programmed using two paradigms: CUDA and OpenCL.

CUDA

CUDA is supported by Nvidia GPUs. The current CUDA 5.5 implementation can be added to your environment as follows:


$ module load cuda55/toolkit

Documentation for writing and building CUDA applications is then available from $CUDA_INSTALL_PATH/doc/CUDA_C_Programming_Guide.pdf. An SGE job script to submit a CUDA application on a host with a GTX480 GPU could then look like this:


#!/bin/sh
#$ -S /bin/sh
#$ -l gpu=GTX480
#$ -l h_rt=00:15:00
#$ -cwd

. /etc/bashrc
module load cuda55/toolkit

./cuda-app opts

Environment modules containing settings for additional CUDA BLAS implementation (cuda55/blas), FFT implementation (cuda55/fft), and profiler (cuda55/profiler) are also available.

OpenCL

OpenCL is supported by Nvidia GPUs, AMD GPUs, Xeon Phis, and regular host CPUs. Three implementations are available: opencl-nvidia, opencl-amd, and opencl-intel. All OpenCL implementations employ a common libOpenCL.so dynamic library, so when switching between OpenCL implementations, be sure to use module unload to undefine the previous settings.

When running on a host that provides multiple OpenCL device platforms, be sure when requesting the devices with clGetDeviceIDs to specify either CL_DEVICE_TYPE_CPU (for the host CPU), CL_DEVICE_TYPE_GPU (for the GPU), or CL_DEVICE_TYPE_ACCELERATOR (for the Xeon Phi). Specifically do not rely on using CL_DEVICE_TYPE_ALL and then selecting a static device number, since this can change. It can even be different for hosts that are identically configured.

In the examples below, a simple OpenCL demo application cldemo.c is used that displays the platforms found, selects one, and scales an array of integers by a factor of two using OpenCL.

Nvidia

The Nvidia OpenCL implementation supports only Nvidia GPUs. It can be used as follows:


$ module load opencl-nvidia
$ gcc -I$OPENCL_INCLUDE -c cldemo.c
$ gcc -L$OPENCL_LIB -lOpenCL cldemo.o -o cldemo-nvidia
$ cat cldemo-nvidia.job
#!/bin/sh
#$ -S /bin/bash
#$ -l h_rt=00:10:00
#$ -l gpu=GTX480
#$ -j y
#$ -cwd

. /etc/bashrc
module load opencl-nvidia

./cldemo-nvidia

$ qsub cldemo-nvidia.job

Note that the Nvidia OpenCL implementation also imports the Nvidia CUDA environment.

AMD

The AMD OpenCL implementation supports both AMD GPUs and regular host CPUs. It can be used as follows:


$ module load opencl-amd
$ gcc -I$OPENCL_INCLUDE -c cldemo.c
$ gcc -L$OPENCL_LIB -lOpenCL cldemo.o -o cldemo-amd
$ cat cldemo-amd.job
#!/bin/sh
#$ -S /bin/sh
#$ -l h_rt=00:10:00
#$ -j y
#$ -cwd

. /etc/bashrc
module load opencl-amd

./cldemo-amd

$ qsub cldemo-amd.job

Intel

The Intel OpenCL implementation supports regular host CPUs and the Xeon Phi. It can be used as follows:


$ module load opencl-intel
$ gcc -I$OPENCL_INCLUDE -c cldemo.c
$ gcc -L$OPENCL_LIB -lOpenCL cldemo.o -o cldemo-intel
$ cat cldemo-intel.job
#!/bin/sh
#$ -S /bin/sh
#$ -l h_rt=00:10:00
#$ -j y
#$ -cwd

. /etc/bashrc
module load opencl-intel

./cldemo-intel

$ qsub cldemo-intel.job