DAS-3 Achievements

August, 2008: VU University researchers win two first prizes at DACH 2008

DACH 2008, or the First International Data Analysis Challenge for Finding Supernovae, is a competition which was held in conjuction with the IEEE Cluster/Grid 2008 international conference in Tsukuba, Japan. The competition was organized and supported by the IEEE Technical Committee on Scalable Computing, the Japan MEXT grant-in-aid for priority area research called Info-Plosion, and the Special Interest Group on High Performance Computing (SIGHPC) of the Information Processing Society of Japan (IPSJ).

The competition was driven by the observation that the importance of large scale data analysis increases every year, not only in the scientific domain (e.g. high energy physics, astronomy, biology), but also in industry (e.g. web search engines). The objective of the competition was to follow this emerging trend, and to encourage efficient data analysis efforts in distributed environments.

In the DACH challenge a large distributed database (of several hundreds of GBytes) of scientific data, gathered by the Subaru telescope in Hawaii, had to be searched to find new and unknown supernova candidates. A supernova is a phenomenon in which a star explodes in a spectacular manner, causing a very large amount of light to be emitted. For the calculations, the participants were given access to a supercomputer system comprising of 12 compute clusters distributed over Japan.

• In the Basic Category, the goal was to process all the data as fast as possible. A VU University team consisting of Jason Maassen and Frank Seinstra won the first prize. Their Ibis-based solution obtained by far the fastest result. While their solution required only 36 minutes for all calculations, the second best team used more than 1 hour. All other teams obtained run-times of 3 to even more than 25 times longer.
• In the Fault Tolerant Category, the goal was to process all the data as fast as possible, under artificial node failure. VU University's Kees van Reeuwijk won the first prize there. His solution was implemented in Maestro, a self-organizing data-flow programming model based on the Ibis IPL. Maestro was the only system that participated in the Fault Tolerant challenge. It used a total of 92 nodes, 34 of which were killed, and one of which crashed by itself. Maestro automatically restarted about 10 percent of all tasks, before returning the correct result.

May, 2008: VU University team wins first prize at SCALE 2008

SCALE 2008, or the First IEEE International Scalable Computing Challenge, is a competition organized by the IEEE Technical Committee on Scalable Computing (TCSC), and endorsed by the IEEE Technical Committee on Parallel Processing.

The objective of the competition, held in conjunction with the CCGrid 2008 international conference in Lyon, France, was to highlight and showcase real-world problem solving using scalable computing techniques. The contest focused on end-to-end problem solving using concepts, technologies and architectures (including clusters and grids) relevant to the overall scope of the TCSC. All participants in the challenge were expected to identify significant current real-world problems where scalable computing techniques can be effectively used, and to design, implement, evaluate and demonstrate their solutions.

At SCALE 2008 the VU University Ibis team presented a scalable distributed supercomputing solution for the multimedia domain. Specifically, the team developed an application in which a digital camera is capable of real-time 'recognition' of objects from a set of learned objects, while being connected to a world-wide grid system comprising of clusters in Europe (including DAS-3), the United States, and Australia.

With their application they demonstrated true wall-socket grid computing. The entire application, including all required libraries, were stored on a single memory stick, which could be plugged into any Linux or Windows laptop with an appropriate JVM installed. From there, the application was compiled and started, with the world-wide set of available grid resources being employed entirely transparently.

November, 2007: StarPlane Project Drives Global Innovation and Collaboration with Real-Time Distributed Computing

At SuperComputing 2007 (SC'07), Nortel issued a press release about the StarPlane project. In this project, the University of Amsterdam, VU University and the Dutch National Research and Education Network provider SURFnet investigate the use of dynamic 10G lightpaths for Grid computing, using DAS-3 as a testbed.

Quotes from the press release:

• "The StarPlane project provides researchers with access to massive computing power delivering the equivalent of the processing capacity of 500 personal computers to the desktop. StarPlane uses pure optical technology to link the Distributed ASCI Supercomputer 3 (DAS-3) computer clusters at five locations in The Netherlands into a grid to enable delivery of bandwidth on-demand, e.g. enabling computer scientists to reconfigure the topology of the distributed supercomputer. On-demand service activation of photonic networking is delivered using an extension to Nortel's Dynamic Resource Allocation Controller (DRAC) platform."
• "Our next-generation hybrid optical and packet switching network delivered a paradigm shift in research networking," said Kees Neggers, managing director, SURFnet. "The full photonic implementation of the SURFnet6 network brings alive the possibilities created by coupling the applications and the network and is delivering a flexible application network experience that puts the high-end users and advanced applications in the driver's seat."
• "Traditionally networks are seen as unpredictable resources and this project is changing that picture allowing for a wealth of new research," said Cees de Laat, associate professor at the University of Amsterdam (UvA). "With grid middleware interacting directly on the nationwide photonic layer enabling specification of optimal topologies per computational job, we are able to add another dimension in the resource allocation algorithms."

July, 2007: MultimediaN/UvA researchers win 'Most Visionary Research Award' at AAAI 2007

AAAI 2007 held an exciting new event, which took place during the opening reception: the AI Video Competition. The objective of the competition was to communicate to the world the fun of pursuing research in AI, and illustrate the impact of some of its application areas. Submitters were asked to create narrated videos of 1-5 minutes in length, focused on interesting AI research. Videos were then reviewed by an international program committee. The creators of the best videos were presented with awards named in honor of Shakey, SRI's pioneering robot.

MultimediaN is a Dutch public-private non-profit organization in which the scientific world cooperates with industrial and other non-profit institutions. Together they strive to achieve high-quality multimedia solutions for the digital world of today and tomorrow. MultimediaN researchers Frank Seinstra and Jan-Mark Geusebroek (ISLA, Informatics Institute, University of Amsterdam) submitted a video on their research project " Color-Based Object Recognition by a Grid-Connected Robot Dog" to the AI Video Competition, and won the award in the most prestigeous category: "Most Visionary Research". The video demonstrates their unique application in which a visual task is successfully performed by a robot which is connected to a set of compute clusters located around the world.

The application presented in the video is a close integration between a parallel image and video processing library implemented in C++ and MPI, and a wide-area communication and deployment framework, implemented in Java and Ibis. The system itself has been shown live at several international conferences, such as ICME 2005 (Amsterdam, The Netherlands), ECCV 2006 (Graz, Austria), SC2007 (Reno, NV, USA). Clearly, without the benefits of the Ibis system, moving the application from a controlled laboratory setting to a real-world and hostile grid environment would have been close to impossible.