Header Header
Home Projects Documentation Downloads Publications Demos Awards Users News Contact
gradient

JavaGAT 2.1.1 released

June 2011: JavaGAT 2.1.1 is now available from its downloads page. It contains a.o. the following changes with respect to version 2.1.0:

  • reduced the number of jar-files in the lib directory by combining them where possible. This reduces the number of open files in any JavaGAT application significantly.
  • gLite adaptor: use string constants from HardwareResourceDescription where possible.
  • GridFTPFileAdaptor: create a lock when an FTP client has succesfully been created.
  • GlobusJob: there were scenarios where starttime was never set.
  • GlobusResourceBroker: now also looks at the SoftwareDescription.JOB_QUEUE attribute, in addition to the globus.queue attribute, which still is supported for backwards compatibility.
  • WSGT4newResourceBrokerAdaptor: now also looks at the SoftwareDescription.JOB_QUEUE attribute, in addition to the machine.queue attribute, which still is supported for backwards compatibility.
  • GT42ResourceBrokerAdaptor: now also looks at the SoftwareDescription.JOB_QUEUE attribute, in addition to the machine.queue attribute, which still is supported for backwards compatibility.
  • LocalFileAdaptor made to work on Windows.
  • LocalJob: when stopped, actually destroy Java process before waiting until the output streams are finished (otherwise, you can wait for a long time ...).
  • modifications to SshTrilead and SftpTrilead adaptors to support a new version of the sshtrilead library, which we modified to have some ssh-agent support.
  • Fixed SftpTrileadFileInputStreamAdaptor to recognize EOF properly.
  • Various improvements to the SGEResourceBroker (we had someone use it ...)
  • prevent SshTrileadFileAdaptor from executing stuff through ssh when it can do it locally.
  • URI: improved refersToLocalHost method.
  • GATEngine: try to terminate jobs before ending adaptors.
  • FileCpi: copyDirectory path manipulation was wrong for Windows.
  • JobCpi: improved shutdown hook to make a second invocation wait until the first one is done. This prevents GAT.end to call end() on the adaptors before the job shutdown is done.

JavaGAT 2.1.0 released

July 2010: JavaGAT 2.1.0 is now available from its downloads page. It contains a.o. the following changes with respect to version 2.0.5:

  • local file adaptor: rename() did not work properly.
  • modified axis client config mechanism for Globus and Gridsam adaptors: the mechanism no longer depends on a System property. Instead, each adaptor has its own jndi.properties file.
  • Added "getSupportedScheme()" methods to adaptors, so that engine can check which adaptors are applicable for a specific scheme, without actually invoking the adaptor constructor.
  • Added adaptor-specific schemes, which, for instance, allows one to use the "glite" scheme to select the gLite adaptor.
  • A ScheduledExecutor class has been added to the GATEngine utilities. This is a useful class for adaptors that have to execute a job periodically. Using this class, they don't have to create their own thread.
  • various predefined String constants have been introduced, see the Javadoc of the SoftwareDescription API.
  • commandlineSsh file adaptor now implements createNewFile() and length().
  • there have been various fixes and additions to the Glite adaptors.
  • RandomAccessFile adaptors have been added for Globus and SftpTrilead.
  • adaptor order was not quite deterministic yet, so that different copies of the same javagat release could have different behaviour.
  • Globus File adaptors have been made far more robust (thanks to Stefan Bozic and Bastian Boegel).
  • The GT4 file adaptor now actually seems to work.
  • FTP file adaptor copy method did CR-LF translation on copy.
  • FTP adaptor did not deal with symbolic links properly.
  • FileCpi.getParentFile() was wrong in case of an absolute URI with a relative path where the host part refers to the local host.
  • Zorilla adaptor now sets up connection lazily and not in the resource broker constructor. This avoids 30-second delay.
  • gat.adaptor.path System property now again indicates a File.pathSeparator- separated path of directories (as it used to do in version 1.7).
  • Unicore adaptor is modified to make a unicore job advertisable, so that another javaGAT instance can later obtain the results.

SCALE Challenge 2010

May 2010: WebPie won 1st place at the SCALE Challenge 2010! See the submission paper.

Ibis Satin 2.2 released

December 2009: Ibis Satin version 2.2 is now available! For more information and downloads, see the Ibis Satin subproject page.

Ibis MPJ 2.2 released

November 2009: Ibis MPJ version 2.2 is now available! For more information and downloads, see the Ibis MPJ subproject page.

Ibis GMI 2.2 released

November 2009: Ibis GMI version 2.2 is now available! For more information and downloads, see the Ibis GMI subproject page.

Ibis RMI 2.2 released

November 2009: Ibis RMI version 2.2 is now available! For more information and downloads, see the Ibis RMI subproject page.

IPL 2.2 released

November 2009: Ibis Portability Layer (IPL) version 2.2 is now available! For more information and downloads, see the IPL subproject page.

JavaGAT 2.0.5 released

April 2009: JavaGAT 2.0.5 is now available from its downloads page. It is mostly a bug-fix release of JavaGAT 2.0.4, and contains a.o. the following changes with respect to version 2.0.4:

  • Alexander Beck-Ratzka contributed a Unicore resourcebroker adaptor.
  • The Ganymed sftp adaptor has been replaced with a Trilead sftp adaptor.
  • Removed sftp and sftpnew adaptors as they are obsoleted by the Trilead sftp adaptor.
  • JavaGAT now uses slf4j as logging system. This means that you can use your favorite logging system, as long as there is an slf4j implementation for it. To accomplish that, you need to put the appropriate slf4j jar-file in front of your classpath. If, like before, you want to use log4j, you don't have to do anything, since this is the default.
  • The GATEngine now supports a "shared" jar-file directory, which allows adaptors to share some of their dependencies, so that these are loaded with the same classloader. This is useful for for instance security objects that need to be shared by several adaptors. See the file doc/adaptors/whySharedDirectory.txt.
  • There was an error in the unmarshal method of the LogicalFileCpi, which made it impossible to unmarshal an advertized logicalfile.
  • There was a bug in the fixURI method of the FileCpi which sometimes made it refer to a wrong location.
  • When staging in a file with an absolute source path and no explicit destination, the file was staged to the wrong location (not the sandbox).
  • An explicit check for null job arguments was added, so that an exception is thrown when the job description is created, instead of further on when the job is run.
  • Fixed various problems in the Globus and FTP file adaptor. This is tricky stuff because different servers behave differently! We have tested with various servers, and it seems to work now, but your mileage may vary!
  • The Koala adaptor is now functional.
  • The sshtrilead file adaptor sometimes failed because warnings were printed on stderr. Now, these warnings are ignored (see bug #256).
  • Added a list method for the CommandlineSshFileAdaptor.
  • Protect executable and arguments against shell meta character expansion in the Gridsam resource broker adaptor.
  • Some API has been added for file permissions. This basically is jdk7 stuff. For now, gLite is the only adaptor that implements (some of) it. org.gridlab.gat.io.File now has a method getFileAttributeView().
  • A special version of the GATEngine has been added for Android.
  • There has been a lot of development on the various gLite adaptors.
  • The Zorilla adaptor has been improved considerably.

JavaGAT 2.0.4 released

Februari 2009: JavaGAT 2.0.4 is now available from its downloads page. It is mostly a bug-fix release of JavaGAT 2.0.3, and contains a.o. the following changes with respect to version 2.0.3:

  • fixed a couple of glitches in the tests.
  • Glite adaptor: various updates and fixes, but according to the developers, its file adaptor is still to be considered "experimental".
  • Globus File adaptor: various fixes.
  • SshTrilead File adaptor: a couple of fixes for operations on non-existent files. Also, its getAbsolutePath() implementation was wrong.
  • File.getName() returned null in some cases, which is a no-no. This is fixed, and this also fixes bug 187.
  • re-implemented metrics administration. The old method maintained static lists inside GATEngine which were never cleaned up, so that there was a huge memory leak because the engine kept references to all jobs that were ever run. Now, the lists are maintained within the monitorable object itself and are no longer static. This fixes bug 134.
  • (almost) all ResourceBroker adaptors: fixed stop() method: the method may now be called in any job state, as promised in the javadoc.
  • wsgt4new ResourceBroker adaptor: fixed to actually use a user-supplied credential (bug reported by Ralf Groeper).
  • Sshtrilead and Commandlinessh ResourceBroker adaptors: fixed to prevent expansion of shell-meta-characters, and to make them work in case the shell on the server side is a C-shell (bug reported by Kees van Reeuwijk).
  • fixed classloader issue: use either the classloader of the class that prompted the loading of JavaGAT or the context classloader, for the parent of the URL classloaders for the adaptors (bug reported by Ralf Groeper).
  • fixed globus security utils: if the CredentialSecurityObject already contains a GSSCredential, just return that instead of ignoring it (bug reported by Ralf Groeper).
  • fixed CommandRunner: now uses ProcessBuilder. This protects against local expansion of shell meta characters.
  • Wsgt4new ResourceBroker Adaptor: sometimes gave redundant callbacks when the job was already finished, for instance, a redundant POST_STAGING callback, which caused overwriting of initially correct output files (bug reported by Brian Carpenter).
  • fixed File.copy method to mimic cp -r behaviour when copying directories (copy dir a to b ended up as a copy b/a, even if b did not exist, now it only does so when a directory b exists, otherwise a directory b is created with copy of contents of a).
  • LocalFileAdaptor: fixed bug 221 (getParentFile).
  • Zorilla adaptor: various fixes.

MaRVIN Team wins 3rd prize at Billion Tripple track of Semantic Web Challenge 2008

October 2008: The MaRVIN team used the Ibis Portability Layer to develop a self-organising distributed reasoner that can handle these large amounts of data. The MaRVIN system was demonstrated on the DAS-3 distributed supercomputer and was much appreciated for its innovative ideas, winning the 3rd prize in the Billion Triples track, see the Awards page.

Smartsockets 1.4 released

October 2008: Smartsockets 1.4 is now available from the downloads page. This is the first public release of Smartsockets.

JavaGAT 2.0.1 released

JavaGAT 2.0.1 is a bug-fix release of JavaGAT 2.0. It contains the following changes with respect to version 2.0:

  • GlobusFileAdaptor: renameTo failed if the passed URI wasn't on the same host as the file. (Bug reported by Alexander Beck-Ratzka)
  • LogicalFileCpi: underlying storage of the files belonging to the logicalfile is now done in a Vector and is synchronized (previously it wasn't synchronized in an ArrayList). (Bug reported by Enric Tejedor)
  • javagat.properties: updated to reflect all preferences and their defaults.
  • GliteAdaptor: added a GliteAdaptor developed by Thomas Zangerl (Thanks a lot!)
  • added support for preference <"job.stop.poststage", true|false> which controls whether the job should poststage if the stop method is invoked on this job. (request from Keith Cover)
  • FileCpi: improved method recursiveDeleteDirectory, it only does recursive delete if a direct delete fails, this can speed up poststaging and sandbox removal quite a lot.
  • LocalResourceBroker: fixed accessing of the path of the sandbox (Bug reported by Hans-Martin Adorf)
  • GlobusFileAdaptor: mkdir failed if preference file.chmod wasn't set, it did set the mode to 000, now it takes the umask on the remote machine as default (Bug reported by Hans-Martin Adorf and others)

Press Release: Ibis wins Data Analysis Challenge for Finding Supernovae

In August 2008 two Ibis teams (both from the Computer Systems group of Prof. Henri Bal, VU University, Amsterdam) have won the first prize in the "First International Data Analysis Challenge for Finding Supernovae". The competition was organized in conjunction with the IEEE Cluster/Grid 2008 Conference, held from September 29 to October 1 in Tsukuba, Japan (see: http://www.cluster2008.org/challenge/). The goal of the competition was to promote efficient data analysis techniques that can be applied to large-scale and distributed networks of supercomputer systems.

In the competition an enormous distributed database of scientific data, which was gathered by the Subaru telescope in Hawaii, had to be searched to find so-called 'supernova candidates'. A supernova is a phenomenon in which a star explodes in a spectacular manner, causing a very large mount of light to be emitted. For the calculation of the supernova candidates all fourteen candidates had a supercomputer system at their disposal comprising of 12 computer clusters distributed over Japan.

The competition consisted of two categories:
(1) the 'basic' category, in which the goal was to perform all of the necessary calculations as fast as possible.
(2) the 'fault-tolerant' category, in which the goal was to perform all calculations as fast as possible, but with the added complexity of having one or more of the calculation processes killed by the organizers of the competition.
Both categories represent realistic scenarios that play an important role in computing at a very large scale.

The first VU-team, called 'Ibis Stargazers', led by Dr. Jason Maassen, and further comprising of Dr. Frank Seinstra, Dr. Rob van Nieuwpoort, Drs. Niels Drost, and Roelof Kemp, has won the competition in the basic category by obtaining by far the fastest result. While the VU-team required only 36 minutes for all calculations, the second best team used up more than 1 hour. All other teams obtained run-times of 3 to even more than 25 times longer. In addition, only half of all teams were capable of completing the challenge in this category at all.

The second VU-team, called 'IbisNovae', comprising of Dr. Kees van Reeuwijk, has won the competition in the 'fault tolerance' category, using about 49 minutes of calculation time. This achievement is particularly remarkable as no other team was capable of completing the task in this category at all. This is because, in the research field, the provisioning of solutions to fault tolerance is known to be a very hard problem.

The success of the two teams is largely due to the Ibis Grid Programming and Deployment System (see http://www.cs.vu.nl/ibis/), which has been developed at the VU University in recent years. Ibis greatly simplifies the writing of computer programs for large-scale and distributed applications, by hiding virtually all of the underlying software and hardware complexities. The success in this Data Analysis Challenge is also not the only one: earlier this year a VU-team has won the first prize in the 'First IEEE International Scalable Computing Challenge', held in conjunction with CCGrid'08 in Lyon, France. This earlier result was also obtained with a software system that was implemented using Ibis.

The Ibis software system is freely downloadable from the Ibis website: http://www.cs.vu.nl/ibis/.

Ibis RMI 2.1

May 2008: Ibis RMI 2.1 is available now! For more information and downloads, see the Ibis RMI subproject page.

Ibis MPJ 2.1

May 2008: Ibis MPJ 2.1 is available now! For more information and downloads, see the Ibis MPJ subproject page.

Ibis GMI 2.1

June 2008: Ibis GMI 2.1 is available now! For more information and downloads, see the Ibis GMI subproject page.

SCALE Challenge

May 2008: The Ibis team won 1st place (shared) at the SCALE Challenge 2008!

Tutorial

May 2008: Program and slides for the Ibis tutorial at CCGrid'08 are here

Zorilla 1.0 Beta 1

May 2008: New Zorilla Release (1.0 Beta 1). See the Zorilla subproject page for more information on Zorilla, and downloads.

JavaGAT 2.1-rc1

May 2008: JavaGAT 2.0-rc1 is available now! JavaGAT 2.0-rc1 contains many bug fixes, some changes and additions to the API and new adaptors (among them, Globus 4). For more details, please go to the JavaGAT project page.

Satin 2.1

May 2008: Satin 2.1 is available now! For more information and downloads, see the Satin subproject page.

IPL 2.1.1

May 2008: IPL 2.1.1 is available now! IPL 2.1.1 is mostly a maintainance release, fixing a couple of problems with IPL 2.1. For more information and downloads, see the IPL subproject page.

Ibis Tutorial @ CCGRID

April/May 2008: There will be an Ibis tutorial on the 19th of May at CCGrid in Lyon, France. See this page for more information and registration. We will also participate in the SCALE challenge and show demo's during the conference.

IPL 2.1

April 2008: IPL 2.1 is available now! IPL 2.1 is mostly a maintainance release, fixing various problems with IPL 2.0. For more information and downloads, see the IPL subproject page.

2007

IPL 2.0

November 2007: IPL 2.0 is available now! We now support Smartsockets, and are thus able to work in harsh networking environments with Firewalls, NATs, etc. This marks the first API change in four years. For now, only the IPL and TCP based imlementations are available. For more information and downloads, see the IPL subproject page.

Ibis and JavaGAT demo @ Supercomputing '07

November 2007: There will be live JavaGAT and Ibis demos at Supercomputing'07 in Reno, USA. See this page for more information. We also have a technical paper about the JavaGAT in the conference.

Ibis Tutorial @ VU

April 2007: There will be an Ibis tutorial on the 27th of April at the Vrije Universiteit, Amsterdam. See this page for more information.

2006

JavaGAT 1.6

November 2006: JavaGAT 1.6 is available now. For more details, please go to the JavaGAT project page.

Ibis Tutorial @ Sopha-Antipolis

November 2006: An Ibis tutorial will given at the 3nd Grid Plugtest, which will take place from the 27th of November to the 1st of December 2006 in ETSI Headquarters (Sophia-Antipolis, France).

Ibis 1.4

September 2006: Ibis 1.4 is available now.

Ibis 1.3

February 2006: Ibis 1.3 is available now.

2005

Ibis 1.2.1

December 2005: Ibis 1.2.1 is available now.