Monday, 26 October 2009

IDL (Virtual Machine) and Condor

At our place we have a nice Condor environment set up, and many people use IDL. Until now, our use of IDL with Condor was limited by the number of available licenses at any given time, until...

Until we discovered the IDL Virtual Machine which lets you run an IDL "executable" file without the need for licenses (see here for an example of how to create such a file).

The problem was that the IDL Virtual Machine is meant to be run interactively in a server with X running and Condor is not particularly well suited for this. But with a little ingenuity, we wrote a script (still in beta :-)) that uses Xvfb and Xautomation to submit condor jobs that will create a virtual X server, then automatically click the "Click to continue" button of the IDL virtual machine, and then do whatever the code is supposed to do. We have tested this, and so far it works quite well. There are some glitches with multicore machines and slow machines, but overall it works without any problems, and now we can submit hundreds of IDL jobs simultaneously to our Condor pool!

More info at SIEpedia