Graphing Glue

I've been playing with GraphViz and HyperGraph to draw out the relationships between the objects in the GLUE schema. The results are interesting but especially for the complicated sites, e.g. CERN, GRIF, RAL,... they are hard to read. There are massive png's, vrml and svg formats. Also the html pages load a dynamic applet of the data. I'll request that they are worked into gstat which is the natural place for them to live once I have improved them. Warning: Some of the png images are huge and will most likely crash your browser. The image here is RAL-LCG2. The orange blob top right is the FTS.

Combined SLC4 and SL4 CE.

As part of the WN working group efforts I've been finishing of a vandalised CE which is now part of the CERN_PPS site. It is two node cluster both of which are WNs. One is also a gatekeeper publishing GlueCE, VOViews and CESEBind objects, the other is publishing two GlueCluster and GlueSubClusters objects. ... To cut a long story short I have a single gatekeeper node which can matched to for either SL4 or SLC4 and the jobs route through to the correct node at the back. It is exactly the same for any GlueSubCluster attribute such as memory for instance. Tomorrow we try and thrash how to get the final steps of this into a release.... It is quite possibly not going to be pretty, keeping things backwards compatible may just not be possible or desirable. details.

WN Meeting Kickoff Last Week

The first EGEE WN working group meeting happened last week. It went well. I had previously thought I knew exactly what the end point would be even if the exact details of migration were not together. The oversight for me was software tags. Namely that for a site publishing non-overlapping SubClusters then a software installation job has to publish software tags in the correct SubCluster. Exactly how you determine which SubCluster you are installing for is far from obvious. No one has suggested a better solution than a configuration file different on every batch worker, there has to be a better way....

From the EGEE conference there was an addition to the group's scope. The group will give help, advice support, approval, something, ... to the WMS wrapper script addition of doing something sensible in grid jobs during the SIGTERM -> SIGKILL window given by the batch system. Francesco's presentation
looks to have considered everything but can check with group if any comments.

One of the consequences of offering users better job matching to WNs is that the sites are possibly going to start killing more excessive jobs dead. A timely addition if the job wrapper now handles the SIGTERMs.

FTS Release Progress

At last the FTM node has actually made it to be released on the PPS shortly. I really don't want to think how long it has taken. Longer than you can possibly imagine. Also this week I tried the SL4 build of the FTS for first time. Installation has been fine but configuration now looks to stuck and will need some new builds to be solved. Most of the problems are changes between gLite 3.0 and 3.1 rather than the OS upgrade. Having said that most of the changes between 3.0 and 3.1 look sensible and bits of yaim configuration are being removed basically.

Torque Maui Builds from ETICS.

Finally got around to pushing maui and torque through ETICS. Bang up to date releases for sl3 and sl4 with i386 or X86_64 are all available. Others like SL5 should now be trivial. The torque build was fairly straight forward where as the maui build does some black magic for the torque dependency it needs. Recently there have been many requests for the X86_64 builds and it is needed anyway for the upcoming official gLite X86_64 WN which is around the corner now. Also the Dubliners have been wanting it for ages to build other exotic platforms like MacOS. Releases, CVS Repos. To finish this off I need to close down some of pages hosted at RAL and GridPP with pointers to the new locations. Not bad for a day when I got my notice of contract termination.

Back to School.

After a two week break without a single keystroke pressed I've been back for my first day. Big outstanding items are starting a TCG inspired group to investigate and conclude on finding and utilizing individual worker node resources. My first thoughts is that it is all done but there are massive gaps in flexibility of configuration or deployment to achieve it..., it is not done in other words. Other than this the FTM node I've been trying to deploy was rejected from certification for a lack of timestamps in it's logfiles. This is fair enough, I thought it at the time but ignored the problem. Finally the SL4 FTS must be installed, the compilation is there but who knows if it works.

SAM test CE-sft-posix now running.

After deployment mistakes and corrections the CE-sft-posix test is now running again. The significant change is that now new SEs that appear pass for the first two weeks. Previously they were guaranteed to fail for the first two weeks. The results look promising in that out of 287 CEs tested today 69 or so look to fail. This includes the 10 or so CERN CEs where there may be a problem with the configuration of the atlas-durable and lhcb-durable SRMs but perhaps only with respect to the OPS vo which is probably not well tested. Next step will be to get CERN passing. Need to have my own house in order before proceeding. Following that go through the other failiures to look if they seem fair.