Minutes of the IT strategy steering committee, 02/12/11

Present: Martin Hardcastle (chair: MJH), John Atkinson (systems manager: JA), Tim Gledhill (staff rep: TMG). Mark Thompson (staff rep: MAT), Kieran Forde (student rep: KF), John Barnes (PD/fellow rep: JB)

0. Introduction

The meeting opened at 14:30.

1. Minutes of previous meeting

The minutes had previously been circulated and were approved.

All actions from the previous minutes had been done.

2. System status report

JA gave a report on central IT facilities.

He reported that the web/ftp/gateway server uhssfv2 had developed a fault when rebooted as part of an enforced electrical shutdown. While the fault was not critical, he had felt it prudent to purchase a new server as this one was over 5 years old. A new server (named star-server) had been purchased and is a Dell R510 — a slightly lower spec version than the main server (and cluster head node). Fedora 15 (64bit) is installed on the new server and both http and ftp servers had been successfully transferred to it (the DNS alias star.herts.ac.uk has been redirected to star-server). As part of this upgrade, the /scratch and /www (web hierarchy) filesystems had been upgraded. A RAID which had previously been used by Maria Cruz had now been put to use as the /scratch disc, increasing its size from 6.4 to 11 TB. The web hierarchy was then moved to the RAID originally used for /scratch. This increases the /www filesystem from 1 TB to 6.4 TB, necessary due to the increase of data from Hugh and David Campbell's Bayfordbury all sky project. The old 8 x 200GB RAID originally used for /www had now been retired.

The general backup strategy was discussed, prompted by the investigative use of AMANDA as a backup tool. Current disc usage in CAR is ~250 GB for /home and ~5.5 TB for /data. Consequently, traditional Level 0 backups now take around two whole days to complete. JA reported that the primary advantage of using AMANDA is that it performs load balancing: this means once the backup strategy has been in use for a short time, Level 0 dumps are spread out over time. AMANDA is controlled by rules that state, for example, that a Level 0 dump of a particular disc must be performed at least once a fortnight, with incremental dumps performed all other days. He concluded that results looked promising but that further tests were required. Currently both traditional backups and those performed by ANAMDA are being used. The committee agreed that we should continue to keep a full dump of /home and /data in the fire safe (in the STRI) as a safeguard against the server room getting destroyed. JA and MJH commented this was possible using both traditional scripts (as used currently) and/or AMANDA.

Fedora 14 is now coming to EOL which means security updates will shortly stop being provided. JA proposed two alternatives: either desktop OS should be upgraded to Fedora 16 in the New Year, or we should wait until April for Fedora 17. The committee agreed that upgrades to FC16 in January were preferable. JB reported that he was running FC16 on his laptop without problems.

It was agreed that the system was running smoothly and there were no concerns from users.

3. Cluster report

MJH reported on the status of the cluster, as follows:

The 12 new CAIR nodes reported in the last minutes had been up and running for some time and were being used.
CAR now has a dedicated 16-node, 192-core enclosure with QDR infiniband, bought out of end-of-year money. This meant that the room is basically full — we can't increase computing power in there significantly without straining its thermal capacity. There are four unfilled slots in chassis 6 (CAIR), and some room for extra SMP machines if absolutely necessary. There is room in the racks for extra storage.
Some firmware upgrades over the summer meant that the QDR links between the CAIR nodes in different chassis are working to spec now (this solves a long-standing problem).
CAIR had purchased another 24 Tb (Raw). That fills up the capacity of the existing RAID unit (8 x 12 x 2 = 192 Tb raw, around 150 Tb actual). Discussions with CAIR about the purchase of more storage were ongoing at the time of the meeting.
Usage of the cluster was running at about 30% -- the actual usage has stayed about the same but the capacity has increased. There has been increased use by astronomers and decreased use by Computer Science over the period, though.

In terms of future development, MJH hoped to be able to incorporate /home on the cluster into the backup schedule soon. Other, more long-term development would include the deployment of Lustre and the use of a unified Infiniband network.

4. Money left on the rolling grants

MJH explained that there was some money left on the rolling grants that would need to be spent by the end of the year: some of this was earmarked for computer equipment. The committee felt that the best approach was to bring forward some of the items proposed for the new Consolidated Grant, since there was no guarantee that these would be funded in STFC's current financial position. Accordingly, the committee selected for purchase a new RAID head unit, which would start a new stack and would be filled with 12 3-Tb discs (36 Tb: about £13k) and sufficient gigabit Ethernet switches to provide gigabit connections to all gigabit-capable desktop machines (budget estimate about £6k).

Action: MJH to pass this on to the forthcoming MC meeting.

There was also some money on some of the research training grants (RTGs) which support graduate students: Janet expects that we will be able to buy student desktop PCs from this source (rather than from end-of-year PAM money as we have for the last couple of years). The committee discussed loan laptops for students: one of the two notebook PCs that have been set up for this purpose is currently in use and we should get a report back from the user at some point, but if these machines turn out to be too old then it might be sensible also to purchase some (mid-range) laptops from the same source.

At TMG's suggestion, the committee also discussed IT-related training. MJH said that the rolling grants were probably not appropriate for this for most people but that there should be funds to send people on external training courses if there was a need (e.g. the RTGs for STFC students; general funds for UH students and fellows). There was no mechanism for doing this other than an ad-hoc approach to Janet on a case-by-case basis, and external courses were generally too expensive for everyone to attend them. After discussion it was agreed that MJH would pass the committee's feeling that it should be possible to pay for external training on to Janet/the MC.

Action: MJH to discuss training at the MC.

5. Mobile device purchase

The committee agreed that mobile devices (e.g. iPads) could be useful in some circumstances, but that it was not appropriate to commit to purchasing them for everyone who wanted them. The committee agreed to amend the existing laptop policy to cover such devices: the essential point remains that users should have a demonstrable need for such a purchase that cannot be met by any existing equipment.

Action: MJH to amend the laptop policy appropriately.

6. AOB

There was no other business. The reps had nothing to pass on: KF said that everyone was happy.

7. Date of next meeting

The next meeting should be in Feb/March 2012.

Action: MJH to circulate a Doodle nearer the time.

The meeting closed at 15:40.