EGI-InSPIRE:Globus integration task force

From EGIWiki
Jump to: navigation, search
EGI Activity groups Special Interest groups Policy groups Virtual teams Distributed Competence Centres


Alert.png This article is Deprecated and should no longer be used, but is still available for reasons of reference.


Interoperations menu: Home Unicore Globus Mapper/QCG ARC Desktop Grids EGI Federated Cloud XSEDE Contact



Coordinator: M. Krakowian/EGI.eu

Meetings page: https://www.egi.eu/indico/categoryDisplay.py?categId=53

Mailing list: globus-integration-tf@mailman.egi.eu

Mailing list page: https://www.egi.eu/sso/groupView/globus-integration-tf

NGIs with Globus RC's: NGI_HR, NGI_DE, NGI_NL, NGI_UK

Current status

Status of integration

Status Comments
Management

Completed

  • globus-GRIDFTP
  • globus-RLS
  • globus-GSISSHD
  • GRAM5
Monitoring

Completed

  • globus-GRIDFTP - hr.srce.GridFTP-Transfer, org.nagios.GridFTP-Check
  • globus-GSISSHD - org.nagios.gsissh-Check
  • GRAM5 - hr.srce.GRAM-Auth, hr.srce.GRAM-Command, hr.srce.GRAM-CertLifetime

In EGI OPS Availability and Reliability Profile:

  • GRAM5 - hr.srce.GRAM-CertLifetime, hr.srce.GRAM-Auth, hr.srce.GRAM-Command
Accounting

Completed

Manual: MAN09

Support

Completed

Globus support is provided by the EGCF product team through the EGI Helpdesk with base level of service support
Dashboard

Completed

  • GGUS-81278
    • hr.srce.GRAM-Auth
    • hr.srce.GRAM-CertLifetime
    • hr.srce.GRAM-Command
    • hr.srce.GridFTP-Transfer
    • org.nagios.GridFTP-Check
    • org.nagios.gsissh-Check
Documentation

Completed

Information System

Completed

Manual: MAN08

Actions

Open action points

Closed action points

September 2012

Action

Descripton Assigned to Deadline Concerning
ACTION 1 Check the integration of sgutils perl library into epel repository. Emir Imamagic
Monitoring
ACTION 2 Find out the best location for source code of probes. Helmut Heller
Monitoring
ACTION 3
Provide update on accounting. Steve Crouch, John Robinson
Accounting
ACTION 4
Organize the next meeting in first 2 weeks of November (5th – 16th) Małgorzata Krakowian

June 2012

Action

Descripton Assigned to Deadline Concerning
ACTION 1 Modify certification procedure for Globus sites.

PROC09 HOWTO04#Globus_checks

Małgorzata Krakowian
Documentation
ACTION 4
Find out GLUE 2.0 support plans in WMS. Peter Solagna


June 2012

ACTION 3: Distribute materials on installation of GRAM5. (David Groep)

  • Email with instructions was sent to the list on June 15th

ACTION 2: Move probes to IGE repositories. (Daniel Waldmann)

March 2012

ACTION 1: Propose on the next OMB inclusion of Globus tests to Operations tests. (Tiziana Ferrari, Emir Imamagic)

Old action points

  • AP: SC to keep us updated on Grid-SAFE/accounting efforts within Globus. Especially:
    • Summary of the meeting between JR and Steven Booth on 110607 and the status of the LRZ testbed.
    • More exact time frames for moving the complete code to SourceForge and the release process afterwards. Howto and details on the release process.
    • Keeping us updated on Globus relevant UR fields analysis within the EMI ComputeAccounting working group.
    Update: See mail from SC on 110620.
    Update 110714: SC to circulate the IGE accounting mechanisms document giving a quick overview on GridSAFE including archictecture, components, planned release process,... People can send their DN to SC to access and test the testbed. Current Deadline to get GridSAFE testbed fully operational: September. After that code on SourceForge will be updated to mirror this working version. In August: Plan for a comparison between GridSAFE client and NGS RUS client (any RUS client _should_ work out of the box). When testbed works: SC will check UK plans to publish from GridSAFE to the RUS in Manchester (RUS part in production, passing on to APEL in prototype stage). JG will ask the UK Globus sites.
    Postponing a decision on whether GridSAFE should be part of an IGE release taken into account by UMD until after the testbed is fully operational. IGE development cycles: second January 2012, third July 2012. Actual deployment might take a long time. Investigate possiblity to sqeaze it into IGEv2 instead. No plans for intermediate IGE releases.
    Update: SC circulated IGE accounting mechanisms on our mailinglist 110714
  • AP: MB to ask NGIs on desired timelines for Globus integration to determine need for a second Early Adopter team for Globus at current stage.
    • AP: TA to ask within NGI-DE for a second EA volunteer.
    Update: LRZ will be an EA for Globus. AP on MB to send a reminder to all NGIs to tell their sites to register all their Globus GT5 services in GOCDB, since this is a good time now with the upcoming SAM/Nagios release.
    Update 110714: JG to ask for EA volunteer within the UK, UTCN in Romania volunteering as well (g-Lite based production site which plans to deploy Globus GT5 components in summer 2011). Contacts and Informations should be sent to Mario David.
    Identifying possible problems due to a still missing operational set of SAM/Nagios Globus probes during staged-rollout as we go.
  • AP on JG to ask Dortmund if they are able to make some accounting tests against the new APEL test system.

Actionpoints on hold

  • AP: (on request of COO after a discussion at the TCB): Which type of OS for GLOBUS resources will be used by the NGIs that plan to integrate their GLOBUS resources into EGI? This information is important for the integration of GLOBUS into the UMD.
    Update: "if we do not have sites that volunteer for EA activity in StagedRollout then there is no point in including that particular platform (e.g. CentOS on x86_64) in the UMD!"
    Update: AP on MB to update Michel Drescher on the OS discussion we had at the meeting 110606. Done 110609.
    Update: Currently just SL5 supported by UMD. Requests on further plattforms should go over OMB to TCB. Every supported plattform increases workload for verification and staged-rollout.
    Freeze: Taking up this actionpoint again when everything works in staged-rollout and discuss an official request through the OMB and the TCB. --> on hold.

Closed action points

  • AP on next meeting: MB to create a doodlepoll for the next meeting in the second week of July.
If another meeting is necessary before the TF in Lyon will be decided during this meeting.
Update: http://www.doodle.com/dk532e78uerp84nh
Update 110714: If we need an other meeting during August then a dedicated one about staged-rollout: AP on MD to decide whether we need a dedicated meeting on staged-rollout during August.
  • AP on EI: to share his notes and technical details on the gLite based BDII hack for Globus as a possibility to customize a gLite BDII to act as an information system for Globus 5. IS to eventually test this and probably compare with IIS (to derive a basic set of what is needed to be compatible?).
Update 110728: EI provided a link to http://wiki.cro-ngi.hr/en/index.php/Globus_BDII_integration
  • AP on Nagios Probe support responsibilities: HH to answer Emirs email on IGE officially taking over probe support in the future. The details should be fixed within a month.
    Update 110714: AP on HH to resend the mail with the technical details and the names.
    Update: Ilya will take over the probes.

after meeting on 110714

  • AP: MB to circulate link of MS409 as soon as it comes out to answer questions about the new staged-rollout process.
    Update: https://documents.egi.eu/secure/ShowDocument?docid=478 as in mail of 110613
    Update: MS409 almost final and currently undergoing AMB review, so the procedures in it for the staged rollout can be considered valid. You find them either here in this document or on the wiki. Any questions now or later via email to Mario David are welcome. --> closed.
  • AP: MB to ask OK for a nice summary of status of Globus integration into Argus at the next meeting.
    Update: A status summary has been given --> closed.


after meeting on 110606

An update has been given by IS during the meeting 110606. IIS is still in alphastate.


  • AP: MB to send link with suggested extensions to OGF-UR by APEL to HH. HH to check within IGE if those extensions are sufficient for use in Grid-SAFE.
    Update: MB, checked again with John Gordon about current valid APEL extensions:
    • a) extensions to the basic UR, eg it didn't include the sitename for example. Cristina del Cano Novales wrote a comparison of our UR with the standard as input to OGF, after the Catania meeting. It is probably somewhere in GridForge but a quick browse didn't find it.
    • b) a variation on the UR that represented total CPU time etc for a number of jobs. We made the mistake of calling this the aggregated accounting record. Some people had a semantic problem with this name and expected it to contain a concatenation of all the job records. We should have called it a summary accounting record. http://forge.gridforum.org/sf/docman/do/listDocuments/projects.ur-wg/docman.root.current_drafts.aggregate_ur_schema
    • c) Cristina has proposed an updated APEL UR schema with the fields aligned more closely to the standard. This is list of fieldnames used in a relational database, not an XML DTD. This is being discussed in a special group in EMI chaired by Cristina: https://twiki.cern.ch/twiki/bin/view/EMI/ComputeAccounting
    Update: Update: IGE (SC) contacts the EMI ComputeAccounting over their mailinglist and actively participates in the UR fields analysis.
Created a new AP on SC to keep us updated on the efforts within the EMI ComputeAccounting wg.


  • AP: EP to report on installation of Grid-SAFE in the LRZ testbed to our mailinglist when done.
Update: JR and other people on the LRZ testbed to get the missing Globus log part of the RUS publisher client, the harvester, from the original NeSCForge (or Steven Booth) and adept it according to their needs.
Update: Meeting between JR and Steven Booth on 110607 to get Grid-SAFE working on the testbed and to discuss the moving of the complete code to SourceForge to create a complete distribution there and the best ways to release it and advertise. Probable timeframe for move: 2 weeks.
Created a new AP on SC to keep us updated on it and provide us with some more details on the exact roadmap for the release on our mailinglist.


  • AP: MB to test GGUS leaf to IGE: what is the mailinglist and who is on it.
Update: Ticket created: https://gus.fzk.de/ws/ticket_info.php?ticket=70499
Update: Ticket was confirmed and solved. Some people needed more rights to access, update and close the ticket in GGUS. No more explicit test tickets needed. Another ticket already open: https://ggus.eu/tech/ticket_show.php?ticket=70857


  • AP: HH to pass on question to IGE about which transport mechanism is intended to be used in Grid-SAFE.
Answer: SOAP over https.
  • AP: MB to organize the next meeting as a dedicated accounting meeting with the accounting experts: either as a Phoneconference slot during the EGCF1 or in week 21 before the 2nd EMI allhands F2F in Lund.
    Update: Current status: Phonemeeting during EGCF1 on Wednesday afternoon possible. Details still defined between MB and HH, otherwise Monday the 23rd as backup option.
    Update: Meeting held on Wednesday as intended.


Update: Event passed successfully. A follow-up event will be held in about a year. Collocation with another event is planned.

after first meeting

  • AP: Fill mailinglist
  • AP: Decide on date for first meeting and put together an agenda.

https://www.egi.eu/indico/conferenceDisplay.py?confId=469