Difference between revisions of "EGI-XSEDE:Collaboration"

From EGIWiki
Jump to: navigation, search
Line 14: Line 14:
== Administrative  ==
== Administrative  ==
*Mailinglist:  [mailto:egi-xsede@xsede.org egi-xsede@xsede.org] (managed by Victor Hazlewood vhazlewo@utk.edu)
*Mailinglist:  [mailto:egi-xsede@xsede.org egi-xsede@xsede.org] (managed by Victor Hazlewood vhazlewo@utk.edu)  
*Meetings: [https://indico.egi.eu/indico/conferenceTimeTable.py?confId=1222 EGI-CF 2013], Telcon, [https://www.xsede.org/web/xsede13/schedule XSEDE13-BoF], [https://indico.egi.eu/indico/conferenceTimeTable.py?confId=1417#20130916 EGI TF 2013] + [https://wiki.egi.eu/w/images/7/7f/17.09_EGI_XSEDE_OSG_meeting.doc minutes (doc)], <br> [https://indico.egi.eu/indico/contributionDisplay.py?contribId=179&confId=1994 EGI CF 2014]
**[https://indico.egi.eu/indico/conferenceTimeTable.py?confId=1222 EGI-CF 2013]
**[https://www.xsede.org/web/xsede13/schedule XSEDE13-BoF]
**[https://indico.egi.eu/indico/conferenceTimeTable.py?confId=1417#20130916 EGI TF 2013] + [https://wiki.egi.eu/w/images/7/7f/17.09_EGI_XSEDE_OSG_meeting.doc minutes (doc])
**[https://indico.egi.eu/indico/contributionDisplay.py?contribId=179&confId=1994 EGI CF 2014]
== Goal  ==
== Goal  ==

Revision as of 14:29, 2 June 2014

Welcome to the page of the EGI-XSEDE collaboration



The goal of this collaboration is to identify and exchange best practices and solutions between the XSEDE and EGI e-infrastructures so they can operate more efficiently to serve scientists in the U.S. and Europe. The collaboration is focussed on four areas of work:

  • Operation of e-infrastructure services
  • Cloud services for science and education
  • Champions to engage with new users
  • User support and joint use cases

Each area has a named contact point from EGI and from XSEDE. Further details about the work under these four areas are provided below.

Background information on XSEDE

The Extreme Science and Engineering Discovery Environment (XSEDE) is the most advanced, powerful, and robust collection of integrated advanced digital resources and services in the world. It is a single virtual system that scientists can use to interactively share computing resources, data, and expertise. XSEDE is a five-year, $121-million project supported by the National Science Foundation. It replaces and expands on the NSF TeraGrid project.

Background information on EGI

The European Grid Infrastructure (EGI) delivers integrated computing services to European researchers, driving innovation and enabling new solutions to answer the big questions of tomorrow. EGI is a federation of over 340 resource centres, set up to provide computing services and resources to European researchers and their international collaborators. EGI supports research collaborations of all sizes: from the large teams behind the Large Hadron Collider at CERN and Research Infrastructures in the ESFRI roadmap, to the individuals and small research groups that equally contribute to innovation in Europe.

Area 1: Operations

  • EGI: Tiziana Ferrari <tiziana.ferrari@egi.eu>, Malgorzata Krakowian <malgorzata.krakowian@egi.eu>
  • XSEDE: Victor Hazlewood <victor@utk.edu>


  1. Organisational benchmarking - compare operational processes and services – so we can identify the good practices and learn these from each other
  2. Helpdesk and ticket procedures (e.g. Escalation processes)
  3. Resource monitoring
  4. Use the multi-infrastructure use cases to decide what should be changed in the infrastructures and on which side (Compchem and WeNMR)
  5. Integration of helpdesk and accounting to support communities that in the future will jointly use XSEDE and involve OSG for those communities like wenmr that will also consume OSG resources


  1. Accounting (17.09.2013)on Rob Quick: OSG will share with XSEDE information how they publish accounting data to EGI
  2. [DONE] Support (17.09.2013)on Małgorzata Krakowian: Send EGI Helpdesk team contact to Victor Hazlewood
  3. Support (17.09.2013)on Alessandro Costantini: review WeNMR ticketing workflow if it meets Compchem requirements
  4. Support (17.09.2013)on Alessandro Costantini and Marco Verlato: Describe ticketing use case (tools + procedures) which are now in place
  5. Authentication (17.09.2013) on TBD: identify the IGTF CAs that XSEDE needs to authorize
  6. Authentication (17.09.2013) on TBD (XSEDE): get the XSEDE Operations Security team to add the identified CAs to the XSEDE accepted CAs list



  • The EGI Helpdesk GGUS. It is based on Remedy, authentication of users is based on X.509 certificates, access through Shibboleth is being implemented. All users with a valid certificate released by a IGTF CA can have read access to all tickets. GGUS already supports an interface to RT, as various National Grid Initiatives have RT as local helpdesk system. GGUS Interfaces
  • The GGUS Report Generator is the system we are using to collect statistics about usage, distribution of tickets, time to respond and solve tickets etc.


The EGI accounting infrastructure is distributed with a central accounting database gathering usage records either directly from each individual resource centres or from national databases. The central accounting database is based on APEL. Accounting records are being collected for computing jobs that are successfully DONE. Storage and cloud accounting will be rolled to production in 2013 and are being tested.


EGI monitoring is distributed and is based on Service Availability Monitoring (SAM). Monitoring data is centrally gathered for access to historical information and for the computation of performance indicators (availability and reliability). SAM is installed by individual National Grid Initiatives, monitoring data is exchanged through messaging (ActiveMQ).

  • EGI Monitoring information can be consulted via MyEGI.

Procedures and procedures

Resource allocation

  • Contacts: Ken Hackworth, XSEDE Allocations Manager; Dave Hart, POPS

XSEDE's current allocations system, called "POPS", already supports allocations of storage resources. In fact, POPS is general enough that it will allow to specify any kind of resource (and associated "billing unit") for allocations.

"POPS 2.0", which is a re-engineering of the POPS system to disentangle it from some legacy decisions, re-work legacy code that dates back to the late 1990s (along with 15+ years of incremental changes), and better integrate it with other components of the XSEDE infrastructure. As part of that process we are designing POPS as "allocations software as a service" such that non-XSEDE client organizations could spin up their own allocations process within the XSEDE service, with minimal effort/investment for the client organization. We're still fairly early in the design and implementation stage, but if EGI might be interested in being a 'customer' of this XSEDE service, we should certainly talk.


Area 2: Cloud

  • EGI: David Wallom <david.wallom@oerc.ox.ac.uk>
  • XSEDE: David Lifka <lifka@cac.cornell.edu>


  1. VMI preparation – collaboration on creating endorsed images of common software
  2. Identify questions that the technical support teams will have to be able to answer
  3. Hybrid cloud setup – internal cloud that is kept busy, outburst some load to external clouds when needed.

Area 3: Champions

  • EGI: Sara Coelho <sara.coelho@egi.eu>
  • XSEDE: Kay Hunt <kay@purdue.edu>

Area 4: User support

  • EGI: Gergely Sipos <gergely.sipos@egi.eu>
  • XSEDE: Sergiu Sanielevici <sergiu@psc.edu>, Suresh Marru <smarru@iu.edu>


  1. Support the implementation of the use cases that have been submitted to the 'Collaborative Use Examples' call
    1. Computational Chemistry use case (CMMST): See related gateway activity below.
    2. WeNRM use case: Active use of OSG by WeNMR achieved. WeNMR put the activities with XSEDE on hold.
  2. Identify science gateways that would benefit from resources from XSEDE and EGI, and facilitate such integration activities.
    1. New gateway for CMMST based on SCI-BUS technology. Development of a setup is ongoing. Technical contacts: Peter Kacsuk (SCI-BUS), Andrew Grimshaw (XSEDE), Antonio Lagana (CMMST)
  3. Exchange information, best practices and tools from the user support and technical outreach areas.
    1. EGI Applications Database: http://appdb.egi.eu