EGI-Engage:WP3 (JRA1) E-Infrastructure Commons

From EGIWiki
(Redirected from EGI-Engage:WP3)
Jump to: navigation, search
EGI-Engage project: Main page WP1(NA1) WP3(JRA1) WP5(SA1) PMB Deliverables and Milestones Quality Plan Risk Plan Data Plan
Roles and
WP2(NA2) WP4(JRA2) WP6(SA2) AMB Software and services Metrics Project Office Procedures


WP leader: Diego Scardaci/ at INFN

WP contact (all members):egi-engage-wp3 AT

WP taskleaders: egi-engage-wp3-taskleaders AT


This workpackage will coordinate the development of the e-Infrastructure Commons - an ecosystem of services that constitute the foundation layer of any distributed e-Infrastructures. The technical development of the e-Infrastructure Commons services will be user-driven to ensure full interoperability with other e-Infrastructures and RIs. The current set of services will be expanded to include a Service Registry and Marketplace. The main targets of this task are:

Task Leaders

To contact all task leaders (see below), send mail to 

Task Name Task Leader Deputy Mailing list
Authentication and Authorisation Infrastructure
Christos Kanellopoulos, GRNET
Peter Solagna,
JRA1.2 Service Registry and Marketplace
Dean Flanders, SWING
JRA1.3 Accounting
Stuart Pullinger, STFC
Adrian Coveney
JRA1.4 Operations Tools
Cyril Lorphelin, CNRS
Christos Kanellopoulos, GRNET
JRA1.5 Resource Allocation – e-GRANT
Tomasz Szepieniec, CYFRONET
Roksana Różańska

Involved partners


TASK JRA1.1 Authentication and Authorisation Infrastructure

(Lead: GRNET, M3 – M27)

Estimated task effort: 24.5PM
Provide viable methods for authentication and authorisation (AA) in the EGI ecosystem, addressing current shortcomings. The task will explore how to integrate suggested AA methods with current middleware and community services, guaranteeing a sufficient Level of Assurance, and the use of credentials issued by other infrastructures and services:

TASK JRA1.2 Service Registry and Marketplace

(Lead: SWING, M1 – M30)

Estimated task effort: 22.5PM (SWISS 7PM; GR NGI (IASA) 3PM; IT NGI (INFN) 6.5PM; CYFRONET 3PM; STFC 3PM)
The main target of this task is the design and the development of a proof-of-concept of a Service Registry and a Marketplace for the EGI infrastructure. The Service Registry will record the list of all the services offered by EGI and will be available for consultation by the EGI customers. The Marketplace will allow customers to acquire services through a payment or for free, through SLAs with service providers. An important role will be covered also by, as broker/intermediary to define SLA and policies and to identify the best services for the customers and the best users for the service providers. This design will be used to drive the development of the Service Registry and Marketplace proof-of-concept prototype. The implementation will be based on already existing open-source tools, identified during the analysis phase, and extensions of EGI tools (AppDB, GOCDB and e-GRANT).

TASK JRA1.3 Accounting

(Lead: STFC, M1 – M30)

Estimated task effort: 33PM
The target of this task is the evolution of the EGI accounting system. Its main components are the accounting repository and the Accounting Portal. The development guidelines will be:

This task will collaborate with the OGF Usage Record Working Group, in particular to agree a schema for a data usage record. Moreover, the support for the OGF standard UR2 will be improved.

TASK JRA1.4 Operations Tools

(Lead: CNRS, M1 – M30)

Estimated task effort: 56PM
The operations tools have to be continued improved to adapt them to technology evolution and to satisfy new requirements emerging from service providers and user communities. The aim of this task is to coordinate tool development to:

The work plan, described below, includes a set of subtasks, focused on specific tool components.
Operations Portal
The Operations Portal provides information to various actors along with related facilities, such as the VO administration tool and the VO management module, the different dashboards (Operations dashboard, security dashboard, VO Operations dashboard) and different communication tools: broadcast tool and downtime system announcement. The activity will:

GOCDB is an information system for recording and managing e-Infrastructure topology data. The GOCDB evolution foresees work to:

The ARGO platform is the continuation and evolution of the SAM framework. The development of ARGO in EGI-Engage will focus in three main branches: modularise the Core Monitoring Engine, monitoring availability & reliability as a Cloud Service and provide clean interfaces for end-users, operations and developers. The first branch, the Core Monitoring Engine, includes the following activities:

The Messaging Service is a critical core service provided through a Network of Message Brokers resilient to isolated failures. The Messaging Service will be evolved providing a Restful HTTP API as a layer on top of the existing Message Broker Network. The change will be backwards compatible, as the operation of the STOMP interfaces for direct usage of the Message Broker Network will be continued. Developers, which will move to the new Restful Service layer, simplify the maintenance of their client implementations. The introduction of a Restful Service layer on top of the Message Broker Network will give EGI the ability to implement also other requested features such as access controls or message accounting.
Security Monitoring
Security monitoring tools developed in the scope of EGI focused mainly on traditional grid sites and operations, which is not enough nowadays. Current technologies, namely federated clouds, bring new security challenges that must be addressed by new approaches. In this task we will identify the new areas and provide solutions for proper monitoring of them. The solutions delivered by the task will be modular and individual components could be utilised independently and on several levels in the infrastructure.

TASK JRA1.5 Resource Allocation – e-GRANT

(Lead: CYFRONET, M1 – M30)

Estimated task effort: 17PM
e-GRANT is a platform that enables EGI customers to apply and get allocation of compute and storage resources. In this task, e-GRANT functionalities will be extended to cover the complete SLA lifecycle-enabling activities such as tracing of resource centre configuration, monitoring of resources delivery, availability analysis for agreed services, capacity reports analysis and billing. Another area of e-GRANT development includes facilitating activities taking place before customer request resources. The platform will enable customers to investigate the services and resources available to them by publishing this information on the EGI Service Registry and Marketplace, including special offers or promotion actions, such as seed-resources for new communities, demonstrators or hackathons. Furthermore, e-GRANT will be extended to play an important role in the new mode of service delivery like pay-for-use, which would require extension towards negotiating the price. e-Grant will support the integration of EGI with other e-Infrastructures enabling customers to jointly request resources owning to different infrastructures.


The following gives an overview of deliverables. Schedule

Code Title
Technical design of the new Accounting Portal and implementation plan (R)
D3.2 Design of the EGI Service Registry and Marketplace (R)
D3.3 Accounting Repository release (OTHER)
D3.4 First release of the Operational tools (OTHER)
D3.6 (3.5)
Analysis on techniques to manage big data on the EGI accounting system (R)
D3.5 (3.6) First release of the new Accounting Portal deployed in production (OTHER)
D3.7 First release of the EGI Service Registry and Marketplace prototype (DEM)
D3.8 First data accounting prototype (DEM)
D3.9 Identity Management for Distributed User Communities report (R)
D3.10 Second release of the new Accounting Portal deployed in production (OTHER)
D3.11 Second release of the Operational tools (OTHER)
D3.12 Second release of the Accounting Repository (OTHER)
D3.14 (3.13) Report on Data Accounting (R)
D3.13 (3.14) Second release of the EGI Service Registry and Marketplace prototype (DEM)
D3.15 Second data accounting prototype (DEM)
D3.16 Final report on EGI Service Registry and Marketplace (R)
D3.17 Final release of the Accounting Repository (OTHER)
D3.18 Final release of the Operational tools (OTHER)
D3.19 Final release of the new Accounting Portal deployed in production (OTHER)


The following gives an overview of milestones. Schedule

Milestone Title
M3.1 Operational tools development roadmap agreed   
M3.2 User requirements on data accounting collected
M3.3 Operational tools development roadmap revised
M3.4 Pilot services and best practices to enable federated AAI solutions released
M3.5 The first version of the EGI Marketplace is demonstrated
M3.6 The second version of the EGI Marketplace is demonstrated



Description Type How measured Target PM12 Target PM24 Target PM30
KPI.6.JRA1.AAI Number of users adopting federated IdP Cumulative Number of users accessing EGI services through the IdP Proxy/broker TBD

Activity Metrics

Description Type Task
M.JRA1.AAI.1 Number of communities whose IdP framework integrates with EGI AAI Cumulative 3.1
M.JRA1.Marketplace.1 Number of entries in the EGI Marketplace (i.e. services, applications etc.) Cumulative 3.2
M.JRA1.Accounting.1 Number of kinds of data repository systems integrated with the EGI accounting software Cumulative 3.3
M.JRA1.Accounting.2 Number of kinds of storage systems integrated with the EGI accounting software Cumulative 3.3
M.JRA1.OpsTools.1 Number of new requirements introduced in the roadmap Per period 3.4
M.JRA1.OpsTools.2 Number of probes developed to monitor cloud resources Per period 3.4
M.JRA1.eGrant.1 Number of user requests handled in e-GRANT Per period 3.5

Yearly plan

Internal Documents

Personal tools