Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI-Engage:WP5 (SA1) Operations

From EGIWiki
(Redirected from EGI-Engage:WP5)
Jump to navigation Jump to search
EGI-Engage project: Main page WP1(NA1) WP3(JRA1) WP5(SA1) PMB Deliverables and Milestones Quality Plan Risk Plan Data Plan
Roles and
responsibilities
WP2(NA2) WP4(JRA2) WP6(SA2) AMB Software and services Metrics Project Office Procedures



WP leader:  Peter Solagna/EGI.eu

WP contact: egi-engage-wp5@mailman.egi.eu


Objective

This work package will:

  1. coordinate the operational activities of the EGI production infrastructure, ensuring a secure and reliable provisioning of grid, cloud and storage resources, harmonised between resource providers and peer e-Infrastructures.
  2. Evolve the security activities in EGI to support the new technologies and resource provisioning paradigms, maintaining a secure and trustworthy infrastructure while supporting new use cases and new ways to access the resources;
  3. Integrate and deploy platforms on cloud and grid resources to support new use cases for the existing and new EGI users. These platforms will include services for the long-tail of science that will reduce both the barriers for new users to access EGI resources and the learning curve to efficiently use them.

Task Leaders

To contact all task leaders (see below), send mail to 

Task Name Task Leader Deputy Email
SA1.1
Operations Coordination
Peter Solagna (EGI.eu)
SA1.2
Development of Security Operations
David Kelsey (STFC)
SA1.3
Integration, Deployment of Grid and Cloud Platforms
Peter Solagna (EGI.eu)

Involved partners

  • EGI.eu   
  • CESNET   
  • CNRS    
  • INFN   
  • CNR   
  • SURF sara
  • CYFRONET   
  • UU-SNIC   
  • STFC   
  • CERN

Tasks

TASK SA1.1 Operations Coordination

(Lead: EGI.eu, M1 – M30) 

Estimated task effort: 60PM
The main goal of this task is to coordinate the EGI infrastructure operational activities, liaising with the NGI Operations Managers, to ensure that the operations of the partners providing resources and services to the users are harmonised among each other and with the e-Infrastructure Commons services. The operational procedures and processes will be extended to support diverse technologies and paradigms for service provisioning such as IaaS, PaaS and SaaS.
Operations coordination will ensure that the core services (funded by the NGIs through participation in EGI.eu) are delivered and fulfil their targets, and that their evolution is aligned with the evolution of the EGI production infrastructure. Requirements will be gathered from the service providers and the user communities.
Other activities of this task include: to evolve the portfolio of technologies offered through the Unified Middleware Distribution (UMD), which is used by the resource providers to deploy the software, and to liaise with UMD software contributors and initiate agreements with new technology providers, in particular to include the software products that are needed to support the new upcoming technologies in the Federated Cloud Platform.

TASK SA1.2 Development of Security Operations

(Lead: STFC, M1 – M30)

Estimated task effort: 24PM
This task will develop security operations, including policies, procedures and best practices, to meet the requirements of new trust models, new developments and new usage scenarios as these evolve in EGI-Engage. The work is split into the following activities:
Security requirements and risk assessment for new services, technology, and deployments
The new developments and evolving usage scenarios in EGI-Engage will involve trust models different from the core infrastructure used in EGI-InSPIRE. The task will ensure that the security requirements and the trust model are defined. Any security problems will be addressed and risk assessment associated with new deployments will be developed, to drive operational security in the evolved environment, to keep services secure and available and to mitigate the serious risks
The evolution of operational security procedures, including forensics
Refine and extend the current security procedures and tools for incident response and forensics, for example: to take into account new kinds of players (e.g. cloud resource providers), or to extend the emergency suspension mechanism to cover new kinds of services. The security procedures related to other EGI operational procedures will also be modified as required.
Develop a new trust framework and develop new policies
In collaboration with other infrastructures, we will define new additions to a new policy framework to handle the new deployment and usage scenarios as they evolve in EGI-Engage.
In collaboration with JRA1.1 the task will validate the architecture assumptions through testing in partnership with user communities under realistic production conditions and provide support on AAI security issues in close coordination with the EGI CSIRT and SVG. The task will provide recommendations on how best to sustain this important activity beyond the end of EGI-Engage.
Develop the security challenge framework
Experience from EGI-InSPIRE has shown that performing security service challenges on the operational infrastructure is useful confirm that there is sufficient audit information for traceability of any incident, that procedures and tools are sufficient and that participants are trained and aware of the need to participate in incident response. The framework for these security challenges will be modified and extended to meet the evolving scenarios.
Develop the software vulnerability handling process to adapt to new technology and deployments
Software vulnerability issues in the EGI core infrastructure have been handled through a close relationship with the technology providers, many of whom supply members of the Software Vulnerability Group (SVG). The general principles will remain, including the assessment of risks and the issuing of advisories. In the evolving scenarios of EGI-Engage there are, however, likely to be different types of relationship with the technology providers, especially when this does not involve membership of SVG. The procedures and methods for handling vulnerabilities in EGI-Engage will evolve accordingly.

TASK SA1.3 Integration, Deployment of Grid and Cloud Platforms

(Lead: CNR, M1 – M30)

Estimated task effort: 22PM
This task will deploy, operate and integrate new platforms and frameworks, in the EGI production infrastructure, to enable new use cases and support new user communities.
Fishery and Marine Sciences VREs deployment and operations (CNR) (M6 - M30)
Deploy and maintain a set of fishery and marine sciences VREs to offer innovative working environments with the as-a-Service paradigm of both gCube and EGI. These facilities are offered by exploiting EGI resources, namely hosting of services and data. A number of representative VREs will be pro-actively created to act as a sort of typical environment ready to use for a class of users (e.g. biodiversity students, data managers). In addition VREs will be specifically created to serve the needs of specific use cases identified during the project. This activity will also support the adaptation of existing applications and data to the VREs.
The e-Collaboration for Earth Observation (e-CEO) platform
The e-Collaboration for Earth Observation (e-CEO) platform developed by the European Space Agency (ESA) to support online contexts where researchers can work collaboratively and compare and evaluation different problem-solving approaches. EGI will support these challenges by provision capacity to the e-CEO platform onto the EGI Federated Cloud. The required capacity will be negotiated between ESA and the resource providers via EGI.eu for each challenge and provided as unfunded contribution by the EGI Community as described in the section 3.4.1.

Platform for the long-tail of science (CESNET, CNRS, CYFRONET, INFN)
This task will also develop services and tools to support the long-tail of science, removing the barriers that prevent new users to quickly start using the EGI resources and decreasing the learning curve.
The services are:

  • Short-term credential service. This service will provide to the long-tail of science users with short-term X509 credentials, in case of difficulties to access to an IGTF Certification Authority. With this service users will be able to access the EGI resources using credentials that they already own, without the need to start a process for new credentials.
  • User Management Portal. This portal will be deployed to allow users to require access to the EGI resources, and to submit a request for resources. The requests will be approved by NGI representatives or outreach teams. The portal will allow the users to use short-term X509 credentials and will prevent users to submit more tasks to the EGI services once they consumed all the capacity allocated for their request or for security reasons.
  • Integrated VREs, to lower the learning curve of the EGI services and reduce the time required for new users to become productive and make the most out of the granted resources. The VREs to be integrated are services extensively used by several communities (e.g. DIRAC and SCI-BUS).

Deliverables

The following gives an overview of deliverables. Schedule

Code Title
D5.1
Report on the evolution of the EGI Operations (R)
D5.2 Platforms for the long-tail of science (R)
D5.3 Evolution of security policies, procedures and best practices (R)
D5.4 Roadmap of the evolution of the EGI Operations beyond EGI-Engage (R)

Milestones

The following gives an overview of milestones. Schedule

Milestone Title
M5.1
Platform for the long-tail of science is available
M5.2
e-CEO challenges run on EGI resources
M5.3 VREs for the fishery and marine sciences community are deployed in production

Metrics

KPIs

Metrics
Description Type How measured Target PM12 Target PM24 Target PM30
KPI.2.SA1.Intergation Number of RIs and e-Infrastructures integrated with EGI Cumulative Number of RIs and e-Infrastructures that are NOT participants of egi using at least one service from either Core, Collaboration or a Community Platform (via MoU or OLA) 9
11
13
KPI.3.SA1.Software Number of new registered software items and VM appliances Per period Numbers of new registered software and VM Appliances in AppDB 50/50
60/60
70/70
KPI.4.SA1.Cloud Number of providers offering compute and storage capacity accessible through open standard interfaces Cumulative Number of Cloud resource centres registering in GOCDB interfaces exposing standard API: OCCI, CDMI... 25
30
35
KPI.8.SA1.Users Number of VO SLAs established Cumulative Number of VO SLAs established regarding to HTC, Cloud and Operations tools 4
8
10
KPI.11.SA1.Users User satisfaction Average Satisfaction of Long tail of science and VO managers with whom EGI has SLA (1 to 5 scale ) 4
5
5
KPI.14.SA1.Size Number of compute available to international research communities and long tail of science Cumulative
TBD TBD TBD
KPI.15.SA1.Size Number of storage available to international research communities and long tail of science Cumulative
TBD TBD TBD
KPI.17.SA1.Size Number of compute resources available to the long tail of science Cumulative Amount of resources (Cores) supporting the long-tail VO 300
500
500



Activity Metrics

Metrics
Description Type Task
M.SA1.Operations.1 Amount of federated HTC compute capacity (EGI participants and integrated) Cumulative 5.1
M.SA1.Operations.2 Amount of federated HTC storage capacity (EGI participants and integrated): (Disk, Tape) Cumulative 5.1
M.SA1.Operations.3 Amount of allocated resources (storage) allocated through a EGI centrally managed pool of resources Cumulative 5.1
M.SA1.Operations.4 Amount of allocated resources (logical cores) allocated through a EGI centrally managed pool of resources Cumulative 5.1
M.SA1.Operations.5 Number of new products distributed with UMD Per period
5.1
M.SA1.SecurityOperations.1 Number of security policies and procedures updated reviewed and adapted to support new services Per period 5.2
M.SA1.Platforms.1 Number of gCUBE VREs instantiated on the Federated Cloud for the iMARINE community Cumulative 5.3
M.SA1.Platforms.2 Number of CPU time consumed by e-CEO challenges (hours * cores) Per period 5.3

Yearly plan

Internal Documents