Difference between revisions of "PROC19"

From EGIWiki
Jump to: navigation, search
(Integration steps)
(Integration steps)
Line 183: Line 183:
 
| &nbsp; EGI Ops'''<br>'''  
 
| &nbsp; EGI Ops'''<br>'''  
 
| Verify integration  
 
| Verify integration  
| '''Alessandro Paolini, Enol Fernandez, Baptiste Grenier, '''Operations check documentation
+
| '''Alessandro Paolini, Enol Fernandez, Baptiste Grenier, '''Operations checks documentation
 
|- valign="top"
 
|- valign="top"
| colspan="4" | '''Availability Monitoring'''
+
| colspan="4" | '''Monitoring'''
 
|- valign="top"
 
|- valign="top"
 
| 3a  
 
| 3a  
Line 204: Line 204:
 
| NGI/EGI Ops  
 
| NGI/EGI Ops  
 
| Deploy probe in production nagios and documentation.  
 
| Deploy probe in production nagios and documentation.  
| Ops support check documentation
+
| Operations checks documentation
 
|- valign="top"
 
|- valign="top"
 
| colspan="4" | '''OPS Dashboard'''
 
| colspan="4" | '''OPS Dashboard'''

Revision as of 11:05, 9 January 2019

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators




Title Integration of new cloud management framework and grid middleware in EGI Production Infrastructure
Document link https://wiki.egi.eu/wiki/PROC19
Last modified
Policy Group Acronym OMB
Policy Group Name Operations Management Board
Contact Group operations@egi.eu
Document Status DRAFT
Approved Date
Procedure Statement A procedure for the steps to integrate new cloud management framework (Cloud platform) or grid middleware (Grid Platform) in EGI Production Infrastructure.
Owner Alessandro Paolini



Overview

To assure production quality of EGI Infrastructure every cloud management framework (Cloud platform) or middleware (Grid Platform) supported by Production Resource Centres needs to fulfil certain requirements. The goal of this procedure is to assure EGI Infrastructure compliance.

Definitions

Types of Technology Products:

  • cloud management framework: software for creating, managing, and deploying infrastructure cloud services.
  • grid middleware: software which allows the users to execute jobs in grid infrastructure.


Please refer to the EGI Glossary for the definitions of the terms used in this procedure.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", “MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Entities involved in the procedure

  • Technology Provider (TP): person representing or leading Technology Provider team
  • EGI Operations (EGIOps)
  • Operations Centre (OC)
  • Resource Centre (RC)
  • Operations Management Board: EGI operations policy board

Prerequisites

Before sending a request:

  • OC has to have
    • the support of TP with effort to integrate with EGI Infrastructure (information system, accounting, monitoring etc), provide support via GGUS and maintain software via UMD
    • one or more RC available to deploy the new platform
  • TP has to have
    • effort to integrate with EGI Infrastructure (information system, accounting, monitoring etc), provide support via GGUS and maintain software via UMD
    • the support of one or more OC, with one or more RC available to deploy the new platform and the integration-software developed by the TP

Steps

Request submission and validation

The request can be send by:

  1. Operations Centre
  2. EGI Operations
  3. Technology Provider

Resource Centre can also request integration of new cloud management framework or grid middleware. Such request should be first approved by Operations Centre, it belongs to. In such case OC is responsible to create a ticket on behalf of RC.


Step Action on Action
1 Applicant
Opens a GGUS ticket to Operations to start the process.
Subject: Request for integration of XXX to EGI Production Infrastructure (PROC19)

Dear Operations,

We would like to request for starting procedure of integrating XXX to EGI Production Infrastructure
https://wiki.egi.eu/wiki/PROC19

Prerequisite data:
* name of Technology Product:
* Technology Provider (person representing or leading the team) contact details(name, email):
* customers of the Product (eg. user community, Operations Centre):
* motivation:


Best Regards
XXX
2 EGIOps

Operations contacts the OMB to request the approval of the request.

Functional requirements

Functional requirements for new product to be integrated:

  • support VO concept
  • support X.509 certificates

Integration steps

Integration covers following areas (where possible steps can be done in parallel):

# Responsible Action Additional temporary comments
0a EGIOps When Approved, EGIOps and TP should agree on Underpinning Agreement (UA)
0b EGIOps

Set up an integration Task force for given Technology Product composed of:

  • Technology Provider representative
  • Operations tools representative
  • NGI representatives (wanting to deploy Technology Product) with Pilot Site
  • EGI Operations representative
  • User communities representative (interested in deployment of Technology Product)
  • EGI Security team representative
  • UMD representative

Configuration Management

1a

GOCDB 

Add new service types agreed within Task Force.


1b Pilot Site Deploy technical service instance and register in GOCDB.
Information System
2a Technology Provider

Develop software for integration with BDII.


Analyse the use cases for deciding if the new technology has to be published in the BDII or not, and the relevant set of information to publish.

  • Must the new technology be published in the BDII?
    • it has to be created the information providers
  • Is it necessary any modification to the Glue Schema for properly publishing the new technology information?
    • any modification to the Glue Schema has to be discussed with the Glue Working Group
2b Pilot Site Deploy software for integration with BDII and documentation.
2c   EGI Ops
Verify integration Alessandro Paolini, Enol Fernandez, Baptiste Grenier, Operations checks documentation
Monitoring
3a Technology Provider Develop nagios probe with support from SAM team and documentation.
3b ARGO/SAM EGI Support  

Check probe, verify results, add to SAM release.

Add test to ROC profile.


3c NGI/EGI Ops Deploy probe in production nagios and documentation. Operations checks documentation
OPS Dashboard
4 EGI Ops  Add test to Operations profile Setting a Nagios test status to OPERATIONS
Support
5a Technology Provider

Declare Quality of Support for 3rd level Support Unit (SU) and name of SU

FAQ GGUS-New-Support-Unit


5b GGUS   Create Support Unit under "Product Teams" category

Accounting
6a Technology Provider

Develop software for integration with APEL


Define integration and what data should be published.

  • if the new technology is using computing or storage services for which accounting data are already collected, there is no need of new parser/software for integration with APEL
6b APEL  Validate integration Ops support check documentation
6c EGI Accounting Portal  Display data
VM image Marketplace (Cloud management framework ONLY)
7a Technology Provider Implement subscription to VM image lists from EGI MarketPlace and create documentation.
7b Pilot Site Add service endpoint to GOCDB (type: eu.egi.cloud.vm-metadata.vmcatcher)
7c NGI/EGI Ops Check eu.egi.cloud.vm-metadata.vmcatcher is passing  https://cloudmon.egi.eu/nagios/ 
7d EGI Cloud VM Image Management SU Validate integration Ops support check documentation
Documentation
8a EGI Ops Support Update relevant documentation

8b Technology Provider Develop documentation for users and admins where missing

8c EGI Ops Support Validate Documentation
Resource Allocation
9
Resource Allocation

Add new access method in e-GRANT(if needed)

Define if the middleware is a new way of accessing resources


Security
10a
Technology Provider Fill in Technology Provider security questionary (to be provided)
10b
Security team
Provide recommendations based on provided input

10c Technology Provider Implement recommendations
10d
Security Team Validate implementation of recommendations
UMD
11a Technology Provider Ensure software developed for the integration of the new Technology Product satisfies UMD Minimal Requirements
11b EGI Ops (UMD representative) Technology Provider info is added in TechnologyProviders List and UMD Product ID card
11c EGI Software provisioning Team Applies the UMD Software Provisioning process to assess the quality of the new product
11d EGI Ops (UMD representative)
 Once confirmed a successful provisioning (step 11c) includes the new product/products into an UMD release and makes it available to the production infrastricture, in the UMD repositories
Final
12a EGI Ops Announces the availability of new product to OMB and includes the announcement in the monthly EGI Broadcast to communicate the availability of the new product to NGIs, VOs, RCs managers




Revision History

Version Authors Date Comments





A. Paolini 2016-06-03 Trying to define some rules for integrating the new technology with the information and the accounting system
Alessandro Paolini 2016-06-08 "EGI Operations Support" was decommissioned, changed all the references to "Operations"