Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "PROC23"

From EGIWiki
Jump to navigation Jump to search
(Log changes.)
(15 intermediate revisions by 3 users not shown)
Line 4: Line 4:
|Doc_title = Services and Service components release and deployment process
|Doc_title = Services and Service components release and deployment process
|Doc_link = https://wiki.egi.eu/wiki/PROC23
|Doc_link = https://wiki.egi.eu/wiki/PROC23
|Version =  
|Version = 2019-09-18
|Policy_acronym = OMB
|Policy_acronym = OMB
|Policy_name = Operations Management Board
|Policy_name = Operations Management Board
|Contact_group =  operations at egi.eu  
|Contact_group =  operations at egi.eu  
|Doc_status = FINAL
|Doc_status = FINAL
|Approval_date =  
|Approval_date = 2018-11-29
|Procedure_statement = The procedure describes the process of release and deployment of services and service components in EGI production infrastructure
|Procedure_statement = The procedure provides steps for service updates and releases of the centrally-provided production services.
|Owner = Baptiste Grenier
}}  
}}  


= Overview  =
= Overview  =


The procedure describes the process of release and deployment of service and service components in EGI production infrastructure. It is applicable to all production instances of the services and service components mentioned in the following list:
The procedure describes the release and deployment of the centrally-provided IT service and service components part of the EGI Service Catalogue.
 
*Accounting repository and portal
*AppDB
*ARGO Messaging Services
*ARGO/SAM
*CheckIn (AAI Support)
*Collaboration tools
*DIRAC4EGI
*EGI Catch-all services
*EGI Marketplace
*GGUS
*GOCDB
*Operations portal and VAPOR
*PERUN
*Security monitoring
*UMD and CMD Software provisioning infrastructure
 
Adding and removing tools from this list can be done by EGI central operations office, upon request, and every change is communicated to the OMB ML before being implemented. Please, open a GGUS ticket (ticket category: service request) to the Operations SU to request any change.


= Definitions  =
= Definitions  =
Line 43: Line 26:
= Entities involved in the procedure  =
= Entities involved in the procedure  =


*'''Service Provider team''': Team responsible for development, release and deployment of the tool <br>
*'''Service Provider team''': Team responsible for development, release and deployment of the service or service component
*'''Operations team''': Oversees all the process and may provide further people for testing the tool
*'''Operations team''': Oversees all the process and may provide further people for testing the service or service component
*'''Noc-Managers''': they are informed regarding the release of the new tool and may provide further people for testing it
*'''Noc-Managers''': Is informed regarding the release of the new service or service component and may provide further people for testing it


= Requirements  =
= Requirements  =
Line 51: Line 34:
The prerequisites are:  
The prerequisites are:  


*Tool is used by EGI production Infrastructure<br>
* Centrally-provided Service or service component part of the EGI production Infrastructure;
*Development team should follow [https://wiki.egi.eu/wiki/Instructions_for_Production_Tools_teams#Release_and_deployment_management https://wiki.egi.eu/wiki/Instructions_for_Production_Tools_teams#Release_and_deployment_management]
* Development team should refer to the [[EGI_Change_Management|EGI Change management]] documentation.
* In case of deployment of a major update of a tool, a contingency plan must be agreed with the team who operates the tool, to handle major disruptions caused to production activities by problems in the new version of the tool.


= Acceptance criteria =  
= Acceptance criteria =  


A new release of a Tool can be deployed in production if:
A new release of a Service or service component can be deployed in production if:
* All the prerequisites are satisfied
* All the prerequisites are satisfied
* The results of the testing in step ''3b'' do not show any major problem:
* The results of the testing in step ''3b'' do not show any major problem:
** The tool is usable in all its features
** The Service or service component is usable with all its features
** The interoperability with the other tools is not broken
** The interoperability with the other Service or service component is not broken


= Steps =
= Steps for the release of a service or service component update =


{| width="906" height="305" class="wikitable"
{| width="906" height="305" class="wikitable"
|-
|-
! <br>
!
! Responsible  
! Responsible  
! Action  
! Action  
! Notes
! Notes
|- valign="top"
|- valign="top"
| 1<br>
| 1
| Service Provider team<br>
| Service Provider team
|  
|  
Once release is ready the team opens a GGUS ticket ('''selecting category: Release''') to Operations with the following information:
Once release is ready the team [https://ggus.eu/?mode=ticket_submit opens a GGUS ticket] ('''selecting category: Release''') to Operations SU with the following information:


*Name of the tool
* Name of the service or service component;
*Date of release
* Date of release;
*Release notes
* Release notes;
*Suggested deployment date
* Suggested deployment date;
*Testing instance url and testing instructions  
* Testing instance url and testing instructions;
*Names of testers if testing is manual (if not defined Development team may ask to appoint testers)
* Names of testers if testing is manual (if not defined Development team may ask to appoint testers);
* If the changes is one orginating from a Change Record tracked in an RT ticket as part of CHM, then add the link to the respective RT ticket for that change
* If the release is originating from one or multiple Change Records tracked in RT tickets as part of Change Management (CHM), then add the links to the respective RT tickets.
|
|
|-
|-
Line 89: Line 71:
| Operations Team
| Operations Team
|
|
*Inform the Noc-Managers about the upcoming release, asking if there is anyone else interested in performing the tests
* Assess if ticket came from a change record tracked in RT:
*can add further people for performing the tests
** if yes check that the GGUS ticket contains links to the changes (RT tickets) that are implemented in this release;
*The suggested duration of the test phase is two weeks
** If not validate that the change was allowed to by-pass Change Record tracking.
*Update the ticket
* Inform the Noc-Managers about the upcoming release, asking if there is anyone else interested in performing the test;
* can add further people for performing the tests;
* The suggested duration of the test phase is two weeks;
* Update the ticket.
|
|
|-
|-
| 3a
| 3a
| Operations Team / Noc-Managers
| Operations Team / Noc-Managers
| Update the ticket with the information on the performed tests and their result
| Update the ticket with the information on the performed tests and their result.
|
|
|-
|-
| 3b
| 3b
| Service Provider team
| Service Provider team
| Update the ticket with information about results of the overall testing phase<br>
| Update the ticket with information about results of the overall testing phase.
| <br>
|
|-
|-
| 4<br>
| 4
| Service Provider team
| Service Provider team
| Provide in the ticket the link to updated documentation<br>
| Provide in the ticket the link to updated documentation.
| <br>
|
|-
|-
| 5
| 5
| Service Provider team and Operations team<br>
| Service Provider team and Operations team
| Agree on deployment date and update the ticket
| Agree on an appropriate deployment date and update the ticket.
| <br>
|
|-
|-
| 6
| 6
| Operations team
| Operations team
| 10 days before the upcoming deployment, inform the Noc-Managers.
| 10 days before the upcoming deployment, inform the Noc-Managers.
Update the ticket
Update the ticket.
|
|
|-
|-
| 7
| 7
| Service Provider team
| Service Provider team
| Schedule a downtime of the service in case it is needed
| Schedule a downtime of the service in case it is needed.
|
|
|-
|-
| 8
| 8
| Service Provider team
| Service Provider team
| Deploy release and update the ticket
| Deploy release and update the ticket.
| <br>
|
|-
|-
|9
|9
|Product team or Operations team
| Operations team
|Check that the GGUs tickets contains links to the changes (RT tickets) that are implemented in this release. ''Added on Nov 15th 2017''
| Close the GGUS ticket after a week of the deployment only if the release was successful.
|<br>
|
|}
 
= Emergency release =
 
== Definition of emergency release ==
 
An emergency release is the process of releasing one Service or service component, or a set of Services or service components, with the targeted goal of solving a specific problem affecting the EGI infrastructure. To qualify the problem for an emergency release the problem should be classified in at least one of the following categories:
* Major incident;
* Critical security vulnerability.
 
== Steps for emergency release ==
 
{| width="906" height="305" class="wikitable"
|-
!
! Responsible
! Action
! Notes
|- valign="top"
| 1
| Service Provider team
|  
Once the release is ready the team [https://ggus.eu/?mode=ticket_submit opens a GGUS ticket] (Selecting '''category''': '''Release''' and '''priority''': '''Top priority''') to Operations SU with the following information:
* Name of the Service or service component;
* Date of the release;
* Release notes;
* Justification of the need for the emergency release;
* If the release is originating from one or multiple Change Records tracked in RT tickets as part of Change Management (CHM), then add the links to the respective RT tickets.
|
|-
|-
| 10
| 2
| Operations Team
|
* Acknowledge the need for the emergency release;
* Update the ticket.
|
|-
| 3
| Service Provider team
| Record an unplanned downtime of the service in case it is needed.
|
|-
| 4
| Service Provider team
| Deploy the release and update the ticket.
|
|-
| 5
| Operations team
| Operations team
| Close the GGUS ticket after a week of the deployment only if the release was successful
| Inform the Noc-Managers about the update.
|<br>
Update the ticket.
|
|-
| 6
| Operations team
| Close the GGUS ticket after a week of the deployment only if the release was successful.
|
|}
|}


Line 175: Line 212:
| 2018-11-20
| 2018-11-20
| Existing CHM ticket should be mentioned when applicable; rephrase to use services and services components terms instead of tools.
| Existing CHM ticket should be mentioned when applicable; rephrase to use services and services components terms instead of tools.
|-
|
| Baptiste Grenier
| 2018-11-29
| Introduce specific steps for emergency releases. Slightly re-organise steps for standard releases.
|-
|
| Baptiste Grenier
| 2019-09-18
| Various clarifications/simplification using formal terms. Update procedure statement. Drop old instructions for providers that have been replaced by CHM documentation.
|}
|}


[[Category:Operations_Procedures]]
[[Category:Operations_Procedures]]

Revision as of 10:56, 22 January 2020

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Documentation menu: Home Manuals Procedures Training Other Contact For: VO managers Administrators



Title Services and Service components release and deployment process
Document link https://wiki.egi.eu/wiki/PROC23
Last modified 2019-09-18
Policy Group Acronym OMB
Policy Group Name Operations Management Board
Contact Group operations at egi.eu
Document Status FINAL
Approved Date 2018-11-29
Procedure Statement The procedure provides steps for service updates and releases of the centrally-provided production services.
Owner Baptiste Grenier


Overview

The procedure describes the release and deployment of the centrally-provided IT service and service components part of the EGI Service Catalogue.

Definitions

Please refer to the EGI Glossary for the definitions of the terms used in this procedure.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", “MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Entities involved in the procedure

  • Service Provider team: Team responsible for development, release and deployment of the service or service component
  • Operations team: Oversees all the process and may provide further people for testing the service or service component
  • Noc-Managers: Is informed regarding the release of the new service or service component and may provide further people for testing it

Requirements

The prerequisites are:

  • Centrally-provided Service or service component part of the EGI production Infrastructure;
  • Development team should refer to the EGI Change management documentation.

Acceptance criteria

A new release of a Service or service component can be deployed in production if:

  • All the prerequisites are satisfied
  • The results of the testing in step 3b do not show any major problem:
    • The Service or service component is usable with all its features
    • The interoperability with the other Service or service component is not broken

Steps for the release of a service or service component update

Responsible Action Notes
1 Service Provider team

Once release is ready the team opens a GGUS ticket (selecting category: Release) to Operations SU with the following information:

  • Name of the service or service component;
  • Date of release;
  • Release notes;
  • Suggested deployment date;
  • Testing instance url and testing instructions;
  • Names of testers if testing is manual (if not defined Development team may ask to appoint testers);
  • If the release is originating from one or multiple Change Records tracked in RT tickets as part of Change Management (CHM), then add the links to the respective RT tickets.
2 Operations Team
  • Assess if ticket came from a change record tracked in RT:
    • if yes check that the GGUS ticket contains links to the changes (RT tickets) that are implemented in this release;
    • If not validate that the change was allowed to by-pass Change Record tracking.
  • Inform the Noc-Managers about the upcoming release, asking if there is anyone else interested in performing the test;
  • can add further people for performing the tests;
  • The suggested duration of the test phase is two weeks;
  • Update the ticket.
3a Operations Team / Noc-Managers Update the ticket with the information on the performed tests and their result.
3b Service Provider team Update the ticket with information about results of the overall testing phase.
4 Service Provider team Provide in the ticket the link to updated documentation.
5 Service Provider team and Operations team Agree on an appropriate deployment date and update the ticket.
6 Operations team 10 days before the upcoming deployment, inform the Noc-Managers.

Update the ticket.

7 Service Provider team Schedule a downtime of the service in case it is needed.
8 Service Provider team Deploy release and update the ticket.
9 Operations team Close the GGUS ticket after a week of the deployment only if the release was successful.

Emergency release

Definition of emergency release

An emergency release is the process of releasing one Service or service component, or a set of Services or service components, with the targeted goal of solving a specific problem affecting the EGI infrastructure. To qualify the problem for an emergency release the problem should be classified in at least one of the following categories:

  • Major incident;
  • Critical security vulnerability.

Steps for emergency release

Responsible Action Notes
1 Service Provider team

Once the release is ready the team opens a GGUS ticket (Selecting category: Release and priority: Top priority) to Operations SU with the following information:

  • Name of the Service or service component;
  • Date of the release;
  • Release notes;
  • Justification of the need for the emergency release;
  • If the release is originating from one or multiple Change Records tracked in RT tickets as part of Change Management (CHM), then add the links to the respective RT tickets.
2 Operations Team
  • Acknowledge the need for the emergency release;
  • Update the ticket.
3 Service Provider team Record an unplanned downtime of the service in case it is needed.
4 Service Provider team Deploy the release and update the ticket.
5 Operations team Inform the Noc-Managers about the update.

Update the ticket.

6 Operations team Close the GGUS ticket after a week of the deployment only if the release was successful.

Revision History

Version Authors Date Comments
Alessandro Paolini 2016-03-22 procedure revised and made some changes
Alessandro Paolini 2016-07-25 added the new AAI in the tools list
Baptiste Grenier 2018-10-01 Introduced usage of new ticket category "release"
Alessandro Paolini 2018-10-11 Updated the list of services under scope; some typos corrections.
Baptiste Grenier 2018-11-20 Existing CHM ticket should be mentioned when applicable; rephrase to use services and services components terms instead of tools.
Baptiste Grenier 2018-11-29 Introduce specific steps for emergency releases. Slightly re-organise steps for standard releases.
Baptiste Grenier 2019-09-18 Various clarifications/simplification using formal terms. Update procedure statement. Drop old instructions for providers that have been replaced by CHM documentation.