Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Staged Rollout process"

From EGIWiki
Jump to navigation Jump to search
Line 42: Line 42:
  protect=1
  protect=1


=== Staged Rollout Workflow ===
= Staged Rollout Workflow =


The staged rollout start when the '''"RolloutProgress'''" custom field of RT ticket in the queue '''sw-rel''' changes from "'''InVerification"''' to "'''StageRollOut"''', this means that the SW component as passed the verification process, i.e. was "'''ACCEPTED'''" in the verification phase.<br>  
The staged rollout start when the '''"RolloutProgress'''" custom field of RT ticket in the queue '''sw-rel''' changes from "'''InVerification"''' to "'''StageRollOut"''', this means that the SW component as passed the verification process, i.e. was "'''ACCEPTED'''" in the verification phase.<br>  

Revision as of 15:11, 15 September 2011

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security



New Software Release Workflow

The full description of the New Software Release Workflow (NSRW) includes all stages from software release by a Technology Provider to release for deployment in the EGI production infrastructure.

The remainder of this document describes the part of the workflow corresponding to the Staged Rollout phase.

Repository Configuration

This document assumes SL5 x86_64, for other architectures the actual repositories to use may change

Base, EPEL and EGI Trustanchors repositories

You should set up these repositories according to the description given in http://repository.egi.eu/

Product Repository

The URL for the staged rollout of each "EMI product" repository can be found at the " Custom Fields" of the RT ticket within attribute RepositoryURL, note that you should click "Expand Release Info" that contains the the URL to the repository and a pointer to the documentation. The easiest way of adding the repositoy is using the files available at the repofiles directory.

Example config file for CREAM release, has the repository located at:

http://repository.egi.eu/sw/stagerollout/emi.cream.sl5.x86_64/1

and the repofile is available at:

http://repository.egi.eu/sw/stagerollout/emi.cream.sl5.x86_64/1/repofiles/EMI.cream.sl5.x86_64.repo

Contents of this file is shown below:

# EGI Software Repository - REPO META (releaseId,repositoryId,repofileId) - (2206,239,166)

[EMI.cream.sl5.x86_64]
name=EMI.cream.sl5.x86_64
baseurl=http://repository.egi.eu/sw/stagerollout/emi.cream.sl5.x86_64/1/
gpgkey=http://emisoft.web.cern.ch/emisoft/dist/EMI/1/RPM-GPG-KEY-emi 
gpgcheck=1  
enabled=1
protect=1

Staged Rollout Workflow

The staged rollout start when the "RolloutProgress" custom field of RT ticket in the queue sw-rel changes from "InVerification" to "StageRollOut", this means that the SW component as passed the verification process, i.e. was "ACCEPTED" in the verification phase.

It is assumed that the staged rollout repository is fixed for any given major release of any SW, this is publicly available in the repository and EGI wiki for Early Adopters. This state change triggers the following actions:

  1. The SW packages are moved from the InVerification repository to the StageRollOut repository.
  2. The sw-rel ticket creates a new child ticket in the staged-rollout queue. All the Staged Rollout process occurs in this queue/ticket.
    1. Advisable to change to the RT ticket tab "Jumbo".
    2. The "Owner" of the "staged-rollout" ticket is "(Nobody)".
    3. The "Status" of the ticket is "New".
    4. Change the ownership of the ticket to the "Staged Rollout Manager" responsible for that MW component: ARC, gLite, UNICORE, Globus, Operational Tools.
  3. (Staged Rollout Manager) The staged rollout manager is notified and takes the actions below. Advisable to change to the RT ticket tab "Jumbo".
    1. The staged-rollout ticket should contain the links to:
      1. Release notes, contains information about install/upgrade and configuration.
      2. Bugs or issues fixed.
      3. Documentation
      4. Staged rollout repository (needed by the EA teams).
    2. The status of the ticket is "Open".
  4. (Staged Rollout Manager) The queue staged-rollout has a custom field (dropped down) where the staged rollout managers select all the EA teams to assign the staged rollout test.
  5. (Staged Rollout Manager) When the "Save changes" button is pressed an automatic notification mail is sent to all EA . The mail notification should contain the links to release notes, installation/upgrade information, configuration, documentation and the repository URL. All EA teams are added to the "Administrative Cc" field.
  6. (EA teams) Each EA team has to acknowledge the reception of the notification within 1 working day. Either by replying to the mail sent by the RT or directly in the ticket with: <accept|reject> <NGI>-<Site-name>
  7. (Staged Rollout Manager) The staged rollout manager will check the ticket and if there are no EAs accepting the staged rollout test, it will pool the early-adopters-XXX.mailman.egi.eu mailing lists, and other (s)he see’s fit to get other EA sites.
  8. (EA teams) (NOTE this point needs more clarification) The EA team has the option to put the service node into downtime in the GOCDB, special beta tag. This tag may be set only during the staged rollout phase, if/when the component is release into production, this tag (or downtime), should be removed from the gocdb.
  9. (EA teams) The EA teams do the staged rollout: install/upgrade, configure, and some tests as they see fit.
  10. (EA teams) If the EA finds problems or issues, either they are clarified within the ticket by the staged rollout managers and other EA teams, OR , a GGUS ticket should be opened. If a GGUS ticket is opened, this ticket will be assigned to the DMSU support unit, which will then route it to the technology providers as they see fit.
    1. (EA teams) When opening a ggus ticket, you should let DMSU decide on the priority, but should describe the criticality of the issue in the ticket body. If it's a show stopper or not, or if there are possible workarounds, etc.
    2. (EA teams) A link to the GGUS ticket should be created in the staged rollout ticket.
  11. (EA teams) The service should ideally be exposed to production load/environment and users. This period lasts between 5 to 7 days, but may be extended depending on the cases or components under test.
  12. Each EA team should fill the staged rollouttemplates after the last point, and send it to the Staged Rollout Manage'r (see table below):
    1. (EA teams) The report should contain as much information as possible, specially the correctness of the release notes, test that have been preformed, and possible metrics when the service is exposed to production (like number of jobs per day, or number of transfers, what VOs are configured for that service, etc.)
    2. (EA teams) The name of the file should be:
      1. ea-<NGI>-<Site-name>-<MW stack>-<component>-<version>.EXT (doc, docx, odt)
  13. (Staged Rollout Manager) The staged rollout managers:
    1. Collect all reports and produce a summary report.
    2. Create a document in the EGI Doc server with a given ID, which will contain all reports for the staged rollout of that component, as well as the summary report.
    3. Insert a link in the RT ticket to the Dec server ID of the reports.
    4. Select from the custom field"Outcome" the outcome < ACCEPT | REJECT >.
    5. Set the ticket status to resolved.
  14. (Staged Rollout Manager) When the staged rollout child ticket is set to resolved the parent ticket is notified, and it gets the outcome and the doc server ID of the reports.

Resources

  • The Staged Rollout process is supported by the RT ticketing system (see tickets in the staged-rollout queue).
  • Report templates: each Early Adopter has to fill in a report (see the template on DocumentDB.

Naming conventions

  • The EA team names are SSO groups containing the members of each team:
    • ea-< NGI >-< Site-name >
  • The custom fields provided in the staged-rollout queue are:
    • Drops down box containing all EA teams, on the EGI SSO: possibility to select several teams, the button Save Changes notifies those teams and they will be added to the AdminCC field.
    • Outcome of the staged rollout, drop down box with: < ACCEPT | REJECT >.
  • The title of the ticket is of the form:
    • “Staged Rollout < SW stack-MajorVersion > < COMPONENT > < VERSION > < OS > < ARCH >”
    • Examples:
      • Staged Rollout CA 1.38
      • Staged Rollout EMI-1.0 SE-DPM_mysql 1.8.0-1 SL5 x86_64
SW Stack EA SSO group Staged Rollout Managers
gLite (+dCache) https://www.egi.eu/sso/groupView/early-adopters-glite Mario David, Michaela Barth
ARC https://www.egi.eu/sso/groupView/early-adopters-arc Christian Ulrik Søttrup, Sergio Maffioletti
UNICORE https://www.egi.eu/sso/groupView/early-adopters-unicore Mathilde Romberg
Globus https://www.egi.eu/sso/groupView/early-adopters-globus
Operational Tools https://www.egi.eu/sso/groupView/early-adopters-opstools Daniele Cesini



  • The name of the file should be:
    • Filename: ea-< NGI >-< Site-name >-< MW stack >-< component >-< version >.doc(odt)
  • The staged rollout manager will provide a summary and the name of the file is:
    • Filename of summary: summary-< MW stack >-< component >-< version >.doc(odt)

All reports and the summary will be in: https://documents.egi.eu with a given ID, that will be referenced in the respective rt ticket.

  • The description in the document database should have the following naming convention:
    • Staged rollout < MW stack > < component > < version >

Back Staged-Rollout