Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Resource Allocation Procedure and Work Instruction"

From EGIWiki
Jump to navigation Jump to search
 
(33 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This is draft of procedure and related work instructions for User, Provider and Broker  
{{Op_menubar}} {{EGI_RA_menubar}}
{{TOC_right}}
This is draft of procedure and related work instructions for User, Provider and Broker<br>


<br>
= '''Terms''' =
 
= Terms  =
 
Request – Service Level Agreement (SLA)
 
OLA – Operations Level Agreement – between EGI.eu and Resource Provieder


SLA Sections – Service Level Agreement Sections related to OLA with given Resource Provider
[[https://wiki.egi.eu/wiki/Resource_Allocation_Terminology Terminology]]


Customer - person signing SLA with Resource Provider
In RA process there are three parties involved:


User - person using resources on base of binding SLA (could be Customer itself)  
*Broker (B) - matchmaker and coordinator of RA process
*Resource Provider (P) - Site Manager or NGI Manager responsible for resource allocation on sites in their scope
*Customer (C) - scientist involved in a science project requesting storage or computing resources in EGI, person signing SLA with Resource Provider


= Introduction  =
= '''Introduction''' =


The effective usage of IT&nbsp;resources requires from Customers and Resource Provider reaching agreement on how much resources, with some specific requirements will be available to User at specific time period.  
The effective usage of IT&nbsp;resources requires from Customers and Resource Provider reaching agreement on how much resources, with some specific requirements will be available to User at specific time period.  
Line 21: Line 19:
In heterogeneous IT environment it is difficult for Customers to find matching resources and for Resource Providers to manage their IT&nbsp;capacity.  
In heterogeneous IT environment it is difficult for Customers to find matching resources and for Resource Providers to manage their IT&nbsp;capacity.  


The Resource Allocation process, if possible,&nbsp; facilitates reaching agreement between Customers and Resource Providers.  
The Resource Allocation process facilitates reaching agreement between Customers and Resource Providers. <br>
 
<br>


Starting point are:  
Starting point are:  
Line 27: Line 27:
*set of Resource Pools defined by Resource Providers describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to deliver to Users  
*set of Resource Pools defined by Resource Providers describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to deliver to Users  
*set of Customers expectations (Request) describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to use
*set of Customers expectations (Request) describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to use
<br>


As a result of the process, if the Request was agreed by all parties, there is binding SLA signed, that is basis for resource configuration, allocation usage, support and allocation accounting.  
As a result of the process, if the Request was agreed by all parties, there is binding SLA signed, that is basis for resource configuration, allocation usage, support and allocation accounting.  
Line 32: Line 34:
In case there were not matched Requests, the output of the process is report to EGI.eu allowing to manage Pool Capacity (add new resources).  
In case there were not matched Requests, the output of the process is report to EGI.eu allowing to manage Pool Capacity (add new resources).  


The role of Broker is to handle Customer requests, match request to pool and negotiate SLA (with Customers) and OLA&nbsp;(with Providers). Broker also informs EGI.eu operations team about shortage of&nbsp; resources.


<br>


= Resource Allocation activities: =
= '''Resource Allocation Process activities:'''  =


#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;" />Pool creation - P : Creation of Resource Pool [https://wiki.egi.eu/wiki/Pool_Allocation_For_Providers instruction] (for Resource Providers)
#Pool creation - P&nbsp;: Creation of Resource Pool [https://wiki.egi.eu/wiki/Pool_Allocation_For_Providers instruction] (''for Resource Providers'')  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">User Request creation - C </span>: Creation of Request [[Resource Allocation Request Handling#Request_creation|instruction]] (for Customers)
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">User Request creation - C </span>: Creation of Request [[Resource Allocation Request Handling#Request_creation|instruction]] (''for Customers'')  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">User Request validation - C and B</span>
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">User Request validation - C and B</span>  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA creation based on pool matching - B</span>
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA creation based on pool matching - B</span>  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA (re)negotiation - P and B</span>
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA (re)negotiation - P and B</span>  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA confirmation/rejection &nbsp;- P</span>
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA confirmation/rejection &nbsp;- P</span>  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">SLA creation/updating &nbsp;- B </span>
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">SLA creation/updating &nbsp;- B </span>  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">SLA negotiation step - B and C</span>
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">SLA negotiation step - B and C</span>  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"> SLA confirmation/signing - C  
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"> SLA confirmation/signing - C  
</span>
</span>


= Resource Allocation Procedure (draft)<br>  =


== Request validation (non blocking 3 days, executed in parallel) ==
 
<br>
 
<br>
 
= '''Resource Allocation Procedure (draft)'''<br>  =
 
== '''Request validation (non blocking 3 days, executed in parallel)'''  ==


Broker checks that Customer has valid x.509 certificate supported by EGI.eu. (e.g. EGI EUGridPMA organization). If not, Broker suggest Customer to obtain certificate.  
Broker checks that Customer has valid x.509 certificate supported by EGI.eu. (e.g. EGI EUGridPMA organization). If not, Broker suggest Customer to obtain certificate.  
Line 64: Line 74:
<br>  
<br>  


== Request vs resource pool matching (executed in parallel)  ==
== '''Request vs resource pool matching (executed in parallel)''' ==


Broker matches resources. Broker choose pool according to [https://wiki.egi.eu/wiki/Resource_Pool https://wiki.egi.eu/wiki/Resource_Pool] scenario (Free hands, Right to revoke, Negotiated)  
Broker matches resources. Broker choose pool according to [https://wiki.egi.eu/wiki/Resource_Pool https://wiki.egi.eu/wiki/Resource_Pool] scenario (Free hands, Right to revoke, Negotiated)  
Line 70: Line 80:
<br>  
<br>  


== SLA/OLA negotiations <br>  ==
== '''SLA/OLA negotiations '''<br>  ==


Broker checks wheter there was EGI.eu veto announced. If yes, Request is Rejected and Customer notified about reason.  
Broker checks wheter there was EGI.eu veto announced. If yes, Request is Rejected and Customer notified about reason.  


If not, Broker sends to Customer SLA proposal.  
If not, Broker sends to Customer SLA proposal.<br>
 
Customer and Broker negotiate values of the metrics.
 
If agreement is reached, Customer signes SLA and the offer is binding for both parties.
 
 
 
== Binding OLA configured on site (VO on site, resource allocation)  ==


Resource Provider configures resources according to the binding Request.&lt;span style="background: #ffff00" /&gt;<br>  
Customer and Broker negotiate values of the metrics. If needed Broker and Provider negotiate related OLA.<br>  


<br>  
If agreement is reached, Customer signes SLA and the offer is binding for both parties.<br> <br>  


== Offer exploitation ==
== '''Binding OLA configured on site (VO on site, resource allocation)''' ==


User exploits the allocation. In case of problems creates ticket in GGUS assigned to "Resource Allocation" Support Unit. Then ticket may be assigned to related Resource Provider support unit.  
Resource Provider configures resources according to the binding Request.  


<br>  
<br>  


== Offer review ==
= '''Work Instructions (draft)''' =
 
Customer evaluates the cooperation with EGI.eu
 
<br>


= Work Instructions (draft)  =
== '''1. User Side''' ==
 
== 1. User Side  ==


=== Request creation  ===
=== Request creation  ===
Line 109: Line 105:
#Click “Request new allocation” button.  
#Click “Request new allocation” button.  
#Chose Start (at least 7 days ahead), end (no more than 1 year), user community, vo and enter description of activity field. Enter Computing time in HEPSPEC-hours, and Storage capacity (one of them may be zero). [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics]  
#Chose Start (at least 7 days ahead), end (no more than 1 year), user community, vo and enter description of activity field. Enter Computing time in HEPSPEC-hours, and Storage capacity (one of them may be zero). [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics]  
#You may safe your request in draft state (is not beeing processed) and you can edit it later on.  
#You may save your request in draft state (is not beeing processed) and you can edit it later on.  
#Before sending request you may, by clicking on “Invalidate Request”.  
#Before sending request you may invalidate it, by clicking on “Invalidate Request”.  
#When satisfied with your Request, click “Submit to EGI” button. The mail to Broker is sent. EGI RA team should contact you within 3 days.
#When satisfied with your Request, click “Submit to EGI” button. The mail to Broker is sent. EGI RA team should contact you within 3 days.


<br>  
<br>  


== 2. Provider Side  ==
== '''2. Provider Side''' ==


=== New OLA handling  ===
=== New OLA handling  ===
Line 124: Line 120:
#RPmay reject, negotiate and accept proposal.  
#RPmay reject, negotiate and accept proposal.  
#When RP wants to negotiate, clicks on “Negotiate” button.  
#When RP wants to negotiate, clicks on “Negotiate” button.  
#RP enters new proposed values. Then RP can save a draft or Send proposal. The mail to the Broker is sent.
#RP enters new proposed values. Then RP can save a draft or Send proposal. The mail to the Broker is sent.<br>
 
<br>


== 3. Broker Side  ==
== '''3. Broker Side''' ==


=== New request handling  ===
=== New request handling  ===
Line 136: Line 134:
#Broker clicks “Find pools”. There are possible pools shown.  
#Broker clicks “Find pools”. There are possible pools shown.  
#Broker checks metric values and decides on usage of given pool by clicking on the select box and click “Save” button.  
#Broker checks metric values and decides on usage of given pool by clicking on the select box and click “Save” button.  
#Broker clicks on related underpinned OLA, and clicks Send “OLA”. The mail to Resource Provider is sent.
#Broker clicks on related underpinned OLA, and clicks Send “OLA”. The mail to Resource Provider is sent.<br>


=== Request negotiation (vs Provider)  ===
=== Request negotiation (vs Provider)  ===
Line 143: Line 141:
#In “Underpinned OLAs” clicks Version link  
#In “Underpinned OLAs” clicks Version link  
#Takes appropriate action.  
#Takes appropriate action.  
#If Broker agrees, then click on “Accept” and “Home” to get to the list of all Requests.
#If Broker agrees, then click on “Accept” and “Home” to get to the list of all Requests.<br>
 
<br>
 
= '''Contact'''  =
 
mail:&nbsp;'''resource-allocation-support at mailman.egi.eu'''


= Contact  =
GGUS: "Resource Allocation"&nbsp;support unit (in preparation)


mail:&nbsp;resource-allocation-support at mailman.egi.eu
[[Resource Allocation|&lt;&lt; Go back to Resource Allocation Page]]


GGUS: "Resource Allocation"&nbsp;support unit (in preparation)
[[Category:ResourceAllocation]]

Latest revision as of 18:00, 25 November 2014

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


EGI Resource Allocation menu:


This is draft of procedure and related work instructions for User, Provider and Broker

Terms

[Terminology]

In RA process there are three parties involved:

  • Broker (B) - matchmaker and coordinator of RA process
  • Resource Provider (P) - Site Manager or NGI Manager responsible for resource allocation on sites in their scope
  • Customer (C) - scientist involved in a science project requesting storage or computing resources in EGI, person signing SLA with Resource Provider

Introduction

The effective usage of IT resources requires from Customers and Resource Provider reaching agreement on how much resources, with some specific requirements will be available to User at specific time period.

In heterogeneous IT environment it is difficult for Customers to find matching resources and for Resource Providers to manage their IT capacity.

The Resource Allocation process facilitates reaching agreement between Customers and Resource Providers.


Starting point are:

  • set of Resource Pools defined by Resource Providers describing in common metrics resources they are willing to deliver to Users
  • set of Customers expectations (Request) describing in common metrics resources they are willing to use


As a result of the process, if the Request was agreed by all parties, there is binding SLA signed, that is basis for resource configuration, allocation usage, support and allocation accounting.

In case there were not matched Requests, the output of the process is report to EGI.eu allowing to manage Pool Capacity (add new resources).

The role of Broker is to handle Customer requests, match request to pool and negotiate SLA (with Customers) and OLA (with Providers). Broker also informs EGI.eu operations team about shortage of  resources.


Resource Allocation Process activities:

  1. Pool creation - P : Creation of Resource Pool instruction (for Resource Providers)
  2. User Request creation - C : Creation of Request instruction (for Customers)
  3. User Request validation - C and B
  4. OLA creation based on pool matching - B
  5. OLA (re)negotiation - P and B
  6. OLA confirmation/rejection  - P
  7. SLA creation/updating  - B
  8. SLA negotiation step - B and C
  9. SLA confirmation/signing - C




Resource Allocation Procedure (draft)

Request validation (non blocking 3 days, executed in parallel)

Broker checks that Customer has valid x.509 certificate supported by EGI.eu. (e.g. EGI EUGridPMA organization). If not, Broker suggest Customer to obtain certificate.

Broker checks that Customer  is member of VO. If not, Broker points Customer to documentation on creation of new Virtual Organization (VO): https://wiki.egi.eu/wiki/PROC14

Broker notifies EGI.eu (support@egi.eu Gergely Sipos). Veto to request(with explanations) should be e-mailed to resource-allocation-support at mailman.egi.eu as soon as possible, no more than 3 days. This description will be sent to Customer.


Request vs resource pool matching (executed in parallel)

Broker matches resources. Broker choose pool according to https://wiki.egi.eu/wiki/Resource_Pool scenario (Free hands, Right to revoke, Negotiated)


SLA/OLA negotiations

Broker checks wheter there was EGI.eu veto announced. If yes, Request is Rejected and Customer notified about reason.

If not, Broker sends to Customer SLA proposal.

Customer and Broker negotiate values of the metrics. If needed Broker and Provider negotiate related OLA.

If agreement is reached, Customer signes SLA and the offer is binding for both parties.

Binding OLA configured on site (VO on site, resource allocation)

Resource Provider configures resources according to the binding Request.


Work Instructions (draft)

1. User Side

Request creation

  1. Enter e-GRANT, login,
  2. Click “Request new allocation” button.
  3. Chose Start (at least 7 days ahead), end (no more than 1 year), user community, vo and enter description of activity field. Enter Computing time in HEPSPEC-hours, and Storage capacity (one of them may be zero). metrics
  4. You may save your request in draft state (is not beeing processed) and you can edit it later on.
  5. Before sending request you may invalidate it, by clicking on “Invalidate Request”.
  6. When satisfied with your Request, click “Submit to EGI” button. The mail to Broker is sent. EGI RA team should contact you within 3 days.


2. Provider Side

New OLA handling

  1. RP check (should be notified by email), that new OLA appeared (negotiation scenario)
  2. RP logs in to e-GRANT tool with GOCBD Admin role.
  3. In“Requests being processed” panel clicks on SLA Name to be processed.
  4. RPmay reject, negotiate and accept proposal.
  5. When RP wants to negotiate, clicks on “Negotiate” button.
  6. RP enters new proposed values. Then RP can save a draft or Send proposal. The mail to the Broker is sent.


3. Broker Side

New request handling

  1. Broker checks (should be notified by email), that new request appeared.
  2. Broker logs in to e-GRANT tool with Broker role.
  3. In “Requests being processed” panel clicks on SLA Name to be processed.
  4. Broker may reject, find pools or negotiate request.
  5. Broker clicks “Find pools”. There are possible pools shown.
  6. Broker checks metric values and decides on usage of given pool by clicking on the select box and click “Save” button.
  7. Broker clicks on related underpinned OLA, and clicks Send “OLA”. The mail to Resource Provider is sent.

Request negotiation (vs Provider)

  1. In “Requests being processed” panel clicks on SLA Name to be processed.
  2. In “Underpinned OLAs” clicks Version link
  3. Takes appropriate action.
  4. If Broker agrees, then click on “Accept” and “Home” to get to the list of all Requests.


Contact

mail: resource-allocation-support at mailman.egi.eu

GGUS: "Resource Allocation" support unit (in preparation)

<< Go back to Resource Allocation Page