Difference between revisions of "Resource Allocation Procedure and Work Instruction"

From EGIWiki
Jump to: navigation, search
 
(39 intermediate revisions by 3 users not shown)
Line 1: Line 1:
This is draft of procedure and related work instructions for User, Provider and Broker  
+
{{Op_menubar}} {{EGI_RA_menubar}}
 +
{{TOC_right}}
 +
This is draft of procedure and related work instructions for User, Provider and Broker<br>
  
<br>
+
= '''Terms'''  =
  
= Terms  =
+
[[https://wiki.egi.eu/wiki/Resource_Allocation_Terminology Terminology]]
  
Request – Service Level Agreement (SLA)
+
In RA process there are three parties involved:
  
OLA – Operations Level Agreement – between EGI.eu and Resource Provieder
+
*Broker (B) - matchmaker and coordinator of RA process
 +
*Resource Provider (P) - Site Manager or NGI Manager responsible for resource allocation on sites in their scope
 +
*Customer (C) - scientist involved in a science project requesting storage or computing resources in EGI, person signing SLA with Resource Provider
  
SLA Sections – Service Level Agreement Sections related to OLA with given Resource Provider
+
= '''Introduction''' =
 
 
Customer - person signing SLA with Resource Provider
 
 
 
User - person using resources on base of binding SLA (could be Customer itself)
 
 
 
= Introduction  =
 
  
 
The effective usage of IT&nbsp;resources requires from Customers and Resource Provider reaching agreement on how much resources, with some specific requirements will be available to User at specific time period.  
 
The effective usage of IT&nbsp;resources requires from Customers and Resource Provider reaching agreement on how much resources, with some specific requirements will be available to User at specific time period.  
Line 21: Line 19:
 
In heterogeneous IT environment it is difficult for Customers to find matching resources and for Resource Providers to manage their IT&nbsp;capacity.  
 
In heterogeneous IT environment it is difficult for Customers to find matching resources and for Resource Providers to manage their IT&nbsp;capacity.  
  
The Resource Allocation process, if possible,&nbsp; facilitates reaching agreement between Customers and Resource Providers.  
+
The Resource Allocation process facilitates reaching agreement between Customers and Resource Providers. <br>
 +
 
 +
<br>
  
 
Starting point are:  
 
Starting point are:  
Line 27: Line 27:
 
*set of Resource Pools defined by Resource Providers describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to deliver to Users  
 
*set of Resource Pools defined by Resource Providers describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to deliver to Users  
 
*set of Customers expectations (Request) describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to use
 
*set of Customers expectations (Request) describing in common [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics] resources they are willing to use
 +
 +
<br>
  
 
As a result of the process, if the Request was agreed by all parties, there is binding SLA signed, that is basis for resource configuration, allocation usage, support and allocation accounting.  
 
As a result of the process, if the Request was agreed by all parties, there is binding SLA signed, that is basis for resource configuration, allocation usage, support and allocation accounting.  
  
In case there were not matched Requests, the output of the process is report to EGI.eu allowing to manage Pool Capacity (add new resources).
+
In case there were not matched Requests, the output of the process is report to EGI.eu allowing to manage Pool Capacity (add new resources).  
  
Creation of Resource Pool [https://wiki.egi.eu/wiki/Pool_Allocation_For_Providers instruction] (for Resource Providers) <br>
+
The role of Broker is to handle Customer requests, match request to pool and negotiate SLA (with Customers) and OLA&nbsp;(with Providers). Broker also informs EGI.eu operations team about shortage of&nbsp; resources.
  
Creation of Request [[Resource Allocation Request Handling#Request_creation|instruction]] (for Customers)
+
<br>
  
= Request Handling Procedure (draft)<br> =
+
= '''Resource Allocation Process activities:''' =
  
== User authentication  ==
+
#Pool creation - P&nbsp;: Creation of Resource Pool [https://wiki.egi.eu/wiki/Pool_Allocation_For_Providers instruction] (''for Resource Providers'')
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">User Request creation - C </span>: Creation of Request [[Resource Allocation Request Handling#Request_creation|instruction]] (''for Customers'')
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">User Request validation - C and B</span>
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA creation based on pool matching - B</span>
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA (re)negotiation - P and B</span>
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">OLA confirmation/rejection &nbsp;- P</span>
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">SLA creation/updating &nbsp;- B </span>
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;">SLA negotiation step - B and C</span>
 +
#<span style="font-size:15px;font-family:Arial;color:#000000;background-color:transparent;font-weight:normal;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;"> SLA confirmation/signing - C
 +
</span>
  
Broker checks that users has valid x.509 certificate supported by EGI.eu. (e.g. EGI EUGridPMA organization). If not, Broker suggest User to obtain certificate.
 
  
<span lang="en-US"> Broker checks that
 
</span><span lang="en-US">User </span><span lang="en-US">is</span><span lang="en-US">
 
member of VO. </span><span lang="en-US">If not, Broker points U</span><span lang="en-US">ser
 
</span><span lang="en-US">to documentation on</span><span lang="en-US">
 
creat</span><span lang="en-US">ion of ne</span><span lang="en-US">w
 
</span><span lang="en-US">vo:</span><span lang="en-US">
 
</span>[https://wiki.egi.eu/wiki/PROC14 https://wiki.egi.eu/wiki/PROC14]
 
  
 
<br>  
 
<br>  
 
== Request validation (non blocking 3 days, executed in parallel)  ==
 
 
Broker notifies Gergely Sipos (and mailing list). <span style="background: #ffff00">When
 
Gergely veto is taken into account later in the procedure? At any
 
step?</span>
 
  
 
<br>  
 
<br>  
  
== Request vs resource pool matching (executed in parallel==
+
= '''Resource Allocation Procedure (draft)'''<br> =
  
Broker matches resources. Broker choose pool according to [https://wiki.egi.eu/wiki/Resource_Pool https://wiki.egi.eu/wiki/Resource_Pool] scenario (Free hands, Right to revoke, Negotiated)  
+
== '''Request validation (non blocking 3 days, executed in parallel)'''  ==
  
<br>
+
Broker checks that Customer has valid x.509 certificate supported by EGI.eu. (e.g. EGI EUGridPMA organization). If not, Broker suggest Customer to obtain certificate.
  
== Request tuning / fitting activities (matched or „around” matched) ==
+
<span lang="en-US"> Broker checks that
 +
</span><span lang="en-US">Customer&nbsp; </span><span lang="en-US">is</span><span lang="en-US">
 +
member of VO. </span><span lang="en-US">If not, Broker points Customer</span><span lang="en-US">
 +
</span><span lang="en-US">to documentation on</span><span lang="en-US">
 +
creat</span><span lang="en-US">ion of ne</span><span lang="en-US">w Virtual Organization (VO)</span><span lang="en-US">:</span><span lang="en-US">
 +
</span>[https://wiki.egi.eu/wiki/PROC14 https://wiki.egi.eu/wiki/PROC14]
  
=== OLA creation – description of Activity 1 https://documents.egi.eu/document/2030  ===
+
Broker notifies EGI.eu (support@egi.eu Gergely Sipos). Veto to request(with explanations) should be e-mailed to resource-allocation-support at mailman.egi.eu as soon as possible, no more than 3 days. This description will be sent to Customer.
  
=== OLA confirmation/rejection – description of Activity 2 https://documents.egi.eu/document/2030  ===
+
<br>
  
=== OLA renegotiation – description of Activity 3 https://documents.egi.eu/document/2030 ===
+
== '''Request vs resource pool matching (executed in parallel)''' ==
  
=== SLA Section creation – description of Activity 4 https://documents.egi.eu/document/2030  ===
+
Broker matches resources. Broker choose pool according to [https://wiki.egi.eu/wiki/Resource_Pool https://wiki.egi.eu/wiki/Resource_Pool] scenario (Free hands, Right to revoke, Negotiated)
  
=== SLA Section negotiation step – description of Activity 5 https://documents.egi.eu/document/2030<br> ===
+
<br>  
 
 
== Binding OLA configured on site (VO on site, resource allocation)  ==
 
  
Resource Provider configures resources according to the binding Request.
+
== '''SLA/OLA negotiations '''<br>  ==
  
<span style="background: #ffff00">Broker
+
Broker checks wheter there was EGI.eu veto announced. If yes, Request is Rejected and Customer notified about reason.  
checks that resources are configured and available to User? and notifies User, that allocation is ready.</span>
 
  
<br>  
+
If not, Broker sends to Customer SLA proposal.<br>  
  
== Offer exploitation  ==
+
Customer and Broker negotiate values of the metrics. If needed Broker and Provider negotiate related OLA.<br>
  
User exploits the allocation. In case of problems creates ticket in GGUS assigned to "Resource Allocation" Support Unit<span style="background: #ffff00" /><br>  
+
If agreement is reached, Customer signes SLA and the offer is binding for both parties.<br> <br>  
  
 +
== '''Binding OLA configured on site (VO on site, resource allocation)'''  ==
  
 
+
Resource Provider configures resources according to the binding Request.  
== Offer review  ==
 
 
 
<span style="background: #ffff00"> User
 
evaluates the cooperation with EGI.eu</span>
 
  
 
<br>  
 
<br>  
  
= Work Instructions (draft)  =
+
= '''Work Instructions (draft)''' =
  
== 1. User Side  ==
+
== '''1. User Side''' ==
  
=== [[Request creation]] ===
+
=== Request creation  ===
  
 
#Enter e-GRANT, login,  
 
#Enter e-GRANT, login,  
 
#Click “Request new allocation” button.  
 
#Click “Request new allocation” button.  
#Chose Start (at least 7 days ahead), end (no more than 1 year), user community, vo and enter description of activity field. Enter Computing time in HEPSPEC-hours, and Storage capacity (one of them may be zero).  
+
#Chose Start (at least 7 days ahead), end (no more than 1 year), user community, vo and enter description of activity field. Enter Computing time in HEPSPEC-hours, and Storage capacity (one of them may be zero). [https://wiki.egi.eu/wiki/Resource_Allocation_Request_Metrics_Description metrics]
#You may safe your request in draft state (is not beeing processed) and you can edit it later on.  
+
#You may save your request in draft state (is not beeing processed) and you can edit it later on.  
#Before sending request you may, by clicking on “Invalidate Request”.  
+
#Before sending request you may invalidate it, by clicking on “Invalidate Request”.  
 
#When satisfied with your Request, click “Submit to EGI” button. The mail to Broker is sent. EGI RA team should contact you within 3 days.
 
#When satisfied with your Request, click “Submit to EGI” button. The mail to Broker is sent. EGI RA team should contact you within 3 days.
  
 
<br>  
 
<br>  
  
== 2. Provider Side  ==
+
== '''2. Provider Side''' ==
  
 
=== New OLA handling  ===
 
=== New OLA handling  ===
Line 124: Line 120:
 
#RPmay reject, negotiate and accept proposal.  
 
#RPmay reject, negotiate and accept proposal.  
 
#When RP wants to negotiate, clicks on “Negotiate” button.  
 
#When RP wants to negotiate, clicks on “Negotiate” button.  
#RP enters new proposed values. Then RP can save a draft or Send proposal. The mail to the Broker is sent.
+
#RP enters new proposed values. Then RP can save a draft or Send proposal. The mail to the Broker is sent.<br>
 +
 
 +
<br>
  
== 3. Broker Side  ==
+
== '''3. Broker Side''' ==
  
 
=== New request handling  ===
 
=== New request handling  ===
Line 136: Line 134:
 
#Broker clicks “Find pools”. There are possible pools shown.  
 
#Broker clicks “Find pools”. There are possible pools shown.  
 
#Broker checks metric values and decides on usage of given pool by clicking on the select box and click “Save” button.  
 
#Broker checks metric values and decides on usage of given pool by clicking on the select box and click “Save” button.  
#Broker clicks on related underpinned OLA, and clicks Send “OLA”. The mail to Resource Provider is sent.
+
#Broker clicks on related underpinned OLA, and clicks Send “OLA”. The mail to Resource Provider is sent.<br>
  
 
=== Request negotiation (vs Provider)  ===
 
=== Request negotiation (vs Provider)  ===
Line 143: Line 141:
 
#In “Underpinned OLAs” clicks Version link  
 
#In “Underpinned OLAs” clicks Version link  
 
#Takes appropriate action.  
 
#Takes appropriate action.  
#If Broker agrees, then click on “Accept” and “Home” to get to the list of all Requests.
+
#If Broker agrees, then click on “Accept” and “Home” to get to the list of all Requests.<br>
 +
 
 +
<br>
 +
 
 +
= '''Contact'''  =
 +
 
 +
mail:&nbsp;'''resource-allocation-support at mailman.egi.eu'''
  
= Contact =
+
GGUS: "Resource Allocation"&nbsp;support unit (in preparation)
  
mail:&nbsp;resource-allocation-support at mailman.egi.eu
+
[[Resource Allocation|&lt;&lt; Go back to Resource Allocation Page]]
  
GGUS: "Resource Allocation"&nbsp;support unit (in preparation)
+
[[Category:ResourceAllocation]]

Latest revision as of 19:00, 25 November 2014

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


EGI Resource Allocation menu:


This is draft of procedure and related work instructions for User, Provider and Broker

Terms

[Terminology]

In RA process there are three parties involved:

  • Broker (B) - matchmaker and coordinator of RA process
  • Resource Provider (P) - Site Manager or NGI Manager responsible for resource allocation on sites in their scope
  • Customer (C) - scientist involved in a science project requesting storage or computing resources in EGI, person signing SLA with Resource Provider

Introduction

The effective usage of IT resources requires from Customers and Resource Provider reaching agreement on how much resources, with some specific requirements will be available to User at specific time period.

In heterogeneous IT environment it is difficult for Customers to find matching resources and for Resource Providers to manage their IT capacity.

The Resource Allocation process facilitates reaching agreement between Customers and Resource Providers.


Starting point are:

  • set of Resource Pools defined by Resource Providers describing in common metrics resources they are willing to deliver to Users
  • set of Customers expectations (Request) describing in common metrics resources they are willing to use


As a result of the process, if the Request was agreed by all parties, there is binding SLA signed, that is basis for resource configuration, allocation usage, support and allocation accounting.

In case there were not matched Requests, the output of the process is report to EGI.eu allowing to manage Pool Capacity (add new resources).

The role of Broker is to handle Customer requests, match request to pool and negotiate SLA (with Customers) and OLA (with Providers). Broker also informs EGI.eu operations team about shortage of  resources.


Resource Allocation Process activities:

  1. Pool creation - P : Creation of Resource Pool instruction (for Resource Providers)
  2. User Request creation - C : Creation of Request instruction (for Customers)
  3. User Request validation - C and B
  4. OLA creation based on pool matching - B
  5. OLA (re)negotiation - P and B
  6. OLA confirmation/rejection  - P
  7. SLA creation/updating  - B
  8. SLA negotiation step - B and C
  9. SLA confirmation/signing - C




Resource Allocation Procedure (draft)

Request validation (non blocking 3 days, executed in parallel)

Broker checks that Customer has valid x.509 certificate supported by EGI.eu. (e.g. EGI EUGridPMA organization). If not, Broker suggest Customer to obtain certificate.

Broker checks that Customer  is member of VO. If not, Broker points Customer to documentation on creation of new Virtual Organization (VO): https://wiki.egi.eu/wiki/PROC14

Broker notifies EGI.eu (support@egi.eu Gergely Sipos). Veto to request(with explanations) should be e-mailed to resource-allocation-support at mailman.egi.eu as soon as possible, no more than 3 days. This description will be sent to Customer.


Request vs resource pool matching (executed in parallel)

Broker matches resources. Broker choose pool according to https://wiki.egi.eu/wiki/Resource_Pool scenario (Free hands, Right to revoke, Negotiated)


SLA/OLA negotiations

Broker checks wheter there was EGI.eu veto announced. If yes, Request is Rejected and Customer notified about reason.

If not, Broker sends to Customer SLA proposal.

Customer and Broker negotiate values of the metrics. If needed Broker and Provider negotiate related OLA.

If agreement is reached, Customer signes SLA and the offer is binding for both parties.

Binding OLA configured on site (VO on site, resource allocation)

Resource Provider configures resources according to the binding Request.


Work Instructions (draft)

1. User Side

Request creation

  1. Enter e-GRANT, login,
  2. Click “Request new allocation” button.
  3. Chose Start (at least 7 days ahead), end (no more than 1 year), user community, vo and enter description of activity field. Enter Computing time in HEPSPEC-hours, and Storage capacity (one of them may be zero). metrics
  4. You may save your request in draft state (is not beeing processed) and you can edit it later on.
  5. Before sending request you may invalidate it, by clicking on “Invalidate Request”.
  6. When satisfied with your Request, click “Submit to EGI” button. The mail to Broker is sent. EGI RA team should contact you within 3 days.


2. Provider Side

New OLA handling

  1. RP check (should be notified by email), that new OLA appeared (negotiation scenario)
  2. RP logs in to e-GRANT tool with GOCBD Admin role.
  3. In“Requests being processed” panel clicks on SLA Name to be processed.
  4. RPmay reject, negotiate and accept proposal.
  5. When RP wants to negotiate, clicks on “Negotiate” button.
  6. RP enters new proposed values. Then RP can save a draft or Send proposal. The mail to the Broker is sent.


3. Broker Side

New request handling

  1. Broker checks (should be notified by email), that new request appeared.
  2. Broker logs in to e-GRANT tool with Broker role.
  3. In “Requests being processed” panel clicks on SLA Name to be processed.
  4. Broker may reject, find pools or negotiate request.
  5. Broker clicks “Find pools”. There are possible pools shown.
  6. Broker checks metric values and decides on usage of given pool by clicking on the select box and click “Save” button.
  7. Broker clicks on related underpinned OLA, and clicks Send “OLA”. The mail to Resource Provider is sent.

Request negotiation (vs Provider)

  1. In “Requests being processed” panel clicks on SLA Name to be processed.
  2. In “Underpinned OLAs” clicks Version link
  3. Takes appropriate action.
  4. If Broker agrees, then click on “Accept” and “Home” to get to the list of all Requests.


Contact

mail: resource-allocation-support at mailman.egi.eu

GGUS: "Resource Allocation" support unit (in preparation)

<< Go back to Resource Allocation Page