Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Resource Allocation Process

From EGIWiki
Revision as of 14:36, 12 March 2013 by Krakow (talk | contribs)
Jump to navigation Jump to search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


<< Resource Allocation Task Force


This page is created by Resource Allocation Task Force. It contains proposal for RA process.

Introduction

Definitions

Pool - broad concept of defining/declaring for EGI Resource Allocation.

Pools of resources repository -

Scientific review -


Process

Resource allocation process

Step 1 Pools definition preparation

  • NGIs/Sites define pools and register them in EGI " Pools of resources repository".
  • Pools definition may be subject of change according to specific Resource alloction policy.
  • NGIs/Sites are free to modify this repository.

Pool concept

Definition: Let a pool be a specific capacity of resources available for EGI allocation according to defined process under defined conditions

Note: Pools covers any scenarios mentioned ever in RATF including “free hand”, “right to revoke”, “negotiation (elastic pool)”

The pool are defined in the Site/NGI declaration:

  • previously referred as pre-agreement, but it should not require EGI to agree/negotiate
  • can be subject of change in time (some limitation apply)

Pools is just a broad concept of defining/declaring for EGI Resource Allocation.

Pool concept: specification

  1. Part 1: Pool Offer specification
    • Description of what is offered (capacity)
    • Validity in time (when pool is available)
    • Who is responsible for allocation process (NGI / Site)  
    • Process definition (scenario) - selection of predefined processes
    • QoS (opportunistic | assigned share | reserved ) - how resources are guaranteed for a specific VO
    • Resource specification (technical details)
  2. Part 2: Acceptance criteria for requests
    • Scope for VO (international | national, membership for NGI, discipline)
    • Scientific review (not-required | at least (fair | good | excellent))
    • Publications: required/not required
      • Type of acknowledgements: EGI/NGI/Site
    • Request size, incl. % of allocation eg.:
      • resources should not be allocated more than 10% of the pool per VO
      • resources should not cover more than 50% of the VO request

Pool concept: example

Pool-1 SiteA will offer:

  • up to 100 cores in 2013 as reserved pool (as a whole -> reservation applies to the whole pool not given usage), opportunistic inside pool
  • “free hand” to EGI - EGI has free choice to allocate resources 
  • each VO should not be allocated more than 10% of the pool
  • can be suitable for small resource requests

Pool-2 SiteB will offer:

  • up to 4000 cores between Jun’13 – Feb’14 with assigned fair-share
  • allocation based on negotiations
  • for requests that proved to be:
    • at least good in EGI scientific review
    • has members in NGI X
    • each requests can be supported up to 50%
  • can be suitable for big resource requests

Pool concept: Process definition (scenarios)

Scenario

Type of pool

(temporary/permanent)

Scope
EGI role
NGIs role
Review
Usage (guaranteed allocation/opportunistic)
Actions/steps
in free hand
permanent central collector for requests; set up and coordinate the usage of the permanent pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers
no review as long as the pool's resources are enough to satisfy all requests. If a review is needed it is up to EGI.eu only to decide



right to revoke
permanent central collector for requests; set up and coordinate the usage of the permanent pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers + option to withdraw if national/local policies are conflicting
same as above, but  if a review is needed a very lightweight procedure should be defined, involving EGI.eu and representatives of the RPs contributing to the pool



negotiated (elastic pools)
temporary (on demand) on a per request basis
central collector for requests; broker for the RPs resources; define all the technical aspects of the requests; open calls to RPs in order to set up the temporary pool  through the NGIs
broker for the RCs resources; provide support to RCs in order to ease the creation of a temporary pool; monitor the effectiveness of the pool resources usage by the requester
not at all; it is up to the different RPs (NGIs and/or RCs) to decide how many resources (if any) they want to provide to that specific request/user community




Step 2 Request Submission

Resource allocation process in initiated by VO. VO specifies and submits a resource request.

Step 3 Pool matching

EGI Operator matches the resources request with currently available pools and decide on actions (may be more that one).

Step 4 Actions

Depending on the result from Step 3 "Pool matching" (what is available in "Pools of resources repository") EGI Operator can take following actions: 

Step 4a

In case some pools are matching the request, EGI operator can proceed with allocation according to process specified in the selected pool definition ("free hand", allocation based on negotiations, etc).

This is the step where NGI come into play. It can be only informed about matching result or work as a broker. 

Step 4b

In case the are missing resources EGI Operator can forward the request to scientific review.

After scientific review the request goes to step 3 again. 

This step take place only if it is needed, for example:

  • if in "Pools of resources repository" are pools which match but require scientific review
  • to make request more attractive for resource providers

Step 4c

EGI Operator sends back the offer to the VO, in case:

  • it covers the request,
  • it covers the request partially or dosen't cover at all, but there is no option to improve the situation

VO can accept the offer or reject it (or continue the negotiation with modified request – step 2 again)