Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Resource Allocation Process

From EGIWiki
Revision as of 09:09, 10 April 2013 by Krakow (talk | contribs)
Jump to navigation Jump to search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


<< Resource Allocation Task Force


This page is created by Resource Allocation Task Force. It contains proposal for RA process.

Introduction

Definitions

Pool - broad concept of defining/declaring for EGI Resource Allocation.

Pools of resources repository -

Scientific review -


Process

Resource allocation process

Step 1 Pools definition preparation

  • NGIs/Sites define pools and register them in EGI " Pools of resources repository".
  • Pools definition may be subject of change according to specific Resource alloction policy.
  • NGIs/Sites are free to modify this repository.

Pool concept

Definition: Let a pool be a specific capacity of resources available for EGI allocation according to defined process under defined conditions

Note: Pools covers any scenarios mentioned ever in RATF including “free hand”, “right to revoke”, “negotiation (elastic pool)”

The pool are defined in the Site/NGI declaration:

  • previously referred as pre-agreement, but it should not require EGI to agree/negotiate
  • can be subject of change in time (some limitation apply)

Pools is just a broad concept of defining/declaring for EGI Resource Allocation.

Pool concept: specification

  1. Part 1: Pool Offer specification
    • Description of what is offered (capacity)
    • Validity in time (when pool is available)
    • Who is responsible for allocation process (NGI / Site)  
    • Process definition (scenario) - selection of predefined processes
    • QoS (opportunistic | assigned share | reserved ) - how resources are guaranteed for a specific VO
    • Resource specification (technical details)
  2. Part 2: Acceptance criteria for requests
    • Scope for VO (international | national, membership for NGI, discipline)
    • Scientific review (not-required | at least (fair | good | excellent))
    • Publications: required/not required
      • Type of acknowledgements: EGI/NGI/Site
    • Request size, incl. % of allocation eg.:
      • resources should not be allocated more than 10% of the pool per VO
      • resources should not cover more than 50% of the VO request

Pool concept: example

Pool-1 SiteA will offer:

  • up to 100 cores in 2013 as reserved pool (as a whole -> reservation applies to the whole pool not given usage), opportunistic inside pool
  • “free hand” to EGI - EGI has free choice to allocate resources 
  • each VO should not be allocated more than 10% of the pool
  • can be suitable for small resource requests

Pool-2 SiteB will offer:

  • up to 4000 cores between Jun’13 – Feb’14 with assigned fair-share
  • allocation based on negotiations
  • for requests that proved to be:
    • at least good in EGI scientific review
    • has members in NGI X
    • each requests can be supported up to 50%
  • can be suitable for big resource requests

Pool concept: process definition (scenarios)

List of examples of scenarios which can be implemented in the process.

Scenario
Scope
EGI role
NGIs role
Review
Usage (guaranteed allocation/opportunistic)
Actions/steps
in free hand
address requests below a defined threshold central collector for requests; set up and coordinate the usage of the pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers
no review as long as the pool's resources are enough to satisfy all requests. If a review is needed it is up to EGI.eu only to decide
guaranteed allocation for a subset of the pool's resources; opportunistic for the rest After the pool has been set up, EGI collects requests, defines the required operational steps to support the requests; RPs configure their resources that are part of the pool accordingly
right to revoke
address request above the threshold central collector for requests; set up and coordinate the usage of the pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers + option to withdraw if national/local policies are conflicting
same as above, but  if a review is needed a very lightweight procedure should be defined, involving EGI.eu and representatives of the RPs contributing to the pool
guaranteed allocation for a subset of the pool's resources; opportunistic for the rest After the pool has been set up, EGI collects requests, defines the required operational steps to support the requests; RPs configure their resources that are part of the pool accordingly
negotiated (elastic pools)
address 'heavy' requests central collector for requests; broker for the RPs resources; define all the technical aspects of the requests; open calls to RPs in order to set up the pool  through the NGIs
broker for the RCs resources; provide support to RCs in order to ease the creation of a pool; monitor the effectiveness of the pool resources usage by the requester
not at all; it is up to the different RPs (NGIs and/or RCs) to decide how many resources (if any) they want to provide to that specific request/user community
the call shall look for resources whose allocation could be guaranteed; in addition opportunistic usage is an added value for each request EGI investigates all the operational aspects, providing a technical descritption of the request (and recipes if necessary) and sets up a call to RPs (throug NGIs) to provide resources for the temporary (elastic) pool dedicated to that request. Then coordinates the setting up of the pools and informs the requester about the conditions for accessing and using the resources


Step 2 VO Request Submission

Resource allocation process in initiated by VO. VO specifies and submits a resource request.

Step 3 Resource and request matching

EGI Operator matches the resources request with currently available pools and decide on actions (may be more that one).

Step 4 Actions

Depending on the result from Step 3 "Pool matching" (what is available in "Pools of resources repository") EGI Operator can take following actions: 

Step 4a

In case some pools are matching the request, EGI operator can proceed with allocation according to process specified in the selected pool definition ("free hand", allocation based on negotiations, etc).

This is the step where NGI come into play. It can be either informed about matching result or start negotiation, depending on the pool definition. Depending on NGI internal procedures NGI can work as a broker or apply different model.

Step 4b

In case the are missing resources EGI Operator can forward the request to scientific review.

After scientific review the request goes to step 3 again. 

This step take place only if it is needed, for example:

  • if in "Pools of resources repository" are pools which match but require scientific review
  • to make request more attractive for resource providers

Step 4c

EGI Operator sends back the offer to the VO, in case:

  • it covers the request,
  • it covers the request partially or dosen't cover at all, but there is no option to improve the situation

VO can accept the offer or reject it (or continue the negotiation with modified request – step 2 again)