Resource Pool

From EGIWiki
Revision as of 15:35, 5 September 2013 by Krakow (talk | contribs) (Terminology)
Jump to: navigation, search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Introduction

Terminology

Pool - broad concept of defining/declaring resources availability for EGI Resource Allocation.

Pool of resources - capacity of resources available for EGI allocation according to defined process under defined conditions. Pool definition may be subject to change by EGI or RP according to a specific policy


Resource allocation QoS levels

Source: https://rt.egi.eu/rt/Ticket/Attachment/219099/80552/QoS_RA-TF_v6_AC.docx

Opportunistic (basic support level provided by a Pool)

  • Resources not guaranteed and subject to availability: CPU slots are granted when no other job is submitted by a VO with a higher priority.
    • The Pool supports the VO for a fixed period of time. The duration of the agreement has to be declared in the VO Request submitted to the Operator and is subject to negotiation
    • Before expiration, the agreement can be renewed and renegotiated directly with the RPP or NGIP
    • 3 different queue levels are proposed
      • short: 30 minutes max - for NAGIOS probes or short test
      • medium : 12 hours max
      • long : 48 hours max
  • The opportunistic model is commonly the default model for RPs that wish to support a VO without any formal agreement. Formalizing this model can help VOs to be more aware of who supports them, and may foster resource acknowledgement.

Time allocation

  • Resources available in fair share-like mode for a fixed time period: during the agreed fixed period, the VO is allocated a guaranteed minimum number of CPU hours per day/week/month/..., the dayly/weekly/monthly/... period is hereafter called Fine Time Scale (FTS). When the VO reaches this threshold, the allocation mode is converted in Opportunistic until the next FTS slot starts.
    • The Pool supports the VO for a fixed period of time. The duration of the agreement has to be declared in the VO Request submitted to the Operator and is subject to negotiation
    • After expiration a new Request has to be submitted to the Operator
    • Fine Time Scale (FTS) may be declared in the VO Request submitted to the Operator, or may be proposed by the Pool operator, and is subject to negotiation by the Pool
    • 3 different queue levels are proposed
      • short: 30 minutes max - for NAGIOS probes or short test
      • medium : 12 hours max
      • long : 48 hours max

Reserved allocation

  • Resources are exclusively reserved to the VOand the job will be executed immediately after submission, unless all the available job slots are already occupied by the VO. The resources made available to the VO are expressed in cores, or job slots.. .
    • The duration of the agreement has to be declared in the VO Request together with the amount of resources. Subject to negotiation
    • available at any time during a fixed period of time
    • Since the resources are fully dedicated to the VO, at least 2 different queue levels are proposed
      • short: 30 minutes max - for NAGIOS probes or short test
      • production: negotiated with the RPP or NGIP
    • The model may be appropriate for well organized large communities, that are able to make 100% use of the dedicated cores during the agreed period.


Resource allocation scenarios

Source: https://wiki.egi.eu/wiki/Resource_Allocation_Process

Scenario
Scope
EGI role
NGIs role
Review
Usage (guaranteed allocation/opportunistic)
Actions/steps
in free hand
address requests below a defined threshold central collector for requests; set up and coordinate the usage of the pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers
no review as long as the pool's resources are enough to satisfy all requests. If a review is needed it is up to EGI.eu only to decide
guaranteed allocation for a subset of the pool's resources; opportunistic for the rest After the pool has been set up, EGI collects requests, defines the required operational steps to support the requests; RPs configure their resources that are part of the pool accordingly
right to revoke
address request above the threshold central collector for requests; set up and coordinate the usage of the pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers + option to withdraw if national/local policies are conflicting
same as above, but  if a review is needed a very lightweight procedure should be defined, involving EGI.eu and representatives of the RPs contributing to the pool
guaranteed allocation for a subset of the pool's resources; opportunistic for the rest After the pool has been set up, EGI collects requests, defines the required operational steps to support the requests; RPs configure their resources that are part of the pool accordingly
negotiated (elastic pools)
address 'heavy' requests central collector for requests; broker for the RPs resources; define all the technical aspects of the requests; open calls to RPs in order to set up the pool  through the NGIs
broker for the RCs resources; provide support to RCs in order to ease the creation of a pool; monitor the effectiveness of the pool resources usage by the requester
not at all; it is up to the different RPs (NGIs and/or RCs) to decide how many resources (if any) they want to provide to that specific request/user community
the call shall look for resources whose allocation could be guaranteed; in addition opportunistic usage is an added value for each request EGI investigates all the operational aspects, providing a technical descritption of the request (and recipes if necessary) and sets up a call to RPs (throug NGIs) to provide resources for the temporary (elastic) pool dedicated to that request. Then coordinates the setting up of the pools and informs the requester about the conditions for accessing and using the resources


Contribute resources to the EGI distributed pool