Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Resource Pool"

From EGIWiki
Jump to navigation Jump to search
Line 2: Line 2:
[[Category:Operations]]
[[Category:Operations]]
= Introduction =
= Introduction =
This page provides information about resource allocation in EGI and instructions on how to contribute resources to the EGI Pool.


= Terminology =
= Terminology =

Revision as of 10:42, 10 September 2013

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Introduction

This page provides information about resource allocation in EGI and instructions on how to contribute resources to the EGI Pool.

Terminology

Resource Provider

The Resource Provider (RP) offers access to ICT resources through service abstractions (e.g., computing power, storage, etc.). NGIs and or individual Resource Centres can be Resource Providers to EGI.

Resource Provider Pool

The Resource Provider Pool (RP Pool) is defined to be the specific resource capacity available for allocation by one Resource Provier and available to EGI for allocation. The RP pool is a dynamic entity as it can change over time: for example, the capacity offered, the resource technical specifications and the allocation policies can be changed at any point in time by the Resource Provider.

RP Pool Manager

The RP Pool Manager is the contact point for a given pool, liaises with the resource broker (EGI.eu) and is responsible of defining the local pool allocation policies.

EGI Pool

The EGI Resource Pool is the federation of one or more RP Pools. EGI federates resources, i.e. in order to satisfy a given demand, resources can be provided by one or multiple federated Resource Providers.

Resource allocation QoS levels

Source: https://wiki.egi.eu/w/images/3/33/List_of_possible_3_QoS_levels_v_3.pdf

Opportunistic (basic support level provided by a Pool)

  • Resources not guaranteed and subject to availability: CPU slots are granted when no other job is submitted by a VO with a higher priority.
    • The Pool supports the VO for a fixed period of time. The duration of the agreement has to be declared in the VO Request submitted to the Operator and is subject to negotiation
    • Before expiration, the agreement can be renewed and renegotiated directly with the RPP or NGIP
    • 3 different queue levels are proposed
      • short: 30 minutes max - for NAGIOS probes or short test
      • medium : 12 hours max
      • long : 48 hours max
  • The opportunistic model is commonly the default model for RPs that wish to support a VO without any formal agreement. Formalizing this model can help VOs to be more aware of who supports them, and may foster resource acknowledgement.

Time allocation

  • Resources available in fair share-like mode for a fixed time period: during the agreed fixed period, the VO is allocated a guaranteed minimum number of CPU hours per day/week/month/..., the dayly/weekly/monthly/... period is hereafter called Fine Time Scale (FTS). When the VO reaches this threshold, the allocation mode is converted in Opportunistic until the next FTS slot starts.
    • The Pool supports the VO for a fixed period of time. The duration of the agreement has to be declared in the VO Request submitted to the Operator and is subject to negotiation
    • After expiration a new Request has to be submitted to the Operator
    • Fine Time Scale (FTS) may be declared in the VO Request submitted to the Operator, or may be proposed by the Pool operator, and is subject to negotiation by the Pool
    • 3 different queue levels are proposed
      • short: 30 minutes max - for NAGIOS probes or short test
      • medium : 12 hours max
      • long : 48 hours max

Reserved allocation

  • Resources are exclusively reserved to the VOand the job will be executed immediately after submission, unless all the available job slots are already occupied by the VO. The resources made available to the VO are expressed in cores, or job slots.. .
    • The duration of the agreement has to be declared in the VO Request together with the amount of resources. Subject to negotiation
    • available at any time during a fixed period of time
    • Since the resources are fully dedicated to the VO, at least 2 different queue levels are proposed
      • short: 30 minutes max - for NAGIOS probes or short test
      • production: negotiated with the RPP or NGIP
    • The model may be appropriate for well organized large communities, that are able to make 100% use of the dedicated cores during the agreed period.


Resource allocation scenarios

Source: https://wiki.egi.eu/wiki/Resource_Allocation_Process

Scenario
Scope
EGI role
NGIs role
Review
Usage (guaranteed allocation/opportunistic)
Actions/steps
in free hand
address requests below a defined threshold central collector for requests; set up and coordinate the usage of the pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers
no review as long as the pool's resources are enough to satisfy all requests. If a review is needed it is up to EGI.eu only to decide
guaranteed allocation for a subset of the pool's resources; opportunistic for the rest After the pool has been set up, EGI collects requests, defines the required operational steps to support the requests; RPs configure their resources that are part of the pool accordingly
right to revoke
address request above the threshold central collector for requests; set up and coordinate the usage of the pool; investigate all the operational aspects; allocate resources;  monitor the effectiveness of the pool resources usage
provide resources directly; act as broker towards the Resource Centers + option to withdraw if national/local policies are conflicting
same as above, but  if a review is needed a very lightweight procedure should be defined, involving EGI.eu and representatives of the RPs contributing to the pool
guaranteed allocation for a subset of the pool's resources; opportunistic for the rest After the pool has been set up, EGI collects requests, defines the required operational steps to support the requests; RPs configure their resources that are part of the pool accordingly
negotiated (elastic pools)
address 'heavy' requests central collector for requests; broker for the RPs resources; define all the technical aspects of the requests; open calls to RPs in order to set up the pool  through the NGIs
broker for the RCs resources; provide support to RCs in order to ease the creation of a pool; monitor the effectiveness of the pool resources usage by the requester
not at all; it is up to the different RPs (NGIs and/or RCs) to decide how many resources (if any) they want to provide to that specific request/user community
the call shall look for resources whose allocation could be guaranteed; in addition opportunistic usage is an added value for each request EGI investigates all the operational aspects, providing a technical descritption of the request (and recipes if necessary) and sets up a call to RPs (throug NGIs) to provide resources for the temporary (elastic) pool dedicated to that request. Then coordinates the setting up of the pools and informs the requester about the conditions for accessing and using the resources


Contribute resources to the EGI distributed pool