Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Resource Allocation Terminology

From EGIWiki
Jump to navigation Jump to search
Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security


Back to the Resource Pool main page

Resource Provider

The Resource Provider (RP) offers access to ICT resources through service abstractions (e.g., computing power, storage, etc.). NGIs and or individual Resource Centres can be Resource Providers to EGI.

Resource Provider Pool

The Resource Provider Pool (RP Pool) is defined to be the specific resource capacity available for allocation by one Resource Provier and available to EGI for allocation. The RP pool is a dynamic entity as it can change over time: for example, the capacity offered, the resource technical specifications and the allocation policies can be changed at any point in time by the Resource Provider.

RP Pool Manager

The RP Pool Manager is the contact point for a given pool, liaises with the resource broker (EGI.eu) and is responsible of defining the local pool allocation policies.

EGI Pool

The EGI Resource Pool is the federation of one or more RP Pools. EGI federates resources, i.e. in order to satisfy a given demand, resources can be provided by one or multiple federated Resource Providers.

Broker

The broker is the central entity (EGI.eu) responsible of identifying Resource Providers who collectively are able to satisfy a given demand. In the proposed process providers are free to choose one of the following models as applicable:

  • (A) EGI.eu is the Broker acting at a central level, while the NGI is the Broker responsible of appointing Resource Providers at a national level; EGI.eu liaises with the NGI to define the local providers offering resources to meet a given demand.
  • (B) EGI.eu is the Broker responsible of liaising directly with Resource Centres at a national level.

The process is flexible enough to allow the coexistence of (1) and (2). For example EGI.eu can be delegated the role of Broker at a national level for resource allocations of small entity (where small can be defined by the NGI) with the purpose of keeping the matchmaking process easy.

Matchmaker

Matchmaker is the model that best suites user communities. EGI.eu will play the Federator as well as the Matchmaker, i.e. is the entity responsible of

  • processing demands,
  • negotiating an offer,
  • identifying the Resource Providers willing to offer resources that match the demand,
  • establishing and enforcing an Operational Level Agreement (OLA) with the Resource Providers: the OLA involves the Federator and the Resource Provider.
  • establishing and enforcing a Service Level Agreement (SAL) with the Customer: the SLA involves the Customer and the Federator.
  • providing monitoring information (collected from the Resource Provider) and periodic service reports.

Computing: Resource allocation QoS levels

By providing resources through a RP pool to the EGI Pool, a Resource Provider can decide to offer different quality of service levels.

Level C1: Opportunistic

This is the basic level which can be provided. Resources are not guaranteed and are subject to local availability at any time: job slots are granted when no other job is submitted by a VO with a higher priority.

The Pool supports the VO for a fixed period of time. The duration of the agreement has to be declared in the VO Request submitted to the Operator and is subject to negotiation. Before expiration, the agreement can be renewed and renegotiated directly with the Resource Provider. Three different queue levels are proposed, each associated to a maximum waiting time.

  • short: 30 minutes max - for NAGIOS probes or short test
  • medium: 12 hours max
  • 48 hours max

The opportunistic model is commonly the default model for RPs that wish to support a VO. Formalizing this model can help VOs to be more aware of who supports them, and fosters resource acknowledgement.

Level C2: Time allocation

In this intermediate service level, resources are available in fair share-like mode for a fixed time period: during the agreed fixed period, the VO is allocated a guaranteed minimum number of normalized wall time hours per day/week/month/.

The allocated resources can be expressed in normalized Wall Clock Time HEP-SPEC06 hours.

The daily/weekly/monthly/... period is hereafter called Fine Time Scale (FTS). When the VO reaches this threshold, the allocation mode is converted in Opportunistic until the next FTS slot starts.

The Pool supports the VO for a fixed period of time. The duration of the agreement has to be declared in the VO Request submitted to the Operator and is subject to negotiation.

Fine Time Scale (FTS) may be declared in the VO Request submitted to the Operator, or may be proposed by the Pool Manager, and is subject to negotiation.

Three different queue levels are proposed

  • short: 30 minutes max - for NAGIOS probes or short tests
  • medium: 12 hours max
  • long: 48 hours max

Level C3: Reserved allocation

Resources are exclusively reserved to the VO and the job will be executed immediately after submission, unless all the available job slots are already occupied by jobs of the VO. The resources made available to the VO are expressed in cores, or job slots.

The duration of the agreement has to be declared in the VO Request together with the amount of resources. Subject to negotiation.

In this service, resources are available at any time during a fixed period of time. Since the resources are fully dedicated to the VO, at least 2 different queue levels are proposed:

  • short: 30 minutes max - for NAGIOS probes or short test
  • production: negotiated with the Resource Provider.

The model may be appropriate for well organized large communities, that are able to make 100% use of the dedicated cores during the agreed period.

Resources