Difference between revisions of "Resource Allocation Task Force brainstorming page"
Line 8: | Line 8: | ||
<br> | <br> | ||
= The high level concept= | = The high level concept = | ||
[[Image:RA overview.png|400px|RA overview.png]] | |||
=Resource allocation components= | EGI as a place where EGI customer and EGI provider meet and negotiate resource allocation. | ||
= Resource allocation components = | |||
[[Image:RA processes.png|600px|RA processes.png]] | |||
= Actors = | = Actors = | ||
Line 31: | Line 33: | ||
*New or existing VO | *New or existing VO | ||
<br> | |||
Regional VO: | |||
*Should be up to NGIs to help the VO and allocate resources within NGI | |||
*Should be up to NGIs to help the VO and allocate resources within NGI | |||
*EGI provides single point of contact for resource allocation for Customers | *EGI provides single point of contact for resource allocation for Customers | ||
International VO: | International VO: | ||
*EGI provides | |||
**Single point of contact for resource allocation for their Customers | *EGI provides | ||
**Single point of contact for resource allocation for their Customers | |||
**Negotiation and signing procedures | **Negotiation and signing procedures | ||
**Monitoring | **Monitoring | ||
**Reporting | **Reporting | ||
New VOs might require additional support. | New VOs might require additional support. | ||
= Prerequisites = | = Prerequisites = | ||
'''EGI Customer''' - registered and in production status in Operations Porta'''l''' | |||
'''EGI Provider '''(NGI and site) - registered and certified in GO DB | |||
= Models = | = Models = | ||
Different models can be applied for different requests: | Different models can be applied for different requests: | ||
* Mature VO vs New VO | |||
* International or regional VO | *Mature VO vs New VO | ||
* „Big” request vs „Small” request | *International or regional VO | ||
or NGIs. | *„Big” request vs „Small” request | ||
or NGIs. | |||
== Model 1 == | == Model 1 == | ||
EGI Providers’ resources and EGI Customers’ requests are published on an open market – are visible for everyone. | [[Image:RA Model1.png|300px|RA Model1.png]] | ||
EGI Providers’ resources and EGI Customers’ requests are published on an open market – are visible for everyone. | |||
EGI regulates and supervises the market. It determines the rules (e.g. procedures) of the market and acts as a referee in case of agreement violation/negotiations | |||
=== Pros === | === Pros === | ||
*EGI knows what is going on in the infrastructure | |||
*There are clear rules for resource allocation | |||
*In case of agreement violation/negotiations EGI is a referee<br> | |||
*EGI Providers knows what are the EGI Customers' current needs | |||
*EGI Customer knows which resources are now available<br> | |||
=== Cons === | === Cons === | ||
*For EGI Provider - negotiation process may not be able to be adapted to its needs | |||
=== Use cases === | === Use cases === | ||
Line 70: | Line 90: | ||
=== Role of Actors === | === Role of Actors === | ||
NGI, EGI and RP | NGI, EGI and RP | ||
== Model 2 == | == Model 2 == | ||
EGI Providers’ resources and EGI Customers’ requests are visible to EGI. | [[Image:RA Model2.png|500px|RA Model2.png]] | ||
EGI Providers’ resources and EGI Customers’ requests are visible to EGI. | |||
EGI matches EGI Providers’ resources to EGI Customers’ requests needs. | EGI matches EGI Providers’ resources to EGI Customers’ requests needs. | ||
Interaction EGI Customer | Interaction EGI Customer <-> EGI provider limited. | ||
=== Pros<br> === | === Pros<br> === | ||
*Single point of contact for EGI Customers | |||
=== Cons<br> === | === Cons<br> === | ||
*EGI has to get and manage the knowledge what is available in the infrstructure (each site) | |||
*Time consuming for EGI | |||
*Some EGI providers want to talk with EGI customers directly | |||
=== Use cases<br> === | === Use cases<br> === | ||
*big international EGI Customers | |||
*specialised EGI Customer requests | |||
*new international EGI Customers | |||
=== Role of Actors<br> === | === Role of Actors<br> === | ||
NGI, EGI and RP | NGI, EGI and RP | ||
== Model 3 == | == Model 3 == | ||
EGI Providers’ resources and EGI Customers’ requests transparent - visible for everyone. | [[Image:RA Model3.png|300px|RA Model3.png]] | ||
EGI Providers’ resources and EGI Customers’ requests transparent - visible for everyone. | |||
The parties have freedom to decide how they want to negotiate resources and under which condidions (no regulation on EGI side). | The parties have freedom to decide how they want to negotiate resources and under which condidions (no regulation on EGI side). | ||
EGI role is to support EGI Providers and/or EGI Customers only if needed. E.g. providing tool for resource allocation, helping in request fulfilment, specialised requests etc. | EGI role is to support EGI Providers and/or EGI Customers only if needed. E.g. providing tool for resource allocation, helping in request fulfilment, specialised requests etc. | ||
<br> | |||
=== Pros<br> === | === Pros<br> === | ||
*each EGI provider may want to apply own procedures - pros for EGI Provider<br> | |||
=== Cons<br> === | === Cons<br> === | ||
*each EGI provider may want to apply own procedures - cons for EGI Customer | |||
*EGI Customer has to negotiate with each of the EGI providers separately | |||
=== Use cases<br> === | === Use cases<br> === | ||
* Regional | |||
*Regional EGI Customer<br> | |||
=== Scientific request review === | |||
*it is up to EGI provider if it is needed | |||
=== Role of Actors<br> === | === Role of Actors<br> === | ||
NGI, EGI and RP | NGI, EGI and RP | ||
= Issues for discussion = | = Issues for discussion = | ||
Line 116: | Line 159: | ||
#The role of NGI, EGI, RC | #The role of NGI, EGI, RC | ||
#Need for scientific request review | #Need for scientific request review | ||
#With whom should be signed SLA? EGI, NGI, RP? | #With whom should be signed SLA? EGI, NGI, RP? | ||
#The demand / offer management. Use cases: | #The demand / offer management. Use cases: | ||
#*All VO/VRCs do say that they will acknowledge the use of resources in all publications. However, it is extremely difficult to find a list of such publications for a crosscheck. Such lists are very important for the production of yearly reports and justify the resources. Some of the communities do not even provide or compile such information, leaving a void in this area. | #*All VO/VRCs do say that they will acknowledge the use of resources in all publications. However, it is extremely difficult to find a list of such publications for a crosscheck. Such lists are very important for the production of yearly reports and justify the resources. Some of the communities do not even provide or compile such information, leaving a void in this area. | ||
#*Let us say if I, as an NGI, give a directive to my national centres to support a given VO/VRC, I also would like to know what is the impact on my national users of my adherence. If I provide resources to a VO but my national users do not see an advantage of this investment, maybe it is not worthwhile. VOs need to try to understand how they can compensate national users (credits, fairshare, ...) that bring new resources to the VO infrastructure. A clear model is needed here. | #*Let us say if I, as an NGI, give a directive to my national centres to support a given VO/VRC, I also would like to know what is the impact on my national users of my adherence. If I provide resources to a VO but my national users do not see an advantage of this investment, maybe it is not worthwhile. VOs need to try to understand how they can compensate national users (credits, fairshare, ...) that bring new resources to the VO infrastructure. A clear model is needed here. | ||
#*The VO/VRCs should stress their top resource providers in the same way has boinc platforms do it, making very visible who are their top contributors. If I go the biomed homepage, I do not see any kind of acknowledgement, or references to the sites contributing to biomed. | #*The VO/VRCs should stress their top resource providers in the same way has boinc platforms do it, making very visible who are their top contributors. If I go the biomed homepage, I do not see any kind of acknowledgement, or references to the sites contributing to biomed. | ||
Revision as of 13:17, 23 January 2013
Main | EGI.eu operations services | Support | Documentation | Tools | Activities | Performance | Technology | Catch-all Services | Resource Allocation | Security |
The main goals are:
- Attract new users and allocate them some grid resources
- Provide users with a simplified procedure to find grid resources for their needs and by means of a central unique reference point
The high level concept
EGI as a place where EGI customer and EGI provider meet and negotiate resource allocation.
Resource allocation components
Actors
EGI
EGI Provider
- NGIs that can directly decide on resource allocation on a pool of resources on specific sites
- NGIs that can coordinate resources allocation with their sites
- sites that are identified to allocate resources by passing its NGI
EGI Customer
- Regional or international VO
- New or existing VO
Regional VO:
- Should be up to NGIs to help the VO and allocate resources within NGI
- EGI provides single point of contact for resource allocation for Customers
International VO:
- EGI provides
- Single point of contact for resource allocation for their Customers
- Negotiation and signing procedures
- Monitoring
- Reporting
New VOs might require additional support.
Prerequisites
EGI Customer - registered and in production status in Operations Portal
EGI Provider (NGI and site) - registered and certified in GO DB
Models
Different models can be applied for different requests:
- Mature VO vs New VO
- International or regional VO
- „Big” request vs „Small” request
or NGIs.
Model 1
EGI Providers’ resources and EGI Customers’ requests are published on an open market – are visible for everyone.
EGI regulates and supervises the market. It determines the rules (e.g. procedures) of the market and acts as a referee in case of agreement violation/negotiations
Pros
- EGI knows what is going on in the infrastructure
- There are clear rules for resource allocation
- In case of agreement violation/negotiations EGI is a referee
- EGI Providers knows what are the EGI Customers' current needs
- EGI Customer knows which resources are now available
Cons
- For EGI Provider - negotiation process may not be able to be adapted to its needs
Use cases
Role of Actors
NGI, EGI and RP
Model 2
EGI Providers’ resources and EGI Customers’ requests are visible to EGI.
EGI matches EGI Providers’ resources to EGI Customers’ requests needs.
Interaction EGI Customer <-> EGI provider limited.
Pros
- Single point of contact for EGI Customers
Cons
- EGI has to get and manage the knowledge what is available in the infrstructure (each site)
- Time consuming for EGI
- Some EGI providers want to talk with EGI customers directly
Use cases
- big international EGI Customers
- specialised EGI Customer requests
- new international EGI Customers
Role of Actors
NGI, EGI and RP
Model 3
EGI Providers’ resources and EGI Customers’ requests transparent - visible for everyone.
The parties have freedom to decide how they want to negotiate resources and under which condidions (no regulation on EGI side).
EGI role is to support EGI Providers and/or EGI Customers only if needed. E.g. providing tool for resource allocation, helping in request fulfilment, specialised requests etc.
Pros
- each EGI provider may want to apply own procedures - pros for EGI Provider
Cons
- each EGI provider may want to apply own procedures - cons for EGI Customer
- EGI Customer has to negotiate with each of the EGI providers separately
Use cases
- Regional EGI Customer
Scientific request review
- it is up to EGI provider if it is needed
Role of Actors
NGI, EGI and RP
Issues for discussion
- The role of NGI, EGI, RC
- Need for scientific request review
- With whom should be signed SLA? EGI, NGI, RP?
- The demand / offer management. Use cases:
- All VO/VRCs do say that they will acknowledge the use of resources in all publications. However, it is extremely difficult to find a list of such publications for a crosscheck. Such lists are very important for the production of yearly reports and justify the resources. Some of the communities do not even provide or compile such information, leaving a void in this area.
- Let us say if I, as an NGI, give a directive to my national centres to support a given VO/VRC, I also would like to know what is the impact on my national users of my adherence. If I provide resources to a VO but my national users do not see an advantage of this investment, maybe it is not worthwhile. VOs need to try to understand how they can compensate national users (credits, fairshare, ...) that bring new resources to the VO infrastructure. A clear model is needed here.
- The VO/VRCs should stress their top resource providers in the same way has boinc platforms do it, making very visible who are their top contributors. If I go the biomed homepage, I do not see any kind of acknowledgement, or references to the sites contributing to biomed.
Ideas received
- Freedom to express what is possible for NGIs (law,resources etc.). Not all of them are able to support given VO, even if they want to.
- The SLA should be easy enough to be filled in, which means that many service levels should be made optional, as we do now in the VO ID card.
- The resource allocation process should not only allow to address a resource request with an offer, but also vice versa a site with free resources to advertise it.
- National VOs need to talk to their NGIs, however, the same processes and interfaces used for EGI international users could be adopted for regional ones.
- The internal allocation processes and liaison to individual resource contributors should be made transparent.
- The same processes that are being defined for resource allocation, could be generalized to request services, i.e. to allow new user communities to request services to be provided by NGIs, such as application porting.