Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Resource Allocation Task Force"

From EGIWiki
Jump to navigation Jump to search
 
(34 intermediate revisions by 4 users not shown)
Line 1: Line 1:
{{Template:Op menubar}} {{TOC_right}}  
{{EGI_Activity_groups_menubar}}


'''Coordinator''': T. Ferrari/EGI.eu<br>
{{TOC_right}} 


'''Meetings page''': https://indico.egi.eu/indico/categoryDisplay.py?categId=86  
[[Category:Special Interest groups]]
 
<br> '''Coordinator''': M. Krakowian '''Meetings page''': [https://indico.egi.eu/indico/categoryDisplay.py?categId=86 https://indico.egi.eu/indico/categoryDisplay.py?categId=86]


'''Mailing list''': resource-allocation@mailman.egi.eu  
'''Mailing list''': resource-allocation@mailman.egi.eu  


'''Mailing list page''': https://www.egi.eu/sso/groupView/resource-allocation  
'''Mailing list page''': [https://www.egi.eu/sso/groupView/resource-allocation https://www.egi.eu/sso/groupView/resource-allocation]
 
'''Action list''': [http://go.egi.eu/Resource_Allocation_Actions_RT http://go.egi.eu/Resource_Allocation_Actions_RT]


'''Members:''' <!--*https://www.egi.eu/sso/group/resource-allocation -->  
'''Members:''' <!--*https://www.egi.eu/sso/group/resource-allocation -->  


*NGIs: NGI_PL, NGI_France, NGI_IberGrid, NGI_IT, NGI_AEGIS, NGI_CZ  
*NGIs: NGI_PL, NGI_France, NGI_IberGrid, NGI_IT, NGI_AEGIS, NGI_CZ  
*VOs: WeNMR, BIOMED, (DCH-RP)?, COMPCHEM VO
*VOs: WeNMR, BIOMED, COMPCHEM VO
 
'''<span lang="en" id="result_box" class="short_text"><span class="hps">Next meeting: 24.01.2013 15:3o-17:oo</span></span>'''


[[Resource Allocation Task Force brainstorming page|<br>]]  
[[Resource Allocation Task Force brainstorming page|<br>]]  
Line 20: Line 22:
[[Resource Allocation Task Force brainstorming page|Resource Allocation Task Force brainstorming page]] - page with ideas, issues for discussion and ideas  
[[Resource Allocation Task Force brainstorming page|Resource Allocation Task Force brainstorming page]] - page with ideas, issues for discussion and ideas  


= Introduction =
[[Resource Allocation Process|Resource Allocation Process]] - page with draft proposal for RA process
 
[https://documents.egi.eu/document/2030 Draft version of RA procedure]
 
[https://documents.egi.eu/document/2032 List of possible 3 QoS levels]
 
= Task force PHASE 2 =
 
July 2013-Dec 2013
 
== Mandate  ==
 
*support the Resource Allocation [[VT Resource Allocation Tools|mini project]] in defining requirements, processes and procedures needed for the implementation of the tools
*participate in testing activities
 
== Milestones  ==
 
*M1: provide guidance on the specification (attributes) defining a resource pool
*M2: provide guidance on the specification (attributes) definin a user request. This action includes the definition of how to qualify a user community entitles to use a given resource reservation (VO, VO group etc.). Any user with valid SSO account will be allowed to submit a request
*M3: provide guidance on the specification of a SLA and its related metrics
*M4: participate to testing activities of the tool prototype that will be available in October
 
All milestones (except M4) are due by the end of the September
 
= Task Force PHASE 1  =
 
Jan 2013-April 2013


EGI needs to develop a model and the related processes for allocation of resources as a service to international collaborations. Offering resources as a services is relevant to all communities that do not contribute own resources to the infrastructure.  
EGI needs to develop a model and the related processes for allocation of resources as a service to international collaborations. Offering resources as a services is relevant to all communities that do not contribute own resources to the infrastructure.  
Line 35: Line 63:
This task force aims at focusing on the processes and the operational aspects of resource application and allocation. The process for assessment is out of scope and will be dwelt with by other groups.  
This task force aims at focusing on the processes and the operational aspects of resource application and allocation. The process for assessment is out of scope and will be dwelt with by other groups.  


= Mandate  =
== Mandate  ==


*Collect information about best practices and processes that are already in place at a national level  
*Collect information about best practices and processes that are already in place at a national level  
Line 44: Line 72:
*Coordinate the implementation of a testbed with the involvement of VOs and Resource Providers
*Coordinate the implementation of a testbed with the involvement of VOs and Resource Providers


= Actions  =
== Actions  ==


See the task force [http://go.egi.eu/resource-allocation-rt actions] on the resource-allocation queue of RT (access restricted to the members of the task force) <!--
{| class="wikitable"
{| class="wikitable"
|-
|-
Line 51: Line 80:
! scope="col" | Action  
! scope="col" | Action  
! scope="col" | Date  
! scope="col" | Date  
! scope="col" | Responsible
! scope="col" | Responsible  
! scope="col" | Status
|-
|-
| 1  
| 1  
| Contact with Prace and ask for document describing RA process in Prace.  
| Contact with Prace and ask for document describing RA process in Prace.  
| 17.01.2013  
| 17.01.2013  
| Tiziana
| Tiziana  
| done
|-
|-
| 2  
| 2  
| Prepare for second phone call: 3 RA models with pros, cons and use cases when the model can be used. Role of NGI, EGI and RP in each model. Is in the model a place for scientific review.  
| Prepare for second phone call: 3 RA models with pros, cons and use cases when the model can be used. Role of NGI, EGI and RP in each model. Is in the model a place for scientific review.  
| 17.01.2013  
| 17.01.2013  
| Małgorzata
| Małgorzata  
| done
|}
|}
--> <br>


<br>
== How to participate  ==
 
= How to participate  =


== For NGIs  ==
=== For NGIs  ===


*Participate to the task force to contribute to the definition of application and allocation processes  
*Participate to the task force to contribute to the definition of application and allocation processes  
Line 74: Line 105:
*Send mail to resource-allocation at mailman.egi.eu
*Send mail to resource-allocation at mailman.egi.eu


== For NGIs and Resource Centres  ==
=== For NGIs and Resource Centres  ===


*Contribute to the pool of shared resources for the implementation of a testbed. Purpose of the testbed is try processes in a production environment. Minimum requirements to participate to the testbed:  
*Contribute to the pool of shared resources for the implementation of a testbed. Purpose of the testbed is try processes in a production environment. Minimum requirements to participate to the testbed:  
Line 82: Line 113:
*Send mail to resource-allocation at mailman.egi.eu
*Send mail to resource-allocation at mailman.egi.eu


= Milestones  =
== Milestones  ==


*(DONE) 11-01-2013: setup the task force with the participation of users and NGIs  
*(DONE) 11-01-2013: setup the task force with the participation of users and NGIs  
*22-01-2013: collect expressions of interest from resource providers to participate to a testbed.  
*(DONE) 22-01-2013: collect expressions of interest from resource providers to participate to a testbed.  
*29-01-2013 (Evolving EGI workshop): propose processes around application and offering for discussion  
*(DONE) 29-01-2013 (Evolving EGI workshop): propose processes around application and offering for discussion  
*15-02-2013: setup a testbed with the participation of resource providers and VOs
*(DONE) 15-02-2013: setup a testbed with the participation of resource providers and VOs  
*(DONE) 10-04-2013: at CF13 finalize models and procedures
*15-06-2013: define storage QoS levels
*15-06-2013:
**specification of the SLA/OLA (includes metric definition and profile - semantics, unit of measure).
**specification of the RP pool, and testing of the definition through the existing testbed (to be processed through manual allocation)
*provide feedback to the implementers on details of the procedure (timing of answers, etc.)
*decide on assignment of roles
 
== Resources  ==


= Resources  =
NGI proposals<br>
 
*[https://indico.egi.eu/indico/materialDisplay.py?contribId=3&materialId=0&confId=1298 NGI&nbsp;IT]
*[https://indico.egi.eu/indico/getFile.py/access?contribId=2&resId=0&materialId=0&confId=1298 NGI PL]
 
<br>


*[http://documents.egi.eu/document/1415 Demonstrating Excellent European Science on EGI's shared resources], Nov 2012  
*[http://documents.egi.eu/document/1415 Demonstrating Excellent European Science on EGI's shared resources], Nov 2012  
*[https://indico.egi.eu/indico/materialDisplay.py?contribId=3&materialId=0&confId=1100 Draft processes]: initial draft document for discussion
*[https://indico.egi.eu/indico/materialDisplay.py?contribId=3&materialId=0&confId=1100 Draft processes]: initial draft document for discussion  
 
*PRACE T1 resource allocation:
[[Category:Task_forces]]
**[http://prace-ri.eu/IMG/pdf/D4-3-1_1ip.pdf D4-3-1]
**[http://prace-ri.eu/IMG/pdf/d4.3.2_1ip.pdf D4.3.2]  
*XSEDE:
**[https://www.xsede.org/documents/234989/378230/XNUT_XSEDE12_July15.pdf High level presentation for users (from slide 9)]
**[https://www.xsede.org/web/xup/allocations-overview XSEDE allocation main page]

Latest revision as of 12:51, 19 June 2015

EGI Activity groups Special Interest groups Policy groups Virtual teams Distributed Competence Centres



Coordinator: M. Krakowian Meetings page: https://indico.egi.eu/indico/categoryDisplay.py?categId=86

Mailing list: resource-allocation@mailman.egi.eu

Mailing list page: https://www.egi.eu/sso/groupView/resource-allocation

Action list: http://go.egi.eu/Resource_Allocation_Actions_RT

Members:

  • NGIs: NGI_PL, NGI_France, NGI_IberGrid, NGI_IT, NGI_AEGIS, NGI_CZ
  • VOs: WeNMR, BIOMED, COMPCHEM VO


Resource Allocation Task Force brainstorming page - page with ideas, issues for discussion and ideas

Resource Allocation Process - page with draft proposal for RA process

Draft version of RA procedure

List of possible 3 QoS levels

Task force PHASE 2

July 2013-Dec 2013

Mandate

  • support the Resource Allocation mini project in defining requirements, processes and procedures needed for the implementation of the tools
  • participate in testing activities

Milestones

  • M1: provide guidance on the specification (attributes) defining a resource pool
  • M2: provide guidance on the specification (attributes) definin a user request. This action includes the definition of how to qualify a user community entitles to use a given resource reservation (VO, VO group etc.). Any user with valid SSO account will be allowed to submit a request
  • M3: provide guidance on the specification of a SLA and its related metrics
  • M4: participate to testing activities of the tool prototype that will be available in October

All milestones (except M4) are due by the end of the September

Task Force PHASE 1

Jan 2013-April 2013

EGI needs to develop a model and the related processes for allocation of resources as a service to international collaborations. Offering resources as a services is relevant to all communities that do not contribute own resources to the infrastructure.

Policies, a service model and the related processes are needed to:

  • provide resources in a streamlined fashion making it procurement easy for users, while taking into account the different Resource Centre-level and NGI-level resource allocation policies
  • provide resources as a service through service level agreements in order to give some structure to resource offering for VOs relying on opportunistic usage of resources
  • ensure mechanisms are in place to acknowledge usage of resources by VOs that rely on opportunistic usage of the infrastructure. This can happen through acknowledgement of infrastructure usage in published research work and it is important for Resource Centres and NGIs to provide evidence of their contribution to national and international science
  • develop processes to express resource demand, and resource offer.

In Nov 2012 the EGI Council approved the start of experimental activities around the development of policies and processes for the application, scientific review and pooled resource allocation of NGI resources. More information

This task force aims at focusing on the processes and the operational aspects of resource application and allocation. The process for assessment is out of scope and will be dwelt with by other groups.

Mandate

  • Collect information about best practices and processes that are already in place at a national level
  • Propose processes around central management of demand and offer of pooled NGI resources, through the establishment of
    • SLAs to express user demand (involved parties: customers, EGI.eu)
    • OLAs to express resource offer (involved parties: EGI.eu, Resource Providers)
  • Identify the needs of development in the existing operational tools for the automation of the identified processes
  • Coordinate the implementation of a testbed with the involvement of VOs and Resource Providers

Actions

See the task force actions on the resource-allocation queue of RT (access restricted to the members of the task force)

How to participate

For NGIs

  • Participate to the task force to contribute to the definition of application and allocation processes
  • Deadline: 11-01-2013
  • Send mail to resource-allocation at mailman.egi.eu

For NGIs and Resource Centres

  • Contribute to the pool of shared resources for the implementation of a testbed. Purpose of the testbed is try processes in a production environment. Minimum requirements to participate to the testbed:
    • Compute: 10 cores available at any time with HIGH PRIORITY. Jobs arriving to that site will enter in execution at the earliest possibility the cluster occupation permits.
    • Storage: 1 TB
  • Deadline: 22 Jan 2013
  • Send mail to resource-allocation at mailman.egi.eu

Milestones

  • (DONE) 11-01-2013: setup the task force with the participation of users and NGIs
  • (DONE) 22-01-2013: collect expressions of interest from resource providers to participate to a testbed.
  • (DONE) 29-01-2013 (Evolving EGI workshop): propose processes around application and offering for discussion
  • (DONE) 15-02-2013: setup a testbed with the participation of resource providers and VOs
  • (DONE) 10-04-2013: at CF13 finalize models and procedures
  • 15-06-2013: define storage QoS levels
  • 15-06-2013:
    • specification of the SLA/OLA (includes metric definition and profile - semantics, unit of measure).
    • specification of the RP pool, and testing of the definition through the existing testbed (to be processed through manual allocation)
  • provide feedback to the implementers on details of the procedure (timing of answers, etc.)
  • decide on assignment of roles

Resources

NGI proposals