Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "VT EGI Pay-for-Use PoC"

From EGIWiki
Jump to navigation Jump to search
 
(107 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{TOC}
{{EGI_Activity_groups_menubar}}
{{Menubar_VT}}
{{Template:EGI-PayForUse-PoC_menubar}}
{{TOC_right}} 
[[Category:Virtual_Teams]]




This page is dedicated to the EGI Pay-for-Use Proof of Concept and Activities.


This page is dedicated to the EGI Pay-for-Use Pilot Group and Activities.
= Overview  =


Group members, rationale, objectives, information references and technical details to be added.
== Motivation  ==


{{Fedcloud-tf:Menu}} {{TOC_right}}
EGI currently operates within a publicly funded research and academic environment providing services free at point of delivery with resources bought from grants dedicated to certain groups or disciplines either by direct allocation or by peer review. With the advent of cloud computing, business models and user expectations are shifting towards on-demand and pay-per-use service provision increasing flexibility and agility. This new paradigm provides motivation for EGI to explore new service definitions by enabling the possibility to provide ICT services that can be paid for the use, along with the more traditional procurement of resources to be managed and offered for free to the owners.<br> This approach also allows researchers and resource providers to better understand costs to access individual services and would enable the creation of innovative business models and pricing schemes (e.g. pay-per-use).


== Mandate  ==


== Mandate ==
In early 2013, the EGI Council approved a policy to explore business models for pay-for-use service delivery to couple together with the traditional method of free-at-point-of-use. The goal of this activity is to support the implementation of this policy in collaboration with NGIs through the definition and execution of proof of concepts. The mandate of the group is to create a proof of concept pay-for-use prototype.
In 2014, activities have continued in order to move the prototype into production, which is expected for the end of 2016.


EGI is a federation of national and domain specific resource infrastructure providers comprised of individual resource centres. Many of these resource centres have been experimenting with the deployment of virtualised management environments to improve the local delivery of services. Many of EGI’s current and new user communities would like to access the flexibility provided by virtualisation across the infrastructure on demand in a ‘cloud like’ environment. Federating these individual virtualised resources is a major priority for EGI that has started with the [http://go.egi.eu/uvw1 EGI User Virtualisation Workshop], and the drafting of the [http://go.egi.eu/435 EGI Cloud Integration Profile].
== Objectives  ==


Objectives:  
The objectives are the group are to:  


* write a [[Fedcloud-tf:Blueprint|blueprint document]] for EGI Resource Providers that wish to [[Fedcloud-tf:WorkGroups:_Federated_AAI|securely federate]] and share their virtualised environments as part of the EGI production infrastructure;
#Articulate appropriate business and responsibility models through defined business cases
* deploy a [[Fedcloud-tf:Testbed|test bed]] to evaluate the integration of virtualised resources within the existing EGI production infrastructure for [[Fedcloud-tf:WorkGroups:Scenario5|monitoring]], [[Fedcloud-tf:WorkGroups:Scenario4|accounting]] and [[Fedcloud-tf:WorkGroups:Scenario3|information services]];
#Define prices for services from the participating sites (both compute and storage)
* investigate and catalogue the [[Fedcloud-tf:WorkGroups:_Outreach#Requirements|requirements]] for community facing services based on or deployed through virtualised resources;
#Define agreements and service management processes and procedures
* provide [[Fedcloud-tf:Blueprint:Solutions_Intentory|feedback]] to relevant technology providers on their implementations and any changes needed for deployment into the production infrastructure;
#Identify the tools required and necessary development to faciliate pay-for-use service provisioning (e.g. billing function)
* identify and work with [[Fedcloud-tf:WorkGroups:_Outreach|user communities]] willing to be early adopters of the test bed infrastructure to help prioritise its future development;
#Analyse the changes within a pre-production environment that would be needed to support and roll out the new functionalities in the production environment
* identify [[Fedcloud-tf:Blueprint:Security_and_Policy|issues]] that need to be addressed by other areas of EGI (e.g. policy, operations, support &amp; dissemination).
#Evaluate legal, policy, and organisational issues around the full implementation of the pay-for-use model
#Develop a full production service.


== Activities ==
= Group Members  =
<div style="width: 60%"><div style="column-count:2;-moz-column-count:2;-webkit-column-count:2">
*Sy Holsinger, EGI.eu (Chair)
*Sergio Andreozzi, EGI.eu
*Luis Alves, CSC
*Owen Appleton, EGI.eu
*Emanouil Atanassov, IICT-BAS
*Bruce Becker, CSIR
*David Blundell, 100 Percent IT Ltd.
*Wilhelm Buehler, KIT-G
*Isabel Campos, CSIC
*Daniele Cesini, INFN-CNAF
*Alessandro Costantini, INFN/Master-Up
*Ivan Diaz, CESGA
*Miroslav Dobrucky, IISAS
*Feyza Eryol, TUBITAK ULAKBIM
*Carlos Fernandez Sanchez, CESGA
*Enol Fernández del Castillo, EGI.eu
*Tiziana Ferrari, EGI.eu
*Luciano Gaido, INFN
*André Gemünd, Fraunhofer SCAI
*Vaidas Giedrimas, Šiauliai University
*John Gordon, STFC
*Todor Gurov, IICT-BAS
*Sigve Haug, ETH ZURICH
*Helmut Heller, BADW
*Ladislav Hluchý, IISAS
*Janne Ignatius, CSC
*Javier Jimenez, EGI.eu
*Kostas Koumantaros, GRNET
*Kaspars Krampis, IMCS UL
*Antonio Lagana, UNIPG/Master-Up
*Panos Louridas, GRNET
*Giorgio Maggi, INFN-Bari
*Jesus Marco, IFCA/CSIC
*Per Öster, CSC
*Stuart Pullinger, STFC
*Roksana Różańska, CYFRONET
*François Ruty, Luna Technologies
*Serge Salamanka, UIIP NASB
*Diego Scardaci, INFN
*Horst Schwichtenberg, Fraunhofer SCAI
*Alvaro Simon, CESGA
*Vincenzo Spinoso, INFN-Bari
*Mariusz Sterzel, CYFRONET
*Tomasz Szepieniec, CYFRONET
*Jelena Tamulienė, VU
*Onur Temizsoylu, TUBITAK ULAKBIM
*Viet Tran, IISAS
</div></div>


The Task Force mandate lasts eighteen months, from Sept 2011 to March 2013. The Task Force activities are organised in three, six-months long phases. During each phase, the Task Force evaluates a set of [[Fedcloud-tf:WorkGroups|scenarios]] that an EGI federation of clouds should support. The scenarios are chosen by collecting use cases and requirements among user communities, resource providers and technology providers that have already adopted cloud computing or are planning to do so in a near future.
= Technical Details  =


One or more [[Fedcloud-tf:WorkGroups|work group]] is created inside the Task Force in order to evaluate each scenario. A leader is appointed for each work group and one or more collaborators are chosen among the Task Force members. The scenario evaluation performed by each work group is recorded in [[Fedcloud-tf:WorkGroups|workbenches]] and consists into:
The following [https://wiki.egi.eu/wiki/EGI_Pay-for-Use_PoC_Technical_Details link] provides an overview of the [https://wiki.egi.eu/wiki/EGI_Pay-for-Use_PoC_Technical_Details technical details] of the resource and technology providers for the PoC.  
* defining the set of capabilities that an EGI cloud infrastructure should have in order to support the given scenario;
* evaluating whether and what standards are available to implement the required capabilities;
* evaluating whether and what software solution is available to implement the required standards;
* evaluating the level of support of such software solutions by the Resource Providers members of the Task Force;
* evaluating the procedures to deploy the required software solutions into the infrastructure of the Resource Providers;
* devising tests to be run on the cloud infrastructure of the Resource Providers in order to simulate the given scenario.


Once the scenario has been evaluated and, when possible, the required software and tests have been deployed in the Task Force [[Fedcloud-tf:Testbed|test bed]] federated cloud, the members of the work group proceed to write a section of the [[Fedcloud-tf:Blueprint|blueprint]] document. Altogether, the blueprint document describes the set of capabilities that define an EGI Cloud Federation. The blueprint focuses on the infrastructure layer of the federation and each work group is tasked with documenting at least one core capability. Each section of the blueprint is structured as follow.
= How to Join  =
* '''Overview'''. The scenario and use cases from which the capability has been investigated are briefly introduced.
* '''Definition of the capability'''. The functional properties, interfaces and available standards for implementation are described and analysed. A reasoned choice is made about what standards and implementations should be endorsed by an EGI cloud federation.
* '''Deployment scenarios'''. The deployment infrastructure(s) and topology(ies) for the implementation of the capabilities are described and analysed.
* '''Testing the capability'''. When possible, a description is given of the tests used to simulate the scenario requiring the capability under exam.
* '''Security considerations'''. Security is a particularly sensitive topic in a federated infrastructure. Each capability has its own security evaluation with references to the EGI policy documents and security guidelines.
* '''Solution inventory'''. A critical review of how the capability under exams is implemented – or not implemented - by the Virtual Machine Management platforms deployed by the Resource Providers participating to the Task Force.


The progress of the activity of each work group is available in the [[Fedcloud-tf:WorkGroups:Workbenches|workbenches summary]] section.
If you would like to participate in the Proof of Concept, contact policy@egi.eu


== Operations ==
= References  =


The Task Force operates on its [mailto:fedcloud-tf@mailman.egi.eu mailing list] and on a [https://www.egi.eu/indico/categoryDisplay.py?categId=56 weekly conference call]. At the moment, our effort is organised in [[Fedcloud-tf:WorkGroups:Workbenches| work groups]] that any member of the Task Force is free to lead or to contribute to. Each member of Task Force can organise a meeting dedicated to further progress a specific workgroup activity. 
*EGI Pay-for-Use Exploratory Report for the EGI Council - https://documents.egi.eu/document/1391
 
*Pay-for-Use Survey Summary - https://documents.egi.eu/document/1391
The Task Force meet every Tuesday at 11:00 CET in a [https://www.egi.eu/indico/categoryDisplay.py?categId=56 conference call]. Minutes of the meetins are [[Fedcloud-tf:Administrativa#Task_Force_Conference_Calls_Agendas_and_Minutes|publicly available]].
*Pay-for-Use Presentation at Evolving EGI workshop: http://indico.egi.eu/indico/contributionDisplay.py?sessionId=3&amp;contribId=9&amp;confId=1252
<!--
*JRA1 Recommendations in D2.7 Sustainability Plan - https://documents.egi.eu/document/313
<div style="width:100%; height:100%; overflow:hidden;>
*EGI Federated Cloud Task Force - https://wiki.egi.eu/wiki/Fedcloud-tf:FederatedCloudsTaskForce
<div style="width:56%; float:left; text-align:center;">
*EGI Accounting Portal - http://accounting.egi.eu/egi.php
High attendance
<pBars ymin='0' ymax='100' angle='60' yunit='%' size='510x400' titlecolor='#000000' boxcolor='#000000' axiscolor='#000000' colors='#00AF33' marginx='15'>
OeRC (UK NGI),100
Chair,100
EGI-InSPIRE SA2,100
STFC (UK NGI),100
STFC (e-Science),100
EGI.eu,100
Operations Officer,92
TCD (NGI IE),90
Technology Manager,84
CESGA (IBERgrid),80
Community Officer,76
IGI,75
CYFRONET (NGI PL),69
CloudSigma,66
INFN/CNAF,66
EGI-InSPIRE JRA1,66
FZ Julich,66
</pBars>
</div>
<div style="width:23%; float:left; text-align:center;">
Medium attendance
<pBars ymin='0' ymax='100' angle='60' yunit='%'  size='210x400' titlecolor='#000000' boxcolor='#000000' axiscolor='#000000' colors='#FCD116' marginx='15'>
CESNET (NGI CZ),61
DANTE,57
GWDG,54
GRNET,50
WeNMR,45
KTH,41
IN2P3 (NGI FR),40
</pBars>
</div>
<div style="width:20%; float:left; text-align:center;">
Low attendance
<pBars ymin='0' ymax='100' angle='60' yunit='%' size='180x385' titlecolor='#000000' boxcolor='#000000' axiscolor='#000000' colors='#FF0000' marginx='15'>
SARA (NGI NL),30
StratusLab,23
OGF DCIFed,18
SIENA Project,9
IPHC,0
GRIF,0
</pBars>
<br>
</div>
</div>
 
Full details on meeting attendance are published in the [[Fedcloud-tf:Administrativa#Meeting_attendance| administrative]] section
-->
 
== Roadmap ==
 
The Task Force roadmap is organised in three, six-months long phases. Every six months, the output of the Task Force activities are disseminated through the EGI.eu conferences and a new phase is started.
 
{| border="1" cellspacing="5" cellpadding="5" class="wikitable" style="border-collapse: collapse; border:1px solid black; text-align:center;"
|- style="background-color: lightgray;"
! colspan="3" |  Phase 1. Setup: Sept 2011 - March 2012
|- style="background-color: lightgray;"
! Scenarios
! Workbenches
! Capabilities
|-
| 1
| align="left" | [[Fedcloud-tf:WorkGroups:Scenario1|Running a pre-defined VM Image]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:VMManagement|VM Management]]
|-
| 2
| align="left" | [[Fedcloud-tf:WorkGroups:Scenario2|Managing users' data and VMs]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:Data_Management|Data management]]
|-
| 3
| align="left" | [[Fedcloud-tf:WorkGroups:Scenario3|Integrating information from multiple resource providers]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:InformationDiscovery|Information discovery]]
|-
| 4
| align="left" | [[Fedcloud-tf:WorkGroups:Scenario4|Accounting across Resource Providers]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:Accounting|Accounting]]
|-
| 5
| align="left" | [[Fedcloud-tf:WorkGroups:Scenario5|Reliability/Availability of Resource Providers]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:Monitoring|Monitoring]]
|-
| 6
| align="left" | [[Fedcloud-tf:WorkGroups:Scenario6|VM/Resource state change notification]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:Notification|Notification]]
|-
| 7
| align="left" | [[Fedcloud-tf:WorkGroups:_Federated_AAI|AA across Resource Providers]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:Authentication_and_Authorisation|Authentication and Authorisation]]
|-
| 8
| align="left" | [[Fedcloud-tf:WorkGroups:VM_Marketplace|VM images across Resource Providers]]
| align="left" | [[Fedcloud-tf:Blueprint:Capabilities:VM_Sharing|VM sharing]]
|- style="background-color: lightgray;"
! colspan="3" |  Phase 2. Consolidation: April 2012 - Sept 2012
|-
| colspan="3" align="left" |
<ul>
<li>Engagement of [[Fedcloud-tf:UserCommunities| user communities]].
<li>Implementing [https://indico.egi.eu/indico/getFile.py/access?contribId=91&sessionId=12&resId=0&materialId=slides&confId=1019| interoperability] among different cloud platforms (OpenNebula, OpenStack) federated into the [[Fedcloud-tf:Testbed| test bed]].
<li>Supporting the first use case: [https://wiki.egi.eu/wiki/FedCloudBNCweb| BNCweb].
<li>Draft of the [[Fedcloud-tf:Blueprint|Blueprint document]].
<li>[https://indico.egi.eu/indico/contributionDisplay.py?sessionId=12&contribId=91&confId=1019| Live demo] at the [http://tf2012.egi.eu/| EGI Technical Forum 2012].
</ul>
|- style="background-color: lightgray;"
! colspan="3" |  Phase 3. Integration: Sept 2012 - Mar 2013
|-
| colspan="3" align="left" |
<ul>
<li>Set up of a User [[Fedcloud-tf:UserCommunities| Community Support Team (UCST)]] dedicated to the task force use cases.
<li>Integration of the federation test bed into the EGI infrastructure: [https://goc.egi.eu/portal/| GOCDB], [https://grid-monitoring.egi.eu/myegi| SAM], [http://accounting.egi.eu/egi.php| APEL].
<li>Open the federation [[Fedcloud-tf:Testbed| test bed]] to early adopters.
<li>Publication of the [[Fedcloud-tf:Blueprint| Blueprint document]].
<li>Demo and poster for each supported use case.
<li>Tutorials and live demo of the federation test bed.
</ul>
|}
 
== Resources ==
 
# [https://wiki.egi.eu/wiki/Fedcloud-tf Wiki]
# [mailto:fedcloud-tf@mailman.egi.eu Mailing list] (incl. [https://mailman.egi.eu/mailman/private/fedcloud-tf/ Mailing list archive])
# [https://www.egi.eu/indico/categoryDisplay.py?categId=56 Meeting planner]
# [https://oerc.basecamphq.com/projects/7732005-egi-federated-cloudstask-force/log Task Force Management &amp; Actions tracker]
 
== References ==
 
#'''Task Force Mandate''': [https://documents.egi.eu/document/720 https://documents.egi.eu/document/720]
#'''Task Force Roadmap''': [https://documents.egi.eu/secure/RetrieveFile?docid=803&version=1&filename=lyon_20092011_v3.pdf https://documents.egi.eu/secure/RetrieveFile?docid=803&version=1&filename=lyon_20092011_v3.pdf]
#'''Other EGI Virtualisation activities''': https://wiki.egi.eu/wiki/EGI_Virtualisation_Activities
 
[[Category:Fedcloud-tf]]

Latest revision as of 11:53, 3 June 2016

EGI Activity groups Special Interest groups Policy groups Virtual teams Distributed Competence Centres


EGI Virtual teams: Main • Active Projects • Closed Projects • Guidelines
Pay-For-Use menu: Home • Processes/Instructions • Service/Price Overview • Service Management • Legal/Policy • Actions • Meetings •



This page is dedicated to the EGI Pay-for-Use Proof of Concept and Activities.

Overview

Motivation

EGI currently operates within a publicly funded research and academic environment providing services free at point of delivery with resources bought from grants dedicated to certain groups or disciplines either by direct allocation or by peer review. With the advent of cloud computing, business models and user expectations are shifting towards on-demand and pay-per-use service provision increasing flexibility and agility. This new paradigm provides motivation for EGI to explore new service definitions by enabling the possibility to provide ICT services that can be paid for the use, along with the more traditional procurement of resources to be managed and offered for free to the owners.
This approach also allows researchers and resource providers to better understand costs to access individual services and would enable the creation of innovative business models and pricing schemes (e.g. pay-per-use).

Mandate

In early 2013, the EGI Council approved a policy to explore business models for pay-for-use service delivery to couple together with the traditional method of free-at-point-of-use. The goal of this activity is to support the implementation of this policy in collaboration with NGIs through the definition and execution of proof of concepts. The mandate of the group is to create a proof of concept pay-for-use prototype. In 2014, activities have continued in order to move the prototype into production, which is expected for the end of 2016.

Objectives

The objectives are the group are to:

  1. Articulate appropriate business and responsibility models through defined business cases
  2. Define prices for services from the participating sites (both compute and storage)
  3. Define agreements and service management processes and procedures
  4. Identify the tools required and necessary development to faciliate pay-for-use service provisioning (e.g. billing function)
  5. Analyse the changes within a pre-production environment that would be needed to support and roll out the new functionalities in the production environment
  6. Evaluate legal, policy, and organisational issues around the full implementation of the pay-for-use model
  7. Develop a full production service.

Group Members

  • Sy Holsinger, EGI.eu (Chair)
  • Sergio Andreozzi, EGI.eu
  • Luis Alves, CSC
  • Owen Appleton, EGI.eu
  • Emanouil Atanassov, IICT-BAS
  • Bruce Becker, CSIR
  • David Blundell, 100 Percent IT Ltd.
  • Wilhelm Buehler, KIT-G
  • Isabel Campos, CSIC
  • Daniele Cesini, INFN-CNAF
  • Alessandro Costantini, INFN/Master-Up
  • Ivan Diaz, CESGA
  • Miroslav Dobrucky, IISAS
  • Feyza Eryol, TUBITAK ULAKBIM
  • Carlos Fernandez Sanchez, CESGA
  • Enol Fernández del Castillo, EGI.eu
  • Tiziana Ferrari, EGI.eu
  • Luciano Gaido, INFN
  • André Gemünd, Fraunhofer SCAI
  • Vaidas Giedrimas, Šiauliai University
  • John Gordon, STFC
  • Todor Gurov, IICT-BAS
  • Sigve Haug, ETH ZURICH
  • Helmut Heller, BADW
  • Ladislav Hluchý, IISAS
  • Janne Ignatius, CSC
  • Javier Jimenez, EGI.eu
  • Kostas Koumantaros, GRNET
  • Kaspars Krampis, IMCS UL
  • Antonio Lagana, UNIPG/Master-Up
  • Panos Louridas, GRNET
  • Giorgio Maggi, INFN-Bari
  • Jesus Marco, IFCA/CSIC
  • Per Öster, CSC
  • Stuart Pullinger, STFC
  • Roksana Różańska, CYFRONET
  • François Ruty, Luna Technologies
  • Serge Salamanka, UIIP NASB
  • Diego Scardaci, INFN
  • Horst Schwichtenberg, Fraunhofer SCAI
  • Alvaro Simon, CESGA
  • Vincenzo Spinoso, INFN-Bari
  • Mariusz Sterzel, CYFRONET
  • Tomasz Szepieniec, CYFRONET
  • Jelena Tamulienė, VU
  • Onur Temizsoylu, TUBITAK ULAKBIM
  • Viet Tran, IISAS

Technical Details

The following link provides an overview of the technical details of the resource and technology providers for the PoC.

How to Join

If you would like to participate in the Proof of Concept, contact policy@egi.eu

References

Retrieved from "https://wiki.egi.eu/w/index.php?title=VT_EGI_Pay-for-Use_PoC&oldid=87841"