Difference between revisions of "Fedcloud-tf:WorkGroups:Scenario4"

From EGIWiki
Jump to: navigation, search
(Cloud Accounting Draft Usage Record)
(Redirected page to Federated Cloud Accounting)
(150 intermediate revisions by 7 users not shown)
Line 1: Line 1:
{{Fedcloud-tf:Menu}} {{Fedcloud-tf:WorkGroups:Menu}} {{TOC_right}}
== Scenario 4: Accounting across Resource Providers ==
<font color="red">Leader: Alison Packer and John Gordon, STFC</font>
== Scenario collaborators ==
{| border="1"
|Scenario leader
|Alison Packer
| OeRC
| Matteo Turilli
| Daniele Lezzi
== Plan ==
To account for resource usage across multiple resource providers we need to define:
# the elements to be accounted for and how accounting data may be gathered. (This relates to the statement  “Resource providers agree and follow the same rules for accounting the resource usage”.)
# how accounting data will be published
# what is required from this data (for VM users, Resource Providers, VO Managers, others?)
== Cloud Accounting Draft Usage Record ==
The latest draft schema attempts to map the Cloud Accounting Usage Record fields to fields used by Venus-C and OpenNebula. [https://oerc.basecamphq.com/projects/7732005/file/115130463/Current%20Version-CloudAccountingUR.pdf Draft - Cloud Accounting Usage Record]
# Many of the properties defined in the Usage Record may be set to NULL, this is based on the assumption that different Resource Providers may want to publish different sets of data, but we want only one Usage Record, usable by all.
# The "Charge" property is no longer included in the Cloud Accounting Usage Record as the cloud accounting records may be used by Resource Providers to charge for their service, but the charging mechanism would be Resource Provider specific and should therefore be separated from accounting.
# The fields for GlobalUserName, VirtualOrganization, Projectname and Group relate to an existing method of authorizing grid users, if this authorization mechanism were to be used for cloud usage as well then these would provide a method of associating VM usage with a user and then querying usage in a portal could be based on user, VO etc.
# A VM might be in one of three states: started, stopped and suspended.  I think we would expect to gather many VM records whilst a VM is running, so I propose using the existing UR “Status” property for this – it may be possible to extend the implementation of this property to include some VM-specific statuses if these existing ones are not deemed appropriate.
# We need a method of identifying where a specific running VM is – there are many ways to do this, I suggest that the RecordIdentity might contain a concatenation of some of the other fields in the record – e.g. TimeStamp SiteName MachineName ImageID  - other suggestions welcome.
# We have job accounting and storage accounting systems and their related usage records already - we therefore account for jobs and storage using these existing records/methods.
==== Compute Accounting Record ====
To help define what we will account for the existing [https://twiki.cern.ch/twiki/pub/EMI/ComputeAccounting/CAR-EMI-tech-doc-0.8.doc Compute Accounting Record] could be extended. 
==== Previous version: ====
[https://oerc.basecamphq.com/projects/7732005/file/105129409/Draft-CloudAccountingUsageRecord-v1.pdf Previous Draft - Cloud Accounting Usage Record]).
== Publishing Accounting Data ==
==== Cloud Accounting System ====
The software and technology choices for the new APEL system (which receives compute accounting records) will be used to provide a ‘testbed’ so that the resource providers can send records to a Cloud Accounting System at STFC.  The software is written in Python and the data is stored in a MySQL database, the component parts of the system are:
# Record loader
# MySQL Database
More detail available in the Cloud Accounting System document here:
[https://oerc.basecamphq.com/projects/7732005/file/111685266/The%20Cloud%20Accounting%20System.pdf Cloud Accounting System]
==== Work on providing OpenNebula Accounting Records ====
Contact made with "VENUS" project developers who have developed a system for extracting accounting data.  Detail on the Cloud Accounting System (above) provided to them, requested more detail on their implementation.
== Feedback from Resource Providers ==
{| border="1"
!What is measured
|Oxford/Eucalyptus (no longer running)
|Number of VM instantiated - Type of VM - Amount of time each VM has been running - Number of Elastic Block Storage volumes created - Amount of time the EBS volumes have been owned - Times the EBS volumes have been mounted - Binding between EBS volumes and VMs - Number of OS images loaded in the S3 repository - Amount of time the OS images have been loaded into the S3 repository
| (Have reviewed UR)

Latest revision as of 13:09, 8 June 2015