Difference between revisions of "Fedcloud-tf:WorkGroups:Scenario4"
Line 3: | Line 3: | ||
== Scenario 4: Accounting across Resource Providers == | == Scenario 4: Accounting across Resource Providers == | ||
Leader: Stuart Pullinger, STFC | |||
== Scenario collaborators == | == Scenario collaborators == |
Revision as of 00:35, 8 December 2014
Main | Roadmap and Innovation | Technology | For Users | For Resource Providers | Media |
Scenario 4: Accounting across Resource Providers
Leader: Stuart Pullinger, STFC
Scenario collaborators
Role | Institution | Name |
---|---|---|
Scenario leader | STFC | Alison Packer |
Collaborator | OeRC | Matteo Turilli |
Collaborator | BSC | Daniele Lezzi |
Plan
To account for resource usage across multiple resource providers we need to define:
- the elements to be accounted for and how accounting data may be gathered. (This relates to the statement “Resource providers agree and follow the same rules for accounting the resource usage”.)
- how accounting data will be published
- what is required from this data (for VM users, Resource Providers, VO Managers, others?)
Latest meeting to discuss Cloud Accounting Data
A team from the Fedcloud task met on 25th July and 1st August 2013 to discuss the data received so far from the Resource Providers and where improvements should be made. The outcome of these meetings is here: [1]
Cloud Accounting Usage Record
A new version of the Cloud Accounting Usage Record should be used with a new version of SSM (Version 2.0), further information is here: New Cloud Accounting Usage Record
Previous versions
The previous version of the Cloud Accounting Usage Record is here: Previous Cloud Accounting Usage Record v.1
This draft schema attempts to map the Cloud Accounting Usage Record fields to fields used by Venus-C and OpenNebula. Version 3 - Cloud Accounting Usage Record
Compute Accounting Record
To help define what we will account for the existing Compute Accounting Record has been used.
Publishing Cloud Accounting Records
Cloud Accounting System
The software and technology choices for the new APEL system (which receives compute accounting records) will be used to provide a ‘testbed’ so that the resource providers can send records to a Cloud Accounting System at STFC. The software is written in Python and the data is stored in a MySQL database, the component parts of the system are:
- SSM
- Record loader
- MySQL Database
More detail available in the Cloud Accounting System document here: Cloud Accounting System
Publishing Records
To publish records to the central Cloud Accounting Database resource providers will need to install and configure SSM 2.0 and also create messages containing records which are sent using SSM. The information below has links to scripts for creating these messages for OpenNebula and Openstack sites. There is also information on the installation of SSM 2.0 below.
SSM Package for publishing records
The new version of SSM (2.0) should be retrieved from here [2]
There are versions for SL5 and SL6.
The EGI test message broker network will be switched off in April 2014. The FedCloud will move to using a test queue on the production brokers from 1st April 2014. Sites will need to change 3 values in their SSM configuration file (/etc/apel/sender.cfg) on 1st April:
[broker] ...network: TEST-NWOBnetwork: PROD ...use_ssl: falseuse_ssl: true ... [messaging] ...destination: /queue/global.accounting.cloudtest.CENTRALdestination: /queue/global.accounting.test.cloud.central ...
1st April Update: Sites are reporting problems using 'use_ssl: true' They receive a Java error message like this:
ssm.ssm2 - WARNING - Error message received: java.lang.SecurityException: User name [null] or password is invalid. No user for client certificate: CN=...
This reflects a problem with the configuration of the message broker network. Please use 'use_ssl: false' in your configuration until further notice.
A full System Administrator's guide is here [3]. A more detailed guide is here [4]
Terminology:
- A message is one file which is sent and received by the SSM. Usually a message will contain a number of records (eg 1000)
- A record corresponds to one row in the database. It contains a number of key-value pairs as specified in the table below
- The header in each message tells the server which type of records are in that message. You need one header per message, so one header per file.
Cloud Accounting Message Format for use with SSM 2.0
Header APEL-cloud-message: v0.2
Key | Value | Description | Mandatory |
---|---|---|---|
VMUUID | string | Virtual Machine's Universally Unique IDentifier | Yes |
SiteName | string | Sitename, e.g. GOCDB Sitename | Yes |
MachineName | string | VM Id | |
LocalUserId | string | Local username | |
LocalGroupId | string | Local groupname | |
GlobalUserName | string | User's X509 DN | |
FQAN | string | User's VOMS attributes | |
Status | string | Completion status - started, completed, suspended | |
StartTime | int | Must be set if Status = Started (epoch time) | |
EndTime | int | Must be set if Status = completed (epoch time) | |
SuspendDuration | int | Set when Status = suspended (seconds) | |
WallDuration | int | Wallclock - actual time used (seconds) | |
CpuDuration | int | CPU time consumed (seconds) | |
CpuCount | int | Number of CPUs allocated | |
NetworkType | string | Description | |
NetworkInbound | int | GB received | |
NetworkOutbound | int | GB sent | |
Memory | int | Memory allocated to the VM (MB) | |
Disk | int | Disk allocated to the VM (GB) | |
StorageRecordId | string | Link to associated storage record | |
ImageId | string | Image ID | |
CloudType | string | e.g. OpenNebula, Openstack |
End of record: %%
Example Message
APEL-cloud-message: v0.2 VMUUID: https://cloud.cesga.es:3202/compute/47f74797-e9c9-46d7-b28d-5f87209239eb 2013-02-25 17:37:27+00:00 SiteName: CESGA MachineName: one-2421 LocalUserId: 19 LocalGroupId: 101 GlobalUserName: NULL FQAN: NULL Status: completed StartTime: 1361813847 EndTime: 1361813870 SuspendDuration: NULL WallDuration: NULL CpuDuration: NULL CpuCount: 1 NetworkType: NULL NetworkInbound: 0 NetworkOutbound: 0 Memory: 1000 Disk: NULL StorageRecordId: NULL ImageId: NULL CloudType: OpenNebula %% ...another cloud record... %% ... %%
OpenNebula resource providers
Records may be retrieved from OpenNebula databases using the cloudacc script from EGI-FCTF/opennebula-cloudacc which creates a file in the message format which can be sent using the SSM 2.0.
Current stable version: 4.6.1
https://github.com/EGI-FCTF/opennebula-cloudacc/archive/4.6.1.tar.gz https://github.com/EGI-FCTF/opennebula-cloudacc/archive/4.6.1.zip
Openstack resource providers
Steps
- Install the software
- Check out the doc (man osssmrc)
- Edit /etc/osssmrc
- activate the osssm service
Openstack VM statuses
Openstack status | Mapped SSM status | Description |
---|---|---|
active | started | VM is booted and running |
build | started | VM is in building phase, not yet available |
deleted | completed | the VM was just deleted (transient state) |
error | error | the VM is in error state, the VM may still consume resources |
hard_reboot | started | the VM is in a reset phase |
migrating | started | the VM is moving from one host to another |
paused | paused | the VM process was stopped (as with a kill -STOP). It does not consume any CPU more, but still RAM and disk |
reboot | started | the VM is proceeding an ACPI power stop/start |
rebuild | started | performing a shutdown, a re-image, and a reboot |
confirming_resize | started | the VM has been resized and nova waits confirmation from the user to leave it in latest state |
rescue | started | the VM has been put into **rescue** mode (VM disk attached as second drive to a newly booted image) |
resize | started | the VM is being resized (changing its flavor) |
revert_resize | started | the user did not confirm the resize and the VM is falling back to the previous state |
password | started | xxx |
verify_resize | started | openstack is performing checks on the resized VM |
shutoff | completed | the VM is powered off: no more CPU/RAM consumed, still disk |
suspended | suspended | the VM has performed a suspend to disk, it consumes no more CPU/RAM, still disk (plus memory image) |
terminated | completed | xxx |
stopped | completed | the VM has performed a proper shutdown on user request: no more CPU/RAM consumed, still disk |
saving | started | the VM is being snapshotted to create a new image |
Resources
- binary packages for RHEL and Debian based systems
- Installation documentation
- Changelog
- osssm sources (Github).
Resource Providers published to Cloud Accounting Database
Using SSM 2.0
- Current list of Sites: http://goc-accounting.grid-support.ac.uk/cloudtest/cloudsites2.html
- VMs during last hour: http://goc-accounting.grid-support.ac.uk/cloudtest/vmshour2.html
- VMs during last day (long list may make your browser choke): http://goc-accounting.grid-support.ac.uk/cloudtest/vms2.html
Feedback from Resource Providers
VM | What is measured |
---|---|
Oxford/Eucalyptus (no longer running) | Number of VM instantiated - Type of VM - Amount of time each VM has been running - Number of Elastic Block Storage volumes created - Amount of time the EBS volumes have been owned - Times the EBS volumes have been mounted - Binding between EBS volumes and VMs - Number of OS images loaded in the S3 repository - Amount of time the OS images have been loaded into the S3 repository |
OpenNebula | (Have reviewed UR) |