Difference between revisions of "EGI-Engage:TASK JRA1.3 Accounting"
Jump to navigation
Jump to search
(36 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
{{Template:EGI-Engage menubar}} {{TOC_right}} | {{Template:EGI-Engage menubar}} {{TOC_right}} | ||
[[Category:EGI-Engage]] | [[Category:EGI-Engage]] | ||
'''Task leader: ''' | '''Task leader: '''Adrian Coveney/STFC <BR> | ||
'''Task leader deputy: ''' | '''Task leader deputy: '''Greg Corbett/STFC | ||
'''Mailing list:'''TBD | '''Mailing list:'''TBD | ||
Line 22: | Line 22: | ||
Task number | Task number | ||
( | (3.1.X) | ||
! Task name | ! Task name | ||
Line 47: | Line 47: | ||
! Issues | ! Issues | ||
|- | |- | ||
| 3.1.1 | |||
| | | '''Cloud Accounting: Usage Record V0.4 (Benchmarks etc)''' | ||
| 03/15 | |||
| 07/16 | |||
| Closed | |||
| | | | ||
| | |||
|- | |||
| 3.1.2 | |||
| '''Storage Accounting: Contact sites and test of the current solution''' | |||
| 03/15 | |||
| 10/15 | |||
| Closed | |||
| | |||
| Fix rolled out and new format being loaded successfully. | |||
|- | |||
| 3.1.3 | |||
| '''Cloud Accounting: Long running VMs''' - Adding support for VMs that run for a period that crosses a month boundary so that their usage is apportioned to each month. | |||
| 03/15 | |||
| 07/16 | |||
| Open | |||
| | |||
| Several attempts at a solution proposed. All rejected by developers. New solution being tested at central repository. | |||
|- | |||
| 3.1.4 | |||
| '''ARC Parser''' | |||
| 04/15 | |||
| 07/16 | |||
| Open | |||
| | |||
| Parser code written and tested against test data. Contacting sites to test. | |||
|- | |||
| 3.1.5 | |||
| '''Data Accounting: proof of concept''' | |||
| 06/15 | |||
| 01/16 | |||
| Closed | |||
| | |||
| | |||
|- | |||
| 3.1.6 | |||
| '''Storage Accounting: Usage Record V2.0''' | |||
| 07/16 | |||
| 10/16 | |||
| Open | |||
| 3.1.2 | |||
| | |||
|- | |||
| 3.1.7 | |||
| '''XML Parser''' | |||
| 07/16 | |||
| 11/16 | |||
| Planned | |||
| | |||
| Requirements need developing. | |||
|- | |||
| 3.1.8 | |||
| '''D3.3: Accounting Repository Release''' | |||
| 02/16 | |||
| 02/16 | |||
| Closed | |||
| 3.1.2, 3.1.5 | |||
| | |||
|- | |||
| 3.1.9 | |||
| '''Data Accounting: minimal requirements implemented''' | |||
| 02/16 | |||
| 09/16 | |||
| Closed | |||
| 3.1.5 | |||
| | |||
|- | |||
| 3.1.10 | |||
| '''Storage Accounting: support new storage systems''' | |||
| 09/16 | |||
| 02/17 | |||
| Planned | |||
| 3.1.6 | |||
| | |||
|- | |||
| 3.1.11 | |||
| '''D3.6: Analysis on techniques to manage big data on the EGI accounting system''' | |||
| 02/16 | |||
| 02/16 | |||
| Closed | |||
| | |||
| Date moved much earlier than originally expected. | |||
|- | |||
| 3.1.12 | |||
| '''Investigating Big Data tools to improve accounting repository''' | |||
| 07/16 | |||
| 02/17 | |||
| Closed | |||
| 3.1.11 | |||
| | |||
|- | |||
| 3.1.13 | |||
| '''SQL parsers''' | |||
| 08/16 | |||
| 03/17 | |||
| Open | |||
| | |||
| Requirements need developing. | |||
|- | |||
| 3.1.14 | |||
| '''D3.8: First data accounting prototype''' | |||
| 10/16 | |||
| 10/16 | |||
| Closed | |||
| 3.1.9 | |||
| | |||
|- | |||
| 3.1.15 | |||
| '''Data Accounting: production requirements''' | |||
| 10/16 | |||
| 05/17 | |||
| Planned | |||
| 3.1.14 | |||
| | |||
|- | |||
| 3.1.16 | |||
| '''GPGPU Accounting''' | |||
| 10/16 | |||
| 05/17 | |||
| Open | |||
| | |||
| Batch systems still do not support accounting for GPGPU usage. Looking into accounting for VM usage of GPGPUs. | |||
|- | |||
| 3.1.17 | |||
| '''Support AAI evolution''' | |||
| 01/17 | |||
| 08/17 | |||
| Open | |||
| | | | ||
| | | | ||
|- | |||
| 3.1.18 | |||
| '''D3.12: Second release of the Accounting Repository''' | |||
| 02/17 | |||
| 02/17 | |||
| Closed | |||
| 3.1.1, 3.1.3, 3.1.4, 3.1.6, 3.1.8, 3.1.9, 3.1.10, 3.1.12, 3.1.14 | |||
| | | | ||
|- | |||
| 3.1.19 | |||
| '''D3.13: Report on Data Accounting''' | |||
| 02/17 | |||
| 02/17 | |||
| Open | |||
| 3.1.15 | |||
| | |||
|- | |||
| 3.1.20 | |||
| '''D3.15: Second data accounting prototype''' | |||
| 06/17 | |||
| 06/17 | |||
| Planned | |||
| 3.1.15 | |||
| | |||
|- | |||
| 3.1.21 | |||
| '''D3.17: Final release of the Accounting Repository''' | |||
| 08/17 | |||
| 08/17 | |||
| Planned | |||
| 3.1.13, 3.1.15, 3.1.16, 3.1.17, 3.1.18 | |||
| | | | ||
|} | |} | ||
Line 63: | Line 223: | ||
Task number | Task number | ||
( | (3.2.X) | ||
! Task name | ! Task name | ||
Line 79: | Line 239: | ||
Status | Status | ||
( | (Planned/Done) | ||
! | ! | ||
Line 88: | Line 248: | ||
! Issues | ! Issues | ||
|- | |- | ||
| 3.2.1 | |||
| | | Requirements collection | ||
| 03/15 | |||
| 08/15 | |||
| Done | |||
| | | | ||
| | | | ||
|- | |||
| 3.2.2 | |||
| D3.1: Technical design of the new Accounting Portal and implementation plan | |||
| 03/15 | |||
| 08/15 | |||
| Done | |||
| 3.2.1 | |||
| | | | ||
|- | |||
| 3.2.3 | |||
| Modernize the accounting Portal with the adoption of technologies easier to maintain | |||
| 06/15 | |||
| 01/16 | |||
| Planned | |||
| 3.2.1 | |||
| | | | ||
|- | |||
| 3.2.4 | |||
| Simplify access to some basic functionality. Avoid the use of complex forms for common statistics and get accounting information for some common queries | |||
| 07/15 | |||
| 04/16 | |||
| Planned | |||
| | |||
| | |||
|- | |||
| 3.2.5 | |||
| Support Cloud Usage Record V0.4 and accounting of long running VMs | |||
| 07/15 | |||
| 04/16 | |||
| Planned | |||
| 3.1.1, 3.1.3 | |||
| | |||
|- | |||
| 3.2.6 | |||
| EGI Federated AAI Integration | |||
| 04/16 | |||
| 04/16 | |||
| Planned | |||
| Depends on outcome of AAI TF and EGI policy | |||
| | |||
|- | |||
| 3.2.7 | |||
| D3.5: First release of the new Accounting Portal deployed in production | |||
| 04/16 | |||
| 04/16 | |||
| Planned | |||
| 3.2.2, 3.2.3, 3.2.4, 3.2.5 | |||
| | | | ||
|- | |||
| 3.2.8 | |||
| Define a complete API to get accounting data directly from the accounting portal | |||
| 05/16 | |||
| 12/16 | |||
| Planned | |||
| | |||
| | |||
|- | |||
| 3.2.9 | |||
| Integrate data in maps for a graphical distribution of the information | |||
| 07/16 | |||
| 01/16 | |||
| Planned | |||
| | |||
| | |||
|- | |||
| 3.2.10 | |||
| Integrate analytics to extract intelligence operation from the data | |||
| 08/16 | |||
| 05/17 | |||
| Planned | |||
| | |||
| | |||
|- | |||
| 3.2.11 | |||
| D3.10: Second release of the new Accounting Portal deployed in production | |||
| 02/17 | |||
| 02/17 | |||
| Planned | |||
| 3.2.7, 3.2.8, 3.2.9, 3.2.10 | |||
| | |||
|- | |||
| 3.2.12 | |||
| Support Data Accounting | |||
| 01/17 | |||
| 08/17 | |||
| Planned | |||
| 3.1.5, 3.1.8, 3.1.9, 3.1.15, 3.1.18 | |||
| | |||
|- | |||
| 3.2.13 | |||
| Support GPGPU Accounting | |||
| 01/17 | |||
| 08/17 | |||
| Planned | |||
| 3.1.16, 3.1.18 | |||
| | |||
|- | |||
| 3.2.14 | |||
| Support Big Data tools | |||
| 01/17 | |||
| 08/17 | |||
| Planned | |||
| 3.1.12, 3.1.18 | |||
| | |||
|- | |||
| 3.2.15 | |||
| D3.19: Final release of the new Accounting Portal deployed in production | |||
| 08/17 | |||
| 08/17 | |||
| Planned | |||
| 3.2.11, 3.2.12, 3.2.13, 3.2.14 | |||
| | |||
|} | |} |
Latest revision as of 14:00, 23 March 2017
Task leader: Adrian Coveney/STFC
Task leader deputy: Greg Corbett/STFC
Mailing list:TBD
Involved Partner
- STFC
- CESGA
Releases
Roadmap
Accounting Repository
Task number (3.1.X) |
Task name |
Start date (MM/YY) |
Release Date (MM/YY) |
Status (Open/Closed) |
Dependencies from other tasks (insert only task number) |
Issues |
---|---|---|---|---|---|---|
3.1.1 | Cloud Accounting: Usage Record V0.4 (Benchmarks etc) | 03/15 | 07/16 | Closed | ||
3.1.2 | Storage Accounting: Contact sites and test of the current solution | 03/15 | 10/15 | Closed | Fix rolled out and new format being loaded successfully. | |
3.1.3 | Cloud Accounting: Long running VMs - Adding support for VMs that run for a period that crosses a month boundary so that their usage is apportioned to each month. | 03/15 | 07/16 | Open | Several attempts at a solution proposed. All rejected by developers. New solution being tested at central repository. | |
3.1.4 | ARC Parser | 04/15 | 07/16 | Open | Parser code written and tested against test data. Contacting sites to test. | |
3.1.5 | Data Accounting: proof of concept | 06/15 | 01/16 | Closed | ||
3.1.6 | Storage Accounting: Usage Record V2.0 | 07/16 | 10/16 | Open | 3.1.2 | |
3.1.7 | XML Parser | 07/16 | 11/16 | Planned | Requirements need developing. | |
3.1.8 | D3.3: Accounting Repository Release | 02/16 | 02/16 | Closed | 3.1.2, 3.1.5 | |
3.1.9 | Data Accounting: minimal requirements implemented | 02/16 | 09/16 | Closed | 3.1.5 | |
3.1.10 | Storage Accounting: support new storage systems | 09/16 | 02/17 | Planned | 3.1.6 | |
3.1.11 | D3.6: Analysis on techniques to manage big data on the EGI accounting system | 02/16 | 02/16 | Closed | Date moved much earlier than originally expected. | |
3.1.12 | Investigating Big Data tools to improve accounting repository | 07/16 | 02/17 | Closed | 3.1.11 | |
3.1.13 | SQL parsers | 08/16 | 03/17 | Open | Requirements need developing. | |
3.1.14 | D3.8: First data accounting prototype | 10/16 | 10/16 | Closed | 3.1.9 | |
3.1.15 | Data Accounting: production requirements | 10/16 | 05/17 | Planned | 3.1.14 | |
3.1.16 | GPGPU Accounting | 10/16 | 05/17 | Open | Batch systems still do not support accounting for GPGPU usage. Looking into accounting for VM usage of GPGPUs. | |
3.1.17 | Support AAI evolution | 01/17 | 08/17 | Open | ||
3.1.18 | D3.12: Second release of the Accounting Repository | 02/17 | 02/17 | Closed | 3.1.1, 3.1.3, 3.1.4, 3.1.6, 3.1.8, 3.1.9, 3.1.10, 3.1.12, 3.1.14 | |
3.1.19 | D3.13: Report on Data Accounting | 02/17 | 02/17 | Open | 3.1.15 | |
3.1.20 | D3.15: Second data accounting prototype | 06/17 | 06/17 | Planned | 3.1.15 | |
3.1.21 | D3.17: Final release of the Accounting Repository | 08/17 | 08/17 | Planned | 3.1.13, 3.1.15, 3.1.16, 3.1.17, 3.1.18 |
Accounting Portal
Task number (3.2.X) |
Task name |
Start date (MM/YY) |
Release Date (MM/YY) |
Status (Planned/Done) |
Dependencies from other tasks (insert only task number) |
Issues |
---|---|---|---|---|---|---|
3.2.1 | Requirements collection | 03/15 | 08/15 | Done | ||
3.2.2 | D3.1: Technical design of the new Accounting Portal and implementation plan | 03/15 | 08/15 | Done | 3.2.1 | |
3.2.3 | Modernize the accounting Portal with the adoption of technologies easier to maintain | 06/15 | 01/16 | Planned | 3.2.1 | |
3.2.4 | Simplify access to some basic functionality. Avoid the use of complex forms for common statistics and get accounting information for some common queries | 07/15 | 04/16 | Planned | ||
3.2.5 | Support Cloud Usage Record V0.4 and accounting of long running VMs | 07/15 | 04/16 | Planned | 3.1.1, 3.1.3 | |
3.2.6 | EGI Federated AAI Integration | 04/16 | 04/16 | Planned | Depends on outcome of AAI TF and EGI policy | |
3.2.7 | D3.5: First release of the new Accounting Portal deployed in production | 04/16 | 04/16 | Planned | 3.2.2, 3.2.3, 3.2.4, 3.2.5 | |
3.2.8 | Define a complete API to get accounting data directly from the accounting portal | 05/16 | 12/16 | Planned | ||
3.2.9 | Integrate data in maps for a graphical distribution of the information | 07/16 | 01/16 | Planned | ||
3.2.10 | Integrate analytics to extract intelligence operation from the data | 08/16 | 05/17 | Planned | ||
3.2.11 | D3.10: Second release of the new Accounting Portal deployed in production | 02/17 | 02/17 | Planned | 3.2.7, 3.2.8, 3.2.9, 3.2.10 | |
3.2.12 | Support Data Accounting | 01/17 | 08/17 | Planned | 3.1.5, 3.1.8, 3.1.9, 3.1.15, 3.1.18 | |
3.2.13 | Support GPGPU Accounting | 01/17 | 08/17 | Planned | 3.1.16, 3.1.18 | |
3.2.14 | Support Big Data tools | 01/17 | 08/17 | Planned | 3.1.12, 3.1.18 | |
3.2.15 | D3.19: Final release of the new Accounting Portal deployed in production | 08/17 | 08/17 | Planned | 3.2.11, 3.2.12, 3.2.13, 3.2.14 |