Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI-Engage:TASK JRA1.3 Accounting"

From EGIWiki
Jump to navigation Jump to search
 
(16 intermediate revisions by 5 users not shown)
Line 1: Line 1:
{{Template:EGI-Engage menubar}} {{TOC_right}}  
{{Template:EGI-Engage menubar}} {{TOC_right}}  
[[Category:EGI-Engage]]
[[Category:EGI-Engage]]
'''Task leader: '''Stuart Pullinger/STFC <BR>
'''Task leader: '''Adrian Coveney/STFC <BR>
'''Task leader deputy: '''TBD
'''Task leader deputy: '''Greg Corbett/STFC


'''Mailing list:'''TBD  
'''Mailing list:'''TBD  
Line 48: Line 48:
|-
|-
| 3.1.1   
| 3.1.1   
| Cloud Accounting: Usage Record V0.4 (Benchmarks etc)
| '''Cloud Accounting: Usage Record V0.4 (Benchmarks etc)'''
| 03/15
| 03/15
| 04/15
| 07/16
| Planned
| Closed
|  
|  
|  
|  
|-
|-
| 3.1.2   
| 3.1.2   
| Storage Accounting: Contact sites and test of the current solution
| '''Storage Accounting: Contact sites and test of the current solution'''
| 03/15
| 03/15
| 06/15
| 10/15
| Planned
| Closed
|
|  
|  
| Fix rolled out and new format being loaded successfully.
|-
|-
| 3.1.3   
| 3.1.3   
| Cloud Accounting: Long running VMs
| '''Cloud Accounting: Long running VMs''' - Adding support for VMs that run for a period that crosses a month boundary so that their usage is apportioned to each month.
| 03/15
| 03/15
| 09/15
| 07/16
| Planned
| Open
|  
|  
|  
| Several attempts at a solution proposed. All rejected by developers. New solution being tested at central repository.
|-
|-
| 3.1.4   
| 3.1.4   
| SQL Parser
| '''ARC Parser'''
| 04/15
| 04/15
| 11/15
| 07/16
| Planned
| Open
|  
|  
|  
| Parser code written and tested against test data. Contacting sites to test.
|-
|-
| 3.1.5   
| 3.1.5   
| Data Accounting: proof of concept
| '''Data Accounting: proof of concept'''
| 06/15
| 06/15
| 01/16
| 01/16
| Planned
| Closed
|  
|  
|  
|  
|-
|-
| 3.1.6   
| 3.1.6   
| Storage Accounting: Usage Record V2.0
| '''Storage Accounting: Usage Record V2.0'''
| 07/15
| 07/16
| 02/16
| 10/16
| Planned
| Open
| 3.1.2
| 3.1.2
|  
|  
|-
|-
| 3.1.7   
| 3.1.7   
| XML Parser
| '''XML Parser'''
| 12/15
| 07/16
| 07/16
| 11/16
| Planned
| Planned
|  
|  
|  
| Requirements need developing.
|-
|-
| 3.1.8   
| 3.1.8   
| D3.3: Accounting Repository Release
| '''D3.3: Accounting Repository Release'''
| 02/16
| 02/16
| 02/16
| 02/16
| Planned
| Closed
| 3.1.1, 3.1.2, 3.1.3, 3.1.4, 3.1.5, 3.1.6
| 3.1.2, 3.1.5
|  
|  
|-
|-
| 3.1.9
| 3.1.9
| Data Accounting: minimal requirements implemented
| '''Data Accounting: minimal requirements implemented'''
| 02/16
| 02/16
| 09/16
| 09/16
| Planned
| Closed
| 3.1.5
| 3.1.5
|  
|  
|-
|-
| 3.1.10  
| 3.1.10  
| Storage Accounting: support new storage systems
| '''Storage Accounting: support new storage systems'''
| 03/16
| 09/16
| 10/16
| 02/17
| Planned
| Planned
| 3.1.6
| 3.1.6
Line 128: Line 128:
|-
|-
| 3.1.11   
| 3.1.11   
| D3.6: Analysis on techniques to manage big data on the EGI accounting system
| '''D3.6: Analysis on techniques to manage big data on the EGI accounting system'''
| 05/16
| 02/16
| 05/16
| 02/16
| Planned
| Closed
|
|  
|  
| Date moved much earlier than originally expected.
|-
|-
| 3.1.12   
| 3.1.12   
| Investigating Big Data tools to improve accounting repository
| '''Investigating Big Data tools to improve accounting repository'''
| 07/16
| 07/16
| 02/17
| 02/17
| Planned
| Closed
| 3.1.11
| 3.1.11
|  
|  
|-
|-
| 3.1.13   
| 3.1.13   
| Other kinds of parsers
| '''SQL parsers'''
| 08/16
| 08/16
| 03/17
| 03/17
| Planned
| Open
|
|  
|  
| Requirements need developing.
|-
|-
| 3.1.14  
| 3.1.14  
| D3.8: First data accounting prototype
| '''D3.8: First data accounting prototype'''
| 10/16
| 10/16
| 10/16
| 10/16
| Planned
| Closed
| 3.1.9
| 3.1.9
|  
|  
|-
|-
| 3.1.15  
| 3.1.15  
| Data Accounting: production requirements
| '''Data Accounting: production requirements'''
| 10/16
| 10/16
| 05/17
| 05/17
Line 168: Line 168:
|-
|-
| 3.1.16
| 3.1.16
| GPGPU Accounting
| '''GPGPU Accounting'''
| 10/16
| 10/16
| 05/17
| 05/17
| Planned
| Open
|
|  
|  
| Batch systems still do not support accounting for GPGPU usage. Looking into accounting for VM usage of GPGPUs.
|-
|-
| 3.1.17
| 3.1.17
| Support AAI evolution
| '''Support AAI evolution'''
| 01/17
| 01/17
| 08/17
| 08/17
| Planned
| Open
|  
|  
|  
|  
|-
|-
| 3.1.18
| 3.1.18
| D3.12: Second release of the Accounting Repository
| '''D3.12: Second release of the Accounting Repository'''
| 02/17
| 02/17
| 02/17
| 02/17
| Planned
| Closed
| 3.8, 3.9, 3.10, 3.12, 3.14
| 3.1.1, 3.1.3, 3.1.4, 3.1.6, 3.1.8, 3.1.9, 3.1.10, 3.1.12, 3.1.14
|  
|  
|-
|-
| 3.1.19
| 3.1.19
| D3.13: Report on Data Accounting
| '''D3.13: Report on Data Accounting'''
| 02/17
| 02/17
| 02/17
| 02/17
| Planned
| Open
| 3.1.15
| 3.1.15
|  
|  
|-
|-
| 3.1.20
| 3.1.20
| D3.15: Second data accounting prototype
| '''D3.15: Second data accounting prototype'''
| 06/17
| 06/17
| 06/17
| 06/17
| Planned
| Planned
| 3.15
| 3.1.15
|  
|  
|-
|-
| 3.1.21
| 3.1.21
| D3.17: Final release of the Accounting Repository
| '''D3.17: Final release of the Accounting Repository'''
| 08/17
| 08/17
| 08/17
| 08/17
Line 252: Line 252:
| 03/15
| 03/15
| 08/15
| 08/15
| Planned
| Done
|  
|  
|  
|  
Line 260: Line 260:
| 03/15
| 03/15
| 08/15
| 08/15
| Planned
| Done
|  
| 3.2.1
|  
|  
|-
|-
Line 269: Line 269:
| 01/16
| 01/16
| Planned
| Planned
|  
| 3.2.1
|  
|  
|-
|-
Line 285: Line 285:
| 04/16
| 04/16
| Planned
| Planned
|  
| 3.1.1, 3.1.3
|  
|  
|-
|-
| 3.2.6
| 3.2.6
| D3.5: First release of the new Accounting Portal deployed in production
| EGI Federated AAI Integration
| 04/15
| 04/16
| 04/16
| 04/16
| Planned
| Planned
|  
| Depends on outcome of AAI TF and EGI policy
|  
|  
|-
|-
| 3.2.7
| 3.2.7
| Define a complete API to get accounting data directly from the accounting portal
| D3.5: First release of the new Accounting Portal deployed in production
| 05/16
| 04/16
| 12/16
| 04/16
| Planned
| Planned
|  
| 3.2.2, 3.2.3, 3.2.4, 3.2.5
|  
|  
|-
|-
| 3.2.8
| 3.2.8
| Integrate data in maps for a graphical distribution of the information
| Define a complete API to get accounting data directly from the accounting portal
| 07/16
| 05/16
| 01/16
| 12/16
| Planned
| Planned
|  
|  
Line 318: Line 318:
| Planned
| Planned
|  
|  
|
|  
|-
|-
| 3.2.10
| 3.2.10
Line 333: Line 333:
| 02/17
| 02/17
| Planned
| Planned
|  
| 3.2.7, 3.2.8, 3.2.9, 3.2.10
|  
|  
|-
|-
Line 341: Line 341:
| 08/17
| 08/17
| Planned
| Planned
|  
| 3.1.5, 3.1.8, 3.1.9, 3.1.15, 3.1.18
|  
|  
|-
|-
Line 349: Line 349:
| 08/17
| 08/17
| Planned
| Planned
|  
| 3.1.16, 3.1.18
|  
|  
|-
|-
Line 357: Line 357:
| 08/17
| 08/17
| Planned
| Planned
|  
| 3.1.12, 3.1.18
|
|
|-
|-
| 3.2.19
| 3.2.15
| D3.19: Final release of the new Accounting Portal deployed in production
| D3.19: Final release of the new Accounting Portal deployed in production
| 08/17
| 08/17
| 08/17
| 08/17
| Planned
| Planned
|  
| 3.2.11, 3.2.12, 3.2.13, 3.2.14
|
|
|}
|}

Latest revision as of 13:00, 23 March 2017

EGI-Engage project: Main page WP1(NA1) WP3(JRA1) WP5(SA1) PMB Deliverables and Milestones Quality Plan Risk Plan Data Plan
Roles and
responsibilities
WP2(NA2) WP4(JRA2) WP6(SA2) AMB Software and services Metrics Project Office Procedures


Task leader: Adrian Coveney/STFC
Task leader deputy: Greg Corbett/STFC

Mailing list:TBD


Involved Partner

  • STFC
  • CESGA

Releases

Roadmap

Accounting Repository

Task number

(3.1.X)

Task name

Start date

(MM/YY)

Release Date

(MM/YY)

Status

(Open/Closed)

Dependencies from other tasks

(insert only task number)

Issues
3.1.1 Cloud Accounting: Usage Record V0.4 (Benchmarks etc) 03/15 07/16 Closed
3.1.2 Storage Accounting: Contact sites and test of the current solution 03/15 10/15 Closed Fix rolled out and new format being loaded successfully.
3.1.3 Cloud Accounting: Long running VMs - Adding support for VMs that run for a period that crosses a month boundary so that their usage is apportioned to each month. 03/15 07/16 Open Several attempts at a solution proposed. All rejected by developers. New solution being tested at central repository.
3.1.4 ARC Parser 04/15 07/16 Open Parser code written and tested against test data. Contacting sites to test.
3.1.5 Data Accounting: proof of concept 06/15 01/16 Closed
3.1.6 Storage Accounting: Usage Record V2.0 07/16 10/16 Open 3.1.2
3.1.7 XML Parser 07/16 11/16 Planned Requirements need developing.
3.1.8 D3.3: Accounting Repository Release 02/16 02/16 Closed 3.1.2, 3.1.5
3.1.9 Data Accounting: minimal requirements implemented 02/16 09/16 Closed 3.1.5
3.1.10 Storage Accounting: support new storage systems 09/16 02/17 Planned 3.1.6
3.1.11 D3.6: Analysis on techniques to manage big data on the EGI accounting system 02/16 02/16 Closed Date moved much earlier than originally expected.
3.1.12 Investigating Big Data tools to improve accounting repository 07/16 02/17 Closed 3.1.11
3.1.13 SQL parsers 08/16 03/17 Open Requirements need developing.
3.1.14 D3.8: First data accounting prototype 10/16 10/16 Closed 3.1.9
3.1.15 Data Accounting: production requirements 10/16 05/17 Planned 3.1.14
3.1.16 GPGPU Accounting 10/16 05/17 Open Batch systems still do not support accounting for GPGPU usage. Looking into accounting for VM usage of GPGPUs.
3.1.17 Support AAI evolution 01/17 08/17 Open
3.1.18 D3.12: Second release of the Accounting Repository 02/17 02/17 Closed 3.1.1, 3.1.3, 3.1.4, 3.1.6, 3.1.8, 3.1.9, 3.1.10, 3.1.12, 3.1.14
3.1.19 D3.13: Report on Data Accounting 02/17 02/17 Open 3.1.15
3.1.20 D3.15: Second data accounting prototype 06/17 06/17 Planned 3.1.15
3.1.21 D3.17: Final release of the Accounting Repository 08/17 08/17 Planned 3.1.13, 3.1.15, 3.1.16, 3.1.17, 3.1.18

Accounting Portal

Task number

(3.2.X)

Task name

Start date

(MM/YY)

Release Date

(MM/YY)

Status

(Planned/Done)

Dependencies from other tasks

(insert only task number)

Issues
3.2.1 Requirements collection 03/15 08/15 Done
3.2.2 D3.1: Technical design of the new Accounting Portal and implementation plan 03/15 08/15 Done 3.2.1
3.2.3 Modernize the accounting Portal with the adoption of technologies easier to maintain 06/15 01/16 Planned 3.2.1
3.2.4 Simplify access to some basic functionality. Avoid the use of complex forms for common statistics and get accounting information for some common queries 07/15 04/16 Planned
3.2.5 Support Cloud Usage Record V0.4 and accounting of long running VMs 07/15 04/16 Planned 3.1.1, 3.1.3
3.2.6 EGI Federated AAI Integration 04/16 04/16 Planned Depends on outcome of AAI TF and EGI policy
3.2.7 D3.5: First release of the new Accounting Portal deployed in production 04/16 04/16 Planned 3.2.2, 3.2.3, 3.2.4, 3.2.5
3.2.8 Define a complete API to get accounting data directly from the accounting portal 05/16 12/16 Planned
3.2.9 Integrate data in maps for a graphical distribution of the information 07/16 01/16 Planned
3.2.10 Integrate analytics to extract intelligence operation from the data 08/16 05/17 Planned
3.2.11 D3.10: Second release of the new Accounting Portal deployed in production 02/17 02/17 Planned 3.2.7, 3.2.8, 3.2.9, 3.2.10
3.2.12 Support Data Accounting 01/17 08/17 Planned 3.1.5, 3.1.8, 3.1.9, 3.1.15, 3.1.18
3.2.13 Support GPGPU Accounting 01/17 08/17 Planned 3.1.16, 3.1.18
3.2.14 Support Big Data tools 01/17 08/17 Planned 3.1.12, 3.1.18
3.2.15 D3.19: Final release of the new Accounting Portal deployed in production 08/17 08/17 Planned 3.2.11, 3.2.12, 3.2.13, 3.2.14