VT VAPOR:Progress Report

Progress Report

VT Reporting guidelines

Reporting period: 3 September - 17 September

Progress

• Continue the customization of the JobMonitor tool by partner IPHC to support a multi-vo running context. Several phone conferences with the developer.
• Global actions on the web application:
  • Continued to improve the ergonomy and look and feel of web pages: enrich helps, add customizable tooltips.
  • Tests have revealed weaknesses in the application development (parameters not tested, no error trace...): an important action has consisted in improve the safety of the application and adding much more log traces to be able to follow up on the application life.
• VO Operations, reports of availability of resources:
  • Added some reports in the web pages to show the "running ratio" R(R/W)
  • Almost completed the Lavoisier VO-views to consolidate data from the BDII and GOCDB.
• VO data management: continuation of the study on VO Data management, very rich discussion with developers of GFAL2 in GGUS ticket #97076

Plans for next period

• Face to face meeting with the developers of the EGI Operation Portal to discuss future steps.
• Add 2 new web pages to show (1) the current status of resources supporting the VO (CE, SE, WMS, VOMS, LFC), and (2) the list of resources which status is currently not in producton (consolidate BDII/GOCDB statuses)

Problems we encounter, but can solve

The lack of experience of the main developer is being handled and things are imporving, altough the code production remains slower than expected. Action: heavy support is being provided to him.

Problems and issues we need external help with

None.

Reporting period: 21 August - 3 September

Progress

• Follow up of the improvement of the ergonomy and look and feel of web pages.
• VO Operations, reports of availability of resources:
  • completed the web page and code dedicated to the production of white list of CEs based on job monitoring reports.
  • Writing of VO-views to consolidate the data from the BDII and GOCDB, using the Lavoisier data integration service. Numerous interactions with the developers of the tool.
• VO data management: continuation of the study on VO Data management (dark data in particular), discussions with sites admins (FR, UK) to refine the procedures

Plans for next period

• VO Operations, reports of availability of resources: still some reports to add in the web pages to show the "running ratio" R(R/W)

Problems we encounter, but can solve

• The lack of experience of the main developer is being handled and things are imporving, altough the code production remains slower than expected. Action: heavy support is being provided to him.
• Complexity of the Lavoisier data integration service, but the development team is very helpful and reactive.

Problems and issues we need external help with

None.

Reporting period: 7 August - 20 August

Progress

• Work with partner IPHC to upgrade tool JobMonitor to support a multi-vo environment. Several phone conferences.
• Improvements of the ergonomy of web pages.
• Improve the management of misc. errors.
• VO Operations, reports of availability of resources:
  • development of new web pages to report running and waiting jobs, and ratio R(R/W).
• VO data management:
  • testing of scripts to detect and clean up VO dark data
  • discussions held with sites admins (FR IPHC, UK QMUL) in order to refine the procedure to deal with dark data and lost files. Study of good practices from HEP VOs.

Plans for next period

• VO Operations management features (resources availability): add new reports on ratio R(R/W), complete the production of the white list of CEs.
• Continue study on VO Data management (dark data).

Problems we encounter, but can solve

Lack of experience of the main developer in terms of development good practices and application design. This has slown down the activity but this is handled.

Problems and issues we need external help with

None.

Reporting period: 24 July - 6 August

Progress

• This period was used to make a strong focus on code quality including code review, cleaning up of code, and the first commit of the current version after code and environment cleaning up.
• Documentation: write a document to describe the application environment and installation procedure.

Development:

• VO Operations management features
  • resources availability: continue development of reports on running and waiting jobs, and ratio R(R/W).
  • VO data management: start development of scripts to detect and clean up VO dark data.
• Focus on styles and look and feel of web pages

Plans for next period

Complete VO Operations management features (resources availability): reports on running and waiting jobs, and ratio R(R/W); white list of CEs Continue work on VO Data management (dark data).

Problems we encounter, but can solve

Lack of experience of the main developer in terms of development good practices and application design. This has slown down the activity but this is handled.

Problems and issues we need external help with

None.

Reporting period: Jun. 26th - Jul. 23rd

Progress

Low activity due to summer vacation period: 2 weeks vacation for Flavien Forestier (developer) and 3.5 weeks for Franck Michel (project manager) => 10 days work for Flavien and 2 days for Franck.

• VO Operations management features (resources availability) :
  • continue development of web pages to view results of the Job Monitor tool of IPHC partner: chart for the history report, table view by computing element.
  • development of web page to view results of the CE monitor: view of number of running, waiting jobs and ration R(R/W) (chart)
  • discuss the parameters of the white list of CEs with IPHC.
• Deployment of the dev and test environment on a virtual machine.

Plans for next period

• Refine existing with better ergonomy and presentation
• Focus on styles and look and feel of web pages
• Display a white list of CEs

Problems we encounter, but can solve

None.

Problems and issues we need external help with

None.