EGI-InSPIRE:Na3 Scrap room
This page is the "internal wiki" of NA3. It serves as an area where ideas, suggestions, useful information, links can be collected. The activity management contributes, as well as monitors the content of the page and incorporates contents from here into the EGI user support processes.
Please note that because this page can be read by anybody, sensitive information should be collected in separate documents with restricted accessibility (e.g. a GoogleDoc) and only links to these documents should be inserted here!
RT Requirements vs Market place
In terms of requirements solving market, there is a suggestion to make access right to all SSO accounts to take the ticket (aka, become the Owner of the ticket). The Owner could indicate that ticket/requirement was taken and is in the solving process after the solution will be provided by the Owner.
New UCST Web Pages - under construction
Ideas and suggestions for EGI User & Community Support Team web-pages. The UCST’s intent is to create a new set of web pages that are accessed initially from the EGI.eu website, thus simplifying the ‘User’ experience and entry into all Support services through a single entry point. This area is being used for the development of the new site prior to going live.
Check this page.
Owner of this section is Richard McLennan, UCST
This section has been established in order to capture ideas and suggestions for EGI User Community workshops - all inputs should be posted to email@example.com
A key element of the EGI task in its contribution towards “enabling collaboration and exploring research opportunities” is achieved firstly through the twice yearly Technical and User Forums and secondly, through other meetings and workshops. The User Forum held in April 2011 in Vilnius, Lithuania revealed a growing desire amongst the user communities for such ‘collaboration and exploration’ to be achieved through smaller and more focused meetings which should be much more involving for individual specialist users. Furthermore, feedback drawn from requirements submitted through EGI’s Requirements Tracker (RT) system reveals that there is a call for carefully targeted events that must be tailored to the needs of specific parts of the community, either to ‘inform and educate’ or more frequently, to help 'clarify and specify' what are initially no more than vaguely stated requirements.
The EGI RT system has been operating since September 2010. By September 2011 EGI.eu’s User Community Support (UCS) team had acquired numerous user requests for new features and new configurations to be added to the EGI supported infrastructure. The total number of such requests submitted to UCST by Jun 2011 stood at 233. All of the tickets submitted are analysed as a matter of course and distilled into general themes in order to drive them towards successful resolution through appropriate channels. Invariably, there is a great deal of work that still needs to be completed in order to translate these general requirement ‘Themes’ into solutions. In some cases, the solution is one of collective training and education while in others, work must start by producing a clear, tight and unambiguous ‘Statements of Requirement’ that can be effectively passed as guidance to Technology Providers so that they may in turn develop the needed solutions. What is clear is that a most practical way forwards is through the use of targeted Workshops whereby interested parties can be brought together for brainstorming, interactive learning, sharing best practices, building relationships, data gathering and problem solving.
In November 2011 EGI Director launched the NGI International Liaison Officer initiative as a method to improve the coordination between EGI and its user community. Though a shortage of time precluded any discussion on 'Community Workshops', the slides for that session remain relevant and can be found here (Slides).
Aims and Outcomes
In order to contribute to the creation of ‘Sustainable user services for sustainable user communities’, the UCST will continuously plan and conduct targeted workshops with the following two principal outcomes in mind:
- Training, Education & Standards. T&E Workshops shall focus on sharing practices and procedures so that EGI as a service provider and user communities (as service users) can work towards increasingly common standards. As a by-product, UCST will aim to publish the standards as EGI reference documentation.
- Clarification of requirements. UCST shall assemble focus groups to discuss, brainstorm and clarify specific RT ‘Themes’ such that technical details that will eventually be incorporated into ‘Statements of Requirement’ can start to be properly defined. Such SORs will subsequently be submitted to Technology Providers in the normal way via the Technical Coordination Board route.
The programme detailed in this document provides an historical record of past workshops as well as a forward looking plan of scheduled future events.
Workshop Events and Themes
Recognising that all such events represent a cost in terms of resources (be they financial, human or both), every workshop event shall aim to be cost effective by being constrained to be as small and short as practicable given the outcome that needs to be achieved. Once the ‘Theme’ or training purpose of a workshop has been identified, the following key steps shall form the core process for planning the events:
- Definition of Workshop Goal(s) (the desired Outcomes).
- Selection of appropriate target group or specific individual attendees.
- Consider collaboration with another organisation with parallel interests - possibly run dual events 'back to back'
- Selection of Venue for event – there may be advantages in conducting the event centrally near EGI.eu offices or remotely with an NGI.
- 'Communicating a plan to the Audience' - the concept of the event needs to be marketed and attendees need to be enrolled. The event, together with a working Agenda, needs to be published (inc Training Marketplace).
- Documentation of findings and follow up plan – training workshops will result in documented standards and processes; fact finding events will ultimately lead to documented SORs.
Conduct of Workshops
UCST workshop events will normally be limited to a maximum of 30 participants and last no more than 2 working days. Events focused on training can be expected to comprise groups of nearer 30 people and be shorter in duration, while those seeking to gather hard data for generating ‘SORs’ or other documents/guidance are likely to be smaller groups (closer to 12-15 – for larger gatherings, effectiveness will be achieved through the use of ‘breakout sessions’) and may last up to 2 days. Breakout sessions to stimulate critical discussion and hence achieve output are a recognised element of successful workshop events.
- General details of EGI Workshops should be published using:
- The EGI events calendar and blog
- The co-hosting institution's website.
- the Training Marketplace website (Title of event, description, venue, main presenters, organiser URL etc)
- Specific information relating to the programme of an event together with supporting training material will be published through INDICO.
Workshops will routinely start with some form of ‘scene setting’ introduction which will normally be briefed by UCST staff. Case studies will be used as much as possible and participants shall be encouraged to deliver 10-15 minute presentations to present their view/position.
There are a number of tools available to assist in preparation of workshops and these will be used extensively to ensure attendance, productivity and effectiveness of the events are optimised.
- RT: Requirements Tracking system gathers user inputs from which UCST analysis can derive ‘Themes’ and some of the broad issues that need to be resolved.
- User / customer surveys: Zoomerang survey tool will be used to refine the content and agenda that needs to be covered during intended workshops (as well as confirming the broader support for the proposed theme).
- NGIs and VRCs will be directly canvased for their support/perceived need for issues to be more fully explored.
- Scheduling: Doodle polls are used extensively as the means for identifying the optimum timing for such events.
Table 1 beneath details the intended Workshop events for the period up to Dec 2011 and readers are invited to provide constructive comments for additions, changes and deletions as applicable by e-mail to Richard.firstname.lastname@example.org . Events planned for the 2011 Technical Forum are described in more detail in Table 2.
Table 1: UCST Workshops Phase 1 – Period 1 June to 15 Dec 2011
|Date||Theme||Remarks (supporting evidence etc)||Target Audience:||Outcome/Desired Outcome|
|13 May 2011||MAPPER/PRACE/EGI collaboration||Initial presentation agreed to form a ‘Task Force’ to drive for a demonstration of MAPPER through 2 applications||MAPPER Task Force (MTF) and leadership||SOR. Definition of strategic objectives for activity plus clarification of customers and their requirements. Outcome: MTF Wiki as container for MAPPER documents.|
|Sep 2011||Portal technologies for EGI Communities||Explore emerging technologies and requirements for development of portals, portlets and i-frames||TF11 event (see Table 2)||T, E & S. Compiled table of Portal system features.|
|Sep 2011||Data management – best practices and requirements||Present emerging data management issues and explore avenues for further developments as seen from user perspective.||TF11 event (see Table 2); Technology providers (developers) and users.||T, E & S. Draft SORs|
|Sep 2011||Virtual Research Communities||Promote available tools/services; explain RT and requirements gathering; seek VRC involvement in providing / adding local tools to EGI ‘service catalogue’.||TF11 event (see Table 2)- VRC leadership.||T, E & S. Train and Educate – establish best practice|
|Sep 2011||EGI / NGI Roadshows – to include "Training tools, services and requirements”||How to bring in more users and communities? Roadshow type event to promote EGI services and tools, and to seek NGI feedback on expand the customer base. To inc the new Training Marketplace. In particular, to enthuse NGIs to start encouraging their communities to load all potential Training and supporting material.||TF11 event (see Table 2). Multi event type workshop to be delivered broadly. Note this can be seen as a ‘Train the Trainer’ function.||T, E & S|
|How to set-up and manage successful grid software projects. Develop user-centric, usability-conscious software applications (apps) for the grid environment.||Especially relevant within an open-source/open-development process, ie bringing the Google/Apple model to the Grid Dissemination and promotion aspects should be covered||App developers, App consumers||T, E & S|
|Train research and education institutions to implement EGI gadgets and other applications to implement a support resource for users (users with an emphasis on bringing new users to DCI at a campus level.||Discussed with Stephen Winter at Westminster Uni Jun 2011. Explore generating a focal point (website) that institutions could point their students to which would easily answer all questions about getting started even down to - how do I get started with such-and-such an app.
Such a session would provide EGI with some critical feedback on existing gadgets.
|WS is aimed at more senior people within NGIs and institutions responsible for planning the implementation of support solutions.||T, E & S|
|Workload management - brokers, pilot jobs or direct submission||RT 918. How many communities use pilot jobs? Should this really be part of middleware or keep this in the community layer? What are the issues with using pilot jobs today? (in the community layer)
- Maybe what communities need is better support from the infrastructure to install and run their pilot job frameworks in the community area. Can this better support be arranged with VO-site OLAs? Or is modification in the middleware needed?
|Generate SORs and Standards|
|APIs to interface community software with EGI middleware services||RT 928 - LSGC expects a scientific data-flow system to be part of EGI. But, Because data-flow systems can be very specific to a community (what to optimise the data-flow for?) the request is really about "proper, homogeneous, documented API for middleware services"||Create and agree SOR|
|Operating grid services in fail-over mode||Ref RT 792|
|User-driven, community developed, open-source software applications and tools: best processes and practices||Potential workshop.|
|Building communities: trust, communication and contracts - towards a blueprint for creating and successfully running a VRC||Potential workshop.|
Note T, E&S = Training, Education & Standards
Table 2 - UCST Workshops at EGI Technical Forum
NGI User Support Teams
(Format: T&E Session)
The stated goal of EGI is to provide significant added value for existing and new user communities. The main providers of User Support service in EGI are National Grid Initiatives (NGIs), under the coordination of the User Community Support Team (UCST) of EGI.eu in Amsterdam. While NGI teams primarily serve national users, using the communication and coordination services by UCST the needs of structured, multi-national, large scientific collaborations are also addressed by NGIs. The session includes an overview, presented by UCST, on the structure and achievements of EGI user support activities, followed by presentations by a sample of NGIs with active user support teams. The presentations will provide information about various human and software services that NGIs develop and provide. Attendance is recommended for user support personnel from NGIs and VRCs. Duration: one 90 minutes long session.
Presentation by UK NGS: NGS Roadshows - The NGS (UK National Grid Service; www.ngs.ac.uk) has been running roadshow events for more than 2 years with 13 events having taken place. The main aims of the roadshows are to raise awareness of the NGS amongst researchers, post docs and PhD students as well as IT staff and to provide “hands on” training in using NGS resources. The roadshows are mainly held at NGS member sites of which there are currently 24. Roadshow events are also held at institutions that are interested in becoming NGS members or where there is a demand e.g. from a large research group. The roadshows follow a format which can be tailored to each institutions requirements but usually include an introduction to the NGS, presentations from current NGS users and presentations on how to get started using NGS resources. The afternoon session is a “hands on” training session to allow delegates to gain real experience of submitting jobs to the NGS. Feedback on the roadshows has been very positive with 80% of attendees likely to apply for a grid certificate and NGS account. We also carry out follow up feedback approximately 3 months after the roadshow. We are currently planning a large number of roadshow events. The NGS has recently had funding confirmed until the end of July 2012 and we are currently ensuring that all member sites host a roadshow event in the next 11 months if they have not already hosted an event.
Portal technologies for EGI Communities
Portals, portlets and science gateways provide simple mechanisms for users to interact with grid and cloud computing services and with the community-related functions offered within the European Grid Infrastructure. Portal frameworks, portlet repositories, “iframe” encapsulation of content enables EGI user support teams to customise and integrate third party services according to their customers’ needs. EGI aims to support development effort from the area for the benefit of EGI Virtual Research Communities through the sharing of beast practices. The workshop aims to bring together three groups from EGI – portal technology providers, NGI support teams, VRCs – to discuss and answer the following series of questions:
- What kind of portal-related technologies and best practices are the most widely used and are emerging within the EGI community? How many and which VRCs and NGIs are supporting these?
- What requirements do the user and operation communities have concerning the usage, operation, integration and further-development of portals, portlets and iframes?
- How could a wider and harmonised adoption of these technologies be facilitated within EGI? What role should EGI.eu and the NGIs play in this process?
The workshop will consist of 2 x 90 minutes long sessions. The first slot is to be filled by 10 minutes long presentations given by the most active and experienced portal provider and user communities from the NGIs and VRCs. The second slot will be a discussion session where requirements and issues can be analysed and where an action plan concerning EGI portal technologies can be drafted.
1. Jan Just Keijser of BiG Grid/Nikhef, Amsterdam. email@example.com
The LGI Pilot Job portal is developed by BiG Grid, the Dutch NGI. This portal framework is based on the Leiden Grid Initiative framework, which offers a uniform access model to multiple heterogenenous resources, such as clusters, clouds, supercomputers and now also gLite based grid infrastructure. The LGI Pilot Job framework is a single-user pilot job framework, in which grid authentication is done using a robot hardware token. User authentication on the portal itself is done using plain (i.e. non-grid) X509 certificates or using username/password authentication. An extension to support TERENA using SimpleSAML is currently being developed. Several client interfaces are available, including command-line clients written in C++ and Python, as well as PHP and Java clients. An R-interface to the LGI Pilot Job framework is currently being tested.
Data management – best practices and requirements
Identifying, implementing and operating efficient and scalable data management and processing structures is one of the biggest problems one has to face when using a distributed computing infrastructure. These require robust software services with flexible integration and customisation facilities for both system administrators and end users. The middleware software stacks hosted on the European Grid Infrastructure provide several services for data (and metadata) management. Members of the European Grid Infrastructure collaboration have long-running experience of using these services and extending, integrating, customising them for generic or community-specific data processing use cases. This workshop will provide a forum for the developers and operators as well as existing and emerging users of these services to exchange experiences, requirements and best practices concerning the data-related services of EGI. The workshop will be coordinated by the User Community Support Team (UCST) of EGI.eu, the team that is responsible for the gathering and analysis of user requirements in EGI. By bringing the main stakeholders of data-related services together, UCST aims to capture requirements for the next generation of software services that should be provided within EGI. The workshop will consist of 3 x 90 minute long sessions. The first session will consist of presentations given by EGI technology providers on the status and roadmaps of data-related middleware services. The second session will be filled by presentations from VRCs focusing on best practices and issues that these communities have. The third session will be allocated for discussions – the aim is to capture the details of existing best practices and to collect details about community requirements concerning data services.
Grid Data Access: Proxy caches and User views - Jeff Templon, Jan Just Keijser and Cristian Cirstea A new grid data access model has been developed at Nikhef, based on a caching mechanism and WebDAV access. This grid proxy cache can be added independently "in front of" existing LFC and SRM implementations, without having to make any modifications to the LFC or SRM servers. It provides a caching mechanism to speed up access to frequently requested files, as well as a transparent platform-independent access protocol. A secure WebDAV interface is offered, so that no grid-specific client software is needed to access the grid proxy cache. The WebDAV interface is accessible using username and passwords from Windows XP/Vista/7 and Mac OS X clients, and using grid certificates or proxies from Linux worker nodes. The grid proxy cache also includes load balancing and redundancy.
Virtual Research Communities
(Format: T&E Session)
Within EGI the model for scalable user support is the Virtual Research Community (VRC). This model serves both large and small communities by offering structured research communities a sustainable mechanism with which to interact with EGI. VRCs are groups of researchers, possibly widely dispersed, working together effectively through the use of information and communications technology (ICT). With the help of EGI the VRC researchers can collaborate, communicate, share resources, access remote computers or equipment and produce results as effectively as if they, and the resources they require, were physically co-located. EGI through its partners provides a set of services for VRCs, which includes the following:
- Computing, data storage and other types of resource made available by EGI stakeholders through open source middleware software solutions.
- A requirement gathering process and tracking system run by the User Community Support Team of EGI.eu to capture, detail and investigate communities’ requirements and to provide solutions directly or through third parties.
- Assistance and training to become routine user of the infrastructure, to adapt of VRC-specific applications and data to EGI technologies.
While VRCs benefit from these existing services they also develop and operate services that would be in the interest of other VRCs. The 2 x 90 minutes long session will provide opportunity for VRCs to present their experiences of being part of the EGI community and about the services they use and offer within the European Grid Infrastructure. In addition to the VRC presentations, the representatives of the EGI User Community Support activity will introduce the technical services provided for VRCs and NGIs, including the Application Database, the Training Marketplace and the Services and Dashboards for VO Managers.
1st Contribution: A&A VRC - Status, Activities, and Evolution
Authors: Claudio Vuerli & Giuliano Taffoni (A&A VRC Coordinators)
Abstract: The current status of the A&A VRC will be presented, focusing the first part of the presentation on activities carried out during the first year of the EGI-InSPIRE Project. Some issues met during the last year hampering the growth of the VRC in EGI will be listed together with current plans (in preparation) to mitigate this adverse situation. The presentation will also report about an initiative currently in progress to create a liaison group between IVOA, the International Virtual Observatory Alliance, and OGF.
(Format: Training / Workshop)
EGI is a partnership between National Grid Initiatives (NGIs) and a coordinating body, named EGI.eu to operate a sustainable, pan-European Grid infrastructure for international scientific communities. NGIs are national legal entities charged with taking care of grid infrastructure related matters in their own countries. One of the objectives of EGI and particularly of members of the EGI-InSPIRE project is to expand access to the services of the European Grid Infrastructure to new user communities including new potential heavy users of the infrastructure from the ESFRI projects. EGI.eu aims to facilitate this process by developing an “EGI Roadshow” event model that can also be used by NGIs. The model should be usable by NGIs such that they can organise and deliver events within their own countries to attract new communities to EGI, to sign them up as EGI Virtual Research Communities. EGI Roadshow events consist of presentations and optional demos or hands on sessions. The talks introduce the goals and structure and services of EGI, while the demos and hands-on sessions demonstrate some of these services in practice. As well as providing ready-to-use content for Roadshows, the model also includes instructions and best practices on how to attract communities and it is customisable for local needs. The Roadshow model is currently under development by the EGI Training Working Group lead by the EGI.eu User Community Support Team. During the 90 minutes long session the elements of the model will be presented by the members of the EGI Training Working Group and presentations will be given by a sample of partners who have used this or similar models in the past. The session aims to be a “Training the trainers” type of event, where representatives from NGIs can learn how to organise and run EGI information events within their regions. Attendees will also have the opportunity to provide feedback on the model.
Workshop Jotting pad/sand box - Potential Event titles & discussion points
Proposed event titles
- Single sign-on
- Virtual Organisations (VO)
- Persistent storage
- User Support
- Training and consultancy
- Web-service interfaces
- Global scope
- Integration with cloud systems and volunteer desk-top systems
Event titles for discussion
- Science Field: Earth Science
- Event Title: Data management, modelling, simulation and visualization
- Technology providers: Grid - gLite, Cloud computing - StratusLab, Data management - EUDAT,...
- Science Field: Astronomy & Astrophysics
- Event Title: Grid computing and data management
- Technology providers: Grid - Unicore, Data management - Unicore,...
- Science Field: Computational Chemistry
- Event Title: need title
- Technology providers: need details
- Science Field: Life Sciences
- Event Title: need title
- Technology providers: need details
We are investigating if relevant NA3 metrics GGUS related could be gathered from these links:
sorting table xample
Taken from Sun Wiki Publisher homepage
The Sun Wiki Publisher enables you to create Wiki articles on MediaWiki servers without having to know the syntax of the MediaWiki markup language. Publish your new and existing documents transparently with the Writer to a wiki page.
Status : tested and it works reasonable well
Caution : be careful not to overwrite wiki pages
HowToDoIt : ...