|
|
| (168 intermediate revisions by 5 users not shown) |
| Line 1: |
Line 1: |
| {{Template:Op menubar}} {{TOC_right}}
| | #REDIRECT [[Applications_on_Demand_Service_-_Information_pages]] |
| | |
| This page provides information about the 'EGI platform for the Long-tail of science', as well as about the project that develops the platform. If you want to join the platform as a science gateway provider, or cloud resource provider, then jump to [[Long-tail_of_science#Information_for_providers|Information for providers]]. If you are interested in the project that developed and maintains the platform, please jump to the [[Long-tail_of_science_project|[Long-tail of science project]]] page.
| |
| | |
| == Overview ==
| |
| | |
| EGI now offers an [http://access.egi.eu easy-to-use platform] for researchers to access compute, storage and application services. The platform allows individual researchers and small research teams to perform compute and data-intensive simulations on large, distributed networks of computers. The platform is extensible with cloud and HTC compute resources, storage resources, science gateways and application services.
| |
| | |
| == Information for users ==
| |
| | |
| === What can you access in the platform? ===
| |
| | |
| The platform is accessible through [http://access.egi.eu this portal] and offers grid, cloud and application services from across the EGI community for individual researchers and small research teams. The platform offers the following type of resources:
| |
| * High-throughput computing sites for running compute/data-intensive jobs
| |
| * Cloud sites suited for both compute/data intensive jobs and hosting of scientific services
| |
| * Storage resources for storing job input and output data, and for setting up data catalogues
| |
| * Science gateways that provide graphical web environments for building and executing applications in the platform.
| |
| * Applications that are made available ‘as services’ through the science gateways.
| |
| | |
| === Who can access the platform? ===
| |
| | |
| The platform is open for any researcher who needs a simple and user-friendly access to compute, storage and applications services in order to carry out data/compute intensive science and innovation. You need to be affiliated with, or at least have a partner (for example a referee), at a European research institution to qualify for access. The platform is designed to meet the needs of individual researchers and small research groups who have limited or no experience with distributed and cloud computing.
| |
| | |
| === How can you access the platform? ===
| |
| | |
| # Login to the [http://access.egi.eu entry portal] with an EGI SSO, Google or Facebook account.
| |
| # Provide information on your profile page about your affiliation to a research institute or team.
| |
| # Request resources from the platform: Indicate what you would like to achieve with the resources so we can help you find the most suitable ones.
| |
| # After your request is approved, login to any of the science gateways and build or execute compute/data intensive applications.
| |
| | |
| === Presentations about the platform ===
| |
| * Slideset about the concept of the EGI long-tail of science platform: [https://documents.egi.eu/document/2358]
| |
| * Slideset about the authentication & authorization model adopted (incl. per-user subproxies): [https://documents.egi.eu/document/2363]
| |
| | |
| == Information for providers ==
| |
| | |
| === How to connect a science gateway to the platform ===
| |
| | |
| ==== Connecting the SG with the User Registration Portal ====
| |
| | |
| <br>'''Client service Registration'''<br><br>1. Open the GGUS ticket to operations that include return URIs<br><br>2. UNITY team send Client clientID and secretKey<br>
| |
| <div class="moz-forward-container" style="font-size: 13.28px; line-height: 19.92px;"><br><div dir="ltr"><div><div>'''Authorization procedure Unity with Client:'''</div><div>'''<br>'''</div><div>1] The Client sends a request to the OpenID Provider</div><div><br></div><div>address: [https://unity.egi.eu/oauth2-as/oauth2-authz https://unity.egi.eu/oauth2-as/oauth2-authz]</div><div><br></div><div>parameters:</div><div>response_type:code</div><div>redirect_uri: [[Redirect url]]</div><div>client_id:unity-oauth-egrant</div><div>state: [[You should generate your own state eg. md5(uniqid(rand(), TRUE));]]</div><div>scope:profile openid </div><div><br></div><div>example:</div><div> [https://unity.egi.eu/oauth2-as/oauth2-authz https://unity.egi.eu/oauth2-as/oauth2-authz]</div><div> response_type=code</div><div> &client_id=123123123</div><div> &redirect_uri=https%3A%2F%2Fclient.pl%2Fauth</div><div> &scope=openid%20profile</div><div> &state=a123a123a123</div><div><br></div><div><br></div><div>2] Authorization Server authenticates the End-User.</div><div>3] Authorization Server obtains End-User Consent/Authorization.</div><div>4] Authorization Server sends the End-User back to the redirect uri from the first request ([[Redirect url]]) with code.</div><div><br></div><div>example of the response</div><div><br></div><div>Location: [https://client.pl/auth https://client.pl/auth]?</div><div> code=uniquecode123</div><div> &state=a123a123a123</div><div><br></div><div><br></div><div><br></div><div>5] Client sends the code to the Token Endpoint to receive an Access Token and ID Token in the response.</div><div><br></div><div>POST /token HTTP/1.1</div><div> Host: [http://client.pl/ client.pl]</div><div> Authorization: Basic czZCaGRSa3F0MzpnWDFmQmF0M2JW</div><div> Content-Type: application/x-www-form-urlencoded</div><div><br></div><div> grant_type=authorization_code&code=uniquecode123</div><div> &redirect_uri=https%3A%2F%2Fclient.pl%2Fauth</div><div><br></div><div><br></div><div><br></div><div><br></div><div>6] Client validates the tokens and retrieves the End-User's Subject Identifier.</div><div><br></div><div>example:</div><div><br></div><div> HTTP/1.1 200 OK</div><div> Content-Type: application/json</div><div> Cache-Control: no-store</div><div> Pragma: no-cache</div><div> {</div><div> "access_token":"accessToken123",</div><div> "token_type":"Bearer",</div><div> "expires_in":3600,</div><div> "refresh_token":"refreshToken123",</div><div> "id_token":"idToken123123"</div><div> }</div><div><br></div><div>You should decode id_token and make some validation (more information: [http://openid.net/specs/openid-connect-basic-1_0.html http://openid.net/specs/openid-connect-basic-1_0.html])</div><div><br></div><div><br></div><div>7] Client Gets some information from userpoint endpoint ([https://unity.egi.eu/oauth2/userinfo https://unity.egi.eu/oauth2/userinfo])</div><div><br></div><div>example</div><div>[https://unity.egi.eu/oauth2/userinfo?schema=openid&access_token=accessToken123 https://unity.egi.eu/oauth2/userinfo?schema=openid&access_token=accessToken123]"</div><div><br></div><div><br></div><div>8] User gets information about user such as email or name in json format</div><div><br></div><div><br></div><div><br></div><div>important data:</div><div>unity.server.clientId= [YOUR CLIENT ID]<br>unity.server.clientSecret= [YOUR SECRET KEY]<br></div><div>unity.server.authorize=[https://unity.egi.eu/oauth2-as/oauth2-authz https://unity.egi.eu/oauth2-as/oauth2-authz]</div><div>unity.server.token=[https://unity.egi.eu/oauth2/token https://unity.egi.eu/oauth2/token]</div><div>unity.server.base=[https://unity.egi.eu/ https://unity.egi.eu]</div><div><br></div><div>full configuration:</div><div>[https://unity.egi.eu/oauth2/.well-known/openid-configuration https://unity.egi.eu/oauth2/.well-known/openid-configuration]</div></div><div><br></div>
| |
| ==== Connecting the SG with the per-user subproxy ====
| |
| | |
| Diego to add
| |
| | |
| <br>
| |
| | |
| <br>
| |
| | |
| === How to join as a resource provider ===
| |
| | |
| <span style="font-size: 13.28px; line-height: 19.92px; font-weight: normal;">Any EGI</span><span style="font-size: 13.28px; font-weight: normal; line-height: 1.5em;"> </span><span style="line-height: 19.92px; font-size: 13.28px; font-weight: normal; background-color: initial;">resource provider can join the platform to offer capacity for members of the long-tail of science. The site needs to run one of the supported grid or cloud middleware software, enable per-user sub-proxies (for user authentication and authorisation), and join the vo.access.egi.eu Virtual Organisation in EGI. The next subsections provide instructions on how to enable per-user sub-proxies on EGI</span><span style="font-size: 13.28px; font-weight: normal; line-height: 1.5em;"> </span><span style="font-size: 13.28px; font-weight: normal; line-height: 19.92px; background-color: initial;">sites. Please email support@egi.eu if you wish to join as a resource provider.</span> The ID Card of the VO is available at http://operations-portal.egi.eu/vo/view/voname/vo.access.egi.eu.
| |
| | |
| In order to provide authorization to the users of the LToS VO, a couple of DNs (Distinghished Names) are required to be configured on the services to be enabled. For instance, for the CREAM CE the usual grid-mapfile is the place where to add them, for OpenStack it's /etc/keystone/voms.json. You can find below the instructions for each service.
| |
| | |
| Both the following Robot Certificate DNs must be configured:
| |
| | |
| <pre>/DC=EU/DC=EGI/C=HU/O=Robots/O=MTA SZTAKI/CN=Robot:zfarkas@sztaki.hu
| |
| /C=IT/O=INFN/OU=Robot/L=Catania/CN=Robot: Catania Science Gateway - Roberto Barbera</pre>
| |
| | |
| ==== Instructions for OpenStack providers ====
| |
| | |
| Keystone-VOMS has support for PUSP in the special branch called <code>subproxy_support</code> available in the github repository https://github.com/enolfc/keystone-voms (code is in progress of being integrated into the main branch of Keystone-VOMS). You can install the code from the repository following these instructions:
| |
| <pre> git clone -b subproxy_support https://github.com/enolfc/keystone-voms.git
| |
| cd keystone-voms
| |
| pip install .
| |
| </pre>
| |
| Configuration and deployment of the plugin does not change from the normal Keystone-VOMS plugin, follow the [https://keystone-voms.readthedocs.org/en/latest/ Keystone-VOMS documentation] to deploy it.
| |
| | |
| There are new parameters to configure in your keystone config file, under the <code>[voms]</code> section:
| |
| | |
| *<code>allow_subproxy</code>, should be set to <code>True</code> for enabling PUSP support.
| |
| *<code>subproxy_robots</code>, should be set to <code>*</code> (recommended) or to a list of the DNs that are allowed to create PUSP in the system.
| |
| *<code>subproxy_user_prefix</code>, determines the expected prefix for the PUSP user specification. It is safe to leave it undefined so the default value (<code>CN=eToken</code> is used.
| |
| | |
| ==== Instructions for gLite providers ====
| |
| | |
| There is an EGI manual that shows how to set up a per-user sub-proxy (PUSP) to allow identification of the individual users under a common robot certificate. You can find the guide here: https://wiki.egi.eu/wiki/MAN12
| |
| | |
| ==== Instructions for OpenNebula providers ====
| |
| | |
| Development is ongoing. Release is not expected before the EGI Community Forum <br>
| |
| | |
| === How to join the user support team ===
| |
| | |
| If you wish to support users in your country, region or science disciplinary area with the EGI platform, then please email support@egi.eu. We can train you and register you as a supporter.
| |
| | |
| <br>
| |
| | |
| = Architecture details and technical discussions =
| |
| | |
| == <span style="font-size: 19.92px; line-height: 19.92px; background-color: initial;">Virtual Organisation</span> ==
| |
| | |
| Name: vo.access.egi.eu
| |
| | |
| Scope: Global
| |
| | |
| Homepage URL: https://wiki.egi.eu/wiki/Long-tail_of_science (This wiki will evolve further to become a page about the platform.)
| |
| | |
| GGUS dedicated support: No (support will be via email)
| |
| | |
| Acceptable use policy for users: https://documents.egi.eu/document/2635
| |
| | |
| Discipline: Support Activities (to be changed to Multi-disciplinary support as soon as this is possible)
| |
| | |
| VOMS: We will use VOMS+PERUN.
| |
| * perun.cesnet.cz. The enrollment url is https://perun.metacentrum.cz/perun-registrar-cert/?vo=vo.access.egi.eu
| |
| * voms1.egee.cesnet.cz and voms2.grid.cesnet.cz
| |
| | |
| | |
| Resources - Requirements:
| |
| * Deploy support for per-user sub-proxies following the instructions from this page: https://wiki.egi.eu/wiki/Long-tail_of_science#How_to_join_as_a_resource_provider
| |
| * Indicate your interested in joining the platform to support@egi.eu
| |
| | |
| Contacts:
| |
| <long-tail-support@mailman.egi.eu> for all. This is a new email list that will have technology and user supporters
| |
| | |
| Resources:
| |
| * INFN Catania, gLite grid site, WLCG site with opportunistic access: <capacity>
| |
| * INFN Catania, OpenStack site: <capacity>
| |
| | |
| == User Management Portal (UMP) ==
| |
| | |
| ==== Introductory text ====
| |
| | |
| EGI enables researchers to get access to distributed resources, EGI have recognised the need for simpler and more harmonised access to the distributed EGI Infrastructure. This portal allows individual researchers and small research teams to be productive using EGI without barriers and without unnecessary overhead.
| |
| | |
| ===== What can you get through this portal? =====
| |
| | |
| The type of services and resources are driven by the capabilities of the science gateways integrated with the platform. Ideally EGI can offer:
| |
| | |
| *HTC resources
| |
| *Cloud resources
| |
| *Storage resources
| |
| | |
| And through the science gateways:
| |
| | |
| *Run a variety of software and applications already available and
| |
| | |
| used in EGI, from statistics tools to bioinformatics. While the platform grows, more services and capabilities will be extended, feel free to provide feedback!
| |
| | |
| ===== Who can get access to this portal? =====
| |
| | |
| User need to:
| |
| | |
| *Be able to demonstrate affiliation with a research institution within Europe, or to have contacts with a research institution in Europe
| |
| | |
| (e.g. a referee in an institution)
| |
| | |
| *Be able to describe the purpose of his/her research
| |
| **Possibly with medium term goals
| |
| *Be willing to acknowledge the EGI/NGI support in their publications
| |
| | |
| How the registration works?
| |
| | |
| #Register with EGI SSO credential to the portal. eduGAIN support is
| |
| | |
| planned and will be available soon. You will be reidrected to EGI sso or to other IdP supported in the future. Creating an EGI SSO account requires few minutes and it is completely automatic.
| |
| | |
| #Provide information about your affiliation. Your institution or the
| |
| | |
| research team you are member of.
| |
| | |
| #Request access to the resources. To submit a request you will have
| |
| | |
| to describe the research subject and the goals of the activities in EGI.
| |
| | |
| #Once approved login with your credential to the science gateways
| |
| | |
| supported by the platform and start using EGI!
| |
| | |
| Steps 2) and 3) can be performed by the user immediately, but they require approval by EGI team.
| |
| | |
| ===== For the resource providers =====
| |
| | |
| Are you a site manager or an NGI manager and do you want to support with your resources the long tail of science platform? Contact operations@egi.eu!
| |
| | |
| ==== Analysis of the functionalities and architecture ====
| |
| | |
| *Registration of the user. Including the form where to provide information about the user's institution, field of research and the purposes of his/her activity in EGI resources.
| |
| **The request must be approved by authorized users.
| |
| *User registry. The UMP will be a registry of the users who are accessing, or accessed, EGI through the long tail of science platform.
| |
| *User authentication
| |
| **UMP must support a catch all IdP for the homeless users (use of EGI SSO?)
| |
| **Consider in the UMP the possibility to integrate external IdPs.
| |
| **The other services of the long tail of science platform should get hthe user information from the UMP. This will ensure that users are associated to uniform identifiers assigned only by the UMP to facilitate accounting and authorization.
| |
| | |
| As shown in the following figure, the UMP must act as a service proxy, between science gateways and the identity providers, being them EGI SSO or other IdP (e.g. eduGAIN federations). In this way UMP can control the access to the infrastructure for the long tail of science users. UMP acts as unique IdP for the science gateways.
| |
| | |
| This architecture also allows the UMP to be the service provider that needs to be authorized by the IdPs.
| |
| | |
| [[Image:UMD-architecture.png|600px|User Management Portal Architecture]]
| |
| | |
| Once the users' request is authorized on the User Management Portal, they are redirected to one or several science gateways where they can run their computational tasks or manage their data on the grid. A possible workflow to access resources could be the following:
| |
| | |
| #User accesses the Scienge Gateway (SG).
| |
| #The SG redirect the request to the UMP.
| |
| #The UMP redirect the request to the IdP that holds the credentials of the user (e.g. EGI SSO).
| |
| #The User authenticate on his/her IdP.
| |
| #The IdP provides the assertion with some attributes about the user to the UMP (e.g. the user email address).
| |
| #The UMP answers to the SG adding more attributes including the Unique Identifier that identifies the user in the UMP registry, and that is unique for every user using the LTOS platform.
| |
| #The SG uses the UID to ask a credentials that can be univocally associated to the individual user.
| |
| #The credential is used to access EGI resources.
| |
| | |
| <br> [[Image:UMP-workflow.png|600px|User Management Portal workflow]]
| |
| | |
| == Credential services ==
| |
| | |
| The user credential service will be based on the [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Federated_AAI:per-user_sub-proxy per-user sub-proxies (PUSP)].
| |
| | |
| The purpose of a per-user sub-proxy (PUSP) is to allow identification of the individual users that operate using a common robot certificate. A common example is where a web portal (e.g., a scientific gateway) somehow identifies its user and wishes to authenticate as that user when interacting with EGI resources. This is achieved by creating a proxy credential from the robot credential with the proxy certificate containing user-identifying information in its additional proxy CN field. The user-identifying information may be pseudo-anonymised where only the portal knows the actual mapping.
| |
| | |
| This solution will allow LToS users to access EGI resources through their LToS portal credentials (e.g. EGI SSO, Identity Federations, etc.) without owning a personal grid certificate. This will simplify the access to the infrastructure for the final users.
| |
| | |
| == Policy changes ==
| |
| | |
| The long-tail platform requires two policies:
| |
| | |
| #An 'Acceptable Use Policy' (AUP) for the platform. | |
| #A new security policy that describes the conditions of generating and using user-specific proxies from robot certificates
| |
| | |
| === AUP ===
| |
| | |
| Acceptable Use Policy and Conditions of Use of the EGI Platform for the Long-tail of Science: https://documents.egi.eu/document/2635
| |
| | |
| === Security Policy for the Long-tail platform ===
| |
| | |
| [[SPG:Drafts:LToS Service Scoped Security Policy]] | |
| | |
| == Admin links ==
| |
| | |
| * Detailed accounting data about the VO users can be obtained by the VO managers at https://accounting.egi.eu/user/voadm.php
| |
| * Get the list of VO members: https://voms1.egee.cesnet.cz:8443/voms/vo.access.egi.eu/user/search.action
| |
| * Register in the VO (for gateway robot certificates and for support staff): https://perun.metacentrum.cz/cert/registrar/?vo=vo.access.egi.eu
| |
| * VO membership management interface: https://perun.metacentrum.cz/cert/gui/
| |
| | |
| <!--
| |
| = Timeline for deployment =
| |
| | |
| == User Management Portal ==
| |
| | |
| *Week 24-28.11:
| |
| **clarify user-story (input for CYFRNONET) [?]
| |
| *Week 1-5.12:
| |
| **wireframe draft (?) [CYFRONET]
| |
| **(implementation internal [CYFRONET])
| |
| *Week 8-12.12
| |
| **(implementation internal [CYFRONET])
| |
| *Week 15-19.12 portal very early protype [CYFRONET]
| |
| **technology framework UNITY (http://www.unity-idm.eu/)
| |
| | |
| == Scientific gateway ==
| |
| | |
| The following providers expressed interest in providing science gateways in the long-tail platform: WS-PGRADE, DIRAC, Catania Science Gateway Framework, QosCosGrid gateway. The platform is open for other gateway/VRE providers.
| |
| | |
| == User credential service ==
| |
| | |
| See the [https://wiki.egi.eu/wiki/Fedcloud-tf:WorkGroups:Federated_AAI:per-user_sub-proxy per-user sub-proxies (PUSP)] wiki page for more information.
| |
| | |
| == Resources for the long-tail VO ==
| |
| | |
| The following providers expressed interest in contributing with resources in the long-tail platform:
| |
| | |
| *xxx, type of resource [grid/cloud], capacity: xxx
| |
| *xxx, type of resource [grid/cloud], capacity: xxx
| |
| *xxx, type of resource [grid/cloud], capacity: xxx
| |
| | |
| | |
| = References =
| |
| -->
| |
| | |
| [[Category:Task_forces]] [[Category:LToS]]
| |
The wiki is deprecated and due to be decommissioned by the end of September 2022.