Difference between revisions of "Applications on Demand Service - information for providers"
Line 160: | Line 160: | ||
=== Connecting the Science Gateways/Portals with Per-User Sub-Proxies === | === Connecting the Science Gateways/Portals with Per-User Sub-Proxies === | ||
The | The Science Applications on demand Infrastructure uses [https://wiki.egi.eu/wiki/Usage_of_the_per_user_sub_proxy_in_EGI per-user sub-proxies] (PSUPs) for user's authentication. Any connected Science Gateway must generate per-user sub proxies for their users and must use these for any interaction with VO resources on behalf of the users. | ||
A Science Gateway can generate PSUPs in two ways: | A Science Gateway can generate PSUPs in two ways: |
Revision as of 13:44, 30 March 2017
Main | EGI.eu operations services | Support | Documentation | Tools | Activities | Performance | Technology | Catch-all Services | Resource Allocation | Security |
Applications on Demand Service menu: | Home • | Documentation for providers • | Documentation for developers • | Architecture |
This page provides information about the 'EGI Science Applications on-demand' infrastructure, hereafter referred to as the Infrastructure.
The Science Applications on-demand infrastructure aims to address the requirements of a large number of researchers, National Users Support Teams and small laboratories who are scattered across Europe which do not have access to dedicated computational resources, storage and community specific Virtual Research Environments to manage and analyse large amount of data. This infrastructure, makes available generic storage and compute services through a set of high-level, user friendly graphical environments.
Overview
- Science Gateways/Portals and resource providers must accept and follow the platform security policy: https://documents.egi.eu/public/ShowDocument?docid=2734
- Science Gateways/Portals providers must sign OLA with EGI.eu.
- Science Gateway/Portals providers must integrate with the User Registration Portal to enable the single sign-on capability for users.
- Science Gateway/Portals providers must integrate with the per-user subproxy solution to offer traceable user authentication towards the e-infrastructure VO.
- Science Gateways/Portals must implement user resource usage quota (to prohibit a user consuming all the resources from the platform).
- Resource providers must support the per-user subproxy solution and join the e-infrastructure VO.
The below subsections provide guidance to complete these steps.
How to join
How to connect a Science Gateway to the Infrastructure
Connecting the Science Gateway with the User Registration Portal (URP)
Client service Registration
1. Open the GGUS ticket to operations that include return URIs
2. UNITY team send Client clientID and secretKey
response_type:code redirect_uri: [[Redirect url]] client_id:unity-oauth-egrantstate: [[You should generate your own state eg. md5(uniqid(rand(), TRUE));]] scope:profile openid
[https://unity.egi.eu/oauth2-as/oauth2-authz https://unity.egi.eu/oauth2-as/oauth2-authz]response_type=code &client_id=123123123 &redirect_uri=https%3A%2F%2Fclient.pl%2Fauth &scope=openid%20profile
&state=a123a123a123
Location: [https://client.pl/auth https://client.pl/auth]
code=uniquecode123 &state=a123a123a123
POST /token HTTP/1.1Host: [http://client.pl/ client.pl]
Authorization: Basic czZCaGRSa3F0MzpnWDFmQmF0M2JW
Content-Type: application/x-www-form-urlencoded
grant_type=authorization_code&code=uniquecode123&redirect_uri=https%3A%2F%2Fclient.pl%2Fauth
HTTP/1.1 200 OKContent-Type: application/json
Cache-Control: no-store
Pragma: no-cache
{
"access_token":"accessToken123",
"token_type":"Bearer",
"expires_in":3600,
"refresh_token":"refreshToken123",
"id_token":"idToken123123"}
important data:
unity.server.clientId= [YOUR CLIENT ID] unity.server.clientSecret= [YOUR SECRET KEY] unity.server.authorize=[https://unity.egi.eu/oauth2-as/oauth2-authz https://unity.egi.eu/oauth2-as/oauth2-authz] unity.server.token=[https://unity.egi.eu/oauth2/token https://unity.egi.eu/oauth2/token] unity.server.base=[https://unity.egi.eu/ https://unity.egi.eu]
List of attributes released by Unity
for scope 'profile':
- name (string)
- email (string)
- confirmedRegistration (true/false) - tells if user has confirmed affiliation request
- hasActiveSla - (true/false) - tells if user has confirmed resource request
both confirmedRegistration and hasActiveSla have to be set to 'true' to accept user as full ltos user
for scope 'additional':
- persistent (string) - persistent ID of user given by UNITY. Attribute should be used by SG providers as a source of unique ltos userID.
OpenID Connect for Liferay
OpenId Connect for Liferay is a very rough but effective implementation of the OpenId connect protocol for Liferay. Use this module to authenticate with any OpenId Connect provider.
OpenID Connect for Django
To configure Django project to support OpenID Authentication it is possible to use the django-oidc-auth module.
How to use
$ pip install django-oidc-auth
Then configure the module:
INSTALLED_APPS += ['oidc_auth'] AUTHENTICATION_BACKENDS = ('oidc_auth.auth.OpenIDConnectBackend',) + AUTHENTICATION_BACKENDS OIDC_AUTH = { 'DEFAULT_PROVIDER': { 'issuer': 'https://unity.egi.eu/oauth2', 'client_id': AUTH_OIDC_CLIENT_ID, 'client_secret': AUTH_OIDC_SECRET, }, 'SCOPES': ['openid', 'profile'], }
Finally, add this to your urls.py:
urlpatterns = patterns('your.views', # ... url(r'oidc/', include('oidc_auth.urls')), )
Run
python manage.py migrate
Connecting the Science Gateways/Portals with Per-User Sub-Proxies
The Science Applications on demand Infrastructure uses per-user sub-proxies (PSUPs) for user's authentication. Any connected Science Gateway must generate per-user sub proxies for their users and must use these for any interaction with VO resources on behalf of the users.
A Science Gateway can generate PSUPs in two ways:
- From a robot certificate that is physically hosted on the gateway server itself. OR
- From a remote robot certificate that is hosted in the e-Token Server by INFN Catania.
We recommend you to choose the first option. If there is an IGTF CA in your country which issues robot certificates then obtain a robot from this CA. If such robots are not available in your country or region, then EGI can issue a robot for you from the SEEGRID catch-all CA. The next subsections provide details information to complete these steps.
Generic requirements
The Per-User Sub-Proxy (PUSP) and End-Entity Certificate (EEC) must satisfy the following requirements:
- The EEC is a valid robot certificate:
- it either contains OID 1.2.840.113612.5.2.3.3.1, see https://www.eugridpma.org/objectid/?oid=1.2.840.113612.5.2.3.3.1
- or its DN matches the regular expression "/CN=[rR]obot[^/[:alnum:]]" i.e. containing a CN field which starts with robot or Robot and is followed by a non-alphanumerical non-slash character. see https://www.eugridpma.org/guidelines/robot/ section 3.
- The PUSP is RFC 3820 compliant, i.e. no legacy GT2 or GT3 proxies
- The PUSP is the first proxy delegation
- If the same user enters via the same portal, he must get the same PUSP DN
- No two distinct identified users will have the same PUSP DN.
A robot EEC that generates PUSP credentials SHOULD NOT be used for any other purpose; for example, it should not be used to generate non-PUSP proxy credentials and should not be use for direct authenticating.
The machine/service that will take care of PUSP generation and management should respect the following rules:
- Documented response procedures in case of incidents (that are periodically tested).
- A listed/accredited CSIRT team.
- Internal risk assessment and an actuarial team to calculate the effective risk
Using a robot certificate from your national IGTF CA
- Obtain a robot certificate from your national IGTF Certification Authority following the instructions here.
- Register the robot in the vo.access.egi.eu VO: https://perun.metacentrum.cz/cert/registrar/?vo=vo.access.egi.eu
- Generate proxies from the robot using this script: https://ndpfsvn.nikhef.nl/viewvc/mwsec/trunk/lcmaps-plugins-robot/tools/
Obtaining a robot certificate from EGI catch-all CA
- Contact long-tail-support@mailman.egi.eu and send a short description of your gateway service and the way it would be integrated with platform resources. The team will arrange a robot certificate for your gateway from the SEEGRID CA (which operates as a 'catch-all' CA in EGI) and will register this in the VO and in the e-Token Server in Italy.
- Register the robot in the vo.access.egi.eu VO: https://perun.metacentrum.cz/cert/registrar/?vo=vo.access.egi.eu
- Generate proxies from the robot using this script: https://ndpfsvn.nikhef.nl/viewvc/mwsec/trunk/lcmaps-plugins-robot/tools/
Instructions to use the e-Token Server
- Contact long-tail-support@mailman.egi.eu and send a short justification why you would like to use the eToken server (instead of hosting the robot certificate locally). Describe your gateway service and the way it would be integrated with platform resources. The team will arrange a robot certificate for your gateway from the SEEGRID CA and will register it in vo.access.egi.eu.
- Provide long-tail-support@mailman.egi.eu with the static IP address of your gateway server, so proxy requests can be authorized from this address on the e-Token Server.
- Generate proxies from the e-Token server following this guideline:
There are two available e-Token Server instances for availability and reliability reasons:
- etokenserver.ct.infn.it
- etokenserver2.ct.infn.it
The following rest API is available to get a PUSP given a unique identifier:
https://[eToken Server instance]:8443/eTokenServer/eToken/[Robot Certificate ID]?voms=[VO]:/[VO]&proxy-renewal=[true|false]&disable-voms-proxy=[true|false]&rfc-proxy=[true|false]&cn-label=user:[user unique identifier]
- Robot cetificate ID: it is the ID of your robot certificate in the e-Token server. It will be generated after the setup of your robot into the e-Token Server.
- VO: the VO you want to use to perform any action on the EGI infrastructure. The robot certificate must be a member of this VO.
- proxy-renewal: this option is used to enable (true) or disable (false) the automatic registration of a long-term proxy into a MyProxy Server.
- disable-voms-proxy: this option is used to generate plain (true) or VOMS proxy certificate (false).
- rfc-proxy: this option is used to generate standard RFC proxies (true) or legacy proxies (false).
- cn-label: this option is used to generate a PUSP for the given unique identifier.
below an example:
https://[eToken Server instance]:8443/eTokenServer/eToken/27br90771bba31acb942efe4c8209e69?voms=training.egi.eu:/training.egi.eu&proxy-renewal=false&disable-voms-proxy=false&rfc-proxy=true&cn-label=user:test1
Connecting the Science Gateways/portals with the EGI monitoring system
To monitor a new Science Gateway (SG)/portal with the EGI monitoring system based on ARGO, the SG system administrator HAS TO:
- Register the Science Gateway/portal in the GOCDB under EGI.eu NGI
- For that, please contact the EGI Operations Team (<operations@egi.eu>)
- Register the Science Gateway/portal in the GOCDB as service type: egi.Portal
- For that, please contact the EGI Operations Team (<operations@egi.eu>)
- Change in the GOCDB the Certification Status of the SG in Certified
- For that, please contact the EGI Operations Team (<operations@egi.eu>);
ARGO uses a bundle of connectors/sync components to monitor different service components, but if necessary new ones can be easily developed and added.
How to join as a grid/cloud resource provider
Any EGI resource provider can join the Platform to offer capacity for members of the long-tail of science. The site needs to run one of the supported grid or cloud middleware software, enable per-user sub-proxies (for user authentication and authorisation), and join the vo.access.egi.eu Virtual Organisation. Please email long-tail-support@egi.eu if you wish to join as a resource provider.
In order to provide authorization to the users of the LToS VO, a couple of DNs (Distinghished Names) are required to be configured on the services to be enabled. For instance, for the CREAM CE the usual grid-mapfile is the place where to add them, for OpenStack it's /etc/keystone/voms.json. You can find below the instructions for each service.
The following Robot Certificate DNs must be configured:
/DC=EU/DC=EGI/C=HU/O=Robots/O=MTA SZTAKI/CN=Robot:zfarkas@sztaki.hu /C=IT/O=INFN/OU=Robot/L=Catania/CN=Robot: Catania Science Gateway - Roberto Barbera
Instructions for OpenStack providers
Keystone-VOMS has support for per-user subproxies in the special branchs called subproxy/<openstack version>
available in the github repository https://github.com/enolfc/keystone-voms (code is in progress of being integrated into the main branch of Keystone-VOMS). You can install the code from the repository following these instructions (choose newton or mitaka depending on your installation):
git clone -b subproxy/newton https://github.com/enolfc/keystone-voms.git cd keystone-voms pip install .
Configure and deploy as the normal Keystone-VOMS plugin following the Keystone-VOMS documentation. The
The code relies on gridsite for getting extra information about the proxies arriving to your Keystone installation. You can get packages from UMD or if your distribution is not supported you can compile packages following these instructions (for Ubuntu 16.04):
# Install dependencies sudo apt-get install debhelper apache2-dev gsoap libcanl-c-dev libcurl4-openssl-dev doxygen libxml2 \ libxml2-dev libssl-dev libtool-bin pkg-config # get source code curl -k https://emian.zcu.cz:8443/jenkins/view/release/job/gridsite-build-release/platform=ubuntu-14-x86_64/lastSuccessfulBuild/artifact/results/ubuntu/stable/gridsite_2.3.3.orig.tar.gz | tar -xzvf - -C /tmp # make and make install cd /tmp/gridsite-2.3.3/src/ make sudo make install sudo ldconfig
Once gridsite is installed, you just need to add into your apache keystone configuration the following line (location of the module may be different in your installation):
LoadModule gridsite_module /usr/local/lib/httpd/modules/mod_gridsite.so
Instructions for OpenNebula providers
OpenNebula sites are not yet supported in the platform, there is some unofficial instructions (the development is not 100% complete, hence no instructions and partial support is available). Here some hints, please use at your own risks!
For the PUSP support on an up-to-date rOCCI-server, CESGA can enable partial PUSP support by adding
SetEnv ROCCI_SERVER_AUTHN_VOMS_ROBOT_SUBPROXY_IDENTITY_ENABLE yes
to Apache2 VHost configuration for rOCCI-server. This enables PUSP support in accounting but does NOT provide yet isolation between different PUSP users of the same robot.
Instructions for EMI 3/gLite providers
There is an EGI manual that shows how to set up a per-user sub-proxy to allow identification of the individual users under a common robot certificate. You can find the guide here: https://wiki.egi.eu/wiki/MAN12
How to join the user support team
If you wish to support platform users from your country, region or scientific disciplinary area, then please email long-tail-support@egi.eu. We can train you and then register you as a supporter in our team.
How to operate your service
- To operate your service, please refer to the official procedure PROC09 Resource Centre Registration and Certification
- To describe the Intervention in case of unscheduled failure of central operational tool, please refer to the documentation Tool Intervention Management
- To manage Service Intervention, please refer to the following documentation Service Intervention Management
Notifying users about downtimes
Scheduled downtime
- Scheduled interventions MUST be declared at least 24h in advance, specifying reason and duration.
- Existing scheduled interventions CAN be extended, provided that it is done 24h in advance.
1. For Grid/Cloud Resource Providers:
- Resource Provider (RP) has to register the scheduled downtime in the GOCDB by clicking on the 'Add a Downtime' link in the sidebar, specify the downtime information and the full list of impacted services.
- In case there is more than one or select a site to select all the sites associated services.
2. For Science Gateway/portal providers:
- Science Gateway provider has to register the scheduled downtime in the GOCDB by clicking on the 'Add a Downtime' link in the sidebar.
- Display downtime information on the front-page of your Science Gateway.
- The Science Gateway picks up the downtime information from GOCDB via API, and sends out a broadcast email message to active users of the platform. (under development)
- The Science Gateway displays downtime information in a dedicated section of the front page (under development).
- GOCDB Programming Interface documentation can be found here
Unscheduled (unexpected) downtime
- Any intervention declared less than 24h in advance will be considered unscheduled.
- Both grid/cloud resource providers AND science gateway/portal providers MUST declare unscheduled interventions as soon as they are detected to inform the users. Register the scheduled downtime in the GOCDB by clicking on the 'Add a Downtime' link in the sidebar, specify the downtime information and the full list of impacted services in case there is more than one or select a site to select all the sites associated services.
- Unscheduled interventions CAN be declared up to 48h in the past (retroactive information to the user community).
- Unscheduled downtimes can be fetched from the GOCDB via API as well and notified to the end-users.
- GOCDB Programming Interface documentation can be found here