Difference between revisions of "Federated Cloud OpenStack Providers"
Line 69: | Line 69: | ||
</pre> | </pre> | ||
You can | You can refresh the access token and obtain an OpenStack token in a single script similar to this (python requires requests): | ||
<pre> | <pre> | ||
#!/usr/bin/env python | #!/usr/bin/env python | ||
Line 152: | Line 152: | ||
if __name__ == '__main__': | if __name__ == '__main__': | ||
main() | main() | ||
</pre> | |||
The script expects your credentials to be available in the environment: | |||
* <code>CHECKIN_CLIENT_ID</code>: Your Check-in client id (get it from https://aai.egi.eu/fedcloud) | |||
* <code>CHECKIN_CLIENT_SECRET</code>: Your Check-in client secret (get it from https://aai.egi.eu/fedcloud) | |||
* <code>CHECKIN_REFRESH_TOKEN</code>: Your Check-in refresh token (get it from https://aai.egi.eu/fedcloud) | |||
* <code>OS_AUTH_URL</code>: Keystone URL (depends on the provider, you can get it in [https://goc.egi.eu/portal/index.php?Page_Type=Services&serviceType=org.openstack.nova&selectItemserviceType=org.openstack.nova&ngi=&searchTerm=&production=&monitored=&certStatus=&scopeMatch=all&servKeyNames=&servKeyValue= GOC-DB]) | |||
* <code>OS_PROJECT_ID</code>: OpenStack project to use (See [[xx|script above]] for obtaining it) | |||
Sample usage (script named get-token.py): | |||
<pre> | |||
export CHECKIN_CLIENT_ID=xxx | |||
export CHECKIN_CLIENT_SECRET=secret | |||
export CHECKIN_REFRESH_TOKEN=othersecret | |||
export OS_AUTH_URL=OS_AUTH_URL=https://sbgcloud.in2p3.fr:5000 | |||
export OS_PROJECT_ID=a5eb30bba2c2497b90645fb199e34b39 | |||
export OS_TOKEN=$(python get-token.py) | |||
</pre> | </pre> | ||
Revision as of 08:54, 2 August 2018
Overview | For users | For resource providers | Infrastructure status | Site-specific configuration | Architecture |
OpenStack is the cloud management framework powering most of the EGI Federated Cloud providers. These providers offer native OpenStack APIs with EGI-specific AAI integration so you can have Single Sign-On across all the federation.
This page describes how to access OpenStack providers of the EGI Cloud Compute service with different tools. For providers joining the federation, check the integration documentation for details
Authentication
EGI Check-in
Providers natively integrated with EGI Check-in support OpenID Connect for authentication on Keystone with the OS-FEDERATION extension of Keystone v3 API. The OS-FEDERATION has a set of API calls that allow getting an unscoped token and Keystone assume the URLs for these calls to be protected by some authentication mechanism (SAML, OpenID Connect, ...) that will make sure the user is valid. Once the user is granted by the underlying authentication, Keystone will perform the authorisation based on a mapping configured by the provider. Mappings restrict the allowed users and maps them to local groups or specific local accounts. If the mapping is successful, an unscoped token will be returned. This is a regular OpenStack unscoped token that can be scoped to any of the allowed projects as with any other authentication mechanism.
Obtaining an access token
Access tokens can obtained via several mechanisms, usually involving the use of a web server and a browser. Command line clients/APIs without access to a browser or interactive prompt for user authentication can use refresh tokens. A refresh token is a special token that is used to generate additional access tokens. This allows you to have short-lived access tokens without having to collect credentials every single time one expires. You can request this token alongside the access and/or ID tokens as part of a user’s initial authentication flow.
In the case of EGI Check-in, we have created a special client meant to obtain your personal refresh token and client credentials that will allow the obtention of access tokens as needed. You can access the client at https://aai.egi.eu/fedcloud/ and click on 'Authorise' to log in with your Check-in credentials to obtain:
- a client id
- a client secret
- a refresh token
All of them can be used to obtain the needed access token:
$ curl -X POST -u '<client id>':'<client secret>' \ -d 'client_id=<client id>&<client secret>&grant_type=refresh_token&refresh_token=<refresh token>&scope=openid%20email%20profile' \ 'https://aai.egi.eu/oidc/token' | python -m json.tool; % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 2066 0 1743 100 323 2970 550 --:--:-- --:--:-- --:--:-- 2974 { "access_token": "<your access token>, "expires_in": 3599, "id_token": "<your id token>", "scope": "openid profile email", "token_type": "Bearer" }
Authentication using OpenStack Client
Once you have your access token, you can easily use it for authenticating with the OpenStack client using the v3oidcaccesstoken
auth type as follows:
$ openstack --auth-type v3oidcaccesstoken \ --os-protocol oidc --identity-provider egi.eu \ --os-auth-url <keystone url> \ --os-access-token <your access token> \ --os-project-id <your project id> <command..>
Discovering projects
You may not know which projects are available to your user at a given provider, but you can easily discover them using the Keystone API. The following python code will just do that for you (requires requests library):
TBD
Obtaining OpenStack tokens for other tools
Most OpenStack client allow authentication with tokens, so you can easily use them with EGI Cloud providers just doing a first step for obtaining the token. With the OpenStack client you can use the following command to set the OS_TOKEN variable with the needed token:
$ OS_TOKEN=$(openstack --auth-type v3oidcaccesstoken \ --os-protocol oidc --identity-provider egi.eu \ --os-auth-url <keystone url> \ --os-access-token <your access token> \ --os-project-id <your project id> token issue -c id -f value)
You can refresh the access token and obtain an OpenStack token in a single script similar to this (python requires requests):
#!/usr/bin/env python # in case someone is trying this with python 2 from __future__ import print_function import base64 import json import os import requests # URL Parse try: # Python 2.x from urlparse import urlparse, urlunparse except ImportError: # Python 3.x from urllib.parse import urlparse, urlunparse def get_access_token(client_id, client_secret, refresh_token): refresh_data = { 'client_id': client_id, 'client_secret': client_secret, 'grant_type': 'refresh_token', 'refresh_token': refresh_token, 'scope': 'openid email profile', } r = requests.post("https://aai.egi.eu/oidc/token", auth=(client_id, client_secret), data=refresh_data) return r.json()['access_token'] def get_keystone_url(os_auth_url, path): url = urlparse(os_auth_url) prefix = url.path.rstrip('/') if prefix.endswith('v2.0') or prefix.endswith('v3'): prefix = os.path.dirname(prefix) path = os.path.join(prefix, path) return urlunparse((url[0], url[1], path, url[3], url[4], url[5])) def get_unscoped_token(os_auth_url, access_token): url = get_keystone_url( os_auth_url, "/v3/OS-FEDERATION/identity_providers/egi.eu/protocols/oidc/auth") r = requests.post(url, headers={'Authorization': 'Bearer %s' % access_token}) return r.headers['X-Subject-Token'] def get_scoped_token(os_auth_url, os_project_id, unscoped_token): url = get_keystone_url(os_auth_url, "/v3/auth/tokens") token_body = { "auth": { "identity": { "methods": ["token"], "token": {"id": unscoped_token} }, "scope": {"project": {"id": os_project_id}} } } r = requests.post(url, headers={'content-type': 'application/json'}, data=json.dumps(token_body)) return r.headers['X-Subject-Token'] def main(): # read from environment client_id = os.environ.get('CHECKIN_CLIENT_ID', '') client_secret = os.environ.get('CHECKIN_CLIENT_SECRET', '') refresh_token = os.environ.get('CHECKIN_REFRESH_TOKEN', '') os_auth_url = os.environ.get('OS_AUTH_URL', '') os_project_id = os.environ.get('OS_PROJECT_ID', '') access_token = get_access_token(client_id, client_secret, refresh_token) token = get_scoped_token(os_auth_url, os_project_id, get_unscoped_token(os_auth_url, access_token)) print(token, end='') if __name__ == '__main__': main()
The script expects your credentials to be available in the environment:
CHECKIN_CLIENT_ID
: Your Check-in client id (get it from https://aai.egi.eu/fedcloud)CHECKIN_CLIENT_SECRET
: Your Check-in client secret (get it from https://aai.egi.eu/fedcloud)CHECKIN_REFRESH_TOKEN
: Your Check-in refresh token (get it from https://aai.egi.eu/fedcloud)OS_AUTH_URL
: Keystone URL (depends on the provider, you can get it in GOC-DB)OS_PROJECT_ID
: OpenStack project to use (See script above for obtaining it)
Sample usage (script named get-token.py):
export CHECKIN_CLIENT_ID=xxx export CHECKIN_CLIENT_SECRET=secret export CHECKIN_REFRESH_TOKEN=othersecret export OS_AUTH_URL=OS_AUTH_URL=https://sbgcloud.in2p3.fr:5000 export OS_PROJECT_ID=a5eb30bba2c2497b90645fb199e34b39 export OS_TOKEN=$(python get-token.py)
Detailed workflow
You can safely skip this section if not interested in the internals.
Clients willing to authenticate with an OpenStack provider should perform:
- a request to
http://<keystone_url>/v3/OS-FEDERATION/identity_providers/egi.eu/protocols/oidc/auth
with the headerAuthentication: Bearer <access token>
, where<access token>
is a valid OAuth2.0 Access Token obtained from EGI Check-in. This call if successful will return aX-Subject-Token
header with an unscoped token to be used in subsequent calls.
curl -i -H 'Authorization: Bearer <OAuth2.0 acces token>' \ https://<keystone_url>/v3/OS-FEDERATION/identity_providers/egi.eu/protocols/oidc/auth HTTP/1.1 201 Created Date: Wed, 01 Aug 2018 15:23:15 GMT Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_wsgi/3.4 Python/2.7.5 X-Subject-Token: <unscoped token> Vary: X-Auth-Token x-openstack-request-id: req-fae2789d-35ba-4020-9487-378ba1e3fafb Content-Length: 491 Content-Type: application/json {"token": {"issued_at": "2018-08-01T15:23:15.000000Z", "audit_ids": ["j-B2cjhwQ9ePBjJdntWnvw"], "methods": ["oidc"], "expires_at": "2018-08-01T16:23:15.000000Z", "user": {"OS-FEDERATION": {"identity_provider": {"id": "egi.eu"}, "protocol": {"id": "oidc"}, "groups": [{"id": "c526ee5d3d224c0780f7877b3597d014"}]}, "domain": {"id": "Federated", "name": "Federated"}, "id": "84b1c6812ab746c89e282ac854824712", "name": "529a87e5ce04cd5ddd7161734d02df0e2199a11452430803e714cb1309cc3907@egi.eu"}}}
- A request to
https://<keystone url>/v3/auth/projects
get supported projects with the obtained token in aX-Auth-Token
header.
curl -H 'X-Auth-Token: <unscoped token>' https://<keystone-url>/v3/auth/projects | python -mjson.tool % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 384 100 384 0 0 1534 0 --:--:-- --:--:-- --:--:-- 1536 { "links": { "next": null, "previous": null, "self": "https://<keystone url>/v3/auth/projects" }, "projects": [ { "description": null, "domain_id": "default", "enabled": true, "id": "a5eb30bba2c2497b90645fb199e34b39", "is_domain": false, "links": { "self": "https://<keystone url>/v3/projects/a5eb30bba2c2497b90645fb199e34b39" }, "name": "EGI_FCTF", "parent_id": "default" } ] }
- Lastly, the token can be scoped to a project with the information from the previous call by performing a
POST
request tohttps://<keystone url>/v3/auth/tokens
with a json body as shown below and will return the scoped token in aX-Subject-Token
header:
{ "auth": { "identity": { "methods": [ "token" ], "token": { "id": "<unscped token>" } }, "scope": { "project": { "id": "<project id>" } } } }
Sample curl command:
curl -i -H 'content-type: application/json' -X POST \ -d '{"auth": {"identity": {"methods": ["token"], "token":{ "id": "token"}}, "scope": {"project": {"id": "project id"}}}}' \ https://<keystone url>/v3/auth/tokens HTTP/1.1 201 Created Date: Wed, 01 Aug 2018 15:45:34 GMT Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_wsgi/3.4 Python/2.7.5 X-Subject-Token: <your scoped token> Vary: X-Auth-Token x-openstack-request-id: req-e2d54fa1-e3d2-451d-a5a4-e5ee22f7d8c3 Content-Length: 8464 Content-Type: application/json <...extra token information returned by keystone...>
Legacy VOs (VOMS)
Other OpenStack clients
Terraform
Terraform OpenStack provider accepts tokens for authentication and will use the value in the OS_TOKEN
environment variable if defined. Just configure your provider as usual:
provider "openstack" { }