Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

EGI Notebooks

From EGIWiki
Jump to navigation Jump to search

The EGI Jupyter as a Service provides a browser-based tool for interactive analysis of data on EGI infrastructure. The notebooks can combine text, mathematics, computations and their rich media output using Jupyter technology, and can scale to multiple servers and multiple users (hub) with the EGI cloud service.

Key Features

EGI Jupyter provides the well-known Jupyter interface for notebooks with the following added features:

  • Integration with EGI Check-in for authentication, login with any EduGAIN or social accounts (e.g. Google, Facebook)
  • Persistent storage associated to each user, available in the notebooks environment.
  • Customisable with new notebook environments, expose any existing notebook to your users.

Service Modes

We offer different service modes depending on your needs:

  • For individual users, EGI hosts and offers the service within the Applications On Demand Service. Users (after lightweight approval) can login, write and play notebooks using storage and compute capacity from the access.egi.eu VO. Request access via EGI marketplace
  • For user communities EGI offers consultancy and technology to setup a community-specific JupyterHub instance on top of community VO resources and with community-specific storage/data. Contact [support@egi.eu] for more information
  • (under development) BinderHub mode that allows to recreate notebooks from existing repositories making the code immediately reproducible by anyone, anywhere. While under development, this option does not have persistent storage and does not require authentication, there is ongoing work to integrate with the modes described above. Alpha instance available at https://binderhub.fedcloud-tf.fedcloud.eu

Data Management

Persistent storage for the notebooks is available at /persistent linked from the notebooks home directory. This is backed up by a NFS server managed as persistent volume in Kubernetes and automatically mounted at every notebook users create. Access to other kinds of persistent storage, specially for community specific instances that can be tailored to your specific needs and available storage systems.

Getting your data in

Your notebooks should have full outgoing internet connectivity so you can connect to any external service to bring data in for analysis. We are evaluating integration with other existing EGI and EOSC-hub services for facilitating the access to input data, with EGI DataHub as first target.

Deposit output data

As with input data, you can connect to any external service to deposit the notebooks output.

Bring your custom notebooks

Adding new notebooks to the EGI Jupyter just requires a working Docker image accessible from a public repository that follows these rules:

  1. It must install JupyterHub 0.8
  2. It must not run as user root, user with uid 1000 is recommended
  3. It must use $HOME as notebook directory

If you have such image, let us know so we can add it to the configuration.

Once binder integration is complete, you will be able to import any notebook just by providing the URL of a repository which contains your notebook.

Technology Stack

The EGI setup is based in the following components:

The enolfc/egi-jupyterhub github repo contains detailed configuration information on the existing setup at EGI resources.