Competence centre DARIAH

From EGIWiki
Revision as of 10:42, 11 February 2016 by Ddavidov (talk | contribs) (Dissemination activities)
Jump to: navigation, search
EGI-Engage Competence centres: Main page ELIXIR BBMRI MoBrain DARIAH LifeWatch EISCAT_3D EPOS Disaster Mitigation | EGI-Engage Knowledge Commons


CC Coordinator: Karolj Skala, Davor Davidović
Technical Coordinator: Zoltán Farkas
Dissemination Manager: Eveline Wandl-Vogt

CC mailing list: cc-dariah AT

Type of Competence Centre: Science-oriented

Target user communities: Digital Arts, Humanities, and Social Sciences

List of organizations representing the user communities: DARIAH-EU

Duration of the CC: 30 M

Starting at Project Month: 1 M

Ending at Project Month: 30 M

The DARIAH CC aims to widen the usage of the e-Infrastructures for Arts and Humanities research. The CC will develop and provide a workflow-based science gateway based on the generic-purpose WS-PGRADE and gLibrary technologies, adapted and tailored to the needs of users coming from the field of Arts and Humanities. The gateway will provide access and compute services for data residing in distributed grid and cloud storages. The gateway will be validated and enriched with the ‘Multi-Source Distributed Real-Time Search and Information Retrieval’ application (SIR). The CC will engage with Arts and Humanities communities to attract more applications and users to the gateway.


  1. To strengthen the collaboration between DARIAH-EU and EGI using workflow-oriented application gateways and deploying A&H applications in the EGI federated cloud (EGI FedCloud)
  2. To increase the number of accessible e-Science services and applications for the A&H researchers and integration of existing NGI resources into EGI
  3. To raise awareness of A&H researchers of the possible benefits (excellence research) of using e-Infrastructure and e-Science technologies in their research, creating conditions for a sustained increase of the user community coming from A&H and social sciences as well
  4. To widen the work started within DC-NET, INDICATE and DCH-RP project to other A&H communities


  1. Ruđer Bošković Institute (RBI, Leading partner), Croatia
  2. Hungarian Academy of Science, Institute for Computer Science and Control (MTA SZTAKI), Hungary
  3. Italian National Institute of Nuclear Physics (INFN), Italy
  4. Gesellschaft für wissenschaftliche Datenverarbeitung mbH (GWDG), Germany
  5. Data Archiving and Networked Services (DANS), Netherlands
  6. Austria Academy of Science (AAS), Austria


Task 1: User Support and Training

Duration: M6-M30, Leader: RBI

Objectives: Raise awareness of Arts and Humanities (A&H) researchers about the necessity of digital research and to qualify them to work with EGI DARIAH CC services

  • to organize Workshops and Training Courses
  • to provide ICT consulting services to A&H community
  • to provide technical support services

Task 2: DARIAH eScience Gateway on EGI

Duration: M1-M24, Leader: SZTAKI

This task develops a DARIAH eScience Gateway that provides a user friendly web-based environment capable of exploiting EGI cloud and grid resources transparently with the help of WS-PGRADE/gUSE and its customization methodologies (Remote API or Application Specific Module). The specific objectives of this task are:

  • To enable the connection between the existing DARIAH virtual research environments and the EGI infrastructure;
  • To EGI-enable new identified A&H applications (e.g. SADE from Task 3) by creating new custom web-based interfaces.

The work of this task can basically split into five different topics: requirements collection, application interface adaptation and implementation, testing, documentation and eScience gateway operation. WS-PGRADE/gUSE offers different approaches for integrating EGI resources into an application:

  • If the application already has a user interface, than it is possible to set up a WSPGRADE/gUSE gateway, develop workflow(s) running on EGI for the given application on this WS-PGRADE/gUSE gateway, and use the gateway’s Remote API to submit workflows from the already existing user interface. The advantage of this approach is that users still will be able to use the interface they are familiar with, but with the possibility to use EGI resources transparently;
  • If the application doesn’t have a user interface yet, one may create a custom interface for that using some technology (PHP, Ruby on Rails, etc.), which can rely on the Remote API mentioned above to run workflows;
  • If the application doesn’t have a user interface yet, another possibility is to use the Application Specific Module (ASM API) of WS-PGRADE/gUSE to develop a portlet-based user interface.

Task 3: Storing and Accessing DARIAH contents on EGI (SADE)

Duration: M1-M12, Leader: INFN

The overall goal of this mini-project is to create a digital repository of DARIAH contents using gLibrary, the framework developed by the Italian National Institute of Nuclear Physics (INFN) to create and manage archives of digital assets (data and metadata) on local, Grid and Cloud storage resources. The digital repository will be created taking into account the requirements of the DARIAH end-users.

For this specific mini-project the data-sets will be provided by the Austrian Academy of Sciences (AAS), one of the leading Austrian research institutions with a very long-running experience and interest in the Arts and Humanities domain. The AAS datasets represent the work on a 100+ years old collection on Bavarian dialects within the Austrian-Hungarian monarchy from the beginnings of German language to nowadays. Several data types are taken into account: text, multimedia (images, audio files etc.), URIs; as well as primary collection data, interpreted data, secondary background data and geo-data with different license opportunities.

An extract is available at the website of the Database of Bavarian dialects in Austria [1] electronically mapped.

  • Headwords (about 50,000 A-Z)[2];
  • Records (about 40,000 plants; about 70,000 in general)[3];
  • Multimedia with Link to Audio-file (examples; to be improved)[4];
  • Multimedia with Collection (about 3,000; planned to be published within the mini-project)[5];
  • Multimedia connected to Headword (about 3,000; planned to be digitized)[6];
  • Project specific biographies[7];
  • Locations[8].

The AAS datasets will be orchestrated by the INFN gLibrary Digital Repository System whose high-level overview is shown in the following figure:


The repositories will be exposed to end-user through two channels:

  1. As a (series of) portlet(s) integrated both in one of the already existing Science Gateways implemented with the Catania Science Gateway Framework [9] and in the Science Gateway developed by the lighthouse project;
  2. As native apps for mobile appliances based on Android and iOS operating systems and downloadable from the official App Stores. The mobile apps will be coded using a cross-platform development environment so that other mobile operating systems could be supported, if needed. Furthermore, the apps could exploit geo-localisation services available on smartphones and tablets to find “near” contents.

Task 4: Multi-Source Distributed Real-Time Search and Information Retrieval (SIR)

Duration: M1-M12, Leader: GWDG

The Multi-Source Distributed Real-Time Search and Information Retrieval (SIR) pathfinding mini-project investigates and implements the possibility of using distributed real-time search engines built on-top of a big-data search and analytics platform to offer A&H users information retrieval and search functionality on heterogeneous data sets, commonly found in the domain of A&H, similar to industry-like platforms such as Google Search or commercial data analytic dashboards. The multi-source distributed real-time search and analytics aims on bringing a next-generation search and data retrieval platform sourced by different systems and heterogeneous data to the users from the Arts and Humanities community. For this reasons, a data hub is implemented based on big data stream and batch processing techniques. The data hub is the foundation for sourcing the distributed source engine based on elastic search that is driven by a software called Common Data Storage Architecture (CDSTAR). On top of this stack, as an interface for the user, a portlet for the DARIAH infrastructure is created that uses the REST-interface of CDSTAR for interacting with the big-data-enabled search and analytics stack. The portlet will form the user interface for searching and browsing data, and displaying the search results by lists, interactive graphics or dash boards that allow a refinement of data retrieval queries.

Task 5: Exploitation

Duration: M7-M30, Leader: DANS

Objectives: Ensure a successful transfer of the mini-projects’ results to its targeted user community and researchers and to increase the applicability and impact of the mini-projects on research conducted.

  • to increase the awareness of exploiting EGI infrastructure in the domain of A&H
  • to define the usage policy, increase applicability and impact of the mini-projects
  • to involve integrate RI resources and services into EGI
  • to disseminate the mini-project results

DARIAH EU aims to develop and maintain an eInfrastructure that supports of A&H research practices. EGI DARIAH CC will foster this aim by promoting the exploitation of EGI Grid and Cloud infrastructure to DARIAH user community. Systematic plan of approaching towards the DARIAH user community and beyond must take place by applying and integrating national roadmaps that rely on different eInfrastructure and eScience technologies. In this regard, exploitation activities will include horizontal and vertical user support. Horizontal support will be based on the dissemination of knowledge and skills and vertical by establishing of new technological solutions and environments. The grounds for this vertical component will be placed from the mini-project activities and used for designing a roadmap for sustainable A&H community involvement in EGI.

Milestones (M) and Deliverables (D)

The following gives an overview of deliverables/milestones scheduled. The 'Scope' of the deliverable defines the applicability and visibility of a deliverable. The deliverables marked as 'internal' are internal deliverables for EGI DARIAH CC and if 'external' then it is a deliverable of the EGI-Engage project that is presentable to the EC.

Code Title Lead task Lead participant Type Scope (Internal=Not sent to the EC; External=Sent to EC) Delivery PM Delivery CM
User support and training plan
D1.2a Report on dissemination and training activities
Report on dissemination and training activities
D2.1 Requirements collection
T2,T3,T4 RBI
R Internal M04
D2.2 Initial technical concept and integration plan
R Internal M03
D2.3a Mini-projects progress report
R Internal M18
D2.3b Mini-projects progress report
R Internal M30
D5.1 Sustainability and user involvement plan
R Internal M12
Data repository for DARIAH SA2.6 DARIAH INFN
External M11
Final version of Multi-Source Distributed Real-Time Search and Information Retrieval application
Production level gateway for Arts and Humanities
Prototype version of the EGI-enabled application

First version of the repository in place




This deliverable describes the outcome of the “Storing and Accessing DARIAH contents on EGI (SADE)” pilot application of the DARIAH Competence Centre of the EGI-Engage project. The activity developed and deployed two systems: (1) a digital repository based on gLibrary framework and the EGI Federated cloud and grid infrastructure and (2) a semantic search engine. The first system helps digital humanities communities build customised and highly-available digital repositories, while the second enables the discovery and correlation of content across geographically distributed digital repositories. The presented work was developed in EGI by the EGI DARIAH Competence Centre of the EGI Engage project.

Virtual organisation

The Competence Centre has established and operates a Virtual Organisation in EGI to serve users from arts and humanities. Further information is available at: EGI Virtual Organisation for arts and humanities:

Science gateways

The competence centre is in the process of setting up three science gateway environments to provide user friendly environments to carry out data and compute intensive simulations in arts and humanities.

  • WS-PGRADE: To create and run compute intensive applications. The gateway is integrated with the EGI Federated Cloud technology, and is connected to the Virtual Organisation for arts and humanities.
  • gLibrary: To create searchable digital data repositories
  • CDSTAR: Common Data Storage Architecture


  • The competence centre is working on integrating the 'Multi-Source Distributed Real-Time Search and Information Retrieval' application with the EGI Federated Cloud, using CDSTAR.
  • The competence centre is creating a searchable digital repository of Bavarian dialects with the gLibrary science gateway.
  • DARAIH-CC Parallel Semantic Search Engine (SSE)


The Competence Centre has launched the survey to collect information from the digital arts and humanities communities about their experiences, expectations and needs concerning e-Infrastructures. The responses collected through this survey will help the EGI DARIAH Competence Centre select, deploy and sustainably operate appropriate online tools, training and user support services for digital arts and humanities groups within and beyond Europe.

  • DARIAH e-Infrastructure survey: link
  • e-Infrastructure survey results (presentation given at EGI Community Forum, Bari, Nov. 2016): [10]

Dissemination activities

  • Poster presentation at 38. International convention on information and communication technology, electronics and microelectronics (MIPRO), 25-29 May 2015, Opatija, Croatia (poster)
  • Poster and 2-min oral presentation at 12th ESWC2015 conference, May 31st to June 4th 2015, Portoroz, Slovenia ([11])
  • Presentation: DARIAH requirements and roadmap in EGI ([12]), EGI Community Forum, 9-13 Nov 2015, Bari, Italy
  • Presentation: The SADE mini-project of the EGI DARIAH Competence Centre ([13]), EGI Community Forum, 9-13 Nov 2015, Bari, Italy
  • TODO: Poster presentation: "TITLE", XVII EURALEX Congress, 6-10 Sep 2016, Tbilisi, Georgia
  • TODO: Workshop "Open Up! Introducing the new DARIAH CC Science Gateway for Lexicographer" @ XVII EURALEX Congress, 6-10 Sep 2016, Tbilisi, Georgia
  • TODO: Register DARIAH CC services (once established) in this registry:

Useful links