Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Competence centre DARIAH

From EGIWiki
Jump to navigation Jump to search
EGI-Engage Competence centres: Main page ELIXIR BBMRI MoBrain DARIAH LifeWatch EISCAT_3D EPOS Disaster Mitigation | EGI-Engage Knowledge Commons


DARIAH.png


About

The DARIAH Competence Centre (CC) of EGI is broadening the usage of e-Infrastructures in the domain of the Arts and Humanities research. We

  1. Bring together relevant stakeholders from the EGI and DARIAH communities who help DARIAH better understand the e-Infrastructure needs of the Arts and Humanities communities, and who operate as a ‘knowledge hub’ concerning the use of e-Infrastructures in the domain. >> Read about the CC members.
  2. Provide customised e-Infrastructure services for Arts and Humanities, by extending generic EGI e-Infrastructure services according to the unique needs of Arts and Humanities users. >> Try the e-infrastructure services
  3. Create demonstrator applications from the Arts and Humanities domain that use e-Infrastructure services. The demonstrators display state-of-the-art capabilities of what's available in digital science with the use of EGI. >> Learn about and try our demonstrators
  4. Raise awareness within the Arts and Humanities researcher community teams about the benefits of e-Infrastructure technologies. The CC is organising outreach, training and workshop events, and contributes to high-impact events in the field. >> Check our event calendar


Participants

Ruđer Bošković Institute (RBI), Croatia (Leader institute) Davor Davidović (CC coordinator), Karolj Skala
Hungarian Academy of Science, Institute for Computer Science and Control (MTA SZTAKI), Hungary Zoltán Farkas (CC technical coordinator)
Italian National Institute of Nuclear Physics (INFN), Italy

Roberto Barbera,

Antonio Calanducci

Gesellschaft für wissenschaftliche Datenverarbeitung mbH (GWDG), Germany

Philipp Wieder,

Tibor Kalman

Data Archiving and Networked Services (DANS), Netherlands Rene van Horik
Austrian Academy of Science (AAS), Austria Eveline Wandl-Vogt





Generic services for Digital Arts and Humanities

Cloud-based compute and storage infrastructure

The most basic service provided by the DARIAH-CC is a federated 'Infrastructure as a Service' (IaaS) cloud. The system is open for any researcher and research project or institute from the Digital Arts and Humanities domain. The infrastructure is based on the EGI Federated Cloud technology and currently connects OpenStack clouds from two providers: INFN-Catania and INFN-Bari. Three more sites are expected to join later in 2016 (GWDG, IRB, SZTAKI). The participating clouds are joint into a virtually single system through an identity federation that enables researchers and their applications to access the different clouds with a single identity and with single-sign on. Access to the infrastructure is controlled by the Competence Centre.

The infrastructure is offered as a scalable system for data and compute intensive applications. These applications can be deployed on the DARIAH-CC cloud in the form of Virtual Machine images (VMIs). Existing VMIs can be browsed, and new VMIs can be deployed on the DARIAH-CC cloud through the AppDB Cloud Marketplace portal. The distinct clouds of the DARIAH-CC cloud federation can all be accessed through programming APIs, command line tools and the high level gateways and applications offered by the DARIAH-CC.

Access through science gateways

...

Command line and API access

...

Parallel Semantic Search Engine (PSSE) service

...

WS-PGRADE service for computer simulations

...

DARIAH Science Gateway

...

Scientific services offered for Digital Arts and Humanities

gLibrary service

...

CDSTAR optical text recognition software

...

WS-PGRADE service for computer simulations

...

Demonstrator applications

Bavarian dialects repository

...

Optical Character Recognition (OCR)

...

Events

...

CC project background information

CC mailing list: cc-dariah AT mailman.egi.eu

Type of Competence Centre: Science-oriented

Target user communities: Digital Arts, Humanities, and Social Sciences

List of organizations representing the user communities: DARIAH-EU

Duration of the CC: 30 M

Starting at Project Month: 1 M

Ending at Project Month: 30 M

Tasks

Task 1: User Support and Training

Duration: M6-M30, Leader: RBI

Objectives: Raise awareness of Arts and Humanities (A&H) researchers about the necessity of digital research and to qualify them to work with EGI DARIAH CC services

  • to organize Workshops and Training Courses
  • to provide ICT consulting services to A&H community
  • to provide technical support services

Task 2: DARIAH eScience Gateway on EGI

Duration: M1-M24, Leader: SZTAKI

This task develops a DARIAH eScience Gateway that provides a user friendly web-based environment capable of exploiting EGI cloud and grid resources transparently with the help of WS-PGRADE/gUSE and its customization methodologies (Remote API or Application Specific Module). The specific objectives of this task are:

  • To enable the connection between the existing DARIAH virtual research environments and the EGI infrastructure;
  • To EGI-enable new identified A&H applications (e.g. SADE from Task 3) by creating new custom web-based interfaces.

The work of this task can basically split into five different topics: requirements collection, application interface adaptation and implementation, testing, documentation and eScience gateway operation. WS-PGRADE/gUSE offers different approaches for integrating EGI resources into an application:

  • If the application already has a user interface, than it is possible to set up a WSPGRADE/gUSE gateway, develop workflow(s) running on EGI for the given application on this WS-PGRADE/gUSE gateway, and use the gateway’s Remote API to submit workflows from the already existing user interface. The advantage of this approach is that users still will be able to use the interface they are familiar with, but with the possibility to use EGI resources transparently;
  • If the application doesn’t have a user interface yet, one may create a custom interface for that using some technology (PHP, Ruby on Rails, etc.), which can rely on the Remote API mentioned above to run workflows;
  • If the application doesn’t have a user interface yet, another possibility is to use the Application Specific Module (ASM API) of WS-PGRADE/gUSE to develop a portlet-based user interface.

Task 3: Storing and Accessing DARIAH contents on EGI (SADE)

Duration: M1-M12, Leader: INFN

The overall goal of this mini-project is to create a digital repository of DARIAH contents using gLibrary, the framework developed by the Italian National Institute of Nuclear Physics (INFN) to create and manage archives of digital assets (data and metadata) on local, Grid and Cloud storage resources. The digital repository will be created taking into account the requirements of the DARIAH end-users.

For this specific mini-project the data-sets will be provided by the Austrian Academy of Sciences (AAS), one of the leading Austrian research institutions with a very long-running experience and interest in the Arts and Humanities domain. The AAS datasets represent the work on a 100+ years old collection on Bavarian dialects within the Austrian-Hungarian monarchy from the beginnings of German language to nowadays. Several data types are taken into account: text, multimedia (images, audio files etc.), URIs; as well as primary collection data, interpreted data, secondary background data and geo-data with different license opportunities.

An extract is available at the website of the Database of Bavarian dialects in Austria [1] electronically mapped.

  • Headwords (about 50,000 A-Z)[2];
  • Records (about 40,000 plants; about 70,000 in general)[3];
  • Multimedia with Link to Audio-file (examples; to be improved)[4];
  • Multimedia with Collection (about 3,000; planned to be published within the mini-project)[5];
  • Multimedia connected to Headword (about 3,000; planned to be digitized)[6];
  • Project specific biographies[7];
  • Locations[8].

The AAS datasets will be orchestrated by the INFN gLibrary Digital Repository System whose high-level overview is shown in the following figure:

GLibrary.png

The repositories will be exposed to end-user through two channels:

  1. As a (series of) portlet(s) integrated both in one of the already existing Science Gateways implemented with the Catania Science Gateway Framework [9] and in the Science Gateway developed by the lighthouse project;
  2. As native apps for mobile appliances based on Android and iOS operating systems and downloadable from the official App Stores. The mobile apps will be coded using a cross-platform development environment so that other mobile operating systems could be supported, if needed. Furthermore, the apps could exploit geo-localisation services available on smartphones and tablets to find “near” contents.

Task 4: Multi-Source Distributed Real-Time Search and Information Retrieval (SIR)

Duration: M1-M12, Leader: GWDG

The Multi-Source Distributed Real-Time Search and Information Retrieval (SIR) pathfinding mini-project investigates and implements the possibility of using distributed real-time search engines built on-top of a big-data search and analytics platform to offer A&H users information retrieval and search functionality on heterogeneous data sets, commonly found in the domain of A&H, similar to industry-like platforms such as Google Search or commercial data analytic dashboards. The multi-source distributed real-time search and analytics aims on bringing a next-generation search and data retrieval platform sourced by different systems and heterogeneous data to the users from the Arts and Humanities community. For this reasons, a data hub is implemented based on big data stream and batch processing techniques. The data hub is the foundation for sourcing the distributed source engine based on elastic search that is driven by a software called Common Data Storage Architecture (CDSTAR). On top of this stack, as an interface for the user, a portlet for the DARIAH infrastructure is created that uses the REST-interface of CDSTAR for interacting with the big-data-enabled search and analytics stack. The portlet will form the user interface for searching and browsing data, and displaying the search results by lists, interactive graphics or dash boards that allow a refinement of data retrieval queries.

Task 5: Exploitation

Duration: M7-M30, Leader: DANS

Objectives: Ensure a successful transfer of the mini-projects’ results to its targeted user community and researchers and to increase the applicability and impact of the mini-projects on research conducted.

  • to increase the awareness of exploiting EGI infrastructure in the domain of A&H
  • to define the usage policy, increase applicability and impact of the mini-projects
  • to involve integrate RI resources and services into EGI
  • to disseminate the mini-project results

DARIAH EU aims to develop and maintain an eInfrastructure that supports of A&H research practices. EGI DARIAH CC will foster this aim by promoting the exploitation of EGI Grid and Cloud infrastructure to DARIAH user community. Systematic plan of approaching towards the DARIAH user community and beyond must take place by applying and integrating national roadmaps that rely on different eInfrastructure and eScience technologies. In this regard, exploitation activities will include horizontal and vertical user support. Horizontal support will be based on the dissemination of knowledge and skills and vertical by establishing of new technological solutions and environments. The grounds for this vertical component will be placed from the mini-project activities and used for designing a roadmap for sustainable A&H community involvement in EGI.


Milestones (M) and Deliverables (D)

The following gives an overview of deliverables/milestones scheduled. The 'Scope' of the deliverable defines the applicability and visibility of a deliverable. The deliverables marked as 'internal' are internal deliverables for EGI DARIAH CC and if 'external' then it is a deliverable of the EGI-Engage project that is presentable to the EC.

Code Title Lead task Lead participant Type Scope (Internal=Not sent to the EC; External=Sent to EC) Delivery PM Delivery CM
D1.1
User support and training plan
T1 RBI
R
Internal
M09
11.2015
D1.2a Report on dissemination and training activities
T1 RBI
R
Internal
M18
08.2016
D1.2b
Report on dissemination and training activities
T1 RBI
R
Internal
M30
08.2017
D2.1 Requirements collection
T2,T3,T4 RBI
R Internal M04
06.2015
D2.2 Initial technical concept and integration plan
T2 SZTAKI
R Internal M03
05.2015
D2.3a Mini-projects progress report
T2,T3,T4 GWDG
R Internal M18
08.2016
D2.3b Mini-projects progress report
T2,T3,T4 INFN
R Internal M30
08.2017
D5.1 Sustainability and user involvement plan DRAFT
T5 DANS
R Internal M12
02.2016
D6.2
Data repository for DARIAH SA2.6 DARIAH INFN
OTHER
External M11
01.2016
D6.5
Final version of Multi-Source Distributed Real-Time Search and Information Retrieval application
SA2.6 DARIAH GWDG
OTHER
External
M12
02.2016
D6.17
Production level gateway for Arts and Humanities
SA2.6 DARIAH SZTAKI
OTHER
External
M24
02.2017
M2.1
Prototype version of the EGI-enabled application
T2


M12
02.2016
M3.1
First version of the repository in place
T3


M06
08.2015

Outputs

Deliverables

See the list of deliverables in the table above, together with links to the already produced deliverables.

Virtual organisation

The Competence Centre has established and operates a Virtual Organisation in EGI to serve users from arts and humanities. Further information is available at: EGI Virtual Organisation for arts and humanities: vo.dariah.eu

Science gateways

The competence centre is in the process of setting up three science gateway environments to provide user friendly environments to carry out data and compute intensive simulations in arts and humanities.

  • WS-PGRADE: To create and run compute intensive applications. The gateway is integrated with the EGI Federated Cloud technology, and is connected to the Virtual Organisation for arts and humanities.
  • gLibrary: To create searchable digital data repositories
  • CDSTAR: Common Data Storage Architecture

Applications

  • The competence centre is working on integrating the 'Multi-Source Distributed Real-Time Search and Information Retrieval' application with the EGI Federated Cloud, using CDSTAR.
  • The competence centre is creating a searchable digital repository of Bavarian dialects with the gLibrary science gateway.
  • DARAIH-CC Parallel Semantic Search Engine (SSE)

Survey

The Competence Centre has launched the survey to collect information from the digital arts and humanities communities about their experiences, expectations and needs concerning e-Infrastructures. The responses collected through this survey will help the EGI DARIAH Competence Centre select, deploy and sustainably operate appropriate online tools, training and user support services for digital arts and humanities groups within and beyond Europe.

  • DARIAH e-Infrastructure survey: link
  • e-Infrastructure survey results (presentation given at EGI Community Forum, Bari, Nov. 2016): [10]

Dissemination activities

  • Poster presentation at 38. International convention on information and communication technology, electronics and microelectronics (MIPRO), 25-29 May 2015, Opatija, Croatia
  • Poster and 2-min oral presentation at 12th ESWC2015 conference, May 31st to June 4th 2015, Portoroz, Slovenia (Presentation [11], Poster [12], Abstract [13])
  • Presentation: How to innovate Lexicography by means of Research Infrastructures – The European example of DARIAH ([14]), JaDH 2015 (The Japanese Association for Digital Humanities annual meeting), 1-3 Sep 2015, Kyoto, Japan
  • Presentation: DARIAH requirements and roadmap in EGI ([15]), EGI Community Forum, 9-13 Nov 2015, Bari, Italy
  • Presentation: The SADE mini-project of the EGI DARIAH Competence Centre ([16]), EGI Community Forum, 9-13 Nov 2015, Bari, Italy
  • Presentation: e-Infrastructure demonstrators by the DARIAH Competence Centre for digital Arts and Humanities ([17]), EGI Conference, 6-8 April 2016, Amsterdam, Netherlands
  • Presentation: EGI in Digital Humanities ([18] in Croatian), e-Infrastructure Days at University Computing Centre, 23-25 May 2016, Zagreb, Croatia
  • TODO: Poster presentation: "TITLE", XVII EURALEX Congress, 6-10 Sep 2016, Tbilisi, Georgia
  • TODO: Workshop "Open Up! Introducing the new DARIAH CC Science Gateway for Lexicographer" @ XVII EURALEX Congress, 6-10 Sep 2016, Tbilisi, Georgia
  • TODO: Register DARIAH CC services (once established) in this registry: http://www.civic-epistemologies.eu/outcomes/registry-of-resources/v1/

List of publications

The list of publications, presentations and posters is available on google docs (access permission required)

Useful links

Available resources in the DARIAH CC

Current available resources in the Competence Centre

Type Name Description
Cloud and storage

INFN Catania 

INFN-CATANIA-STACK site capacity:

  • Number of Virtual CPU cores: 20 
  • Memory: 50GB
  • Scratch/ephemeral storage: 1TB
  • Public IP addresses: 10
  • Access type: Opportunistic
  • Middleware: Openstack
INFN Bari

RECAS-BARI site capacity:

  • Number of Virtual CPU cores: 10 
  • Memory: 20GB
  • Scratch/ephemeral storage: 1TB
  • Max number of instances: 7VM
  • Public IP address: 7
  • Access type: Opportunistic
  • Middleware: Openstack
GWDG

GWDG site capacity:

  • Number of Virtual CPU cores:  
  • Memory:
  • Scratch/ephemeral storage:
  • Max number of instances:
  • Public IP address:
  • Access type:
  • Middleware:
IRB

IRB site capacity:

  • Number of Virtual CPU cores:  
  • Memory:
  • Scratch/ephemeral storage:
  • Max number of instances:
  • Public IP address:
  • Access type:
  • Middleware:
MTA-SZTAKI

MTA-SZTAKI site capacity:

  • Number of Virtual CPU cores:  
  • Memory:
  • Scratch/ephemeral storage:
  • Max number of instances:
  • Public IP address:
  • Access type:
  • Middleware: