Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI ENVRI"

From EGIWiki
Jump to navigation Jump to search
 
(38 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{| width="100%" style="background: none repeat scroll 0% 0% rgb(255, 255, 255); margin: 1.2em 0px 6px; border: 1px solid rgb(221, 221, 221);"
{{Fedcloud_Menu}}  
|-
| style="width:50%; color:#000;" |
{| style="width:260px; border:none; background:none;"
|-
| style="width:260px; text-align:center; white-space:nowrap; color:#000;" | <div style="font-size:162%; border:none; margin:0; padding:.1em; color:#000;">Welcome to the page of the EGI ENVRI Study Case page</div>
|}


| style="width:60%; font-size:95%;" | <br>
{{FCloudCom_menubar}}
|}


<br> {{TOC_right}}  
{{TOC_right}}


== <span class="toctext">General Project Information</span>  ==
== <span class="toctext">General Project Information</span>  ==


*Leader: Malgorzata Krakowian (malgorzata.krakowian @ egi.eu) <br>  
*'''Leader:''' Malgorzata Krakowian (malgorzata.krakowian @ egi.eu) <br>  
*Start Date:&nbsp;1.02.2013  
*'''Start Date''':&nbsp;1.02.2013<br>
*End Date: <br>  
*'''End Date''':&nbsp;1.08.2014<br>  
*Meetings: [https://indico.egi.eu/indico/categoryDisplay.py?categId=106 Indico page] (slides and minutes from meetings)
*'''Meetings:''' [https://indico.egi.eu/indico/categoryDisplay.py?categId=106 Indico page] (slides and minutes from meetings)  
*[http://envri.eu/eiscat_3d-study-case '''Official webpage''']
 
<br>


== <span class="toctext">Motivation</span><br>  ==
== <span class="toctext">Motivation</span><br>  ==


A study case was set up to identify existing services and solutions from EGI that could address the data pre-processing, post-processing, publishing needs of these two ESFRI projects. The outcome of the pilot is expected to be directly applicable to EISCAT_3D and EURO-ARGO, and indirectly by other ESFRIs of ENVRI.&nbsp; In cooperation with EISCAT-3D and EURO-ARGO representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing. <br>  
A study case was set up to identify existing services and solutions from EGI and EUDAT that could address the data pre-processing, post-processing, publishing needs of EISCAT 3D. The outcome of the pilot is expected to be directly applicable to EISCAT_3D, and indirectly by other ESFRIs of ENVRI.&nbsp; In cooperation with EISCAT-3D representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing. <br>
 
== <span class="mw-headline" id="Progress">Outcome</span>  ==
 
The design of the next generation incoherent scatter radar system, EISCAT_3D, opens up opportunities for physicists to explore many new research fields. On the other hand, it also introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. This challenge is typically referred to as a big data problem and requires solutions from beyond the capabilities of conventional database technologies. To identify existing services and new services that can tackle the EISCAT_3D big data challenge, a collaboration has been formed in February 2013 among EISCAT_3D, EGI and the EUDAT infrastructures under the ENVRI project.


== <span class="mw-headline" id="Members">Members </span>  ==
<br>
 
=== Phase 1 <!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
  <o:RelyOnVML/>
  <o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]-->Proof of concept architecture draft<span> </span><!--[if gte mso 9]><xml>
<w:WordDocument>
  <w:View>Normal</w:View>
  <w:Zoom>0</w:Zoom>
  <w:TrackMoves/>
  <w:TrackFormatting/>
  <w:PunctuationKerning/>
  <w:ValidateAgainstSchemas/>
  <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
  <w:IgnoreMixedContent>false</w:IgnoreMixedContent>
  <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
  <w:DoNotPromoteQF/>
  <w:LidThemeOther>EN-GB</w:LidThemeOther>
  <w:LidThemeAsian>ZH-CN</w:LidThemeAsian>
  <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
  <w:Compatibility>
  <w:BreakWrappedTables/>
  <w:SnapToGridInCell/>
  <w:WrapTextWithPunct/>
  <w:UseAsianBreakRules/>
  <w:DontGrowAutofit/>
  <w:SplitPgBreakAndParaMark/>
  <w:EnableOpenTypeKerning/>
  <w:DontFlipMirrorIndents/>
  <w:OverrideTableStyleHps/>
  <w:UseFELayout/>
  </w:Compatibility>
  <m:mathPr>
  <m:mathFont m:val="Cambria Math"/>
  <m:brkBin m:val="before"/>
  <m:brkBinSub m:val="&#45;-"/>
  <m:smallFrac m:val="off"/>
  <m:dispDef/>
  <m:lMargin m:val="0"/>
  <m:rMargin m:val="0"/>
  <m:defJc m:val="centerGroup"/>
  <m:wrapIndent m:val="1440"/>
  <m:intLim m:val="subSup"/>
  <m:naryLim m:val="undOvr"/>
  </m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
  DefSemiHidden="true" DefQFormat="false" DefPriority="99"
  LatentStyleCount="267">
  <w:LsdException Locked="false" Priority="0" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
  <w:LsdException Locked="false" Priority="9" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 1"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 2"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 3"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 4"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 5"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 6"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 7"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 8"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 9"/>
  <w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
  <w:LsdException Locked="false" Priority="10" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Title"/>
  <w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
  <w:LsdException Locked="false" Priority="11" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
  <w:LsdException Locked="false" Priority="22" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
  <w:LsdException Locked="false" Priority="20" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
  <w:LsdException Locked="false" Priority="59" SemiHidden="false"
  UnhideWhenUsed="false" Name="Table Grid"/>
  <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
  <w:LsdException Locked="false" Priority="1" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 1"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
  <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
  <w:LsdException Locked="false" Priority="34" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
  <w:LsdException Locked="false" Priority="29" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
  <w:LsdException Locked="false" Priority="30" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 1"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 2"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 2"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 3"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 3"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 4"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 4"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 5"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 5"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 6"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 6"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
  <w:LsdException Locked="false" Priority="19" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
  <w:LsdException Locked="false" Priority="21" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
  <w:LsdException Locked="false" Priority="31" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
  <w:LsdException Locked="false" Priority="32" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
  <w:LsdException Locked="false" Priority="33" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
  <w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
  <w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
  </w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-fareast-language:ZH-CN;}
</style>
<![endif]-->  ===
 
A [https://documents.egi.eu/secure/ShowDocument?docid=1839 'Towards a Big Data Strategy for EISCAT-3D' document] is emerging from the collaboration and it outlines a project that would take the first steps towards defining the EISCAT_3D big data strategy.&nbsp;
 
*[https://wiki.egi.eu/w/images/e/e7/ENVRI-EISCAT3D.pdf 'Towards a Big Data Strategy for EISCAT-3D' presentation] (pdf) during [http://eiscat2013.lancs.ac.uk/ 16th EISCAT International Symposium 2013 ]


'''EGI.eu'''<br>
[[Image:EISCAT-3d.png|500px|EISCAT-3d.png]]


*(Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu
=== Phase 2 Requirements gathering <br> ===
*(User Community Support Officer) Nuno Ferreira nuno.ferreir [at] <span>egi</span>.eu<br>


'''EISCAT-3D:'''<br>
Following questionnaires has been used to collect requirements from EISCAT data managers and scientists:  


*Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se<br>
For scientists: https://www.surveymonkey.com/s/ENVRI-EISCAT_Scientists
*Dr Ingrid Mann ingrid.mann [at] eiscat.se


'''EURO-ARGO: '''<br>
For data managers: https://www.surveymonkey.com/s/ENVRI-EISCAT_Data_managers


*Thierry Carval Thierry.Carval [at] ifremer.fr
Responses: https://documents.egi.eu/public/ShowDocument?docid=1983
*Jerome Detoc Jerome.Detoc [at] ifremer.fr (data management, big data)


'''ENVRI:'''
=== Phase 3 Prototype system based on technologies and resources of EGI and EUDAT<br>  ===


*Yin Chen ChenY58 [at] cardiff.ac.uk<br>
Development and pilot deployment of [http://sourceforge.net/projects/osgcat/ OSGC - OpenSource Geospatial Catalogue]<br>  


== <span id="Resources" class="mw-headline">Resources  </span><br>  ==
Presentation: [https://documents.egi.eu/public/ShowDocument?docid=2281 EGI OpenSearch Catalogue Appliances for EISCAT 3D]


*[https://drive.google.com/a/egi.eu/?pli=1#folders/0Bzrt5PnQFpWocjd1V2ZHeXJUamM Google doc for study case (internal for EGI)]
<br>  
*[https://wiki.egi.eu/wiki/File:Euro-Argo_architecture.png Euro-Argo architecture (image file)]
*[https://wiki.egi.eu/wiki/File:EISCAT-3D_architecture.JPG EISCAT-3D architecture (image file)]<br>


'''External tools and presentations (could be useful)'''
OSGC is an Open Source implementation of an OpenSearch GeoSpatial Catalogue compliant to OGC 10-32r3 specification, developed by EGI.eu ([http://www.egi.eu/ http://www.egi.eu/]) under the ENVRI ([http://envri.eu/ http://envri.eu/]) project.<br> <br> OSGC provides a catalogue engine built on top of a PostgreSQL+Postgis database, which exposes a cusmizable OpenSearch interface. Most of the application configuration can be set from the Admin web interface, while Data Administrators have a separated Dropbox interface, which ease the management of the catalog and the data storage, and a Data Gateway interface, which controls access to data and produces data access statistics.<br> <br> A stand-alone client web interface, written in HTML5 and Javascript, can be used to query this catalog and other compliant OpenSearch catalogs.


*[http://cds.cern.ch/record/1512927 The IT Challenges For Research Infrastructures In Physics] presentation (CRISP ESFRI cluster project)
'''Features'''


'''ENVRI wiki'''
*OpenSearch catalogue engine with customizable output formats, products metadata, query schema, input formats (for ingestion).
*Web admin interface (to offer the catalog as a Platform-As-A-Service on the Cloud).
*Dropbox, to automatically extract metadata, register it into the catalogue and optionally push the data file into Cloud or other connected storage.
*Data Gateway interface, to control access to data, produce data access statistics and bridge non-http protocols
*OpenSearch web client interface, with the possibility to execute it remotely or as a standalone application (for integration into Cloud Virtual Laboratories PaaS services) and cumulative download (with shop-chart functionality).


*[http://envri.eu/group/envri/wiki/-/wiki/Main/Euro-Argo Euro-Argo description ]
== <span class="mw-headline" id="Members">Members  </span>  ==
*[http://www.argodatamgt.org/ Ero-Argo data management - ftp server]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/EISCAT_3D EISCAT-3D description ]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Analyse%20Common%20Requirements%20for%20Data%20Processing Analyse Common Requirements for Data Processing ]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Iceland%20Volcano%20Use%20Case Iceland Volcano Study Case ]


== <span class="mw-headline" id="Progress">Progress  </span><br> ==
'''[http://egi.eu EGI.eu]'''<br>  


=== EISCAT-3D  ===
*(Study case leader &amp; EGI Operations Officer) Malgorzata Krakowian malgorzata.krakowian [at] egi.eu
*(Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu
*(User Community Support Officer) Nuno Ferreira nuno.ferreir [at] <span>egi</span>.eu<br>
*(EGI Chief Operations Officer) Tiziana Ferrari


A short document will be prepared that outlines the scope of the EISCAT_3D - EGI collaboration. The document would be used by EGI and by EISCAT_3D to invite contributors into the collaboration (NGIs, institutes)
'''[https://www.eiscat3d.se/node EISCAT-3D:]'''<br>


Key points in the document:
*Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se<br>


- EISCAT archive includes 60TB of correlated data collected during 1981-2013. There are interfaces to interact with the data but the data is not searchable.  
'''[http://envri.eu/ ENVRI:]'''


- The collaboration would make the archive searchable by:
*Yin Chen ChenY58 [at] cardiff.ac.uk<br>
*Malcolm Atkinson
*Alex Hardisty
*Yannick Legré
*Paul Martin
*Alun Preece


1. moving the data to EGI storages (grid or cloud to be decided later)
'''[http://eudat.eu/ EUDAT]/[http://www.csc.com/ CSC:]'''


2. registering the data into EGI file catalogues and metadata services
*Ari Lukkarinen ari.lukkarinen [at] csc.fi
*Antti Pursula
*Ville Savolainen


3. studying, identifying and - if feasible - implementing data mirroring and indexing strategies that would suit to the envisaged processing strategies of EISCAT_3D. The studies could cover areas such as use of Hadoop or other big-data technologies on EGI.
== <span id="Resources" class="mw-headline">Resources  </span><br>  ==


=== Euro-ARGO  ===
*[https://drive.google.com/a/egi.eu/?pli=1#folders/0Bzrt5PnQFpWocjd1V2ZHeXJUamM Google doc for study case (internal for EGI)]<br>
*[https://wiki.egi.eu/wiki/File:EISCAT-3D_architecture.JPG EISCAT-3D architecture (image file)]<br>


A short document will be prepared that outlines the scope of the EURO-ARGO - EGI collaboration. The document would be used by EGI and by EURO-ARGO to invite contributors into the collaboration (NGIs, institutes) Key points in the document:
'''External tools and presentations (could be useful)'''


*EURO-ARGO architecture and interfaces to interact with the data.  
*[http://cds.cern.ch/record/1512927 The IT Challenges For Research Infrastructures In Physics] presentation (CRISP ESFRI cluster project)
*The collaboration would make the archive searchable by
**moving the data to EGI storages (grid or cloud?)  
**registering the data into EGI file catalogues and metadata services
*Advertisement of the data to potentially interested EGI user communities


Todo for Thierry and Jerome&nbsp;:
'''ENVRI wiki'''


*Create the first draft of the document, then circulate the document to EGI.
*[http://envri.eu/group/envri/wiki/-/wiki/Main/EISCAT_3D EISCAT-3D description ]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Analyse%20Common%20Requirements%20for%20Data%20Processing Analyse Common Requirements for Data Processing ]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Iceland%20Volcano%20Use%20Case Iceland Volcano Study Case ]


[[Category:EGI_ENVRI]]
[[Category:EGI_ENVRI]]

Latest revision as of 15:33, 7 May 2015

Overview For users For resource providers Infrastructure status Site-specific configuration Architecture



Federated Cloud Communities menu: Home Production use cases Under development use cases Closed use cases High level tools use cases



General Project Information

  • Leader: Malgorzata Krakowian (malgorzata.krakowian @ egi.eu)
  • Start Date: 1.02.2013
  • End Date: 1.08.2014
  • Meetings: Indico page (slides and minutes from meetings)
  • Official webpage


Motivation

A study case was set up to identify existing services and solutions from EGI and EUDAT that could address the data pre-processing, post-processing, publishing needs of EISCAT 3D. The outcome of the pilot is expected to be directly applicable to EISCAT_3D, and indirectly by other ESFRIs of ENVRI.  In cooperation with EISCAT-3D representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing.

Outcome

The design of the next generation incoherent scatter radar system, EISCAT_3D, opens up opportunities for physicists to explore many new research fields. On the other hand, it also introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. This challenge is typically referred to as a big data problem and requires solutions from beyond the capabilities of conventional database technologies. To identify existing services and new services that can tackle the EISCAT_3D big data challenge, a collaboration has been formed in February 2013 among EISCAT_3D, EGI and the EUDAT infrastructures under the ENVRI project.


Phase 1 Proof of concept architecture draft

A 'Towards a Big Data Strategy for EISCAT-3D' document is emerging from the collaboration and it outlines a project that would take the first steps towards defining the EISCAT_3D big data strategy. 

EISCAT-3d.png

Phase 2 Requirements gathering

Following questionnaires has been used to collect requirements from EISCAT data managers and scientists:

For scientists: https://www.surveymonkey.com/s/ENVRI-EISCAT_Scientists

For data managers: https://www.surveymonkey.com/s/ENVRI-EISCAT_Data_managers

Responses: https://documents.egi.eu/public/ShowDocument?docid=1983

Phase 3 Prototype system based on technologies and resources of EGI and EUDAT

Development and pilot deployment of OSGC - OpenSource Geospatial Catalogue

Presentation: EGI OpenSearch Catalogue Appliances for EISCAT 3D


OSGC is an Open Source implementation of an OpenSearch GeoSpatial Catalogue compliant to OGC 10-32r3 specification, developed by EGI.eu (http://www.egi.eu/) under the ENVRI (http://envri.eu/) project.

OSGC provides a catalogue engine built on top of a PostgreSQL+Postgis database, which exposes a cusmizable OpenSearch interface. Most of the application configuration can be set from the Admin web interface, while Data Administrators have a separated Dropbox interface, which ease the management of the catalog and the data storage, and a Data Gateway interface, which controls access to data and produces data access statistics.

A stand-alone client web interface, written in HTML5 and Javascript, can be used to query this catalog and other compliant OpenSearch catalogs.

Features

  • OpenSearch catalogue engine with customizable output formats, products metadata, query schema, input formats (for ingestion).
  • Web admin interface (to offer the catalog as a Platform-As-A-Service on the Cloud).
  • Dropbox, to automatically extract metadata, register it into the catalogue and optionally push the data file into Cloud or other connected storage.
  • Data Gateway interface, to control access to data, produce data access statistics and bridge non-http protocols
  • OpenSearch web client interface, with the possibility to execute it remotely or as a standalone application (for integration into Cloud Virtual Laboratories PaaS services) and cumulative download (with shop-chart functionality).

Members

EGI.eu

  • (Study case leader & EGI Operations Officer) Malgorzata Krakowian malgorzata.krakowian [at] egi.eu
  • (Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu
  • (User Community Support Officer) Nuno Ferreira nuno.ferreir [at] egi.eu
  • (EGI Chief Operations Officer) Tiziana Ferrari

EISCAT-3D:

  • Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se

ENVRI:

  • Yin Chen ChenY58 [at] cardiff.ac.uk
  • Malcolm Atkinson
  • Alex Hardisty
  • Yannick Legré
  • Paul Martin
  • Alun Preece

EUDAT/CSC:

  • Ari Lukkarinen ari.lukkarinen [at] csc.fi
  • Antti Pursula
  • Ville Savolainen

Resources

External tools and presentations (could be useful)

ENVRI wiki