Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "EGI ENVRI"

From EGIWiki
Jump to navigation Jump to search
 
(22 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{| width="100%" style="background: none repeat scroll 0% 0% rgb(255, 255, 255); margin: 1.2em 0px 6px; border: 1px solid rgb(221, 221, 221);"
{{Fedcloud_Menu}}  
|-
| style="width:50%; color:#000;" |
{| style="width:260px; border:none; background:none;"
|-
| style="width:260px; text-align:center; white-space:nowrap; color:#000;" | <div style="font-size:162%; border:none; margin:0; padding:.1em; color:#000;">Welcome to the page of the EGI ENVRI Study Case page</div>
|}


| style="width:60%; font-size:95%;" | <br>
{{FCloudCom_menubar}}
|}


<br> {{TOC_right}}  
{{TOC_right}}


== <span class="toctext">General Project Information</span>  ==
== <span class="toctext">General Project Information</span>  ==


*'''Leader:''' Malgorzata Krakowian (malgorzata.krakowian @ egi.eu) <br>  
*'''Leader:''' Malgorzata Krakowian (malgorzata.krakowian @ egi.eu) <br>  
*'''Start Date''':&nbsp;1.02.2013<br>  
*'''Start Date''':&nbsp;1.02.2013<br>
*'''Meetings:''' [https://indico.egi.eu/indico/categoryDisplay.py?categId=106 Indico page] (slides and minutes from meetings)
*'''End Date''':&nbsp;1.08.2014<br>  
*'''Meetings:''' [https://indico.egi.eu/indico/categoryDisplay.py?categId=106 Indico page] (slides and minutes from meetings)  
*[http://envri.eu/eiscat_3d-study-case '''Official webpage''']


<br>  
<br>  
'''Mailing lists:'''
*envri-eiscat3d @ mailman.egi.eu - for EISCAT-3D study case


== <span class="toctext">Motivation</span><br>  ==
== <span class="toctext">Motivation</span><br>  ==


A study case was set up to identify existing services and solutions from EGI that could address the data pre-processing, post-processing, publishing needs of these two ESFRI projects. The outcome of the pilot is expected to be directly applicable to EISCAT_3D and EURO-ARGO, and indirectly by other ESFRIs of ENVRI.&nbsp; In cooperation with EISCAT-3D and EURO-ARGO representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing. <br>  
A study case was set up to identify existing services and solutions from EGI and EUDAT that could address the data pre-processing, post-processing, publishing needs of EISCAT 3D. The outcome of the pilot is expected to be directly applicable to EISCAT_3D, and indirectly by other ESFRIs of ENVRI.&nbsp; In cooperation with EISCAT-3D representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing. <br>


== <span class="mw-headline" id="Members">Members  </span>  ==
== <span class="mw-headline" id="Progress">Outcome</span>  ==


'''[http://egi.eu EGI.eu]'''<br>
The design of the next generation incoherent scatter radar system, EISCAT_3D, opens up opportunities for physicists to explore many new research fields. On the other hand, it also introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. This challenge is typically referred to as a big data problem and requires solutions from beyond the capabilities of conventional database technologies. To identify existing services and new services that can tackle the EISCAT_3D big data challenge, a collaboration has been formed in February 2013 among EISCAT_3D, EGI and the EUDAT infrastructures under the ENVRI project.  


*(Study case leader & EGI Operations Officer) Malgorzata Krakowian malgorzata.krakowian [at] egi.eu
<br>  
*(Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu
*(User Community Support Officer) Nuno Ferreira nuno.ferreir [at] <span>egi</span>.eu<br>


'''[https://www.eiscat3d.se/node EISCAT-3D:]'''<br>  
=== Phase 1 <!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
  <o:RelyOnVML/>
  <o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]-->Proof of concept architecture draft<span> </span><!--[if gte mso 9]><xml>
<w:WordDocument>
  <w:View>Normal</w:View>
  <w:Zoom>0</w:Zoom>
  <w:TrackMoves/>
  <w:TrackFormatting/>
  <w:PunctuationKerning/>
  <w:ValidateAgainstSchemas/>
  <w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
  <w:IgnoreMixedContent>false</w:IgnoreMixedContent>
  <w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
  <w:DoNotPromoteQF/>
  <w:LidThemeOther>EN-GB</w:LidThemeOther>
  <w:LidThemeAsian>ZH-CN</w:LidThemeAsian>
  <w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
  <w:Compatibility>
  <w:BreakWrappedTables/>
  <w:SnapToGridInCell/>
  <w:WrapTextWithPunct/>
  <w:UseAsianBreakRules/>
  <w:DontGrowAutofit/>
  <w:SplitPgBreakAndParaMark/>
  <w:EnableOpenTypeKerning/>
  <w:DontFlipMirrorIndents/>
  <w:OverrideTableStyleHps/>
  <w:UseFELayout/>
  </w:Compatibility>
  <m:mathPr>
  <m:mathFont m:val="Cambria Math"/>
  <m:brkBin m:val="before"/>
  <m:brkBinSub m:val="&#45;-"/>
  <m:smallFrac m:val="off"/>
  <m:dispDef/>
  <m:lMargin m:val="0"/>
  <m:rMargin m:val="0"/>
  <m:defJc m:val="centerGroup"/>
  <m:wrapIndent m:val="1440"/>
  <m:intLim m:val="subSup"/>
  <m:naryLim m:val="undOvr"/>
  </m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true"
  DefSemiHidden="true" DefQFormat="false" DefPriority="99"
  LatentStyleCount="267">
  <w:LsdException Locked="false" Priority="0" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Normal"/>
  <w:LsdException Locked="false" Priority="9" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="heading 1"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/>
  <w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 1"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 2"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 3"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 4"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 5"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 6"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 7"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 8"/>
  <w:LsdException Locked="false" Priority="39" Name="toc 9"/>
  <w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/>
  <w:LsdException Locked="false" Priority="10" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Title"/>
  <w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/>
  <w:LsdException Locked="false" Priority="11" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/>
  <w:LsdException Locked="false" Priority="22" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Strong"/>
  <w:LsdException Locked="false" Priority="20" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/>
  <w:LsdException Locked="false" Priority="59" SemiHidden="false"
  UnhideWhenUsed="false" Name="Table Grid"/>
  <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/>
  <w:LsdException Locked="false" Priority="1" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 1"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 1"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 1"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/>
  <w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/>
  <w:LsdException Locked="false" Priority="34" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/>
  <w:LsdException Locked="false" Priority="29" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Quote"/>
  <w:LsdException Locked="false" Priority="30" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 1"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 1"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 2"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 2"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 2"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 2"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 2"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 3"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 3"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 3"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 3"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 3"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 4"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 4"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 4"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 4"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 4"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 5"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 5"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 5"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 5"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 5"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/>
  <w:LsdException Locked="false" Priority="60" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Shading Accent 6"/>
  <w:LsdException Locked="false" Priority="61" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light List Accent 6"/>
  <w:LsdException Locked="false" Priority="62" SemiHidden="false"
  UnhideWhenUsed="false" Name="Light Grid Accent 6"/>
  <w:LsdException Locked="false" Priority="63" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/>
  <w:LsdException Locked="false" Priority="64" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/>
  <w:LsdException Locked="false" Priority="65" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/>
  <w:LsdException Locked="false" Priority="66" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/>
  <w:LsdException Locked="false" Priority="67" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/>
  <w:LsdException Locked="false" Priority="68" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/>
  <w:LsdException Locked="false" Priority="69" SemiHidden="false"
  UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/>
  <w:LsdException Locked="false" Priority="70" SemiHidden="false"
  UnhideWhenUsed="false" Name="Dark List Accent 6"/>
  <w:LsdException Locked="false" Priority="71" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/>
  <w:LsdException Locked="false" Priority="72" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful List Accent 6"/>
  <w:LsdException Locked="false" Priority="73" SemiHidden="false"
  UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/>
  <w:LsdException Locked="false" Priority="19" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/>
  <w:LsdException Locked="false" Priority="21" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/>
  <w:LsdException Locked="false" Priority="31" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/>
  <w:LsdException Locked="false" Priority="32" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/>
  <w:LsdException Locked="false" Priority="33" SemiHidden="false"
  UnhideWhenUsed="false" QFormat="true" Name="Book Title"/>
  <w:LsdException Locked="false" Priority="37" Name="Bibliography"/>
  <w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/>
</w:LatentStyles>
</xml><![endif]--><!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin-top:0cm;
mso-para-margin-right:0cm;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:"Calibri","sans-serif";
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-fareast-language:ZH-CN;}
</style>
<![endif]--> ===


*Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se<br>
A [https://documents.egi.eu/secure/ShowDocument?docid=1839 'Towards a Big Data Strategy for EISCAT-3D' document] is emerging from the collaboration and it outlines a project that would take the first steps towards defining the EISCAT_3D big data strategy.&nbsp;
*Dr Ingrid Mann ingrid.mann [at] eiscat.se


'''EURO-ARGO: '''<br>
*[https://wiki.egi.eu/w/images/e/e7/ENVRI-EISCAT3D.pdf 'Towards a Big Data Strategy for EISCAT-3D' presentation] (pdf) during [http://eiscat2013.lancs.ac.uk/ 16th EISCAT International Symposium 2013 ]


*Thierry Carval Thierry.Carval [at] ifremer.fr
[[Image:EISCAT-3d.png|500px|EISCAT-3d.png]]  
*Jerome Detoc Jerome.Detoc [at] ifremer.fr (data management, big data)


'''[http://envri.eu/ ENVRI:]'''
=== Phase 2 Requirements gathering <br>  ===


*Yin Chen ChenY58 [at] cardiff.ac.uk<br>
Following questionnaires has been used to collect requirements from EISCAT data managers and scientists:
*Malcolm Atkinson
*Alex Hardisty
*Yannick Legré
*Paul Martin
*Alun Preece


'''[http://eudat.eu/ EUDAT]/[http://www.csc.com/ CSC:]'''
For scientists: https://www.surveymonkey.com/s/ENVRI-EISCAT_Scientists
*Ari Lukkarinen ari.lukkarinen [at] csc.fi
*Antti Pursula
*Ville Savolainen


== <span id="Resources" class="mw-headline">Resources  </span><br>  ==
For data managers: https://www.surveymonkey.com/s/ENVRI-EISCAT_Data_managers


*[https://drive.google.com/a/egi.eu/?pli=1#folders/0Bzrt5PnQFpWocjd1V2ZHeXJUamM Google doc for study case (internal for EGI)]
Responses: https://documents.egi.eu/public/ShowDocument?docid=1983
*[https://wiki.egi.eu/wiki/File:Euro-Argo_architecture.png Euro-Argo architecture (image file)]
*[https://wiki.egi.eu/wiki/File:EISCAT-3D_architecture.JPG EISCAT-3D architecture (image file)]<br>


'''External tools and presentations (could be useful)'''
=== Phase 3 Prototype system based on technologies and resources of EGI and EUDAT<br>  ===


*[http://cds.cern.ch/record/1512927 The IT Challenges For Research Infrastructures In Physics] presentation (CRISP ESFRI cluster project)
Development and pilot deployment of [http://sourceforge.net/projects/osgcat/ OSGC - OpenSource Geospatial Catalogue]<br>


'''ENVRI wiki'''
Presentation: [https://documents.egi.eu/public/ShowDocument?docid=2281 EGI OpenSearch Catalogue Appliances for EISCAT 3D]


*[http://envri.eu/group/envri/wiki/-/wiki/Main/Euro-Argo Euro-Argo description ]
<br>
*[http://www.argodatamgt.org/ Ero-Argo data management - ftp server]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/EISCAT_3D EISCAT-3D description ]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Analyse%20Common%20Requirements%20for%20Data%20Processing Analyse Common Requirements for Data Processing ]
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Iceland%20Volcano%20Use%20Case Iceland Volcano Study Case ]


== <span id="Progress" class="mw-headline">Progress  </span><br> ==
OSGC is an Open Source implementation of an OpenSearch GeoSpatial Catalogue compliant to OGC 10-32r3 specification, developed by EGI.eu ([http://www.egi.eu/ http://www.egi.eu/]) under the ENVRI ([http://envri.eu/ http://envri.eu/]) project.<br> <br> OSGC provides a catalogue engine built on top of a PostgreSQL+Postgis database, which exposes a cusmizable OpenSearch interface. Most of the application configuration can be set from the Admin web interface, while Data Administrators have a separated Dropbox interface, which ease the management of the catalog and the data storage, and a Data Gateway interface, which controls access to data and produces data access statistics.<br> <br> A stand-alone client web interface, written in HTML5 and Javascript, can be used to query this catalog and other compliant OpenSearch catalogs.


=== EISCAT-3D  ===
'''Features'''


A short document was prepared that outlines the scope of the EISCAT_3D - EGI collaboration. The document is used by EGI and by EISCAT_3D to invite contributors into the collaboration (NGIs, institutes)
*OpenSearch catalogue engine with customizable output formats, products metadata, query schema, input formats (for ingestion).
[https://documents.egi.eu/secure/ShowDocument?docid=1839 Towards a Big Data Strategy for EISCAT-3D document] (external link)<br>
*Web admin interface (to offer the catalog as a Platform-As-A-Service on the Cloud).  
*Dropbox, to automatically extract metadata, register it into the catalogue and optionally push the data file into Cloud or other connected storage.
*Data Gateway interface, to control access to data, produce data access statistics and bridge non-http protocols
*OpenSearch web client interface, with the possibility to execute it remotely or as a standalone application (for integration into Cloud Virtual Laboratories PaaS services) and cumulative download (with shop-chart functionality).


Key points in the document:
== <span class="mw-headline" id="Members">Members  </span>  ==


*EISCAT archive includes 60TB of correlated data collected during 1981-2013. There are interfaces to interact with the data but the data is not searchable.
'''[http://egi.eu EGI.eu]'''<br>
*The collaboration would make the archive searchable by:  
**moving the data to EGI storages (grid or cloud to be decided later)
**registering the data into EGI file catalogues and metadata services
**studying, identifying and - if feasible - implementing data mirroring and indexing strategies that would suit to the envisaged processing strategies of EISCAT_3D. The studies could cover areas such as use of Hadoop or other big-data technologies on EGI.


<br>
*(Study case leader &amp; EGI Operations Officer) Malgorzata Krakowian malgorzata.krakowian [at] egi.eu
*(Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu
*(User Community Support Officer) Nuno Ferreira nuno.ferreir [at] <span>egi</span>.eu<br>  
*(EGI Chief Operations Officer) Tiziana Ferrari


Those questionnaires are used to collect requirements from EISCAT data managers and scientists:
'''[https://www.eiscat3d.se/node EISCAT-3D:]'''<br>


For scientists:
*Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se<br>
https://www.surveymonkey.com/s/ENVRI-EISCAT_Scientists


For data managers:
'''[http://envri.eu/ ENVRI:]'''
https://www.surveymonkey.com/s/ENVRI-EISCAT_Data_managers


<br>
*Yin Chen ChenY58 [at] cardiff.ac.uk<br>  
*Malcolm Atkinson
*Alex Hardisty
*Yannick Legré
*Paul Martin
*Alun Preece


'''[http://eudat.eu/ EUDAT]/[http://www.csc.com/ CSC:]'''


*Ari Lukkarinen ari.lukkarinen [at] csc.fi
*Antti Pursula
*Ville Savolainen


== <span id="Resources" class="mw-headline">Resources  </span><br>  ==


*[https://drive.google.com/a/egi.eu/?pli=1#folders/0Bzrt5PnQFpWocjd1V2ZHeXJUamM Google doc for study case (internal for EGI)]<br>
*[https://wiki.egi.eu/wiki/File:EISCAT-3D_architecture.JPG EISCAT-3D architecture (image file)]<br>


[[Image:EISCAT-3d.png|500px|EISCAT-3d.png]]
'''External tools and presentations (could be useful)'''


=== Euro-ARGO  ===
*[http://cds.cern.ch/record/1512927 The IT Challenges For Research Infrastructures In Physics] presentation (CRISP ESFRI cluster project)


A short document will be prepared that outlines the scope of the EURO-ARGO - EGI collaboration. The document would be used by EGI and by EURO-ARGO to invite contributors into the collaboration (NGIs, institutes).
'''ENVRI wiki'''


Key points in the document:  
*[http://envri.eu/group/envri/wiki/-/wiki/Main/EISCAT_3D EISCAT-3D description ]
 
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Analyse%20Common%20Requirements%20for%20Data%20Processing Analyse Common Requirements for Data Processing ]
*EURO-ARGO architecture and interfaces to interact with the data.
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Iceland%20Volcano%20Use%20Case Iceland Volcano Study Case ]
*The collaboration would make the archive searchable by
**moving the data to EGI storages (grid or cloud?)
**registering the data into EGI file catalogues and metadata services
*Advertisement of the data to potentially interested EGI user communities
 
[[Image:EuroArgo-scheme.png|500px|EuroArgo-scheme.png]]  


[[Category:EGI_ENVRI]]
[[Category:EGI_ENVRI]]

Latest revision as of 15:33, 7 May 2015

Overview For users For resource providers Infrastructure status Site-specific configuration Architecture



Federated Cloud Communities menu: Home Production use cases Under development use cases Closed use cases High level tools use cases



General Project Information

  • Leader: Malgorzata Krakowian (malgorzata.krakowian @ egi.eu)
  • Start Date: 1.02.2013
  • End Date: 1.08.2014
  • Meetings: Indico page (slides and minutes from meetings)
  • Official webpage


Motivation

A study case was set up to identify existing services and solutions from EGI and EUDAT that could address the data pre-processing, post-processing, publishing needs of EISCAT 3D. The outcome of the pilot is expected to be directly applicable to EISCAT_3D, and indirectly by other ESFRIs of ENVRI.  In cooperation with EISCAT-3D representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing.

Outcome

The design of the next generation incoherent scatter radar system, EISCAT_3D, opens up opportunities for physicists to explore many new research fields. On the other hand, it also introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. This challenge is typically referred to as a big data problem and requires solutions from beyond the capabilities of conventional database technologies. To identify existing services and new services that can tackle the EISCAT_3D big data challenge, a collaboration has been formed in February 2013 among EISCAT_3D, EGI and the EUDAT infrastructures under the ENVRI project.


Phase 1 Proof of concept architecture draft

A 'Towards a Big Data Strategy for EISCAT-3D' document is emerging from the collaboration and it outlines a project that would take the first steps towards defining the EISCAT_3D big data strategy. 

EISCAT-3d.png

Phase 2 Requirements gathering

Following questionnaires has been used to collect requirements from EISCAT data managers and scientists:

For scientists: https://www.surveymonkey.com/s/ENVRI-EISCAT_Scientists

For data managers: https://www.surveymonkey.com/s/ENVRI-EISCAT_Data_managers

Responses: https://documents.egi.eu/public/ShowDocument?docid=1983

Phase 3 Prototype system based on technologies and resources of EGI and EUDAT

Development and pilot deployment of OSGC - OpenSource Geospatial Catalogue

Presentation: EGI OpenSearch Catalogue Appliances for EISCAT 3D


OSGC is an Open Source implementation of an OpenSearch GeoSpatial Catalogue compliant to OGC 10-32r3 specification, developed by EGI.eu (http://www.egi.eu/) under the ENVRI (http://envri.eu/) project.

OSGC provides a catalogue engine built on top of a PostgreSQL+Postgis database, which exposes a cusmizable OpenSearch interface. Most of the application configuration can be set from the Admin web interface, while Data Administrators have a separated Dropbox interface, which ease the management of the catalog and the data storage, and a Data Gateway interface, which controls access to data and produces data access statistics.

A stand-alone client web interface, written in HTML5 and Javascript, can be used to query this catalog and other compliant OpenSearch catalogs.

Features

  • OpenSearch catalogue engine with customizable output formats, products metadata, query schema, input formats (for ingestion).
  • Web admin interface (to offer the catalog as a Platform-As-A-Service on the Cloud).
  • Dropbox, to automatically extract metadata, register it into the catalogue and optionally push the data file into Cloud or other connected storage.
  • Data Gateway interface, to control access to data, produce data access statistics and bridge non-http protocols
  • OpenSearch web client interface, with the possibility to execute it remotely or as a standalone application (for integration into Cloud Virtual Laboratories PaaS services) and cumulative download (with shop-chart functionality).

Members

EGI.eu

  • (Study case leader & EGI Operations Officer) Malgorzata Krakowian malgorzata.krakowian [at] egi.eu
  • (Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu
  • (User Community Support Officer) Nuno Ferreira nuno.ferreir [at] egi.eu
  • (EGI Chief Operations Officer) Tiziana Ferrari

EISCAT-3D:

  • Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se

ENVRI:

  • Yin Chen ChenY58 [at] cardiff.ac.uk
  • Malcolm Atkinson
  • Alex Hardisty
  • Yannick Legré
  • Paul Martin
  • Alun Preece

EUDAT/CSC:

  • Ari Lukkarinen ari.lukkarinen [at] csc.fi
  • Antti Pursula
  • Ville Savolainen

Resources

External tools and presentations (could be useful)

ENVRI wiki