Difference between revisions of "EGI ENVRI"
(25 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
{ | {{Fedcloud_Menu}} | ||
{ | |||
{{FCloudCom_menubar}} | |||
{{TOC_right}} | |||
== <span class="toctext">General Project Information</span> == | == <span class="toctext">General Project Information</span> == | ||
*'''Leader:''' Malgorzata Krakowian (malgorzata.krakowian @ egi.eu) <br> | *'''Leader:''' Malgorzata Krakowian (malgorzata.krakowian @ egi.eu) <br> | ||
*'''Start Date''': 1.02.2013<br> | *'''Start Date''': 1.02.2013<br> | ||
*'''Meetings:''' [https://indico.egi.eu/indico/categoryDisplay.py?categId=106 Indico page] (slides and minutes from meetings) | *'''End Date''': 1.08.2014<br> | ||
*'''Meetings:''' [https://indico.egi.eu/indico/categoryDisplay.py?categId=106 Indico page] (slides and minutes from meetings) | |||
*[http://envri.eu/eiscat_3d-study-case '''Official webpage'''] | |||
<br> | <br> | ||
== <span class="toctext">Motivation</span><br> == | == <span class="toctext">Motivation</span><br> == | ||
A study case was set up to identify existing services and solutions from EGI that could address the data pre-processing, post-processing, publishing needs of | A study case was set up to identify existing services and solutions from EGI and EUDAT that could address the data pre-processing, post-processing, publishing needs of EISCAT 3D. The outcome of the pilot is expected to be directly applicable to EISCAT_3D, and indirectly by other ESFRIs of ENVRI. In cooperation with EISCAT-3D representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing. <br> | ||
== <span class="mw-headline" id=" | == <span class="mw-headline" id="Progress">Outcome</span> == | ||
The design of the next generation incoherent scatter radar system, EISCAT_3D, opens up opportunities for physicists to explore many new research fields. On the other hand, it also introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. This challenge is typically referred to as a big data problem and requires solutions from beyond the capabilities of conventional database technologies. To identify existing services and new services that can tackle the EISCAT_3D big data challenge, a collaboration has been formed in February 2013 among EISCAT_3D, EGI and the EUDAT infrastructures under the ENVRI project. | |||
<br> | |||
=== Phase 1 <!--[if gte mso 9]><xml> | |||
<o:OfficeDocumentSettings> | |||
<o:RelyOnVML/> | |||
<o:AllowPNG/> | |||
</o:OfficeDocumentSettings> | |||
</xml><![endif]-->Proof of concept architecture draft<span> </span><!--[if gte mso 9]><xml> | |||
<w:WordDocument> | |||
<w:View>Normal</w:View> | |||
<w:Zoom>0</w:Zoom> | |||
<w:TrackMoves/> | |||
<w:TrackFormatting/> | |||
<w:PunctuationKerning/> | |||
<w:ValidateAgainstSchemas/> | |||
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid> | |||
<w:IgnoreMixedContent>false</w:IgnoreMixedContent> | |||
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText> | |||
<w:DoNotPromoteQF/> | |||
<w:LidThemeOther>EN-GB</w:LidThemeOther> | |||
<w:LidThemeAsian>ZH-CN</w:LidThemeAsian> | |||
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript> | |||
<w:Compatibility> | |||
<w:BreakWrappedTables/> | |||
<w:SnapToGridInCell/> | |||
<w:WrapTextWithPunct/> | |||
<w:UseAsianBreakRules/> | |||
<w:DontGrowAutofit/> | |||
<w:SplitPgBreakAndParaMark/> | |||
<w:EnableOpenTypeKerning/> | |||
<w:DontFlipMirrorIndents/> | |||
<w:OverrideTableStyleHps/> | |||
<w:UseFELayout/> | |||
</w:Compatibility> | |||
<m:mathPr> | |||
<m:mathFont m:val="Cambria Math"/> | |||
<m:brkBin m:val="before"/> | |||
<m:brkBinSub m:val="--"/> | |||
<m:smallFrac m:val="off"/> | |||
<m:dispDef/> | |||
<m:lMargin m:val="0"/> | |||
<m:rMargin m:val="0"/> | |||
<m:defJc m:val="centerGroup"/> | |||
<m:wrapIndent m:val="1440"/> | |||
<m:intLim m:val="subSup"/> | |||
<m:naryLim m:val="undOvr"/> | |||
</m:mathPr></w:WordDocument> | |||
</xml><![endif]--><!--[if gte mso 9]><xml> | |||
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="true" | |||
DefSemiHidden="true" DefQFormat="false" DefPriority="99" | |||
LatentStyleCount="267"> | |||
<w:LsdException Locked="false" Priority="0" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Normal"/> | |||
<w:LsdException Locked="false" Priority="9" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="heading 1"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 2"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 3"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 4"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 5"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 6"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 7"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 8"/> | |||
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 9"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 1"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 2"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 3"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 4"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 5"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 6"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 7"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 8"/> | |||
<w:LsdException Locked="false" Priority="39" Name="toc 9"/> | |||
<w:LsdException Locked="false" Priority="35" QFormat="true" Name="caption"/> | |||
<w:LsdException Locked="false" Priority="10" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Title"/> | |||
<w:LsdException Locked="false" Priority="1" Name="Default Paragraph Font"/> | |||
<w:LsdException Locked="false" Priority="11" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Subtitle"/> | |||
<w:LsdException Locked="false" Priority="22" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Strong"/> | |||
<w:LsdException Locked="false" Priority="20" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Emphasis"/> | |||
<w:LsdException Locked="false" Priority="59" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Table Grid"/> | |||
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Placeholder Text"/> | |||
<w:LsdException Locked="false" Priority="1" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="No Spacing"/> | |||
<w:LsdException Locked="false" Priority="60" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Shading"/> | |||
<w:LsdException Locked="false" Priority="61" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light List"/> | |||
<w:LsdException Locked="false" Priority="62" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Grid"/> | |||
<w:LsdException Locked="false" Priority="63" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 1"/> | |||
<w:LsdException Locked="false" Priority="64" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 2"/> | |||
<w:LsdException Locked="false" Priority="65" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 1"/> | |||
<w:LsdException Locked="false" Priority="66" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 2"/> | |||
<w:LsdException Locked="false" Priority="67" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 1"/> | |||
<w:LsdException Locked="false" Priority="68" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 2"/> | |||
<w:LsdException Locked="false" Priority="69" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 3"/> | |||
<w:LsdException Locked="false" Priority="70" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Dark List"/> | |||
<w:LsdException Locked="false" Priority="71" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Shading"/> | |||
<w:LsdException Locked="false" Priority="72" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful List"/> | |||
<w:LsdException Locked="false" Priority="73" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Grid"/> | |||
<w:LsdException Locked="false" Priority="60" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Shading Accent 1"/> | |||
<w:LsdException Locked="false" Priority="61" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light List Accent 1"/> | |||
<w:LsdException Locked="false" Priority="62" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Grid Accent 1"/> | |||
<w:LsdException Locked="false" Priority="63" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 1"/> | |||
<w:LsdException Locked="false" Priority="64" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 1"/> | |||
<w:LsdException Locked="false" Priority="65" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 1 Accent 1"/> | |||
<w:LsdException Locked="false" UnhideWhenUsed="false" Name="Revision"/> | |||
<w:LsdException Locked="false" Priority="34" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="List Paragraph"/> | |||
<w:LsdException Locked="false" Priority="29" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Quote"/> | |||
<w:LsdException Locked="false" Priority="30" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Intense Quote"/> | |||
<w:LsdException Locked="false" Priority="66" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 2 Accent 1"/> | |||
<w:LsdException Locked="false" Priority="67" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 1"/> | |||
<w:LsdException Locked="false" Priority="68" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 1"/> | |||
<w:LsdException Locked="false" Priority="69" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 1"/> | |||
<w:LsdException Locked="false" Priority="70" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Dark List Accent 1"/> | |||
<w:LsdException Locked="false" Priority="71" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Shading Accent 1"/> | |||
<w:LsdException Locked="false" Priority="72" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful List Accent 1"/> | |||
<w:LsdException Locked="false" Priority="73" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Grid Accent 1"/> | |||
<w:LsdException Locked="false" Priority="60" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Shading Accent 2"/> | |||
<w:LsdException Locked="false" Priority="61" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light List Accent 2"/> | |||
<w:LsdException Locked="false" Priority="62" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Grid Accent 2"/> | |||
<w:LsdException Locked="false" Priority="63" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 2"/> | |||
<w:LsdException Locked="false" Priority="64" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 2"/> | |||
<w:LsdException Locked="false" Priority="65" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 1 Accent 2"/> | |||
<w:LsdException Locked="false" Priority="66" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 2 Accent 2"/> | |||
<w:LsdException Locked="false" Priority="67" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 2"/> | |||
<w:LsdException Locked="false" Priority="68" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 2"/> | |||
<w:LsdException Locked="false" Priority="69" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 2"/> | |||
<w:LsdException Locked="false" Priority="70" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Dark List Accent 2"/> | |||
<w:LsdException Locked="false" Priority="71" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Shading Accent 2"/> | |||
<w:LsdException Locked="false" Priority="72" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful List Accent 2"/> | |||
<w:LsdException Locked="false" Priority="73" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Grid Accent 2"/> | |||
<w:LsdException Locked="false" Priority="60" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Shading Accent 3"/> | |||
<w:LsdException Locked="false" Priority="61" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light List Accent 3"/> | |||
<w:LsdException Locked="false" Priority="62" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Grid Accent 3"/> | |||
<w:LsdException Locked="false" Priority="63" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 3"/> | |||
<w:LsdException Locked="false" Priority="64" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 3"/> | |||
<w:LsdException Locked="false" Priority="65" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 1 Accent 3"/> | |||
<w:LsdException Locked="false" Priority="66" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 2 Accent 3"/> | |||
<w:LsdException Locked="false" Priority="67" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 3"/> | |||
<w:LsdException Locked="false" Priority="68" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 3"/> | |||
<w:LsdException Locked="false" Priority="69" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 3"/> | |||
<w:LsdException Locked="false" Priority="70" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Dark List Accent 3"/> | |||
<w:LsdException Locked="false" Priority="71" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Shading Accent 3"/> | |||
<w:LsdException Locked="false" Priority="72" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful List Accent 3"/> | |||
<w:LsdException Locked="false" Priority="73" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Grid Accent 3"/> | |||
<w:LsdException Locked="false" Priority="60" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Shading Accent 4"/> | |||
<w:LsdException Locked="false" Priority="61" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light List Accent 4"/> | |||
<w:LsdException Locked="false" Priority="62" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Grid Accent 4"/> | |||
<w:LsdException Locked="false" Priority="63" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 4"/> | |||
<w:LsdException Locked="false" Priority="64" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 4"/> | |||
<w:LsdException Locked="false" Priority="65" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 1 Accent 4"/> | |||
<w:LsdException Locked="false" Priority="66" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 2 Accent 4"/> | |||
<w:LsdException Locked="false" Priority="67" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 4"/> | |||
<w:LsdException Locked="false" Priority="68" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 4"/> | |||
<w:LsdException Locked="false" Priority="69" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 4"/> | |||
<w:LsdException Locked="false" Priority="70" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Dark List Accent 4"/> | |||
<w:LsdException Locked="false" Priority="71" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Shading Accent 4"/> | |||
<w:LsdException Locked="false" Priority="72" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful List Accent 4"/> | |||
<w:LsdException Locked="false" Priority="73" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Grid Accent 4"/> | |||
<w:LsdException Locked="false" Priority="60" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Shading Accent 5"/> | |||
<w:LsdException Locked="false" Priority="61" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light List Accent 5"/> | |||
<w:LsdException Locked="false" Priority="62" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Grid Accent 5"/> | |||
<w:LsdException Locked="false" Priority="63" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 5"/> | |||
<w:LsdException Locked="false" Priority="64" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 5"/> | |||
<w:LsdException Locked="false" Priority="65" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 1 Accent 5"/> | |||
<w:LsdException Locked="false" Priority="66" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 2 Accent 5"/> | |||
<w:LsdException Locked="false" Priority="67" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 5"/> | |||
<w:LsdException Locked="false" Priority="68" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 5"/> | |||
<w:LsdException Locked="false" Priority="69" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 5"/> | |||
<w:LsdException Locked="false" Priority="70" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Dark List Accent 5"/> | |||
<w:LsdException Locked="false" Priority="71" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Shading Accent 5"/> | |||
<w:LsdException Locked="false" Priority="72" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful List Accent 5"/> | |||
<w:LsdException Locked="false" Priority="73" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Grid Accent 5"/> | |||
<w:LsdException Locked="false" Priority="60" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Shading Accent 6"/> | |||
<w:LsdException Locked="false" Priority="61" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light List Accent 6"/> | |||
<w:LsdException Locked="false" Priority="62" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Light Grid Accent 6"/> | |||
<w:LsdException Locked="false" Priority="63" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 1 Accent 6"/> | |||
<w:LsdException Locked="false" Priority="64" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Shading 2 Accent 6"/> | |||
<w:LsdException Locked="false" Priority="65" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 1 Accent 6"/> | |||
<w:LsdException Locked="false" Priority="66" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium List 2 Accent 6"/> | |||
<w:LsdException Locked="false" Priority="67" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 1 Accent 6"/> | |||
<w:LsdException Locked="false" Priority="68" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 2 Accent 6"/> | |||
<w:LsdException Locked="false" Priority="69" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Medium Grid 3 Accent 6"/> | |||
<w:LsdException Locked="false" Priority="70" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Dark List Accent 6"/> | |||
<w:LsdException Locked="false" Priority="71" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Shading Accent 6"/> | |||
<w:LsdException Locked="false" Priority="72" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful List Accent 6"/> | |||
<w:LsdException Locked="false" Priority="73" SemiHidden="false" | |||
UnhideWhenUsed="false" Name="Colorful Grid Accent 6"/> | |||
<w:LsdException Locked="false" Priority="19" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Subtle Emphasis"/> | |||
<w:LsdException Locked="false" Priority="21" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Intense Emphasis"/> | |||
<w:LsdException Locked="false" Priority="31" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Subtle Reference"/> | |||
<w:LsdException Locked="false" Priority="32" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Intense Reference"/> | |||
<w:LsdException Locked="false" Priority="33" SemiHidden="false" | |||
UnhideWhenUsed="false" QFormat="true" Name="Book Title"/> | |||
<w:LsdException Locked="false" Priority="37" Name="Bibliography"/> | |||
<w:LsdException Locked="false" Priority="39" QFormat="true" Name="TOC Heading"/> | |||
</w:LatentStyles> | |||
</xml><![endif]--><!--[if gte mso 10]> | |||
<style> | |||
/* Style Definitions */ | |||
table.MsoNormalTable | |||
{mso-style-name:"Table Normal"; | |||
mso-tstyle-rowband-size:0; | |||
mso-tstyle-colband-size:0; | |||
mso-style-noshow:yes; | |||
mso-style-priority:99; | |||
mso-style-parent:""; | |||
mso-padding-alt:0cm 5.4pt 0cm 5.4pt; | |||
mso-para-margin-top:0cm; | |||
mso-para-margin-right:0cm; | |||
mso-para-margin-bottom:10.0pt; | |||
mso-para-margin-left:0cm; | |||
line-height:115%; | |||
mso-pagination:widow-orphan; | |||
font-size:11.0pt; | |||
font-family:"Calibri","sans-serif"; | |||
mso-ascii-font-family:Calibri; | |||
mso-ascii-theme-font:minor-latin; | |||
mso-hansi-font-family:Calibri; | |||
mso-hansi-theme-font:minor-latin; | |||
mso-fareast-language:ZH-CN;} | |||
</style> | |||
<![endif]--> === | |||
A [https://documents.egi.eu/secure/ShowDocument?docid=1839 'Towards a Big Data Strategy for EISCAT-3D' document] is emerging from the collaboration and it outlines a project that would take the first steps towards defining the EISCAT_3D big data strategy. | |||
'' | *[https://wiki.egi.eu/w/images/e/e7/ENVRI-EISCAT3D.pdf 'Towards a Big Data Strategy for EISCAT-3D' presentation] (pdf) during [http://eiscat2013.lancs.ac.uk/ 16th EISCAT International Symposium 2013 ] | ||
[[Image:EISCAT-3d.png|500px|EISCAT-3d.png]] | |||
=== Phase 2 Requirements gathering <br> === | |||
Following questionnaires has been used to collect requirements from EISCAT data managers and scientists: | |||
For scientists: https://www.surveymonkey.com/s/ENVRI-EISCAT_Scientists | |||
For data managers: https://www.surveymonkey.com/s/ENVRI-EISCAT_Data_managers | |||
Responses: https://documents.egi.eu/public/ShowDocument?docid=1983 | |||
=== Phase 3 Prototype system based on technologies and resources of EGI and EUDAT<br> === | |||
Development and pilot deployment of [http://sourceforge.net/projects/osgcat/ OSGC - OpenSource Geospatial Catalogue]<br> | |||
Presentation: [https://documents.egi.eu/public/ShowDocument?docid=2281 EGI OpenSearch Catalogue Appliances for EISCAT 3D] | |||
<br> | |||
OSGC is an Open Source implementation of an OpenSearch GeoSpatial Catalogue compliant to OGC 10-32r3 specification, developed by EGI.eu ([http://www.egi.eu/ http://www.egi.eu/]) under the ENVRI ([http://envri.eu/ http://envri.eu/]) project.<br> <br> OSGC provides a catalogue engine built on top of a PostgreSQL+Postgis database, which exposes a cusmizable OpenSearch interface. Most of the application configuration can be set from the Admin web interface, while Data Administrators have a separated Dropbox interface, which ease the management of the catalog and the data storage, and a Data Gateway interface, which controls access to data and produces data access statistics.<br> <br> A stand-alone client web interface, written in HTML5 and Javascript, can be used to query this catalog and other compliant OpenSearch catalogs. | |||
'''Features''' | |||
A | *OpenSearch catalogue engine with customizable output formats, products metadata, query schema, input formats (for ingestion). | ||
*Web admin interface (to offer the catalog as a Platform-As-A-Service on the Cloud). | |||
*Dropbox, to automatically extract metadata, register it into the catalogue and optionally push the data file into Cloud or other connected storage. | |||
*Data Gateway interface, to control access to data, produce data access statistics and bridge non-http protocols | |||
*OpenSearch web client interface, with the possibility to execute it remotely or as a standalone application (for integration into Cloud Virtual Laboratories PaaS services) and cumulative download (with shop-chart functionality). | |||
== <span class="mw-headline" id="Members">Members </span> == | |||
'''[http://egi.eu EGI.eu]'''<br> | |||
<br> | *(Study case leader & EGI Operations Officer) Malgorzata Krakowian malgorzata.krakowian [at] egi.eu | ||
*(Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu | |||
*(User Community Support Officer) Nuno Ferreira nuno.ferreir [at] <span>egi</span>.eu<br> | |||
*(EGI Chief Operations Officer) Tiziana Ferrari | |||
'''[https://www.eiscat3d.se/node EISCAT-3D:]'''<br> | |||
*Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se<br> | |||
'''[http://envri.eu/ ENVRI:]''' | |||
<br> | *Yin Chen ChenY58 [at] cardiff.ac.uk<br> | ||
*Malcolm Atkinson | |||
*Alex Hardisty | |||
*Yannick Legré | |||
*Paul Martin | |||
*Alun Preece | |||
'''[http://eudat.eu/ EUDAT]/[http://www.csc.com/ CSC:]''' | |||
*Ari Lukkarinen ari.lukkarinen [at] csc.fi | |||
*Antti Pursula | |||
*Ville Savolainen | |||
== <span id="Resources" class="mw-headline">Resources </span><br> == | |||
*[https://drive.google.com/a/egi.eu/?pli=1#folders/0Bzrt5PnQFpWocjd1V2ZHeXJUamM Google doc for study case (internal for EGI)]<br> | |||
*[https://wiki.egi.eu/wiki/File:EISCAT-3D_architecture.JPG EISCAT-3D architecture (image file)]<br> | |||
'''External tools and presentations (could be useful)''' | |||
*[http://cds.cern.ch/record/1512927 The IT Challenges For Research Infrastructures In Physics] presentation (CRISP ESFRI cluster project) | |||
'''ENVRI wiki''' | |||
*[http://envri.eu/group/envri/wiki/-/wiki/Main/EISCAT_3D EISCAT-3D description ] | |||
*[http://envri.eu/group/envri/wiki/-/wiki/Main/Analyse%20Common%20Requirements%20for%20Data%20Processing Analyse Common Requirements for Data Processing ] | |||
* | *[http://envri.eu/group/envri/wiki/-/wiki/Main/Iceland%20Volcano%20Use%20Case Iceland Volcano Study Case ] | ||
* | |||
[[Category:EGI_ENVRI]] | [[Category:EGI_ENVRI]] |
Latest revision as of 16:33, 7 May 2015
Overview | For users | For resource providers | Infrastructure status | Site-specific configuration | Architecture |
Federated Cloud Communities menu: | Home • | Production use cases • | Under development use cases • | Closed use cases • | High level tools use cases |
General Project Information
- Leader: Malgorzata Krakowian (malgorzata.krakowian @ egi.eu)
- Start Date: 1.02.2013
- End Date: 1.08.2014
- Meetings: Indico page (slides and minutes from meetings)
- Official webpage
Motivation
A study case was set up to identify existing services and solutions from EGI and EUDAT that could address the data pre-processing, post-processing, publishing needs of EISCAT 3D. The outcome of the pilot is expected to be directly applicable to EISCAT_3D, and indirectly by other ESFRIs of ENVRI. In cooperation with EISCAT-3D representatives in ENVRI, EGI.eu will try to find best suitable solutions for data pre-processing of primary data and post-processing toward publishing.
Outcome
The design of the next generation incoherent scatter radar system, EISCAT_3D, opens up opportunities for physicists to explore many new research fields. On the other hand, it also introduces significant challenges in handling large-scale experimental data which will be massively generated at great speeds and volumes. This challenge is typically referred to as a big data problem and requires solutions from beyond the capabilities of conventional database technologies. To identify existing services and new services that can tackle the EISCAT_3D big data challenge, a collaboration has been formed in February 2013 among EISCAT_3D, EGI and the EUDAT infrastructures under the ENVRI project.
Phase 1 Proof of concept architecture draft
A 'Towards a Big Data Strategy for EISCAT-3D' document is emerging from the collaboration and it outlines a project that would take the first steps towards defining the EISCAT_3D big data strategy.
- 'Towards a Big Data Strategy for EISCAT-3D' presentation (pdf) during 16th EISCAT International Symposium 2013
Phase 2 Requirements gathering
Following questionnaires has been used to collect requirements from EISCAT data managers and scientists:
For scientists: https://www.surveymonkey.com/s/ENVRI-EISCAT_Scientists
For data managers: https://www.surveymonkey.com/s/ENVRI-EISCAT_Data_managers
Responses: https://documents.egi.eu/public/ShowDocument?docid=1983
Phase 3 Prototype system based on technologies and resources of EGI and EUDAT
Development and pilot deployment of OSGC - OpenSource Geospatial Catalogue
Presentation: EGI OpenSearch Catalogue Appliances for EISCAT 3D
OSGC is an Open Source implementation of an OpenSearch GeoSpatial Catalogue compliant to OGC 10-32r3 specification, developed by EGI.eu (http://www.egi.eu/) under the ENVRI (http://envri.eu/) project.
OSGC provides a catalogue engine built on top of a PostgreSQL+Postgis database, which exposes a cusmizable OpenSearch interface. Most of the application configuration can be set from the Admin web interface, while Data Administrators have a separated Dropbox interface, which ease the management of the catalog and the data storage, and a Data Gateway interface, which controls access to data and produces data access statistics.
A stand-alone client web interface, written in HTML5 and Javascript, can be used to query this catalog and other compliant OpenSearch catalogs.
Features
- OpenSearch catalogue engine with customizable output formats, products metadata, query schema, input formats (for ingestion).
- Web admin interface (to offer the catalog as a Platform-As-A-Service on the Cloud).
- Dropbox, to automatically extract metadata, register it into the catalogue and optionally push the data file into Cloud or other connected storage.
- Data Gateway interface, to control access to data, produce data access statistics and bridge non-http protocols
- OpenSearch web client interface, with the possibility to execute it remotely or as a standalone application (for integration into Cloud Virtual Laboratories PaaS services) and cumulative download (with shop-chart functionality).
Members
- (Study case leader & EGI Operations Officer) Malgorzata Krakowian malgorzata.krakowian [at] egi.eu
- (Technical Outreach Manager) Gergely Sipos gergely.sipos [at] egi.eu
- (User Community Support Officer) Nuno Ferreira nuno.ferreir [at] egi.eu
- (EGI Chief Operations Officer) Tiziana Ferrari
- Dr. Ingemar Haggstrom Ingemar.Haggstrom [at] eiscat.se
- Yin Chen ChenY58 [at] cardiff.ac.uk
- Malcolm Atkinson
- Alex Hardisty
- Yannick Legré
- Paul Martin
- Alun Preece
- Ari Lukkarinen ari.lukkarinen [at] csc.fi
- Antti Pursula
- Ville Savolainen
Resources
External tools and presentations (could be useful)
- The IT Challenges For Research Infrastructures In Physics presentation (CRISP ESFRI cluster project)
ENVRI wiki