Search results

'''Actions/nagios issues''': see RT queue [https://rt.egi.eu/rt/Search/Results.html?Query=Que *ARC expert team - NGI_CH, NGI_FI, NGI_UA, EMI (Petter Urkedal)

== 2.2 New Nagios Probes == === 2.2.1 CREAM & WN restructured probes ===

* Probes: https://poem.egi.eu/poem/admin/poem/public_probe/ | nordugrid-arc-nagios-plugins

*'''31-08-2015''': dCache 2.6.X 2 version monitoring probes start returning CRITICAL, alarms are displayed in the Operations Dashboard ...rvice end-points according to the version information available from BDII. Probes detecting unsupported service end-points, will generate alarms that can be

* emi-cluster v. 2.0.1 ( 7 months) == 2.2 New Nagios Probes ==

= SAM Nagios probes re-factoring TF = * [[SAM Nagios probes refactoring TF]]

...robes are successfully integrated. At the end of the quarter following SAM/Nagios instances were in production: Detailed list of SAM/Nagios instances can be found on the following page: [[SAM Instances]].

...sion was approved by the OMB in order to enable probe developers to update probes more frequently and independently from SAM releases. ...gridmon) and its dependencies. SAM Update-23 supports only SAM Nagios (sam-nagios). In the future version SAM GridMon will be replaced with the ARGO engine.

=== EMI GFAL does not work with multiple BDIIs === lcg-util 1.11.18 released with EMI 1.0.0 don't support

...within the EMI and EGI projects the monitoring framework will be based on NAGIOS. This has also the advantage to ease the integration of the FedCloud monito |Peter requested enabling alternative AuthN in probes. Need to clarify in which cases is this required and optimal mechanism of d

* Follow up of a problem with StoRM frontend which may prevent the upgrade to EMI-2 ...ion of start of campaign on 04 March. Development of probes for the entire EMI 1 stack in time for the start of the campaign was impossible due to a numbe

* status assessment of EMI 1 probes * discussion of distribution and testing of EMI WN tarball

==== 1.1. Update on the status of EMI updates ==== ** EMI-1 (28.03.2013) [http://www.eu-emi.eu/emi-1-kebnekaise-updates/-/asset_publisher/Ir6q/content/update-24-28-03-2013-v-

=== 1.1 EMI-1 release status (Cristina) === ...h a proxy fetched from a myproxy server fails. This is the case of the SAM/Nagios tests.

** [http://www.eu-emi.eu/releases/emi-3-monte-bianco/updates/-/asset_publisher/5Na8/content/update-19-07-08-2014- ** [http://www.eu-emi.eu/releases/emi-3-monte-bianco/updates/-/asset_publisher/5Na8/content/update-19-07-08-2014-

|Nagios Probes Task Force Meeting (2nd EMI All Hands Meeting) |Presentation and planning of probe handover to EMI product teams.

...Croatian NGI participates in staged rollout of APEL and in some cases SAM/Nagios. At the beginning of project software release procedure was not defined and ...table, but first year saw a lot of releases and changes in components. SAM/Nagios is released through official release process so major bugs are spotted befo

|Arc probes revisited update |glite EMI probe review

COD is involved in the process of retirement the EMI-1 middleware. In near future the EMI-1 dCache service withdrawal, that has extended

...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS. ** [http://www.eu-emi.eu/releases/emi-3-monte-bianco/updates/-/asset_publisher/5Na8/content/update-18-07-07-2014-

(e.g. scripts, SAM probes), or relevant parts of the infrastructure are <![endif]-->'''<span lang="EN-GB">SAM MPI probes</span>'''

*[http://www.eu-emi.eu/releases/emi-3-monte-bianco/updates/-/asset_publisher/5Na8/content/update-22-10-11-2014- * EMI-UI/WN

* EGI/EMI meeting on MEDIA collaboration and URT activities in preparation to CF13 an * preparation of EMI 1/2 DPM decommissioning campaign

* emi-cluster v. 2.0.1 ( 7 months) ** The nagios probe '''eu.egi.sec.DPM-GLUE2-EMI-1''' should be modified because it tries to detect some information that th

==== 1.1. Update on the status of EMI-1 release (Cristina) ==== EAs that did not yet send feedback, please do so. Plans to move to EMI components or not.<br>

* kickoff meeting of working group responsible of assessing the new EMI 2/3 probes before integration into SAM * Preparation of the EMI WN-tarball, by colecting all the interested sites to serve as Early adopter

* emi-cluster v. 2.0.1 ** ngi-de-nagios.gridka.de - [https://ggus.eu/index.php?mode=ticket_info&ticket_id=105128 NG

** EMI apel v. 2.2.0 ** EMI glexec v. 1.2.1

== Notifications from ARGO about the nagios probes failures == == Decommissioning EMI WMS ==

* EMI-UI/WN * emi-wn v. 3.1.0 ('''new''')

== Notifications from ARGO about the nagios probes failures == == Decommissioning EMI WMS ==

...s metapackage contains all the procedures to install and configure the EMI probes. We should assess the impact.

...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS. * emi-cluster v. 2.0.1

...s metapackage contains all the procedures to install and configure the EMI probes. We should assess the impact. * 7 clients now publishing using SSM 2.0/EMI 3 APEL Client;

* organization of EMI WN tarball staged rollout ...G: 1 advisory issued for 'High' risk issue, which was past TD and fixed by EMI but not in UMD - issued as 'Amber'

...aining sites were moved to ROC_LA. At the end of the quarter following SAM/Nagios instances were in production: Detailed list of SAM/Nagios instances can be found on the following page: [[SAM Instances]].

== Notifications from ARGO about the nagios probes failures == == Decommissioning EMI WMS ==

* EMI repository will be shut down on June 15th, broadcast sent on May 31st https * ARC-CE probes are updated in order to mitigate the issue with missing jobs (https://ggus.

...nts</span>'''<span lang="EN-US"> have had their code checked</span> within EMI activities.&nbsp; Probes: it’s up to EMI, we should investigate with them.

== Notifications from ARGO about the nagios probes failures == == Decommissioning EMI WMS ==

...s of gLite decommissioning and of preparation of EMI 1 probes to start the EMI 1 decommissioning campaign ...valuation in preparation to SAM Update 22 which will include the new EMI 2 probes

...I_IE (Ireland) and the new NGI_UK. At the end of the quarter following SAM/Nagios instances were in production: Detailed list of SAM/Nagios instances can be found on the following page: [[SAM Instances]].

* assessment of progress of EMI-1 campaign * definition of new EMI-1 StoRM support calendar with PT

* SAM: Integration of EMI probes ...uires full reinstallation of SAM instance because upgrade from gLite UI to EMI-2 is not supported. Several SAM tests were renamed which required special

* EMI repository shut down on June 15th, see broadcast sent on May 31st https://o * arc-nagios-probes 1.9.1

...bmission of requirement and assessment of progress of readiness for SHA-2 (EMI) *Follow up with an issue with SAM WN probes

== Notifications from ARGO about the nagios probes failures == == Decommissioning EMI WMS ==

...www.egi.eu/indico/conferenceDisplay.py?confId=823 17/02/2012 MPI VT nagios probes meeting] # Nagios probes: New monitoring probes for the EGI Service Availability Monitor (SAM) has been defined. These wil

== Notifications from ARGO about the nagios probes failures == ...guelcg2 https://ggus.eu/index.php?mode=ticket_info&ticket_id=134931 arc-ce probes return UNKNOWN status

***INFN-ROMA1-CMS: still underperforming, but the bug in the nagios probes for the CREAM (ticket GGUS 128151) is then disappeared, == Decommissioning EMI WMS ==

#'''Nagios''' ##NGI_PL developers are writing probes to monitor glite UI

*[http://www.eu-emi.eu/releases/emi-3-monte-bianco/updates/-/asset_publisher/5Na8/content/update-22-10-11-2014- * EMI-UI/WN

...requirements for EMI 2.0 - deadline is 10 February Middleware requirements Nagios requirement SA1 Survey. 1.3.2 EMI 1.0 (release date is 30 April 2011): Major changes with respect to the curr

| ASecureInfrastructure_HowToImprove = Add more probes to security monitoring in order to check known vulnerabilities. ..._Assessment = Croatian NGI participates in staged rollout of APEL and SAM/Nagios. Staged rollout is well organized with well defined procedures.

...for staged rollout. Problematic ggus tickets. ARC probes in Nagios. New CA probes. Ops tools info. |24 attendants. Staged rollout info. Information about EMI 1.0 release. Operational issues now collected on wiki. Ops tools info. COD

* Probes: https://poem.egi.eu/poem/admin/poem/public_probe/ ...RGO instances: https://argo-mon.egi.eu/nagios and https://argo-mon2.egi.eu/nagios. Alarms for these tests are opened directly in the Operations Portal Dashbo

== Notifications from ARGO about the nagios probes failures == ...guelcg2 https://ggus.eu/index.php?mode=ticket_info&ticket_id=134931 arc-ce probes returned UNKNOWN status; Problem with org.nordugrid.ARC-CE-submit-ops test

...leases/2.0/2.10.0/ AppDB info] (CentOS 7): CernVM-FS 2.3.3, dCache 3.0.10, EMI WN 4.0.2, FTS 3.5.8 == Decommissioning EMI WMS ==

== Notifications from ARGO about the nagios probes failures == == Decommissioning EMI WMS ==

*SA1: PQ8 Grid software maintenance and support after EMI/IGE. Issues are being handled and will be dwelt with at the Nov TCB * EMI all hands meeting

Few bugs regarding the notifications mechanism on EMI-3 VOMS have been identified. The VOMS development team was notified, bugs h ...running middleware versions on sites) have been applied. Also several new probes have been added that enable and assist the campaign for unsupported middlew

*EMI WN 2015 -&gt; to be changed together with UI ...NGI ARGUS servers are not properly configured and they are not passing the nagios tests (eu.egi.Argus-DNs probe):

== Notifications from ARGO about the nagios probes failures == ...NGI ARGUS servers are not properly configured and they are not passing the nagios tests (eu.egi.Argus-DNs probe):

waiting for update 17 for NGI-DE Nagios probes - Update of one CE, SE, UI and WNs to SL6 x86_64 and EMI-2

[[UMD-1:UMD-1.3.0#emi.bdii-site.sl5.x86_64| Known issues]] affecting the BDII version 1.0.1 **asked EMI to consider this version the default (it isn't yet)

#Nagios ##Preparation of Unicore Nagios probes for EMI-3

* status assessment of EMI-1 decommissioning campaign (it is proceeding very well) ** EMI px 1.3.32

Regular maintenance of national SAM/Nagios, Top BDII, national operations portal. New LB machine lb1.grid.cesnet.cz installed with EMI middleware, it will

...tOS 7): ARC 15.03 update 12, APEL Client/Server 1.5.1-1, APEL SSM 2.1.7-1, emi-WN-4.0.1-1 == Decommissioning EMI WMS ==

* EMI Vulnerability Assessment of VOMS Core * Update of the EGI Software Vulnerability Group/EMI Vulnerability Assessment plan, including status report. (This is for pro-ac

====1.1. Update on the status of EMI updates ==== ...ous meeting] are all confirmed, and currently have successfully passed the EMI certification.

...d. The first UNICORE services have been added to GOCDB. UNICORE SAM Nagios probes should be included in SAM Release 13 which has been shifted to August. Globus SAM Nagios Probes have been included already by SAM Release 11.

* Status assessment of readiness of infrastructure for start of the EMI 1 decommissioning campaign ...however most of the approved doc is agnostic to whether the 'provider' is EMI or IGE or development teams. However, contact details still need to be mai

== EMI-2 decommissioning == == SAM Nagios probes re-factoring ==

...pment of new probes and extension of the operations dashboard, handling of probes with the upgrade * establishment of a dedicated Nagios box to monitor MW components

...less time consuming. One example of this is the implementation of the A/R nagios probe in 2012. * '''Unknown Issue'''. Nagios probes to generate alarms for the unknown issue.

*** ARC nagios-probes 1.9.1 - bug fixes, see release notes http://www.nordugrid.org/arc/releases/ *** ARC nagios-probes 1.9.1 - bug fixes, see release notes http://www.nordugrid.org/arc/releases/

...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with ...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS.

...D 2.4.0, taking into account what is left now from IGE 3.0 and the several EMI updates, as well as what should be out in the next emi2 updates of December ** EMI.wms.sl6.x86_64-3.4.0

*globus-GRIDFTP - hr.srce.GridFTP-Transfer, org.nagios.GridFTP-Check<br> *globus-GSISSHD - org.nagios.gsissh-Check<br>

=== 1.1 EMI-1 release status (Cristina) === *EMI Update 2 '''23.06.2011'''

...s are found in SR that are not addressed and these impact our smaller VOs. EMI-2 rollout has gone well with minimal disruption. NGI developed a VOMS confi ...ctive in feeding back issues and quickly implementing new versions. Nagios probes and Paikiti actively used and stable. Management for smaller VOs has been s

= EMI 2 retirement campaign = According to our monitoring, your site is running EMI 2 software,

...xec capabilities of the CEs accepting pilot jobs. One of the steps for the nagios administrators is to request the "/pilot" role for the VO ops. In the next *3rd level EMI Support unit -- caNL &nbsp;

...pDB info] (CentOS 7): ARC 15.03 update 12, CernVM-FS 2.3.5, dCache 3.0.12, emi-UI 4.0.2, gfal2 2.13.3, gfal2-utils 1.5.0, glite-ce-cream-client 1.15, glit == Failures with the updated CREAM probes ==

** [http://www.eu-emi.eu/releases/emi-3-monte-bianco/updates/-/asset_publisher/5Na8/content/update-19-07-08-2014- *** contains fixes provided by [http://www.eu-emi.eu/releases/emi-3-monte-bianco/updates/-/asset_publisher/5Na8/content/update-19-07-08-2014-

...f Grid Middleware used in the EGI infrastructure was produced jointly with EMI, availabe at DocDB [https://documents.egi.eu/secure/ShowDocument?docid=563 ===Nagios security monitoring===

...EMI or to glite 3.2. By default there will be backport of new features in EMI 1.0 to glite 3.2. * Gathering of requirements for EMI 2.0 - deadline was 10 February (Peter Solagna)

* EMI repository shut down on June 15th https://operations-portal.egi.eu/broadcas ***INFN-ROMA1-CMS: bug in the nagios probes for the CREAM, ticket GGUS 128151

...he more precise deployment schedules introduced with the official start of EMI 1 will make the process even smoother. ...g. The switchover went essentially smooth, with no major problems. The ARC probes have continuously improved.

* definition of decommissioning calendar for remaining glite 3.2 products and EMI 1 * definition of new probes needed for future mw upgrade campaigns

...f current activities, and promote new ones: revision of current and future probes, support to emerging NGIs and new site administrators, technical revision o ...he decommissioning of unsupported software, followup of missing/incomplete probes, procedures for ticketing of sites and related escalation

* unicore6.Gateway - emi.unicore.Gateway * unicore6.Registry - emi.unicore.Registry

...lacement of old worker nodes) capacities. Migration to the supported glite(EMI) releases. Stable operation with high outcome to the supported vo's. ...to migrate their Globus resources to EGI / NGI_DE; migrated Globus Nagios probes to IGE

#Nagios ##Preparation of Unicore Nagios probes for update SAM17.1

...es and NGIs. A few issues were identified and corrected and several Nagios probes have been updated during this process. An abstract on EGI security monitori ...continue improving co-ordination of fixing and release of advisories, with EMI and EGI DMSU, to ensure advisories issued when software is released. EGI C

Migration of dteam VO from VOMRS to latest EMI-3 VOMS has been completed in Sep 2013. Procedure was discussed and agreed d ...running middleware versions on sites) have been applied. Also several new probes have been added that enable and assist the campaign for unsupported middlew

* Core Grid services provided by NGI_IT have been upgraded to EMI-1 or EMI-2. ...ther consolidated. Discussions with pS-MDM team on possible integration of probes still on going.

...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with ...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS.

== Decommissioning EMI WMS == == webdav probes in production ==

...HEP: undeperforming in the past months for a network issue with one of the nagios server, now it seems solved: https://ggus.eu/index.php?mode=ticket_info&tic == Decommissioning EMI WMS ==

* ongoing extension of security Nagios system for monitoring of sites hosting unsupported gLite 3.1 and 3.2 servic ...the SW provisioning workflow, including repositories, after the end of the EMI and IGE (main EGI Tech providers) on April 2013.

...ery successful. We are now planning for the handling of the end of life of EMI 1 by the end of April 2013. ...nd end of support and also for implementing NGI SAM instances. A dedicated Nagios box was established to monitor middleware components.

...urrently the probes report to both the old and the new servers. New NAGIOS probes were developed Experience of handling vulnerabilities after the end of EMI and IGE shows that the Software Vulnerabiliy Group (SVG) communication and

** new probes and monitoring of site status * handling topics of the SAM nagios WG.

...d in January and February 2011 to provide input to the EMI project for the EMI 2.0 release. This is an important milestone as for the first time the Opera ...security assessment plan which identifies which software components within EMI are going to be assessed and the related timing. The plan also states which

== Decommissioning EMI WMS == == webdav probes in production ==

##Testing UNICORE EMI-1 and EGI-preview releases<br> ##Supporting EGI effort to integrate PL-Grid UNICORE probes with SAM-Nagios

*EMI WN 2015 -&gt; to be changed together with UI == Decommissioning EMI WMS ==

= EMI Review of the EGI QC Documents (Version 2) = Original Document is available at https://twiki.cern.ch/twiki/bin/view/EMI/EmiEgiQcReviewV2

** Central OPS-Monitor instance (https://ops-monitor.cern.ch/nagios/) was deployed in November 2012 after SAM Update-19 release. ...OD and TEST broker networks stability. This serves complimentary to set of nagios endpoints monitoring the PROD MSG infrastructure in case a test needs to be

...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with ...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS.

== Decommissioning EMI WMS == == webdav probes in production ==

* emi-ui v. 3.1.0 (https://ggus.eu/index.php?mode=ticket_info&ticket_id=109527) * emi-wn v. 3.1.0

...In order to assure the correct execution of these applications, monitoring probes that check the proper support for such jobs are available. The Service Avai ...t are currently under development and described at the [[VT_MPI_within_EGI:Nagios]] wiki page.

* emi-wn v. 3.1.0 (including gfal2) * emi-ui v. 3.1.0 (including gfal2)

SAM U21 ongoing: integration of the EMI probes * #196: NAGIOS probes for Accounting Portal - It is done

...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with ...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS.

| Decision to integrate IPv6 testbeds servers and resources of HEPiX, EGI and EMI and to support 3 VOs. ...on June 14-15 it has been agreed to merge testbed resources between HEPiX, EMI and EGI. Testers have been identified for UNICORE components using IPv6, at

...ces (gLite 3.2 DPM, LFC and Worker Nodes) on NGI_AEGIS resource centers to EMI 2. * Maintenance and update of NGI_AEGIS Nagios box, responsible for monitoring of NGI_AEGIS and NGI_GE, to release 19.

...dleware internals and of the mw general framework. Specific training about EMI middleware and its evolution would probably help. ...a dedicated national monitoring framework and the development of security probes) and an auditing activity (in collaboration with other groups doing similar

...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with ...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS.

* EMI 20 update ( APEL Parsers, v. 2.3.1, APEL publiser 4.2.1, BDII core 1.6.0, ...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with

...counting system (BAT), GridSafe, and APEL SSM v0.2 (soon to be replaced by EMI-CAR). ...e QCG platform will integrate with the EGI SAM system by including the QCG Nagios plugins in EGI SAM for NGI-wide deployment.

* emi-wn v. 3.1.0 * emi-ui v. 3.1.0

#'''Nagios''' ##Testing instance: Actualization to version SAM15 and UNICORE probes testing<br>

...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with ...that broke in the previous release, but there is also an extension to the emi-es schema to make it useable for ATLAS.

...e further tested in the forthcoming months, jointly with the deployment of probes at various pilot sites. ...vice. High availability and reliability of the WMS clusters are ensured by Nagios checks and the load balancing based on WMSMonitor [https://cert-wms-01.cnaf

=== Issue with '''eu.egi.sec.DPM-GLUE2-EMI-1''' probe === * The nagios probe '''eu.egi.sec.DPM-GLUE2-EMI-1''' should be modified because it tries to detect some information that th

...in the WLCG monitoring group including help in testing new LCG-SAM Nagios Probes and gathering UK feedback on current monitoring infrastructures. * Additionally Completed the migration of UKI-SCOTGRID-GLASGOW to EMI-3 including the deployment of new management tools.

...the EGI production infrastructure, will terminate in 2013. Among these are EMI (ARC, dCache, gLite, UNICORE) and IGE (Globus). EGI needs to define the lis ...supported will increase with EMI 2.0, which is expected in Spring 2012: in EMI 2.0 it is expected that all products will be released for sl6, and a subset

...fined at the beginning of PY2 to identify which software components within EMI would be assessed and when the assessments would take place. The document a ...improve accuracy of tests and to limit the number of false positives. The Nagios instance that is responsible of security monitoring was moved under the egi

* ARGO: module for probes management under development, new messaging infrastructure under testing * Follow-up of the issues with CREAM-CE and ARC-CE nagios probes (solved)

...13-1/ Revision update UMD-3.13.1] released on 13/07/2015 fixing ARC Nagios probes for SL5 ...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with

...13-1/ Revision update UMD-3.13.1] released on 13/07/2015 fixing ARC Nagios probes for SL5 ...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with

...oject proposal but which did never happen, becomes more realistic. Because EMI funding stops in May 2013, the following 12 months will show the real impac With the end of two of the main development projects in middle 2013, IGE and EMI, there is an uncertainty over what will happen to the product teams of many

...13-1/ Revision update UMD-3.13.1] released on 13/07/2015 fixing ARC Nagios probes for SL5 ...ment about the updates introduced and where they are going to be released (EMI repositories, EPEL, private repositories ecc), linking external pages with

The maui version provided as third party in EMI/UMD (maui-3.2.6p21-snap.1234905291.5.el5) has a bug that prevents the use o ...allow users to compile their applications at the site. Moreover the Nagios probes will try to compile a binary, thus not having a working compiler will make

...key cornerstones relevant for this Capability: Nagios and related service probes as the functional monitoring data, the MyEGI portal to visualise and report ...tion point in that service-specific probes are provided that plug into the Nagios service-monitoring tool. On the monitoring data presentation/query layer, b