Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Preview 2.30.0"

From EGIWiki
Jump to navigation Jump to search
(Created page with "Category:Middleware Back to [https://wiki.egi.eu/wiki/Preview_Repository#Repository_configuration repository configuration] = ARC 6.8.0 and 6.8.1 = The [http://www.nordu...")
 
 
(20 intermediate revisions by the same user not shown)
Line 2: Line 2:
Back to [https://wiki.egi.eu/wiki/Preview_Repository#Repository_configuration repository configuration]
Back to [https://wiki.egi.eu/wiki/Preview_Repository#Repository_configuration repository configuration]


= ARC 6.8.0 and 6.8.1 =
= APEL-SSM 3.0.1 =


The [http://www.nordugrid.org/arc/ Advanced Resource Connector (ARC) middleware] is an Open Source software solution to enable distributed computing infrastructures with the emphasis on processing large volumes of data. ARC provides an abstraction layer over computational resources, complete with input and output data movement functionalities. The security model of ARC is identical to that of Grid solutions, relying on delegation of user credentials and the concept of Virtual Organisations. ARC also provides client tools, as well as API in C++, Python and Java.
Secure STOMP Messenger (SSM) is designed to simply send messages using the [http://stomp.github.io/ STOMP protocol] or via the ARGO Messaging Service (AMS). Messages are signed and may be encrypted during transit. Persistent queues should be used to guarantee delivery.


We are pleased to announce the release of ARC 6.8.0. This release comes with two important changes that affect the update, namely changes in the accounting system, and a dependency on xrootd5. '''Both will need manual steps when updating to ARC 6.8.0''', the latter only if your CE has the nordugrid-plugins-xrootd package installed.  
SSM is written in Python. Packages are available for RHEL 7 and Ubuntu Trusty.


'''Accounting changes'''
The installation and configuration guide is available here: https://github.com/apel/ssm


The largest change in this release is the removal of the pre 6.4.0 accounting system. Since the accounting subsystem changes in 6.4.0 where we went from files to database, the old system has lived alongside the new. From 6.8.0 the legacy code is all removed. For more details of the consequences, and '''especially changes in arc.conf''',
Check also the EGI wiki for more information about [[APEL/SSM2Overview|APEL SSM]].
http://www.nordugrid.org/arc/arc6/admins/reference.html#configuration-options-removed-in-arc-6-8-0


If you want to read more about the accounting changes, please consult this file: http://www.nordugrid.org/documents/jura-ng.html
[https://github.com/apel/ssm/releases Release notes]:
And finally http://www.nordugrid.org/documents/arc6/admins/details/accounting-ng.html for the description of the new system.  
* As part of the migration to Python 3, this release removes support for Python 2.6.
* The ''argo-ams-library'' is now an optional module, depending on whether the Argo Messaging Service is used or not.
* Fixed how the ''use_ssl'' config option is retrieved to avoid "referenced before assignment" errors leading to SSM failing to run.
* Changes made to messaging loops to avoid high CPU usage.


Please be aware that you will need to '''make changes in your arc.conf for ARC to start with 6.8.0'''.
= CVMFS 2.7.5 and egi-cvmfs-2-7.12 =
 
Options and blocks that are no longer supported in arc.conf:
 
* In the [arex/jura] block:
    * urdelivery_keepfailed (in [arex/jura] block)
* In the [arex/jura/sgas:targetname] block:
    * legacy_fallback
* In the [arex/jura/apel:targetname] block:
      * use_ssl
      * benchmark_type
      * benchmark_value
      * benchmark_description
      * legacy_fallback
* [arex/jura/archiving] block  (the whole block is deprecated)
      * archivedir
        * logfile
        * archivettl
 
 
'''Important note on xrootd version for sites using the nordugrid-arc-plugins-xrootd package'''
 
For sites who install the nordugrid-arc-plugins-xrootd package note that ARC 6.8.0 is built with '''xrootd 5'''.
On the date of the release xrootd 5 has not yet reached EPEL-stable which means that to install(update) ARC 6.8.0 on RHEL-systems, you must enable the epel-testing repo:
yum install(update) "nordugrid-arc*" --enablerepo=epel-testing
or alternatively install xrootd 5 before updating ARC.  
 
For more details on bugs and highlights, please consult the '''release notes: http://www.nordugrid.org/arc/releases/6.8/release_notes_6.8.html'''
 
'''Future Support of ARC 5-series'''
 
As of June 2020 no more security updates are provided to ARC 5. Also since the release of ARC 6 in June 2019, no more development on the ARC 5 code base has been provided.
 
Production Sites already running ARC 5 will still get operational site support in form of deployment and configuration troubleshooting help via GGUS until end June 2021. However, we may recommend upgrading to ARC 6 depending on the issues the site is facing.
 
== 6.8.1 ==
This is an express release which targets purely the arc.conf changes related to the deprecated arc.conf values in ARC 6.8.0.
 
In ARC 6.8.0 the ARC validator would prevent ARC from starting up if any of the deprecated blocks and options were present in arc.conf. From 6.8.1 ARC will start even with the deprecated options in place. Note however that they are still deprecated and should be removed. The options will have no effect to your runninginstance of ARC, thus of no value.
 
= BDII 5.5.26 =
The Berkeley Database Information Index (BDII) consists of two or more standard LDAP databases that are populated by an update process. Port forwarding is used to enable one or more databases to serve data while one database is being refreshed. The databases are refreshed cyclically. Any incoming connection is forwarded to the most recently updated database, while old connections are allowed to linger until it is the turn of their database to be refreshed and restarted. The update process obtains LDIF from either doing an ldapsearch on LDAP URLs or by running a local script (given by a URL with "file" protocol) that generates LDIF. The LDIF is then inserted into the LDAP database. Options exist to update the list of LDAP URLs from a web page and to use an LDIF file from a web page to modify the data before it is inserted into the database.
 
See more information [https://github.com/EGI-Foundation/bdii here]
 
'''Release notes''': https://github.com/EGI-Foundation/bdii/releases/tag/v5.2.26
 
= CVMFS 2.7.4 =


The [http://cernvm.cern.ch/portal/filesystem CernVM File System] provides a scalable, reliable and low-maintenance software distribution service. It was developed to assist High Energy Physics (HEP) collaborations to deploy software on the worldwide-distributed computing infrastructure used to run data processing applications. CernVM-FS is implemented as a POSIX read-only file system in user space (a [https://github.com/libfuse/libfuse FUSE] module). Files and directories are hosted on standard web servers and mounted in the universal namespace /cvmfs.
The [http://cernvm.cern.ch/portal/filesystem CernVM File System] provides a scalable, reliable and low-maintenance software distribution service. It was developed to assist High Energy Physics (HEP) collaborations to deploy software on the worldwide-distributed computing infrastructure used to run data processing applications. CernVM-FS is implemented as a POSIX read-only file system in user space (a [https://github.com/libfuse/libfuse FUSE] module). Files and directories are hosted on standard web servers and mounted in the universal namespace /cvmfs.
Line 69: Line 24:
'''Release notes'''
'''Release notes'''


CernVM-FS 2.7.4 is a patch release. It contains bugfixes and improvements for Ubuntu clients and servers. At the same time, we release a new cvmfs-config-default package: please see detailed release notes in the [https://cvmfs.readthedocs.io/en/2.7/cpt-releasenotes.html technical documentation].
CernVM-FS 2.7.5 is a patch release. It contains bugfixes and improvements for Ubuntu clients and servers. Please see detailed release notes in the [https://cvmfs.readthedocs.io/en/2.7/cpt-releasenotes.html technical documentation].


= dCache 5.2.31 =
= dCache 5.2.35 =


[https://www.dcache.org/index.shtml dCache] provides a system for storing and retrieving huge amounts of data, distributed among a large number of heterogeneous server nodes, under a single virtual filesystem tree with a variety of standard access methods.
[https://www.dcache.org/index.shtml dCache] provides a system for storing and retrieving huge amounts of data, distributed among a large number of heterogeneous server nodes, under a single virtual filesystem tree with a variety of standard access methods.
Line 77: Line 32:
'''Detailed release notes''' on the official product site: https://www.dcache.org/downloads/1.9/release-notes-5.2.shtml
'''Detailed release notes''' on the official product site: https://www.dcache.org/downloads/1.9/release-notes-5.2.shtml


= DMLite/DPM 1.14.0 =
= DMLite/DPM 1.14.2 =


The [http://lcgdm.web.cern.ch/dpm Disk Pool Manager] (DPM) is a lightweight storage solution for grid sites. It offers a simple way to create a disk-based grid storage element and supports relevant protocols (SRM, gridFTP, RFIO) for file management and access. It focus on manageability (ease of installation, configuration, low effort of maintenance), while providing all required functionality for a grid storage solution (support for multiple disk server nodes, different space types, multiple file replicas in disk pools)
The [http://lcgdm.web.cern.ch/dpm Disk Pool Manager] (DPM) is a lightweight storage solution for grid sites. It offers a simple way to create a disk-based grid storage element and supports relevant protocols (SRM, gridFTP, RFIO) for file management and access. It focus on manageability (ease of installation, configuration, low effort of maintenance), while providing all required functionality for a grid storage solution (support for multiple disk server nodes, different space types, multiple file replicas in disk pools)
Line 90: Line 45:
as well as the xrootd frontend starting from v 1.12:  dpm-xrootd -> dmlite-dpm-xrootd
as well as the xrootd frontend starting from v 1.12:  dpm-xrootd -> dmlite-dpm-xrootd


= frontier-squid 4.13.1 =
= Dynafed 1.6.0 =
 
The frontier-squid software package is a patched version of the standard [http://squid-cache.org/ squid] http proxy cache software, pre-configured for use by the [https://twiki.cern.ch/twiki/bin/view/Frontier/WebHome Frontier distributed database caching system]. This installation is recommended for use by Frontier in the LHC CMS & ATLAS projects, and also works well with the [http://cernvm.cern.ch/portal/filesystem CernVM FileSystem]. Many people also use it for other applications as well


'''Release notes''': http://frontier.cern.ch/dist/rpms/frontier-squidRELEASE_NOTES
The Dynamic Federations system allows to expose via HTTP and WebDAV a very fast dynamic name space, built on the fly by merging and caching (in memory) metadata items taken from a number of (remote) endpoints. [http://lcgdm.web.cern.ch/dynafed-dynamic-federation-project More information].


= glite-info-update-endpoints 3.0.2 =
= STORM 1.11.19 =
This component is used with Top BDII and is intented to update LDAP endpoits for EGI. BDII documentation is available here: https://gridinfo-documentation.readthedocs.io/


''glite-info-update-endpoints'' is a cron job that runs every hour to download the list of site BDII URLs that are going to be used by the top level BDII to publish their resources.
The StoRM Product Team is pleased to announce the release of [http://italiangrid.github.io/storm/release-notes/StoRM-v1.11.19.html StoRM 1.11.19] that includes the following updated components:
* StoRM Backend [http://italiangrid.github.io/storm/release-notes/storm-backend-server/1.11.19/ v. 1.11.19]
* StoRM WebDAV [http://italiangrid.github.io/storm/release-notes/storm-webdav/1.3.1/ v. 1.3.1]


The script uses the ''/etc/glite/glite-info-update-endpoints.conf'' file which by default is configured to use EGI's list of site BDIIs. The list of site BDIIs is taken from the EGI GOCDBs.
This release mainly:
* fixes a bug introduced with StoRM v1.11.18 about the final update of the status on database of a srmPtG or srmBoL requests;
* introduces new metrics in storm-backend-metrics.log;
* fixes a storm-webdav bug about the returned body content in case of HEAD requests.


See more information [https://github.com/EGI-Foundation/glite-info-update-endpoints here].
Please, follow the [http://italiangrid.github.io/storm/documentation/sysadmin-guide/1.11.19/upgrading/ upgrade instructions].


'''Release notes''': https://github.com/EGI-Foundation/glite-info-update-endpoints/releases/tag/v3.0.2
Read the [http://italiangrid.github.io/storm/release-notes/StoRM-v1.11.19.html release notes] for more details.


= lcg-info 1.12.5 =
= VOMS 10-20 release =


This command line tool allows the user to query the WLCG/EGI LDAP-based information system. More information [https://github.com/EGI-Foundation/lcg-info here].
This new release for several VOMS components fixing outstanding bugs and providing new features. With this release all VOMS components are now ported to CENTOS 7.


'''Release notes''':
The updated components are:
*Bug fix for GGUS 140478: Try all host addresses after a configurable timeout
* [http://italiangrid.github.io/voms/release-notes/voms-admin-server/3.8.0 VOMS Admin server 3.8.0]
* [http://italiangrid.github.io/voms/release-notes/voms-admin-client/2.0.20 VOMS Admin client 2.0.20]
* [http://italiangrid.github.io/voms/release-notes/voms-clients/3.3.2 VOMS clients 3.3.2]
* [http://italiangrid.github.io/voms/release-notes/voms-api-java/3.3.2 VOMS Java APIs 3.3.2]
* [http://italiangrid.github.io/voms/release-notes/voms-server/2.0.15 VOMS server 2.0.15]
* [http://italiangrid.github.io/voms/release-notes/voms-api-c/2.0.15 VOMS C/C++ APIs 2.0.15]
* [http://italiangrid.github.io/voms/release-notes/voms-mysql-plugin/3.1.7-2 VOMS MySQL plugin 3.1.7-2]


= STORM 1.11.18 =
Please have a look at the [http://italiangrid.github.io/voms/releases.html release notes] for more details.
The StoRM Product Team is pleased to announce the release of [http://italiangrid.github.io/storm/release-notes/StoRM-v1.11.18.html StoRM 1.11.18] that includes the following updated components:
*StoRM Backend [http://italiangrid.github.io/storm/release-notes/storm-backend-server/1.11.18/ v. 1.11.18]
*StoRM Frontend [http://italiangrid.github.io/storm/release-notes/storm-frontend-server/1.8.13/ v. 1.8.13]
*StoRM WebDAV [http://italiangrid.github.io/storm/release-notes/storm-webdav/1.3.0/ ]v. 1.3.0
*StoRM GridFTP DSI [http://italiangrid.github.io/storm/release-notes/storm-gridftp-dsi/1.2.4/ v. 1.2.4]
*StoRM Native Libs [http://italiangrid.github.io/storm/release-notes/storm-native-libs/1.0.6/ v. 1.0.6]
*StoRM XMLRPC-C [http://italiangrid.github.io/storm/release-notes/storm-xmlrpc-c/1.39.12/ v. 1.39.12]
*CDMI StoRM [http://italiangrid.github.io/storm/release-notes/cdmi-storm/0.1.1/ v. 0.1.1]
*YAIM StoRM [http://italiangrid.github.io/storm/release-notes/yaim-storm/4.3.13/ v. 4.3.12]
'''This release introduces the support for CentOS 7 for all StoRM components'''.


It also provides fixes to some outstanding bugs, in particular:
= xrootd 4.12.5 and 5.0.3 =
*fixes errors on published storage space occupancy in case multiple storage area shares the same VO;
*fixes not published WebDAV endpoints when latest logic is used;
*fixes not dropped Authorization header in WebDAV TPC redirects;
*fixes leaked file descriptors when Conscrypt is enabled on StoRM WebDAV;
*sets correctly HTTP content-length for large files;
*fixes errors on transferred files through GridFTP that leave empty files with an adler32 checksum for a non-empty file;
*fixes KillMode on GridFTP systemd unit.
It also provides several improvements, in particular:
*fixes wrong ERROR log messages when file does not exist on srmRm requests;
*changes the way info provider checks if Backend is running;
*introduces a Background DU Service (disabled by default) that periodically updates the storage space info for non-GPFS storage areas (read more info [http://italiangrid.github.io/storm/documentation/sysadmin-guide/1.11.18#duserviceconfiguration here]);
*adds Date and thread pools metrics in the metrics logged info;
*updates spring boot to 2.2.6 release for StoRM WebDAV;
*adds SystemD support for StoRM Backend and StoRM Frontend.
Please, follow the [http://italiangrid.github.io/storm/documentation/sysadmin-guide/1.11.18/#upgrading upgrade instructions].


Starting from StoRM v1.11.18 '''StoRM production repository has been migrated'''. Read how to install/upgrade StoRM repositories into the [http://italiangrid.github.io/storm/download.html#stable-releases Downloads] section.
[http://xrootd.org/ XRootD software framework] is a fully generic suite for fast, low latency and scalable data access, which can serve natively any kind of data, organized as a hierarchical filesystem-like namespace, based on the concept of directory. As a general rule, particular emphasis has been put in the quality of the core software parts.


'''Read the [http://italiangrid.github.io/storm/release-notes/StoRM-v1.11.18.html release notes] for more details'''.
'''Detailled release notes''':
* [https://github.com/xrootd/xrootd/blob/v4.12.5/docs/ReleaseNotes.txt 4.12.5]
* [https://github.com/xrootd/xrootd/blob/v5.0.3/docs/ReleaseNotes.txt 5.0.3] '''Only for CentOS 7'''

Latest revision as of 17:05, 27 November 2020

Back to repository configuration

APEL-SSM 3.0.1

Secure STOMP Messenger (SSM) is designed to simply send messages using the STOMP protocol or via the ARGO Messaging Service (AMS). Messages are signed and may be encrypted during transit. Persistent queues should be used to guarantee delivery.

SSM is written in Python. Packages are available for RHEL 7 and Ubuntu Trusty.

The installation and configuration guide is available here: https://github.com/apel/ssm

Check also the EGI wiki for more information about APEL SSM.

Release notes:

  • As part of the migration to Python 3, this release removes support for Python 2.6.
  • The argo-ams-library is now an optional module, depending on whether the Argo Messaging Service is used or not.
  • Fixed how the use_ssl config option is retrieved to avoid "referenced before assignment" errors leading to SSM failing to run.
  • Changes made to messaging loops to avoid high CPU usage.

CVMFS 2.7.5 and egi-cvmfs-2-7.12

The CernVM File System provides a scalable, reliable and low-maintenance software distribution service. It was developed to assist High Energy Physics (HEP) collaborations to deploy software on the worldwide-distributed computing infrastructure used to run data processing applications. CernVM-FS is implemented as a POSIX read-only file system in user space (a FUSE module). Files and directories are hosted on standard web servers and mounted in the universal namespace /cvmfs.

Release notes

CernVM-FS 2.7.5 is a patch release. It contains bugfixes and improvements for Ubuntu clients and servers. Please see detailed release notes in the technical documentation.

dCache 5.2.35

dCache provides a system for storing and retrieving huge amounts of data, distributed among a large number of heterogeneous server nodes, under a single virtual filesystem tree with a variety of standard access methods.

Detailed release notes on the official product site: https://www.dcache.org/downloads/1.9/release-notes-5.2.shtml

DMLite/DPM 1.14.2

The Disk Pool Manager (DPM) is a lightweight storage solution for grid sites. It offers a simple way to create a disk-based grid storage element and supports relevant protocols (SRM, gridFTP, RFIO) for file management and access. It focus on manageability (ease of installation, configuration, low effort of maintenance), while providing all required functionality for a grid storage solution (support for multiple disk server nodes, different space types, multiple file replicas in disk pools)

N.B. the gridftp and http frontends are now build together with the dpm core, so they have new package names

  • lcgdm-dav -> dmlite-apache-httpd
  • dpm-dsi -> dmlite-dpm-dsi

as well as the xrootd frontend starting from v 1.12: dpm-xrootd -> dmlite-dpm-xrootd

Dynafed 1.6.0

The Dynamic Federations system allows to expose via HTTP and WebDAV a very fast dynamic name space, built on the fly by merging and caching (in memory) metadata items taken from a number of (remote) endpoints. More information.

STORM 1.11.19

The StoRM Product Team is pleased to announce the release of StoRM 1.11.19 that includes the following updated components:

This release mainly:

  • fixes a bug introduced with StoRM v1.11.18 about the final update of the status on database of a srmPtG or srmBoL requests;
  • introduces new metrics in storm-backend-metrics.log;
  • fixes a storm-webdav bug about the returned body content in case of HEAD requests.

Please, follow the upgrade instructions.

Read the release notes for more details.

VOMS 10-20 release

This new release for several VOMS components fixing outstanding bugs and providing new features. With this release all VOMS components are now ported to CENTOS 7.

The updated components are:

Please have a look at the release notes for more details.

xrootd 4.12.5 and 5.0.3

XRootD software framework is a fully generic suite for fast, low latency and scalable data access, which can serve natively any kind of data, organized as a hierarchical filesystem-like namespace, based on the concept of directory. As a general rule, particular emphasis has been put in the quality of the core software parts.

Detailled release notes: