Alert.png The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.

Difference between revisions of "Preview 1.29.0"

From EGIWiki
Jump to navigation Jump to search
(Created page with "Category:Middleware Back to [https://wiki.egi.eu/wiki/Preview_Repository#Repository_configuration repository configuration] = ARC 6.8.0 = The [http://www.nordugrid.org/a...")
 
 
Line 2: Line 2:
Back to [https://wiki.egi.eu/wiki/Preview_Repository#Repository_configuration repository configuration]
Back to [https://wiki.egi.eu/wiki/Preview_Repository#Repository_configuration repository configuration]


= ARC 6.8.0 =
= ARC 6.8.0 and 6.8.1 =


The [http://www.nordugrid.org/arc/ Advanced Resource Connector (ARC) middleware] is an Open Source software solution to enable distributed computing infrastructures with the emphasis on processing large volumes of data. ARC provides an abstraction layer over computational resources, complete with input and output data movement functionalities. The security model of ARC is identical to that of Grid solutions, relying on delegation of user credentials and the concept of Virtual Organisations. ARC also provides client tools, as well as API in C++, Python and Java.
The [http://www.nordugrid.org/arc/ Advanced Resource Connector (ARC) middleware] is an Open Source software solution to enable distributed computing infrastructures with the emphasis on processing large volumes of data. ARC provides an abstraction layer over computational resources, complete with input and output data movement functionalities. The security model of ARC is identical to that of Grid solutions, relying on delegation of user credentials and the concept of Virtual Organisations. ARC also provides client tools, as well as API in C++, Python and Java.
Line 50: Line 50:


Production Sites already running ARC 5 will still get operational site support in form of deployment and configuration troubleshooting help via GGUS until end June 2021. However, we may recommend upgrading to ARC 6 depending on the issues the site is facing.
Production Sites already running ARC 5 will still get operational site support in form of deployment and configuration troubleshooting help via GGUS until end June 2021. However, we may recommend upgrading to ARC 6 depending on the issues the site is facing.
== 6.8.1 ==
This is an express release which targets purely the arc.conf changes related to the  deprecated arc.conf values in ARC 6.8.0.
In ARC 6.8.0 the ARC validator would prevent ARC from starting up if any of the deprecated blocks and options were present in arc.conf. From 6.8.1 ARC will start even with the deprecated options in place. Note however that they are still deprecated and should be removed. The options will have no effect to your runninginstance of ARC, thus of no value.


= BDII 5.5.26 =
= BDII 5.5.26 =

Latest revision as of 10:58, 9 October 2020

Back to repository configuration

ARC 6.8.0 and 6.8.1

The Advanced Resource Connector (ARC) middleware is an Open Source software solution to enable distributed computing infrastructures with the emphasis on processing large volumes of data. ARC provides an abstraction layer over computational resources, complete with input and output data movement functionalities. The security model of ARC is identical to that of Grid solutions, relying on delegation of user credentials and the concept of Virtual Organisations. ARC also provides client tools, as well as API in C++, Python and Java.

We are pleased to announce the release of ARC 6.8.0. This release comes with two important changes that affect the update, namely changes in the accounting system, and a dependency on xrootd5. Both will need manual steps when updating to ARC 6.8.0, the latter only if your CE has the nordugrid-plugins-xrootd package installed.

Accounting changes

The largest change in this release is the removal of the pre 6.4.0 accounting system. Since the accounting subsystem changes in 6.4.0 where we went from files to database, the old system has lived alongside the new. From 6.8.0 the legacy code is all removed. For more details of the consequences, and especially changes in arc.conf, http://www.nordugrid.org/arc/arc6/admins/reference.html#configuration-options-removed-in-arc-6-8-0

If you want to read more about the accounting changes, please consult this file: http://www.nordugrid.org/documents/jura-ng.html And finally http://www.nordugrid.org/documents/arc6/admins/details/accounting-ng.html for the description of the new system.

Please be aware that you will need to make changes in your arc.conf for ARC to start with 6.8.0.

Options and blocks that are no longer supported in arc.conf:

* In the [arex/jura] block: 
    * urdelivery_keepfailed (in [arex/jura] block)
* In the [arex/jura/sgas:targetname] block: 
    * legacy_fallback 
* In the [arex/jura/apel:targetname] block: 
      * use_ssl
      * benchmark_type
      * benchmark_value
      * benchmark_description
      * legacy_fallback
* [arex/jura/archiving] block  (the whole block is deprecated)
      * archivedir
       * logfile
       * archivettl


Important note on xrootd version for sites using the nordugrid-arc-plugins-xrootd package

For sites who install the nordugrid-arc-plugins-xrootd package note that ARC 6.8.0 is built with xrootd 5. On the date of the release xrootd 5 has not yet reached EPEL-stable which means that to install(update) ARC 6.8.0 on RHEL-systems, you must enable the epel-testing repo:

yum install(update) "nordugrid-arc*" --enablerepo=epel-testing

or alternatively install xrootd 5 before updating ARC.

For more details on bugs and highlights, please consult the release notes: http://www.nordugrid.org/arc/releases/6.8/release_notes_6.8.html

Future Support of ARC 5-series

As of June 2020 no more security updates are provided to ARC 5. Also since the release of ARC 6 in June 2019, no more development on the ARC 5 code base has been provided.

Production Sites already running ARC 5 will still get operational site support in form of deployment and configuration troubleshooting help via GGUS until end June 2021. However, we may recommend upgrading to ARC 6 depending on the issues the site is facing.

6.8.1

This is an express release which targets purely the arc.conf changes related to the deprecated arc.conf values in ARC 6.8.0.

In ARC 6.8.0 the ARC validator would prevent ARC from starting up if any of the deprecated blocks and options were present in arc.conf. From 6.8.1 ARC will start even with the deprecated options in place. Note however that they are still deprecated and should be removed. The options will have no effect to your runninginstance of ARC, thus of no value.

BDII 5.5.26

The Berkeley Database Information Index (BDII) consists of two or more standard LDAP databases that are populated by an update process. Port forwarding is used to enable one or more databases to serve data while one database is being refreshed. The databases are refreshed cyclically. Any incoming connection is forwarded to the most recently updated database, while old connections are allowed to linger until it is the turn of their database to be refreshed and restarted. The update process obtains LDIF from either doing an ldapsearch on LDAP URLs or by running a local script (given by a URL with "file" protocol) that generates LDIF. The LDIF is then inserted into the LDAP database. Options exist to update the list of LDAP URLs from a web page and to use an LDIF file from a web page to modify the data before it is inserted into the database.

See more information here

Release notes: https://github.com/EGI-Foundation/bdii/releases/tag/v5.2.26

CVMFS 2.7.4

The CernVM File System provides a scalable, reliable and low-maintenance software distribution service. It was developed to assist High Energy Physics (HEP) collaborations to deploy software on the worldwide-distributed computing infrastructure used to run data processing applications. CernVM-FS is implemented as a POSIX read-only file system in user space (a FUSE module). Files and directories are hosted on standard web servers and mounted in the universal namespace /cvmfs.

Release notes

CernVM-FS 2.7.4 is a patch release. It contains bugfixes and improvements for Ubuntu clients and servers. At the same time, we release a new cvmfs-config-default package: please see detailed release notes in the technical documentation.

dCache 5.2.31

dCache provides a system for storing and retrieving huge amounts of data, distributed among a large number of heterogeneous server nodes, under a single virtual filesystem tree with a variety of standard access methods.

Detailed release notes on the official product site: https://www.dcache.org/downloads/1.9/release-notes-5.2.shtml

DMLite/DPM 1.14.0

The Disk Pool Manager (DPM) is a lightweight storage solution for grid sites. It offers a simple way to create a disk-based grid storage element and supports relevant protocols (SRM, gridFTP, RFIO) for file management and access. It focus on manageability (ease of installation, configuration, low effort of maintenance), while providing all required functionality for a grid storage solution (support for multiple disk server nodes, different space types, multiple file replicas in disk pools)

N.B. the gridftp and http frontends are now build together with the dpm core, so they have new package names

  • lcgdm-dav -> dmlite-apache-httpd
  • dpm-dsi -> dmlite-dpm-dsi

as well as the xrootd frontend starting from v 1.12: dpm-xrootd -> dmlite-dpm-xrootd

frontier-squid 4.13.1

The frontier-squid software package is a patched version of the standard squid http proxy cache software, pre-configured for use by the Frontier distributed database caching system. This installation is recommended for use by Frontier in the LHC CMS & ATLAS projects, and also works well with the CernVM FileSystem. Many people also use it for other applications as well

Release notes: http://frontier.cern.ch/dist/rpms/frontier-squidRELEASE_NOTES

glite-info-update-endpoints 3.0.2

This component is used with Top BDII and is intented to update LDAP endpoits for EGI. BDII documentation is available here: https://gridinfo-documentation.readthedocs.io/

glite-info-update-endpoints is a cron job that runs every hour to download the list of site BDII URLs that are going to be used by the top level BDII to publish their resources.

The script uses the /etc/glite/glite-info-update-endpoints.conf file which by default is configured to use EGI's list of site BDIIs. The list of site BDIIs is taken from the EGI GOCDBs.

See more information here.

Release notes: https://github.com/EGI-Foundation/glite-info-update-endpoints/releases/tag/v3.0.2

lcg-info 1.12.5

This command line tool allows the user to query the WLCG/EGI LDAP-based information system. More information here.

Release notes:

  • Bug fix for GGUS 140478: Try all host addresses after a configurable timeout

STORM 1.11.18

The StoRM Product Team is pleased to announce the release of StoRM 1.11.18 that includes the following updated components:

This release introduces the support for CentOS 7 for all StoRM components.

It also provides fixes to some outstanding bugs, in particular:

  • fixes errors on published storage space occupancy in case multiple storage area shares the same VO;
  • fixes not published WebDAV endpoints when latest logic is used;
  • fixes not dropped Authorization header in WebDAV TPC redirects;
  • fixes leaked file descriptors when Conscrypt is enabled on StoRM WebDAV;
  • sets correctly HTTP content-length for large files;
  • fixes errors on transferred files through GridFTP that leave empty files with an adler32 checksum for a non-empty file;
  • fixes KillMode on GridFTP systemd unit.

It also provides several improvements, in particular:

  • fixes wrong ERROR log messages when file does not exist on srmRm requests;
  • changes the way info provider checks if Backend is running;
  • introduces a Background DU Service (disabled by default) that periodically updates the storage space info for non-GPFS storage areas (read more info here);
  • adds Date and thread pools metrics in the metrics logged info;
  • updates spring boot to 2.2.6 release for StoRM WebDAV;
  • adds SystemD support for StoRM Backend and StoRM Frontend.

Please, follow the upgrade instructions.

Starting from StoRM v1.11.18 StoRM production repository has been migrated. Read how to install/upgrade StoRM repositories into the Downloads section.

Read the release notes for more details.