EGI Verification Testbed

From EGIWiki
Revision as of 19:53, 9 December 2014 by Krakow (talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search
Technology Software Component Delivery Software Provisioning UMD Middleware Cloud Middleware Distribution Containers Distribution Technology Glossary


Software Provisioning menu: Software Provisioning Process UMD Release Process Quality Assurance UMD Staged Rollout




Virtual Machine Request

Each released product must be installed in different machines. SA2 Verifiers may request for new virtual machines for verification by submitting an email to grid-admin(at)cesga.es. This email should include:

  • Subject: SA2 Verification Tesbed request.
  • Body:
    • Verifier Name.
    • Product name to verify.
    • OS and hardware requirements (if necessary)
    • Verifier SSH public key (generated id_dsa.pub)

The verification team will submit a confirmation email with the details needed to connect to the new machine (public IP, hostname, etc). New VMs have a limited lifetime of 1 week, after this period the VM is destroyed.

Available Services

Service Host Comments
topbdii test03.egi.cesga.es Only contains the Site UMD-VERIFICATION verified
site bdii sbdii02.ncg.ingrid.pt Site Name = UMD-VERIFICATION verified
topBDII test03.egi.cesga.es
Free test04.egi.cesga.es
WMS test22.egi.cesga.es verified
LB tst05.ific.uv.es verified
UI test13.egi.cesga.es Configured VOs: ops.vo.ibergrid.eu, OPS and DTEAM. Configured WMS/LB: tst04.ific.uv.es, tst05.ific.uv.es, test22.egi.cesga.es Configured TopBDII: topbdii02.ncg.ingrid.pt verified
CREAM-CE test06.egi.cesga.es verified
WN test14.egi.cesga.es, test15.egi.cesga.es Also used for glite-MPI and glexec. verified
APEL Publisher test07.egi.cesga.es verified
DPM test08.egi.cesga.es verified
MyPROXY px02.ncg.ingrid.pt verified
LFC_mysql test09.egi.cesga.es verified
LFC_oracle
rejected
VOMS test10.egi.cesga.es Avaialable VO: ops.vo.ibergrid.eu verified
CREAM/LSF cert-37.pd.infn.it verified
VOMS_oracle
IGE-SAM test12.egi.cesga.es In verification
IGE gsissh test19.egi.cesga.es in verification
storm srmcms.ifca.es verified
Nagios test23.egi.cesga.es verified
OGSA-DAI test18.egi.cesga.es In verification
IGE-rls test05.egi.cesga.es verified
IGE-gridftp test20.egi.cesga.es verified
IGE-myproxy test26.egi.cesga.es verified
IGE GRAM5 test27.egi.cesga.es verified
GridWay test28.egi.cesga.es in verification
IGE-gridsafe test16.egi.cesga.es in verification
Free test11.egi.cesga.es
EMI-ARGUS test30.egi.cesga.es verified
CREAM - SLURM test31.egi.cesga.es
WN - SLURM test01.egi.cesga.es
WN - SLURM test02.egi.cesga.es

Repository Configuration

This document assumes SL5 x86_64 PPAs verification, for other architectures the actual repositories to use may change

Base & EPEL

Verification is done using the SL5 distribution + EPEL repositories. You must have in your yum repositories list the following ones:

  • SL5 BASE and updates:
[sl-base]
name=SL 5 base
baseurl=http://ftp.scientificlinux.org/linux/scientific/55/$basearch/SL
	 http://ftp1.scientificlinux.org/linux/scientific/55/$basearch/SL
	 http://ftp2.scientificlinux.org/linux/scientific/55/$basearch/SL
	 ftp://ftp.scientificlinux.org/linux/scientific/55/$basearch/SL
#mirrorlist=ftp://ftp.scientificlinux.org/linux/scientific/mirrorlist/sl-base-55.txt
enabled=1
gpgcheck=0
# To use priorities you must have yum-priorities installed
priority=10
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-sl 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-sl5 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-csieh
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-dawson 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-jpolok 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-cern 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
[sl-security]
name=SL 5 security updates
baseurl=http://ftp.scientificlinux.org/linux/scientific/55/$basearch/updates/security
http://ftp1.scientificlinux.org/linux/scientific/55/$basearch/updates/security
http://ftp2.scientificlinux.org/linux/scientific/55/$basearch/updates/security
ftp://ftp.scientificlinux.org/linux/scientific/55/$basearch/updates/security
#mirrorlist=ftp://ftp.scientificlinux.org/linux/scientific/mirrorlist/sl-security-55.txt
enabled=1
gpgcheck=0
# To use priorities you must have yum-priorities installed
priority=10
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-sl
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-sl5 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-csieh  
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-dawson 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-jpolok 
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-cern
file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
  • EPEL
[extras]
name=epel
mirrorlist=http://mirrors.fedoraproject.org/mirrorlist?repo=epel-5&arch=$basearch
protect=0
enabled=1
priority=20
  • EGI trust anchors
[EGI-trustanchors]
name=EGI-trustanchors
baseurl=http://repository.egi.eu/sw/production/cas/1/current/
gpgkey=http://repository.egi.eu/sw/production/cas/1/GPG-KEY-EUGridPMA-RPM-3
gpgcheck=1
enabled=1

Product Repository

You must also add the repository with the packages that you are currently verifying. The URL for the repository can be found at the Custom Fields of the RT ticket within attribute RepositoryURL. The easiest way of adding the repositoy is using the files available at the repofiles directory.

Example config file for CREAM release 1.13.0 RT ticket 2206, has the repository located at http://admin-repo.egi.eu/sw/unverified/emi.cream.sl5.x86_64/1/13/0 and the repofile is available at http://admin-repo.egi.eu/sw/unverified/emi.cream.sl5.x86_64/1/13/0/repofiles/EMI.cream.sl5.x86_64.repo

Contents of this file is shown below:

# EGI Software Repository - REPO META (releaseId,repositoryId,repofileId) - (2206,239,166)
[EMI.cream.sl5.x86_64]
name=EMI.cream.sl5.x86_64
baseurl=http://admin-repo.egi.eu/sw/unverified/emi.cream.sl5.x86_64/1/13/0/
enabled=1
protect=1

Issues

Package signing

GPG keys are not provided by repofiles and signature checks of packages cannot be used currently, a solution for this issue is being investigated. In the meantime, make sure to use gpgcheck=0 in the yum repo file.

Extras Repository

For verification of EMI 1.0 Release, an extra repository with compatibility packages is also needed for some products, at least WMS and UI. See information available at RT ticket 2243. Repository file can be obtained from http://repository.egi.eu/sw/stagerollout/emi.extras.sl5.x86_64/1/repofiles/EMI.extras.sl5.x86_64.repo.

[EMI.extras.sl5.x86_64]
name=EMI.extras.sl5.x86_64
baseurl=http://repository.egi.eu/sw/stagerollout/emi.extras.sl5.x86_64/1/
enabled=1
protect=1

Yaim configuration

BDII

The site bdii has currently the following resources configured:

BDII_SBDII_URL="ldap://sbdii02.ncg.ingrid.pt:2170/mds-vo-name=resource,o=grid"
BDII_TBDII_URL="ldap://topbdii02.ncg.ingrid.pt:2170/mds-vo-name=resource,o=grid"
BDII_VOMS_URL="ldap://voms02.ncg.ingrid.pt:2170/mds-vo-name=resource,o=grid"
BDII_WMS_URL="ldap://tst04.ific.uv.es:2170/mds-vo-name=resource,o=grid"
BDII_LB_URL="ldap://tst05.ific.uv.es:2170/mds-vo-name=resource,o=grid"
BDII_CREAM_URL="ldap://test06.egi.cesga.es:2170/mds-vo-name=resource,o=grid"
BDII_DPM_URL="ldap://test08.egi.cesga.es:2170/mds-vo-name=resource,o=grid"

UI

  • test13.egi.cesga.es
# YAIM Verification configuration file for EMI-UI

MY_DOMAIN=egi.cesga.es

LB_HOST=tst05.ific.uv.es
WMS_HOST=tst04.ific.uv.es
RB_HOST=tst04.ific.uv.es
PX_HOST=test02.egee.cesga.es
BDII_HOST=topbdii02.ncg.ingrid.pt
BDII_LIST="lcg-bdii.cern.ch:2170,topbdii02.ncg.ingrid.pt:2170"
LFC_HOST=test09.egi.cesga.es
LFC_CENTRAL="ops.vo.ibergrid.eu iber.vo.ibergrid.eu"
DPM_HOST="test08.egi.cesga.es"   # my-dpm.$MY_DOMAIN
DPMPOOL=egi-pool
DPM_FILESYSTEMS="$DPM_HOST:/storage"

VOS="ops dteam ops.vo.ibergrid.eu iber.vo.ibergrid.eu"
VO_SW_DIR=/opt/exp_soft

# Production VOs
OPS_GROUP_ENABLE="ops /VO=ops/GROUP=/ops/ROLE=lcgadmin"
DTEAM_GROUP_ENABLE="dteam /VO=dteam/GROUP=/dteam/ROLE=lcgadmin"

#IBERGRID VERIFICATION VO GROUPS
GRID_OPSIBEU_GROUP_ENABLE="ops.vo.ibergrid.eu 
                           /VO=ops.vo.ibergrid.eu/GROUP=/ops.vo.ibergrid.eu/ROLE=VO-Admin 
                           /VO=ops.vo.ibergrid.eu/GROUP=/ops.vo.ibergrid.eu/ROLE=Production"
GRID_IBERIBEU_GROUP_ENABLE="iber.vo.ibergrid.eu 
                            /VO=iber.vo.ibergrid.eu/GROUP=/iber.vo.ibergrid.eu/ROLE=VO-Admin
                            /VO=iber.vo.ibergrid.eu/GROUP=/iber.vo.ibergrid.eu/ROLE=Production"

#::::::::::::::
#ops
#::::::::::::::
VO_OPS_SW_DIR=$VO_SW_DIR/ops
VO_OPS_DEFAULT_SE=$DPM_HOST
VO_OPS_STORAGE_DIR=$CLASSIC_STORAGE_DIR/ops
VO_OPS_QUEUES="GRID_ops"
VO_OPS_VOMS_SERVERS="vomss://voms.cern.ch:8443/voms/ops?/ops/"
VO_OPS_VOMSES="'ops voms.cern.ch 15009 /DC=ch/DC=cern/OU=computers/CN=voms.cern.ch ops'"
VO_OPS_VOMS_CA_DN="'/DC=ch/DC=cern/CN=CERN Trusted Certification Authority' 
                   '/DC=ch/DC=cern/CN=CERN Trusted Certification Authority'"

#::::::::::::::
#dteam
#::::::::::::::
VO_DTEAM_SW_DIR=$VO_SW_DIR/dteam
VO_DTEAM_DEFAULT_SE=$DPM_HOST
VO_DTEAM_STORAGE_DIR=$CLASSIC_STORAGE_DIR/dteam
VO_DTEAM_QUEUES="GRID_dteam"
VO_DTEAM_VOMS_SERVERS='vomss://voms.hellasgrid.gr:8443/voms/dteam?/dteam/'
VO_DTEAM_VOMSES="'dteam lcg-voms.cern.ch 15004 /DC=ch/DC=cern/OU=computers/CN=lcg-voms.cern.ch dteam 24' 
                 'dteam voms.cern.ch 15004 /DC=ch/DC=cern/OU=computers/CN=voms.cern.ch dteam 24' 
                 'dteam voms.hellasgrid.gr 15004 /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms.hellasgrid.gr dteam 24' 
                 'dteam voms2.hellasgrid.gr 15004 /C=GR/O=HellasGrid/OU=hellasgrid.gr/CN=voms2.hellasgrid.gr dteam 24'"
VO_DTEAM_VOMS_CA_DN="'/DC=ch/DC=cern/CN=CERN Trusted Certification Authority' 
                     '/DC=ch/DC=cern/CN=CERN Trusted Certification Authority' 
                     '/C=GR/O=HellasGrid/OU=Certification Authorities/CN=HellasGrid CA 2006' 
                     '/C=GR/O=HellasGrid/OU=Certification Authorities/CN=HellasGrid CA 2006'"

APEL

  • test07.egi.cesga.es
MYSQL_PASSWORD=*********
SITE_NAME=UMD-VERIFICATION
# Hostname of the CE
CE_HOST=test06.egi.cesga.es
BATCH_SERVER=$CE_HOST
APEL_DB_PASSWORD="********"
# Hostname of the RGMA server
MON_HOST=test07.egi.cesga.es

WN

MPI Configuration:

#### MPI CONFIGURATION
MPI_OPENMPI_ENABLE="yes"
MPI_SHARED_HOME="no"
MPI_SSH_HOST_BASED_AUTH="no"

gLexec (with Argus) configuration:

GLEXEC_WN_OPMODE="setuid"
GLEXEC_WN_SCAS_ENABLED="no"
GLEXEC_WN_ARGUS_ENABLED="yes"
ARGUS_PEPD_ENDPOINTS="https://test10.egi.cesga.es:8154/authz"

DPM

  • test08.egi.cesga.es

Yaim Variables:

#DPM variables
USERS_CONF=/opt/glite/yaim/etc/users.conf
GROUPS_CONF=/opt/glite/yaim/etc/groups.conf
DPM_HOST="test08.egi.cesga.es"
MYSQL_PASSWORD=******
DPMPOOL=egi-pool
DPM_FILESYSTEMS="$DPM_HOST:/storage"

DPM_DB_USER=dpmmgr
DPM_DB_PASSWORD=*******
DPM_DB_HOST=$DPM_HOST

DPM_INFO_USER=dpm_info
DPM_INFO_PASS=********

LFC_mysql

  • test09.egi.cesga.es

Yaim Variables:

#LFC variables
USERS_CONF=/opt/glite/yaim/etc/users.conf
GROUPS_CONF=/opt/glite/yaim/etc/groups.conf
LFC_HOST=test09.egi.cesga.es
LFC_DB_PASSWORD=*****
LFC_CENTRAL="ops.vo.ibergrid.eu iber.vo.ibergrid.eu"
LFC_LOCAL="ops dteam"
LFC_DB_HOST=$LFC_HOST
LFC_DB=******
MYSQL_PASSWORD=******