UsingCreamUMD3WithApelUMD2
Jump to navigation
Jump to search
Installation of previous APEL
- Add UMD-2 repos
wget http://repository.egi.eu/sw/production/umd/2/repofiles/sl5/UMD-2-base.repo -O /etc/yum.repos.d/UMD-2-base.repo wget http://repository.egi.eu/sw/production/umd/2/repofiles/sl5/UMD-2-updates.repo -O /etc/yum.repos.d/UMD-2-updates.repo
- Install the apel parser package, there is one flavor for each LRMS (pbs, sge and lsf)
$ yum install glite-apel-pbs Loaded plugins: kernel-module, priorities, protectbase Repository EGI-trustanchors is listed more than once in the configuration UMD-2-base | 1.9 kB 00:00 UMD-2-updates | 1.9 kB 00:00 552 packages excluded due to repository priority protections 0 packages excluded due to repository protections Setting up Install Process Resolving Dependencies --> Running transaction check ---> Package glite-apel-pbs.noarch 0:2.0.6-8.sl5 set to be updated --> Processing Dependency: glite-apel-core >= 2.0.14-4 for package: glite-apel-pbs --> Running transaction check ---> Package glite-apel-core.noarch 0:2.0.14-5.sl5 set to be updated --> Finished Dependency Resolution Beginning Kernel Module Plugin Finished Kernel Module Plugin Dependencies Resolved ====================================================================================================== Package Arch Version Repository Size ====================================================================================================== Installing: glite-apel-pbs noarch 2.0.6-8.sl5 UMD-2-updates 24 k Installing for dependencies: glite-apel-core noarch 2.0.14-5.sl5 UMD-2-base 1.0 M Transaction Summary ====================================================================================================== Install 2 Package(s) Upgrade 0 Package(s) Total download size: 1.1 M Is this ok [y/N]: y Downloading Packages: (1/2): glite-apel-pbs-2.0.6-8.sl5.noarch.rpm | 24 kB 00:00 (2/2): glite-apel-core-2.0.14-5.sl5.noarch.rpm | 1.0 MB 00:01 ------------------------------------------------------------------------------------------------------ Total 654 kB/s | 1.1 MB 00:01 Running rpm_check_debug Running Transaction Test Finished Transaction Test Transaction Test Succeeded Running Transaction Installing : glite-apel-core 1/2 Installing : glite-apel-pbs 2/2 Installed: glite-apel-pbs.noarch 0:2.0.6-8.sl5 Dependency Installed: glite-apel-core.noarch 0:2.0.14-5.sl5 Complete!
Configuration
EMI-3 CREAM does not configure the apel parsers with YAIM, but the functions for doing so are still available so it can be executed directly:
$ /opt/glite/yaim/bin/yaim -r -s site-info.def -f config_apel_pbs INFO: Configuring HOST: test06.egi.cesga.es INFO: Using site configuration file: /opt/glite/yaim/etc/site-info.def INFO: ################################################################### . /'.-. ') . yA,-"-,( ,m,:/ ) .oo. oo o ooo o. .oo . / .-Y a a Y-. 8. .8' 8'8. 8 8b d'8 . / ~ ~ / 8' .8oo88. 8 8 8' 8 . (_/ '====' 8 .8' 8. 8 8 Y 8 . Y,-''-,Yy,-.,/ o8o o8o o88o o8o o8o o8o . I_))_) I_))_) current working directory: /opt/glite/yaim/node-info.d site-info.def date: May 6 10:19 /opt/glite/yaim/etc/site-info.def yaim command: -r -s /opt/glite/yaim/etc/site-info.def -f config_apel_pbs log file: /opt/glite/yaim/bin/../log/yaimlog Mon May 6 11:05:16 CEST 2013 : /opt/glite/yaim/bin/yaim Installed YAIM versions: glite-apel-yaim ame:-ame: glite-yaim-bdii 4.3.13-1 glite-yaim-core 5.1.1-1 glite-yaim-cream-ce 4.4.1-4 glite-yaim-mpi 1.1.12-1 glite-yaim-torque-server 5.1.0 -1 glite-yaim-torque-utils 5.1.1-2 #################################################################### INFO: The default location of the grid-env.(c)sh files will be: /usr/libexec INFO: Sourcing the utilities in /opt/glite/yaim/functions/utils INFO: Detecting environment INFO: Assuming the node types: CE_torque INFO: Using hostname: test06.egi.cesga.es WARNING: config_apel_pbs function is not supported by CE_torque! WARNING: Anyway you can force this operation. WARNING: Would you really run config_apel_pbs on CE_torque? [y/]: INFO: Executing function: config_apel_pbs_check INFO: Executing function: config_apel_pbs_setenv INFO: Executing function: config_apel_pbs INFO: YAIM terminated succesfully.
That should create an apel configuration file at /etc/glite-apel-pbs/parser-config-yaim.xml and a cron file for parsing the records (/etc/cron.d/glite-apel-pbs-parser)
The configuration file does not point to the correct prefix for blah logs. The <BlahdLogPrefix> in the configuration file should be changed to:
<BlahdLogPrefix>blahp.log-</BlahdLogPrefix>
Test
Running manually the contents of the cron file:
APEL_HOME=/ /usr/bin/apel-pbs-log-parser -f /etc/glite-apel-pbs/parser-config-yaim.xml >> /var/log/apel.log 2>&1
Log file contents:
Mon May 6 10:32:02 UTC 2013: apel-pbs-log-parser - Read-in configuration: [quiet, p, inspectTables, c, e, recursiveEventFiles, b, recursiveBlahdFiles] [DBPassword=****, site=CESGA-EGEE, pbslog=/var/torque/server_priv/accounting, GIIS=test06.egi.cesga.es, CEhost=test06.egi.cesga.es, blahdlogPrefix=blahp.log-, DBURL=jdbc:mysql://test07.egi.cesga.es:3306/accounting, DBUsername=accounting, timeZone=UTC, blahdlog=/var/log/cream/accounting] Mon May 6 10:32:02 UTC 2013: apel-pbs-log-parser - ------ Starting the apel application ------ Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - **** APEL is examining the schema **** Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the LcgRecords table Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The LcgRecords schema is up-to-date Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the BlahdRecords table Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The BlahdRecords schema is up-to-date Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the LcgProcessedFiles table Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The LcgProcessedFiles schema is up-to-date Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the SpecRecords table for patch 28593 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the SpecRecords table for patch 65723 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The SpecRecords schema is up-to-date Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the GkRecords table Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The GkRecords schema is up-to-date Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the MessageRecords table Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The MessageRecords schema is up-to-date Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - **** Schema checks complete **** Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Finding CPU spec values from GIIS server: test06.egi.cesga.es Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - LDAP URL: ldap://test06.egi.cesga.es:2170 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_dteam:2395 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_ops:2395 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_iberibeu:2395 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_opsibeu:2395 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Reprocess disabled, checking new event logs only Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - **** Updating PBS end event table (EventRecords) **** Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Ignoring already processed event log file: test06.egi.cesga.es /var/torque/server_priv/accounting/20130430 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Processing batch log file: test06.egi.cesga.es /var/torque/server_priv/accounting/20130506 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Event records inserted: 9 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the BlahdRecords table Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - The BlahdRecords schema is up-to-date Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Reprocess disabled, checking new event logs only Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Processing /var/log/cream/accounting/blahp.log-20130506 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Processing /var/log/cream/accounting/blahp.log-20130430 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - Blahd records inserted: 12 Mon May 6 10:32:03 UTC 2013: apel-pbs-log-parser - ------ Processing finished ------