UsingCreamUMD3WithApelUMD2

From EGIWiki
Jump to: navigation, search

Installation of previous APEL

wget http://repository.egi.eu/sw/production/umd/2/repofiles/sl5/UMD-2-base.repo -O /etc/yum.repos.d/UMD-2-base.repo
wget http://repository.egi.eu/sw/production/umd/2/repofiles/sl5/UMD-2-updates.repo -O /etc/yum.repos.d/UMD-2-updates.repo
$ yum install glite-apel-pbs
Loaded plugins: kernel-module, priorities, protectbase
Repository EGI-trustanchors is listed more than once in the configuration
UMD-2-base                                                                     | 1.9 kB     00:00     
UMD-2-updates                                                                  | 1.9 kB     00:00     
552 packages excluded due to repository priority protections
0 packages excluded due to repository protections
Setting up Install Process
Resolving Dependencies
--> Running transaction check
---> Package glite-apel-pbs.noarch 0:2.0.6-8.sl5 set to be updated
--> Processing Dependency: glite-apel-core >= 2.0.14-4 for package: glite-apel-pbs
--> Running transaction check
---> Package glite-apel-core.noarch 0:2.0.14-5.sl5 set to be updated
--> Finished Dependency Resolution
Beginning Kernel Module Plugin
Finished Kernel Module Plugin

Dependencies Resolved

======================================================================================================
 Package                    Arch              Version                  Repository                Size
======================================================================================================
Installing:
 glite-apel-pbs             noarch            2.0.6-8.sl5              UMD-2-updates             24 k
Installing for dependencies:
 glite-apel-core            noarch            2.0.14-5.sl5             UMD-2-base               1.0 M

Transaction Summary
======================================================================================================
Install       2 Package(s)
Upgrade       0 Package(s)

Total download size: 1.1 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): glite-apel-pbs-2.0.6-8.sl5.noarch.rpm                                   |  24 kB     00:00     
(2/2): glite-apel-core-2.0.14-5.sl5.noarch.rpm                                 | 1.0 MB     00:01     
------------------------------------------------------------------------------------------------------
Total                                                                 654 kB/s | 1.1 MB     00:01     
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing     : glite-apel-core                                                                1/2 
  Installing     : glite-apel-pbs                                                                 2/2 

Installed:
  glite-apel-pbs.noarch 0:2.0.6-8.sl5                                                                 

Dependency Installed:
  glite-apel-core.noarch 0:2.0.14-5.sl5                                                               

Complete!

Configuration

EMI-3 CREAM does not configure the apel parsers with YAIM, but the functions for doing so are still available so it can be executed directly:

$ /opt/glite/yaim/bin/yaim -r -s site-info.def -f config_apel_pbs
   INFO: Configuring HOST: test06.egi.cesga.es
   INFO: Using site configuration file: /opt/glite/yaim/etc/site-info.def
   INFO: 
         ###################################################################
         
         .             /'.-. ')
         .     yA,-"-,( ,m,:/ )   .oo.     oo    o      ooo  o.     .oo
         .    /      .-Y a  a Y-.     8. .8'    8'8.     8    8b   d'8
         .   /           ~ ~ /         8'    .8oo88.     8    8  8'  8
         . (_/         '===='          8    .8'     8.   8    8  Y   8
         .   Y,-''-,Yy,-.,/           o8o  o8o    o88o  o8o  o8o    o8o
         .    I_))_) I_))_)
         
         
         current working directory: /opt/glite/yaim/node-info.d
         site-info.def date: May 6 10:19 /opt/glite/yaim/etc/site-info.def
         yaim command: -r -s /opt/glite/yaim/etc/site-info.def -f config_apel_pbs
         log file: /opt/glite/yaim/bin/../log/yaimlog
         Mon May  6 11:05:16 CEST 2013 : /opt/glite/yaim/bin/yaim
         
         Installed YAIM versions:
         glite-apel-yaim ame:-ame:
         glite-yaim-bdii 4.3.13-1
         glite-yaim-core 5.1.1-1
         glite-yaim-cream-ce 4.4.1-4
         glite-yaim-mpi 1.1.12-1
         glite-yaim-torque-server 5.1.0  -1
         glite-yaim-torque-utils 5.1.1-2
         
         ####################################################################
   INFO: The default location of the grid-env.(c)sh files will be: /usr/libexec
   INFO: Sourcing the utilities in /opt/glite/yaim/functions/utils
   INFO: Detecting environment
   INFO: Assuming the node types:   CE_torque
   INFO: Using hostname: test06.egi.cesga.es
   WARNING: config_apel_pbs function is not supported by   CE_torque!
   WARNING: Anyway you can force this operation.
   WARNING: Would you really run config_apel_pbs on   CE_torque? [y/]: 
   INFO: Executing function: config_apel_pbs_check 
   INFO: Executing function: config_apel_pbs_setenv 
   INFO: Executing function: config_apel_pbs 
   INFO: YAIM terminated succesfully.

That should create an apel configuration file at /etc/glite-apel-pbs/parser-config-yaim.xml and a cron file for parsing the records (/etc/cron.d/glite-apel-pbs-parser)

The configuration file does not point to the correct prefix for blah logs. The <BlahdLogPrefix> in the configuration file should be changed to:

<BlahdLogPrefix>blahp.log-</BlahdLogPrefix>

Test

Running manually the contents of the cron file:

APEL_HOME=/ /usr/bin/apel-pbs-log-parser -f /etc/glite-apel-pbs/parser-config-yaim.xml >> /var/log/apel.log 2>&1

Log file contents:

Mon May  6 10:32:02 UTC 2013: apel-pbs-log-parser - Read-in configuration: [quiet, p, inspectTables, c, e, recursiveEventFiles, b, recursiveBlahdFiles]
 [DBPassword=****, site=CESGA-EGEE, pbslog=/var/torque/server_priv/accounting, GIIS=test06.egi.cesga.es, CEhost=test06.egi.cesga.es,
 blahdlogPrefix=blahp.log-, DBURL=jdbc:mysql://test07.egi.cesga.es:3306/accounting, DBUsername=accounting, timeZone=UTC, blahdlog=/var/log/cream/accounting]
Mon May  6 10:32:02 UTC 2013: apel-pbs-log-parser - ------ Starting the apel application ------
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - **** APEL is examining the schema ****
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the LcgRecords table
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The LcgRecords schema is up-to-date 
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the BlahdRecords table
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The BlahdRecords schema is up-to-date 
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the LcgProcessedFiles table
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The LcgProcessedFiles schema is up-to-date 
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the SpecRecords table for patch 28593
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the SpecRecords table for patch 65723
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The SpecRecords schema is up-to-date 
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the GkRecords table
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The GkRecords schema is up-to-date 
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the MessageRecords table
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The MessageRecords schema is up-to-date 
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - **** Schema checks complete ****
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Finding CPU spec values from GIIS server: test06.egi.cesga.es
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - LDAP URL: ldap://test06.egi.cesga.es:2170
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_dteam:2395
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_ops:2395
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_iberibeu:2395
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The ClusterId : SpecInt pair are latest, no update required test06.egi.cesga.es:8443/cream-pbs-GRID_opsibeu:2395
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Reprocess disabled, checking new event logs only
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - **** Updating PBS end event table (EventRecords) ****
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Ignoring already processed event log file: test06.egi.cesga.es /var/torque/server_priv/accounting/20130430
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Processing batch log file: test06.egi.cesga.es /var/torque/server_priv/accounting/20130506
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Event records inserted: 9
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Checking the BlahdRecords table
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - The BlahdRecords schema is up-to-date 
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Reprocess disabled, checking new event logs only
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Processing /var/log/cream/accounting/blahp.log-20130506
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Processing /var/log/cream/accounting/blahp.log-20130430
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - Blahd records inserted: 12
Mon May  6 10:32:03 UTC 2013: apel-pbs-log-parser - ------ Processing finished ------
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Print/export