Difference between revisions of "GOCDB/Regional Module Technical Documentation"

From EGIWiki
Jump to: navigation, search
Line 122: Line 122:
<pre>$ phpunit --version
<pre>$ phpunit --version
PHPUnit 3.7.27 by Sebastian Bergmann.
PHPUnit 3.7.27 by Sebastian Bergmann.
[http://phpunit.de/manual/current/en/automating-tests.html More information on PHPUnit can be found via the PHPUnit documenation here.]
== Deploying GOCDB5  ==
== Deploying GOCDB5  ==

Revision as of 12:38, 14 November 2013

Main EGI.eu operations services Support Documentation Tools Activities Performance Technology Catch-all Services Resource Allocation Security

GOC DB menu: Home Documentation Index

This page is the main install documentation for GOCDB Version 5 



  • A packaged V5 release will also be made available for download soon.
  • In the meantime, early adopters can check out the v5 src from svn:
svn checkout https://www.sysadmin.hep.ac.uk/svn/grid-monitoring/branches/gocdb/Doctrine%20Web%20Portal 

Note, if you are asked to authenticate with a client certificate as below, you can dismiss by pressing Enter

Authentication realm: https://www.sysadmin.hep.ac.uk:443
Client certificate filename: <ENTER to dismiss>

OLD v4.4 Install doc has been relocated


(Under Construction - more details comming soon)

System prerequisites

GOCDB-5 has two components: a database and a web front end. Both components can work on the same machine but it is very likely that in most environments they will be separated. This is what we would advise. Machine requirements for these 2 components are described below.


Version 5 of GocDB uses Doctrine ORM (Object Relational Mapping) as it's interface to the datastore. This allows GocDB to be deployed to either a Oracle, MySQL or SQLite database (Postgres comming soon).


  • Required database: Oracle 11g or higher (note: the free Oracle 11g XE Express Editions which comes with a free license is perfectly suitable. 10g may also work but is untested)
  • Required space: 150 MB Dowload free Oracle XE from the following link and follow the installation instructions: http://www.oracle.com/technetwork/database/express-edition/overview/index.html
  • For gocdb admin tasks, we also recommend the SQL Developer tool.



  • Required database: SQLite 3.x.x or higher. SQLite is produced as an open source database and all versions are free to use.
  • Required: See the SQLite documentation: www.sqlite.org/docs.html

Web frontend

The machine to use as the web frontend will need the following:

  • Apache http server version 2.2 or higher.
  • PHP version 5.3 or higher (version <=5.2 has some OOP related bugs).
    • PHP oci8 extension
  • a X509 host certificate for the machine


  • PHP Unit for testing developments

Preparing your installation

Database preparation

GocDB v5 comes with deploy script which will setup the DB schema for GocDB (i.e. export the DDL to create tables). The only preperation is that you have setup a database user/account in your chosen RDBMS and have the user access details which are required later in the installation when configuring GocDB for deployment. 

GOCDB file system

Place the GocDB source folder on your chosen web server.


Pear is recommended to install Doctrine and PHPUnit, many builds of PHP come pre-bundled with PEAR. 


Verify the PEAR installation by running pear version on the command line.

$ pear version
PEAR Version: 1.9.4
PHP Version: 5.3.8
Zend Engine Version: 2.3.0
Running on: Windows NT ESCPC0095 6.1 build 7601 (Windows 7 Business Edition Service Pack 1) i586


Doctrine can be installed using PEAR. First add the Doctrine and Symfony channels to PEAR:

$pear channel-discover pear.doctrine-project.org
$pear channel-discover pear.symfony.com

Then install Doctrine and it's dependencies:

$pear install --alldeps doctrine/DoctrineORM

Verify the Doctrine installation by running "doctrine --version" on the command line:

$ doctrine --version
Doctrine Command Line Interface version 2.3.3

More information on Doctrine can be found here at the Doctrine2 site.


Apache http server version 2.2 or higher.


PHPUnit is an optional choice for GOCDB users. GOCDB5 includes a test suite of unit tests that can be used to verify buisness logic and database connections for a deployed instance of GOCDB. In addition developers extending and modifying GOCDB can write unit tests for their own developements. To install PHPUnit use PEAR:

$pear clear-cache
$pear config-set auto_discover 1
$pear install pear.phpunit.de/PHPUnit

This install can then be verified using "phpunit --version" on the command line:

$ phpunit --version
PHPUnit 3.7.27 by Sebastian Bergmann.

More information on PHPUnit can be found via the PHPUnit documenation here.

Deploying GOCDB5

Database Connection

You should now have all the required blocks in place to deploy GOCDB V5. To Deploy the database schema to your desired database using Doctrine navigate to to gocDBSrc/lib/Doctrine folder. In this folder you will configure the database connection details for your database. A template file called bootstrap_doctrine_TEMPLATE.php is provided. In this file you will find three blocks of code commented out, once for each of the supported databased, SQLite, Oracle and MySQL:

Un-comment the area for your chosen database and fill in the details for your chosen database. 

        ///////////////////////SQLITE CONNECTION DETAILS/////////////////////////////////////////////
	// $conn = array(
	// 	'driver' => 'pdo_sqlite',
	// 	'path' => __DIR__ . '/db.sqlite',
	// );

	///////////////////////ORACLE CONNECTION DETAILS////////////////////////////////////////////
	//	$conn = array(
	//		'driver' => 'oci8',
	//		'user' => 'docsvn',
	//		'password' => 'doc',
	//		'host' => 'localhost',
	//		'port' => 1521,
        //              /*'service' = true,*/  //This may be needed depending on your Oracle server
	//		'dbname' => 'XE'
	//	);
	//  // Need to explicitly set the Oracle session date format [1]
	//  $evm = new EventManager();
	//  $evm->addEventSubscriber(new OracleSessionInit(array('NLS_TIME_FORMAT' => 'HH24:MI:SS')));	

	///////////////////////MYSQL CONNECTION DETAILS////////////////////////////////////////////
	//$conn = array(
	//	'driver' => 'pdo_mysql',
	//	'user' => 'doctrine',
	//	'password' => 'doc',
	//	'host' => 'localhost',
	//	'dbname' => 'doctrine'

Once you have input your connection details for your database save this file as bootstrap_doctrine.php in the same location as the template file.

Note: Doctrine can use APC caching however this is not suitable for all deployments. To disable comment out the APC configuration lines within the bootstrap_doctrine.php file:

$config->setMetadataCacheImpl(new \Doctrine\Common\Cache\ApcCache());
$config->setQueryCacheImpl(new \Doctrine\Common\Cache\ApcCache());

Compiled Entities

For local test machines this step isn't mandatory. For production servers it is strongly recommended. When Doctrine uses and entity it creates a compiled version of the entity which by default is stored in the machines temporary folder. This is un-suitable for production machines as this may well be emptied and then cause GocDB to cease working. To avoid this uncomment the following line in bootstrap_doctrine.php:


At this point no further action is required, but later we will need to create this folder and populate it with the compiled entities data.

Deploying GOCDB

GocDB can be deployed as a blank instance ready for use or as a sample instance with a small amount of example data to demonstrate GocDB. The deploy script can be found in /lib/Doctrine/deploy folder:


By supplying a flag when executing the script you can deploy either a new empty database or a sample database eg:

$./deploy.sh -s

Deploying GocDB with sample data


$./deploy.sh -n

Deploying new empty GocDB instance

Apache Configuration


Now that your database is deployed the final step is to configure Apache.

A few modules need to be enabled for GOCDB to work, check these are enabled on your Apache and if not enable them. 

In httpd.conf enable ssl_module by un-commenting these lines:

LoadModule ssl_module modules/mod_ssl.so



Depending on the database you are using the supporting module may also need to be enabled. For example Oracle requires the oracle module to be uncommented:


Take the file gocdbssl.conf and place this file in the apache/conf/extra folder. In httpd.conf include the gocdbssl.con by adding the following line under the heading Supplemental Configuration:

Include conf/extra/gocdbssl.conf


This file can be found in GocDBSrc/config. This file will define the alias and SSL settings used by Apache to connect to GocDB.


The first step is to setup the paths for the DocumentRoot, ErrorLog and TransferLog and define the ServerName and ServerAdmin details for your particular web service where you are hosting GocDB.

SSL Keys

The SSLCertificateFile should point to the location of your server SSL key file. The SSLCertificateKeyFile -- The SSLCACertificationPath should point to the location of your CA certificates.

For development the SSLCertificatedFile and SSLCertificateKeyFile can be self signed certificates. However these must not be used in production.


The final step is to set the Alias to map your desired directory for GocDB webportal and Programmatic Interface (PI) to the correct folder and ensure the correct rules are defined for your server.

For the webportal the following settings are an example:

    Alias "/portal" "/gocDBsrc/htdocs/web_portal"
    <Directory "/gocDBsrc/htdocs/web_portal">
            SSLVerifyClient  require
            Options FollowSymLinks Indexes
            Order deny,allow
            Allow from
            deny from all  

The alias should and file location should also be set for the public PI and private PI. Public PI:

Alias "/gocdbpi/public" "/gocDBsrc/htdocs/PI/public"
<Directory "/gocDBsrc/htdocs/PI/public">

Private PI:

Alias "/gocdbpi/private" "/gocDBsrc/htdocs/PI/private"
<Directory "/gocDBsrc/htdocs/PI/private">

Compiled Entities

As described in the previous compiled entities section this step is not required for local test deployments of GocDB but strongly recommend for production deployments. You should have uncommented the line in bootstrap_doctrine.php that sets the proxy directory for the entities:


You now need to create this folder in the same directory as the bootstrap_doctrine.php file:

bootstrap_doctrine.php  bootstrap.php  cli-config.php  deploy  entities  README.txt
$mkdir compiledEntities

Once this folder is created it needs it's ownership changed to be owned by the apache user, the exact command may change depending on your apache server:

$chown apache compiledEntities/

The compiled Entities folder should now be owned by Apache:

$ls -l
total 28
-rw-r--r-- 1 root   root 3043 Nov 13 09:46 bootstrap_doctrine.php
-rwxrwxrwx 1 root   root 1620 Nov  7 12:08 bootstrap.php
-rwxrwxrwx 1 root   root  219 Nov  7 12:08 cli-config.php
drwxr-xr-x 2 apache root 4096 Nov 13 10:21 compiledEntities
drwxrwxrwx 4 root   root 4096 Nov 12 17:46 deploy
drwxrwxrwx 2 root   root 4096 Nov 12 17:46 entities
-rwxrwxrwx 1 root   root  131 Nov 12 17:37 README.txt

Finally command doctrine to generate the compiled entities and store them in the folder:

doctrine orm:generate-proxies compiledEntities/

This folder is now configured with the compiledEntites. It can be copied and used for new instances of GocDB as long as there have been no changes made to any of the entities, or it can be recreated as demonstrated each time.

First Use

You should now be able to navigate to the GocDB webportal on your host using the URL defined in your alias. You will need to install a browser certificate that is suitable for the SSL keys you defined for your host to be able to view GocDB.

Setup an Admin User

To get started with GocDB you will need an admin user. This is done by first registering as a user on GocDB by clicking the 'Register' link on the left menu bar at the bottom. Once you have registered yourself you will then need to set yourself as an admin. To do this you need to change the user record directly in your database. The users table has a field called 'isAdmin' which by default is set to 0. To change a user to admin set this to 1. Below is a sample of the SQL query used when using an Oracle database to set a user as admin.

UPDATE users SET isadmin=1 WHERE forename='John' AND surname='Doe' 

Once you have an admin user you can create Projects and NGI's and grant roles to other users over these entities to allow them to add to GocDB.


GocDB v5 comes with a suite of tests that can be run to validate the install and check that Doctrine and your chosen database are operating as expected. These tests require PHPUnit, the install instructions for this can be found in section 3.6. The tests can be found in the GocDB/tests/ folder.

Deploying a Test Database

We recommend that you deploy a second database that will be used for testing. Once you have created your second database input the connection details into the bootstrap_doctrine_TEMPLATE.php file found in GocDB/tests/doctrine. Within this file there are 3 sets of connection configurations, once for MySQL, Oracle and SQLite. Once you have input your details save this file as bootstrap_doctrine.php. Complete the section for your database and ensure that the other connection details are deleted or commented out. In addition to the standard Doctrine connection to the database the tests make use of a PDO connection to validate that the results that Doctrine is returning are true. To setup the pdo connection open bootstrap_pdo_TEMPLATE.php. Within this file you will again find 3 connection blocks. Complete the details for the connection details for your database and ensure that the other 2 connections are either deleted or commented out and then save this file as bootstrap_pdo.php.

Un-comment the area for your chosen database and fill in the details for your chosen database. 

     ///////////////////////SQLITE CONNECTION DETAILS/////////////////////////////////////////////
     // $sqliteFile = __DIR__ . '/../db.sqlite';
     // $pdo = new PDO("sqlite:" . $sqliteFile);
     // return new PHPUnit_Extensions_Database_DB_DefaultDatabaseConnection($pdo, 'sqlite');
     ///////////////////////ORACLE CONNECTION DETAILS/////////////////////////////////////////////
     // $pdo = new PDO('oci:dbname=//localhost:1521/xe', 'DoctrineUnitTests', 'doc');      
     // return new PHPUnit_Extensions_Database_DB_DefaultDatabaseConnection($pdo, 'USERS');
     ///////////////////////MYSQL CONNECTION DETAILS//////////////////////////////////////////////
     //  $pdo = new PDO('mysql:host=localhost;dbname=doctrine;charset=UTF8', 'doctrine', 'doc');
     //  return new PHPUnit_Extensions_Database_DB_DefaultDatabaseConnection($pdo);

Running the Test Suite

A initial test suite is provided that will excercise core functions of GocDB and Doctrine, this suite can be ran using the command:

phpunit DoctrineTestSuite1.php

You will be prompted to continue as this operation will drop any tables in your test database and then recreate the schema ready to run the tests. These tests should return with no errors to demonstrate that GocDB and Doctrine are installed correctly and are working as expected.

Running Individual Tests

Individual tests can be run by calling phpunit testName.php. It is advised that at the beginning of a testing session you drop and recreate the database by running:


Found in the GocDB/tests/doctrine folder.

Writing Tests

If you intend to develop new functionality for your instance of GocDB you can support this development by writing unit tests for the new features. A simple example of a test can be found in GocDB/tests/exampleTests. 'FirstTest.php' demonstrates how to create data and assert that it is as expected.

A good tutorial for getting started with PHPUnit can be found here.

Writing a Doctrine Test

When working with Doctrine there are a number of setup functions that needed to be included in your test file. A template file has been provided in GocDBSrc/tests/exampleTests called 'DoctrineTestTemplate.php'. When executing any tests based on this template two files need to be present in the same folder as the test being executed:


The first holds your connection details and the second will initialize the database with the correct tables ready for testing.

You can write multiple test functions at the bottom of this skeleton file.

Doctrine Test Example

'DoctrineTestExample.php' shows the following example test. In this test we will create a site, populate the site with some data and then assert that the data is correct.

This test will make use of the 'TestUtil.php' file which contains a few helper methods to quickly create objects. This file is located in GocDBSrc/test/doctrine. When following this tutorial execute 'DoctrineTestExample.php' from GocDBSrc/test/doctrine.

Creating the site:

    public function testDoctrineExample() {
        print __METHOD__ . "\n";

        //Create a site
    	$ourSite = TestUtil::createSampleSite("Our Example Site");

We now have a site and will add some extra information to the site utilizing setters methods defined in the site entity which can be found in GocDBSrc/lib/Doctrine/entites. We will add an email, telephone and location to the site:

	//Create a site
    	$ourSite = TestUtil::createSampleSite("Our Example Site");
	//Set some details of the site
	$ourSite->setLocation("United Kingdom");

We now must persist this object in memory by calling '$this->em->persist($outSite);'

However this now only exist in memory - it is not yet in the database. To check this we can query the database directly using the PDO connection we setup:

	//Get the site ID from the object
	$siteId = $ourSite->getId();   
	//Get a database connection	    	
    	$con = $this->getConnection();
	//Search the database for this site
	$sql = "SELECT 1 FROM sites WHERE ID = '$siteId'";
	$result = $con->createQueryTable('', $sql);

        //We expect this query to return no rows as the site does not exist in the database yet
	$this->assertEquals(0, $result->getRowCount());

When running the test at this point the outcome should be:

OK (1 test, 1 assertion)

Our assertion was correct, there were no sites with that ID in the database yet as it has not been committed, the site only exists in memory at the moment. To commit the site to the database we must call:


Now if we run the query again we expect one result:

	//Search the database for this site again
	$sql = "SELECT 1 FROM sites WHERE ID = '$siteId'";
	$result = $con->createQueryTable('', $sql);
	//We expect this query to return 1 rows as the site now exists in the database
	$this->assertEquals(1, $result->getRowCount());

This shows the process of creating an entity, using its get and set methods to enter and retrieve data and then committing it to the database. It also shows the process of using the PDO connection to check that the data exists in the database as we expect.