Difference between revisions of "GOCDB/Regional Module Technical Documentation"

From EGIWiki
Jump to: navigation, search
(Setup an Admin User)
m (Download)
 
(34 intermediate revisions by 3 users not shown)
Line 3: Line 3:
 
== Download  ==
 
== Download  ==
  
'''LATEST VERSION: GOCDBv5''' <br>  
+
'''LATEST VERSION: GOCDBv5.6''' <br>  
 +
https://github.com/GOCDB/gocdb
  
*A packaged V5 release will also be made available for download soon.  
+
For install see:
*In the meantime, early adopters can check out the v5 src from svn:
+
https://github.com/GOCDB/gocdb/blob/master/INSTALL.md
<pre>svn checkout https://www.sysadmin.hep.ac.uk/svn/grid-monitoring/branches/gocdb/Doctrine%20Web%20Portal
+
 
 +
<!--
 +
*Packaged V5.2: https://www.sysadmin.hep.ac.uk/svn/grid-monitoring/tags/gocdb/GOCDB-5.2/GocDB-5.2.zip
 +
*svn v5.2:
 +
<pre>svn co https://www.sysadmin.hep.ac.uk/svn/grid-monitoring/tags/gocdb/GOCDB-5.2
 
</pre>  
 
</pre>  
Note, if you are asked to authenticate with a client certificate as below, you can dismiss by pressing Enter  
+
*Latest dev/trunk svn:
 +
<pre>svn checkout https://www.sysadmin.hep.ac.uk/svn/grid-monitoring/branches/gocdb/MultipleEndpointsGocDB/</pre>
 +
-->
 +
 
 +
<!--<pre>svn checkout https://www.sysadmin.hep.ac.uk/svn/grid-monitoring/branches/gocdb/Doctrine%20Web%20Portal</pre> -->
 +
<!--Note, for svn co, if you are asked to authenticate with a client certificate as below, you can dismiss by pressing Enter  
 
<pre>Authentication realm: https://www.sysadmin.hep.ac.uk:443
 
<pre>Authentication realm: https://www.sysadmin.hep.ac.uk:443
 
Client certificate filename: &lt;ENTER to dismiss&gt;
 
Client certificate filename: &lt;ENTER to dismiss&gt;
Line 18: Line 28:
  
 
<br>  
 
<br>  
 +
-->
 +
<br>
  
<br>
+
<!--
 
+
== Prerequisites==
== Deployment  ==
 
'''<font color="red">(Under Construction - more details comming soon)</font>'''
 
 
=== System prerequisites  ===
 
=== System prerequisites  ===
  
Line 29: Line 39:
 
==== Database  ====
 
==== Database  ====
  
Version 5 of GocDB uses Doctrine ORM (Object Relational Mapping) as it's interface to the datastore. This allows GocDB to be deployed to either a Oracle, MySQL or SQLite database (Postgres comming soon).  
+
Version 5 of GocDB uses Doctrine ORM (Object Relational Mapping) as it's interface to the datastore. This allows GocDB to be deployed to either a Oracle or MySQL database (Doctrine also supports other DBs but these are untested).  
  
 
Oracle:  
 
Oracle:  
Line 44: Line 54:
 
*Required space: See MySQL install guide:http://dev.mysql.com/doc/refman/5.7/en/installing.html
 
*Required space: See MySQL install guide:http://dev.mysql.com/doc/refman/5.7/en/installing.html
  
<br>  
+
<br>
 
 
SQLite:
 
  
*Required database: SQLite 3.x.x or higher. SQLite is produced as an open source database and all versions are free to use.
 
*Required: See the SQLite documentation: www.sqlite.org/docs.html
 
<br>
 
 
==== Web frontend  ====
 
==== Web frontend  ====
  
Line 57: Line 62:
 
*'''Apache http server'''&nbsp;version 2.2 or higher.  
 
*'''Apache http server'''&nbsp;version 2.2 or higher.  
 
*'''PHP''' version 5.3 or higher (version &lt;=5.2 has some OOP related bugs).  
 
*'''PHP''' version 5.3 or higher (version &lt;=5.2 has some OOP related bugs).  
**'''PHP''' oci8 extension
+
**If using Oracle: '''PHP oci8 extension''' and Oracle Instant client v10 or higher (downloadable from Oracle website)
**libxml2 and DOM support for '''PHP''' (Note: On RHEL, PHP requires the PHP XML RPM to be installed for this component to function).  
+
**'''libxml2 and DOM support''' for PHP (Note: On RHEL, PHP requires the PHP XML RPM to be installed for this component to function).  
**OpenSSL Extension for '''PHP'''
+
**'''OpenSSL Extension''' for PHP  
*'''Oracle Instant client''' version 10 or higher (downloadable from Oracle website)
+
*'''An X509 host certificate''' for the machine
*a X509 host certificate for the machine
 
  
 
<br>
 
<br>
Line 112: Line 116:
 
</pre>  
 
</pre>  
 
[http://docs.doctrine-project.org/en/latest/ More information on Doctrine can be found here at the Doctrine2 site.]
 
[http://docs.doctrine-project.org/en/latest/ More information on Doctrine can be found here at the Doctrine2 site.]
 
=== Apache  ===
 
 
Apache http server version 2.2 or higher.
 
  
 
=== PHPUnit  ===
 
=== PHPUnit  ===
Line 134: Line 134:
 
=== Database Connection ===
 
=== Database Connection ===
 
You should now have all the required blocks in place to deploy GOCDB V5. To Deploy the database schema to your desired database using Doctrine:
 
You should now have all the required blocks in place to deploy GOCDB V5. To Deploy the database schema to your desired database using Doctrine:
* Navigate to to gocDBSrc/lib/Doctrine folder.  
+
* Navigate to to '<gocDBSrcHome>/lib/Doctrine' folder.  
 
* Locate the provided template file: ''bootstrap_doctrine_TEMPLATE.php''. In this file you will find three blocks of code commented out, once for each of the supported databased, SQLite, Oracle and MySQL as shown below:
 
* Locate the provided template file: ''bootstrap_doctrine_TEMPLATE.php''. In this file you will find three blocks of code commented out, once for each of the supported databased, SQLite, Oracle and MySQL as shown below:
 
* '''Copy this file to''' ''bootstrap_doctrine.php'' in the same dir as the template file and modify to specify your chosen DB connection details.  
 
* '''Copy this file to''' ''bootstrap_doctrine.php'' in the same dir as the template file and modify to specify your chosen DB connection details.  
Line 181: Line 181:
 
$config-&gt;setQueryCacheImpl(new \Doctrine\Common\Cache\ApcCache());
 
$config-&gt;setQueryCacheImpl(new \Doctrine\Common\Cache\ApcCache());
 
</pre>
 
</pre>
 
=== Compiled Entities ===
 
For local test machines this step isn't mandatory. For production servers it is strongly recommended. When Doctrine uses and entity it creates a compiled version of the entity which by default is stored in the machines temporary folder. This is un-suitable for production machines as this may well be emptied and then cause GocDB to cease working. To avoid this uncomment the following line in bootstrap_doctrine.php:
 
<pre>
 
$config->setProxyDir(__DIR__."/compiledEntities");
 
</pre>
 
 
At this point no further action is required, but later we will need to create this folder and populate it with the compiled entities data.
 
  
 
=== Deploying GOCDB  ===
 
=== Deploying GOCDB  ===
Line 194: Line 186:
 
GocDB can be deployed as a blank instance ready for use or as a sample instance with a small amount of example data to demonstrate GocDB. The deploy script can be found at ''/lib/Doctrine/deploy/deploy.sh'':<br>
 
GocDB can be deployed as a blank instance ready for use or as a sample instance with a small amount of example data to demonstrate GocDB. The deploy script can be found at ''/lib/Doctrine/deploy/deploy.sh'':<br>
  
'''Windows Users:''' These scripts are designed for deployment on a Linux machine. If you are using Windows or Cygwin you will need to open ''recreate.sh'' and change: ''doctrine'' to ''doctrine.bat''.
+
'''Windows Users:''' These scripts are designed for deployment on a Linux machine (Cygwin on Windows is ok).
 +
 
  
 
By supplying a flag when executing the script you can deploy either a new empty database or a sample database eg:
 
By supplying a flag when executing the script you can deploy either a new empty database or a sample database eg:
Line 212: Line 205:
  
 
=== Apache Configuration  ===
 
=== Apache Configuration  ===
 +
 +
Now that your database is deployed the final step is to configure Apache.
 +
 +
==== /etc/sysconfig/httpd - Oracle configuration ====
 +
 +
The Oracle environment must be set correctly before starting Apache so that PHP OCI8 works correctly. In general you should set the same variables that are set by the ''$ORACLE_HOME/bin/oracle_env.sh'' script. The necessary environment variables can be set in Apache's environment configuration file.
 +
On RedHat Linux and its derivatives with the default httpd package, this is ''/etc/sysconfig/httpd''.
 +
<pre>
 +
export ORACLE_HOME=/u01/app/oracle/product/12.1    # Adjust pathname to match your installation
 +
export LD_LIBRARY_PATH=$ORACLE_HOME/lib:$LD_LIBRARY_PATH
 +
</pre>
  
 
==== httpd.conf  ====
 
==== httpd.conf  ====
 
Now that your database is deployed the final step is to configure Apache.
 
  
 
A few modules need to be enabled for GOCDB to work, check these are enabled on your Apache and if not enable them.&nbsp;  
 
A few modules need to be enabled for GOCDB to work, check these are enabled on your Apache and if not enable them.&nbsp;  
Line 221: Line 223:
 
In ''httpd.conf'' enable ssl_module by un-commenting these lines:  
 
In ''httpd.conf'' enable ssl_module by un-commenting these lines:  
 
<pre>LoadModule ssl_module modules/mod_ssl.so</pre>  
 
<pre>LoadModule ssl_module modules/mod_ssl.so</pre>  
and
+
 
<pre>extension=php_openssl.dll</pre>
+
 
Depending on the database you are using the supporting module may also need to be enabled. For example Oracle requires the oracle module to be uncommented:
+
 
<pre>extension=php_oci8_11g.dll</pre>
 
Take the file ''gocdbssl.conf'' and place this file in the apache/conf/extra folder. In ''httpd.conf'' include the ''gocdbssl.con'' by adding the following line under the heading ''Supplemental Configuration'':
 
<pre>Include conf/extra/gocdbssl.conf
 
</pre>
 
 
==== gocdbssl.conf  ====
 
==== gocdbssl.conf  ====
 
+
A sample ''gocdbssl.conf'' is provided in GocDBSrc/config. This file will define the alias and SSL settings used by Apache to connect to GocDB.   
This file can be found in GocDBSrc/config. This file will define the alias and SSL settings used by Apache to connect to GocDB.   
 
  
 
===== Paths  =====
 
===== Paths  =====
Line 266: Line 263:
  
 
=== Compiled Entities ===
 
=== Compiled Entities ===
As described in the previous compiled entities section this step is not required for local test deployments of GocDB but strongly recommend for production deployments. You should have uncommented the line in ''bootstrap_doctrine.php'' that sets the proxy directory for the entities:
+
This step is optional, but for production servers this step is strongly recommended. When Doctrine uses an entity it creates a compiled version of the entity which by default is stored in the machines temporary folder. This is un-suitable for production machines as this may well be emptied. To avoid this uncomment the following line in bootstrap_doctrine.php:
 
<pre>
 
<pre>
 
$config->setProxyDir(__DIR__."/compiledEntities");
 
$config->setProxyDir(__DIR__."/compiledEntities");
Line 273: Line 270:
 
You now need to create this folder in the same directory as the ''bootstrap_doctrine.php'' file:
 
You now need to create this folder in the same directory as the ''bootstrap_doctrine.php'' file:
 
<pre>
 
<pre>
$ls
 
bootstrap_doctrine.php  bootstrap.php  cli-config.php  deploy  entities  README.txt
 
 
$mkdir compiledEntities
 
$mkdir compiledEntities
$ls
 
 
</pre>
 
</pre>
  
Once this folder is created it needs it's ownership changed to be owned by the apache user, the exact command may change depending on your apache server:
+
Once this folder is created it, your apache server needs write permissions (the exact command may change depending on your apache server):
  
 
<pre>
 
<pre>
Line 285: Line 279:
 
</pre>
 
</pre>
  
The compiled Entities folder should now be owned by Apache:
+
<pre>
<!--<pre>
+
$ls -ltrh
$ls -l
 
 
total 28
 
total 28
-rw-r--r-- 1 root   root 3043 Nov 13 09:46 bootstrap_doctrine.php
+
drwxr--r-- 2 apache root 4.0K Nov 20 16:11 entities
-rwxrwxrwx 1 root  root 1620 Nov  7 12:08 bootstrap.php
+
-rw-r--r-- 1 apache root 131 Nov 20 16:11 README.txt
-rwxrwxrwx 1 root  root  219 Nov 7 12:08 cli-config.php
+
-rw-r--r-- 1 apache root  219 Nov 20 16:11 cli-config.php
drwxr-xr-x 2 apache root 4096 Nov 13 10:21 compiledEntities
+
-rw-r--r-- 1 apache root 1.6K Nov 20 16:11 bootstrap.php
drwxrwxrwx 4 root   root 4096 Nov 12 17:46 deploy
+
-rw-r--r-- 1 apache root 3.0K Nov 20 16:11 bootstrap_doctrine_TEMPLATE.php
drwxrwxrwx 2 root  root 4096 Nov 12 17:46 entities
+
-rw-r--r-- 1 apache root 2.8K Nov 27 09:38 bootstrap_doctrine.php
-rwxrwxrwx 1 root   root  131 Nov 12 17:37 README.txt
+
drwxr--r-- 2 apache root 4.0K Nov 27 09:39 compiledEntities
</pre>-->
+
</pre>
  
Finally command doctrine to generate the compiled entities and store them in the folder:
+
Finally use the doctrine command line client to generate the compiled entities and store them in the folder:
 
<pre>
 
<pre>
 
doctrine orm:generate-proxies compiledEntities/
 
doctrine orm:generate-proxies compiledEntities/
Line 313: Line 306:
 
</pre>
 
</pre>
  
=== Local_Info.xml ===
+
====Local_Info.xml====
 
GocDB picks up a number of its settings and variables from the local_info.xml file located in the ''config'' folder. The ''web_portal_url'' will be output in the PI to create links from PI objects back to Portal views. The ''pi_url'' and ''server_base_url'' will both be used by the monitor. The monitor is a quick look status check for GocDB. The mointor can be found at your web_portal_url/GOCDB_monitor/. If these URL's are not set in the local config this feature will not work correctly.  
 
GocDB picks up a number of its settings and variables from the local_info.xml file located in the ''config'' folder. The ''web_portal_url'' will be output in the PI to create links from PI objects back to Portal views. The ''pi_url'' and ''server_base_url'' will both be used by the monitor. The monitor is a quick look status check for GocDB. The mointor can be found at your web_portal_url/GOCDB_monitor/. If these URL's are not set in the local config this feature will not work correctly.  
  
 
''default_scope'' defines which scope if any will be used if no scope is specified when running PI queries. This can be left blank or set to a scope of your choosing. ''default_scope_match'' is again used if no scope_match parameter is supplied with PI queries. This can be either all or any. The ''minimum_scopes'' section declares how many scopes an NGI, site, service or service group must have at creation. This can be left as 0 or set as 1 if you want all your users entities to belong to at least one scope or more as dictated by your use of GocDB.
 
''default_scope'' defines which scope if any will be used if no scope is specified when running PI queries. This can be left blank or set to a scope of your choosing. ''default_scope_match'' is again used if no scope_match parameter is supplied with PI queries. This can be either all or any. The ''minimum_scopes'' section declares how many scopes an NGI, site, service or service group must have at creation. This can be left as 0 or set as 1 if you want all your users entities to belong to at least one scope or more as dictated by your use of GocDB.
 +
 +
It's important at this point to understand how the scopes work with GocDB v5 especially in relation to the output of the PI. If you specify a default scope within your local_info but none of your entities have this scope then nothing will show in the PI. For an in-depth look at the scopes mechanism the section please read the section 'Multiple Scopes and Projects' in the [https://wiki.egi.eu/w/images/d/d3/GOCDB5_Grid_Topology_Information_System.pdf GOCDB5 Grid Topology Information System] document on page 5.
  
 
====Setup an Admin User====
 
====Setup an Admin User====
Line 336: Line 331:
 
use PDO for their execution to validate that the results that Doctrine is returning are true. To install the OCI8 drivers for pdo. See: http://www.php.net/manual/en/pdo.drivers.php
 
use PDO for their execution to validate that the results that Doctrine is returning are true. To install the OCI8 drivers for pdo. See: http://www.php.net/manual/en/pdo.drivers.php
  
You can test that you have the OCI8 pdo drivers installed via your php_check.php().
+
You can test that you have the OCI8 pdo drivers installed via php info. This can be called on the command line with $php -i or via a script with phpinfo(). There will be a section that lists your enabled pdo drivers, e.g. as below:
There will be a section that lists your enabled pdo drivers, e.g. as below:
 
 
<pre>
 
<pre>
 
             |-----------------------------|
 
             |-----------------------------|
Line 488: Line 482:
 
This shows the process of creating an entity, using its get and set methods to enter and retrieve data and then committing it to the database. It also shows the process of using the PDO connection to check that the data exists in the database as we expect.  
 
This shows the process of creating an entity, using its get and set methods to enter and retrieve data and then committing it to the database. It also shows the process of using the PDO connection to check that the data exists in the database as we expect.  
  
 
+
-->
  
  

Latest revision as of 16:02, 9 November 2016