The wiki is deprecated and due to be decommissioned by the end of September 2022.
The content is being migrated to other supports, new updates will be ignored and lost.
If needed you can get in touch with EGI SDIS team using operations @ egi.eu.
Difference between revisions of "HBP:FZJ-Site-Setup"
(Initial version) |
|||
| Line 15: | Line 15: | ||
~/.ssh/authorized_keys (server @Bari): | ~/.ssh/authorized_keys (server @Bari): | ||
command="$HOME/rrsync .",from="134.94.199.*",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAAAB3Nza...b43 bjoernh@image-service.novalocal | command="$HOME/rrsync .",from="134.94.199.*",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAAAB3Nza...b43 bjoernh@image-service.novalocal | ||
== Monitoring == | |||
The monitoring has been enabled by adding a service to GOCDB. The service type is '''egi.Portal'''. We're using a specific test URL (http://hbp-image.fz-juelich.de/test) that maps to the download of a particular tile. The relevant Nginx configuration is this: | |||
location /test { | |||
proxy_pass http://image-server/image/v0/api/bbic?fname=%2Fsrv%2Fdata%2FHBP%2Ftemplate%2Fhuman%2Fbigbrain_20um%2Fsections%2Fbigbrain.h5&mode=ims&prog=TILE+0+0+0+3702+13+14+none+10+1; | |||
} | |||
image-server in this snippet is the upstream image-server as defined in the upstream section that is used for load balancing. | |||
Results are available from [https://opsmon.egi.eu/nagios/cgi-bin/extinfo.cgi?type=2&host=hbp-image.fz-juelich.de&service=org.nagiosexchange.Portal-WebCheck opsmon]. | |||
Latest revision as of 08:52, 13 July 2016
Basic Information
We have deployed the service according to HBP SP5 documentation. It is running on a virtual machine of four cores and 8GB main memory. Attached to this instance is a virtual block device of currently 3TB, which can be extended on demand. There are currently 10 Docker instances running the Image Service load-balanced by an OpenResty web server that also cached any tiles that are already processed. The service is available at http://hbp-image.fz-juelich.de/image/v0/api/bbic.
Synchronization
Synchronization of data from Bari to FZJ is automated by a Cron job that runs once a day at 22:30. It may be worthwhile to explore the assignment of time slots to the various partners involved, such that everyone gets the most out of the network and disk bandwidth.
Authentication of the Cron job is done via a dedicated SSH key that has been generated on the server machine at FZJ. This key is only allowed to be used for rsync jobs by means of the rrsync (restricted rsync) script in the hbp-user's home directory. There is a very useful description about this approach here.
crontab (client @FZJ):
30 22 * * * rsync -avu --size-only hbp-user@90.147.170.213:data/HBP/* /mnt2/HBP/
~/.ssh/authorized_keys (server @Bari):
command="$HOME/rrsync .",from="134.94.199.*",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAAAB3Nza...b43 bjoernh@image-service.novalocal
Monitoring
The monitoring has been enabled by adding a service to GOCDB. The service type is egi.Portal. We're using a specific test URL (http://hbp-image.fz-juelich.de/test) that maps to the download of a particular tile. The relevant Nginx configuration is this:
location /test {
proxy_pass http://image-server/image/v0/api/bbic?fname=%2Fsrv%2Fdata%2FHBP%2Ftemplate%2Fhuman%2Fbigbrain_20um%2Fsections%2Fbigbrain.h5&mode=ims&prog=TILE+0+0+0+3702+13+14+none+10+1;
}
image-server in this snippet is the upstream image-server as defined in the upstream section that is used for load balancing.
Results are available from opsmon.