Competence centre LifeWatch/Citizen Science/Task 4.2

The objective of this task is to develop an software able to identify plant's images in order to allow citizens to contribute to flora conservation. Image recognition will be implemented through machine learning with deep neural networks (aka deep learning). Caffe is a deep learning framework created at UC Berkeley.

Caffe installation

Caffe installation in Altamira

For the installation of Caffe in Altamira Supercomputer at IFCA (Spain) we have followed both the official installation guide and a specific guide for installing Caffe on a Supercomputer cluster.

Not having root access to Altamira, Caffe has been installed locally at /gpfs/res_scratch/lifewatch/iheredia/.usr/local/src/caffe. For more information on the local installation of software in a supercomputer please check [1]. The software and libraries already available at Altamira are:

• PYTHON 2.7.10
• CUDA 7.0.28
• OPENMPI 1.8.3
• BOOST 1.54.0
• PROTOBUF 2.5.0
• GCC 4.6.3
• HDF5 1.8.10

The remaining libraries have been installed at /gpfs/res_scratch/lifewatch/iheredia/.usr/local/lib. Those libraries are:

• gflags
• glog
• leveldb
• OpenCV
• snappy
• LMDB
• ATLAS

Modules can be loaded all at once by loading /gpfs/res_scratch/lifewatch/iheredia/.usr/local/share/modulefiles/common.

Comments

At this moment Altamira runs with Tesla M2090 GPUs with CUDA capability 2.0. Therefore Caffe has been compiled without CuDNN (the GPU-accelerated library of primitives for deep neural networks) which requires GPUs with CUDA capability of 3.0 or higher.

Caffe installation in Yong

Having root access, installing Caffe is straightforward in Ubuntu. Yong runs with Nvidia's Quadro 4000 GPU which neither enables CuDNN support. This GPU has a very limited memory which enables training in small simple datasets with small networks (eg. MNIST) but is not capable to store (and therefore train) more complex networks needed to learn more involved dataset (eg. ImageNet).

Caffe architecture

Neural networks learn their layer's parameter using backpropagation where the gradients of one layer are used to compute the gradients of the previous one (the communication between layers in Caffe is implemented through blobs which store the values and gradients of each layer of the network). Therefore deep networks are very modular and Caffe reflects this modularity by giving you (almost) complete freedom to compose your network's architecture.

There are several types of layers:

Common layers:

• Inner product
• Splitting
• Flattening
• Reshape
• Concatenation
• Slicing
• Elementwise operations
• Argmax
• Softmax
• Mean Variance Normalization

Vision layers:

• Convolution
• Pooling
• Local Response Normalization

Activation/Neuron layers:'

• ReLU
• Leaky ReLU
• Sigmoid
• Tanh
• Absolute Value
• Power
• Binomial Normal LogLikelihood

Activations functions like sigmoids and tanh are not as popular as the used to be. It is usually a safe assumption to choose ReLUs.

Despite the simplicity of constructing a network, it's correct design often involves a considerable amount of expertise. Therefore for beginners it is usually recommended to train their datasets with existing networks (eg. AlexNet) kindly provided with Caffe.

Useful links

Computer Vision with Deep Learning

• Stanford Course - Convolutional Neural Networks for Visual Recognition: This is by far the most useful link to introduce yourself to the topic of image recognition with neural networks. Has coursenotes and recorded lectures on Youtube.

Deep Learning

• Michael Nielsen's webpage: Introductory notes on deep learning.
• Deep Learning Book: Still on preprint but with open access coursenotes. Written by one of the leading figure in the field.

Machine Learning

• Pattern Recognition and Machine Learning - C. M. Bishop: Clasic reference in machine learning. Useful to have the broad view.