Difference between revisions of "GPGPU-WG KnowledgeBase - Batch Schedulers - Torque MAUI"
m (moved GPGPU-WG:GPGPU Working Group KnowledgeBase:Batch Schedulers:Torque MAUI to GPGPU-WG KnowledgeBase - Batch Schedulers - Torque MAUI) |
|||
(12 intermediate revisions by 2 users not shown) | |||
Line 1: | Line 1: | ||
{{Template:Op menubar}} {{TOC_right}} | |||
[[Category:Task_forces]] | |||
'''[[GPGPU_Working_Group| << GPGPU Working Group main page]]''' | |||
MAUI does not officially support GPGPU scheduling, and is very unlikely to support it in the future. | MAUI does not officially support GPGPU scheduling, and is very unlikely to support it in the future. | ||
Even if a Resource Centre adds the "gpu=X" to the nodes file, MAUI will silently drop batch GPGPU directive: | Even if a Resource Centre adds the "gpu=X" to the nodes file, MAUI will silently drop batch GPGPU directive: | ||
< | <pre> | ||
qsub -l nodes=1:ppn=1:gpus=1 | qsub -l nodes=1:ppn=1:gpus=1 | ||
</ | </pre> | ||
== Experimental MAUI patch == | == Experimental MAUI patch == | ||
A potential workaround to solve this problem (at a batch scheduling level) through the use of a patched version of MAUI 3.3.1. | A potential workaround to solve this problem (at a batch scheduling level) through the use of a patched version of MAUI 3.3.1 (created by Jonathan Michalon at the University of Strasbourg). | ||
This [[http://www.supercluster.org/pipermail/mauiusers/2011-April/004622.html patch ]] implements a Generic Resource | This [[http://www.supercluster.org/pipermail/mauiusers/2011-April/004622.html patch ]] implements a Generic Resource capability in MAUI. | ||
capability in MAUI. | |||
After this patch is applied, the maui.cfg should be updated to include the GRES declartion for all appropriate nodes: | After this patch is applied, the maui.cfg should be updated to include the GRES declartion for all appropriate nodes: | ||
< | <pre> | ||
NODECFG[wn001.example.com] GRES=gpu:2 # Node with two generic resources marked with tag 'gpu' | |||
NODECFG[wn001.example.com] GRES=gpu:2 | </pre> | ||
</ | |||
=== Example (batch | === Example(s) batch usage === | ||
< | <pre> | ||
qsub -W "x=GRES:gpu@1" < test-sl6-gpu.qsub | qsub -W "x=GRES:gpu@1" < test-sl6-gpu.qsub | ||
</ | </pre> | ||
The following examples are provided by way of http://www.sdsc.edu/~hocks/FG/TSCC.torque.html, with thanks to Mariusz Mamonski ( mamonski at man.poznan.pl ) | |||
<pre> | |||
1. one CPU core, one GPU: | |||
qsub -W x='GRES:gpu at 1' #works | |||
2. one CPU core, all two GPUs on one machine: | |||
qsub -lnodes=1:ppn=1 -W x='GRES:gpu at 2' #works | |||
3. two GPUs on two hosts | |||
qsub -lnodes=2:ppn=1 -W x='GRES:gpu at 2' #works | |||
4. you want all GPUs and all CPU cores on two hosts | |||
qsub -lnodes=2:ppn=8 -W x='GRES:gpu at 1' #does not work - because the | |||
job request 16 GPUS on two hosts, but actually if you request | |||
exclusive access to machines you do need to specify GRES at all... | |||
</pre> |
Latest revision as of 15:57, 22 January 2015
Main | EGI.eu operations services | Support | Documentation | Tools | Activities | Performance | Technology | Catch-all Services | Resource Allocation | Security |
<< GPGPU Working Group main page MAUI does not officially support GPGPU scheduling, and is very unlikely to support it in the future. Even if a Resource Centre adds the "gpu=X" to the nodes file, MAUI will silently drop batch GPGPU directive:
qsub -l nodes=1:ppn=1:gpus=1
Experimental MAUI patch
A potential workaround to solve this problem (at a batch scheduling level) through the use of a patched version of MAUI 3.3.1 (created by Jonathan Michalon at the University of Strasbourg). This [patch ] implements a Generic Resource capability in MAUI.
After this patch is applied, the maui.cfg should be updated to include the GRES declartion for all appropriate nodes:
NODECFG[wn001.example.com] GRES=gpu:2 # Node with two generic resources marked with tag 'gpu'
Example(s) batch usage
qsub -W "x=GRES:gpu@1" < test-sl6-gpu.qsub
The following examples are provided by way of http://www.sdsc.edu/~hocks/FG/TSCC.torque.html, with thanks to Mariusz Mamonski ( mamonski at man.poznan.pl )
1. one CPU core, one GPU: qsub -W x='GRES:gpu at 1' #works 2. one CPU core, all two GPUs on one machine: qsub -lnodes=1:ppn=1 -W x='GRES:gpu at 2' #works 3. two GPUs on two hosts qsub -lnodes=2:ppn=1 -W x='GRES:gpu at 2' #works 4. you want all GPUs and all CPU cores on two hosts qsub -lnodes=2:ppn=8 -W x='GRES:gpu at 1' #does not work - because the job request 16 GPUS on two hosts, but actually if you request exclusive access to machines you do need to specify GRES at all...