~ubuntu-branches/ubuntu/vivid/slurm-llnl/vivid

.TH "bluegene.conf" "5" "April 2008" "bluegene.conf 2.0" "Slurm configuration file"
.SH "NAME"
bluegene.conf \- Slurm configuration file for BlueGene systems 
.SH "DESCRIPTION"
\fB/etc/bluegene.conf\fP is an ASCII file which describes BlueGene specific 
SLURM configuration information. This includes specifications for bgblock 
layout, configuration, logging, etc.
The file location can be modified at system build time using the
DEFAULT_SLURM_CONF parameter. The file will always be located in the 
same directory as the \fBslurm.conf\fP file. 
.LP
Paramter names are case insensitive.
Any text following a "#" in the configuration file is treated 
as a comment through the end of that line. 
The size of each line in the file is limited to 1024 characters.
Changes to the configuration file take effect upon restart of 
SLURM daemons, daemon receipt of the SIGHUP signal, or execution 
of the command "scontrol reconfigure" unless otherwise noted.
.LP

There are some differences between Bluegene/L and Bluegene/P in respects to the contents of the bluegene.conf file.  

.SH "The Bluegene/L specific options are:"
.TP
\fBAltBlrtsImage\fR
Alternative BlrtsImage.  This is an optional field only used for 
mulitple images on a system and should be followed by a Groups= with
the user groups allowed to use this image (i.e. Groups=da,jette) if 
Groups= is not stated then this image will be able to be used by all
groups. You can put as many alternative images as you want in the conf file.

.TP
\fBAltLinuxImage\fR
Alternative LinuxImage.  This is an optional field only used for 
mulitple images on a system and should be followed by a Groups= with
the user groups allowed to use this image (i.e. Groups=da,jette) if 
Groups= is not stated then this image will be able to be used by all
groups. You can put as many alternative images as you want in the conf file.

.TP
\fBAltRamDiskImage\fR
Alternative RamDiskImage.  This is an optional field only used for 
mulitple images on a system and should be followed by a Groups= with
the user groups allowed to use this image (i.e. Groups=da,jette) if 
Groups= is not stated then this image will be able to be used by all
groups. You can put as many alternative images as you want in the conf file.

.TP
\fBBlrtsImage\fR
BlrtsImage used for creation of all bgblocks.
There is no default value and this must be specified.

.TP
\fBLinuxImage\fR
LinuxImage used for creation of all bgblocks.
There is no default value and this must be specified.

.TP
\fBRamDiskImage\fR
RamDiskImage used for creation of all bgblocks.
There is no default value and this must be specified.

.SH "The Bluegene/P specific options are:"
.TP
\fBAltCnloadImage\fR
Alternative CnloadImage.  This is an optional field only used for 
mulitple images on a system and should be followed by a Groups= with
the user groups allowed to use this image (i.e. Groups=da,jette) if 
Groups= is not stated then this image will be able to be used by all
groups. You can put as many alternative images as you want in the conf file.

.TP
\fBAltIoloadImage\fR
Alternative IoloadImage.  This is an optional field only used for 
mulitple images on a system and should be followed by a Groups= with
the user groups allowed to use this image (i.e. Groups=da,jette) if 
Groups= is not stated then this image will be able to be used by all
groups. You can put as many alternative images as you want in the conf file.

.TP
\fBCnloadImage\fR
CnloadImage used for creation of all bgblocks.
There is no default value and this must be specified.

.TP
\fBIoloadImage\fR
IoloadImage used for creation of all bgblocks.
There is no default value and this must be specified.

.SH "All options below are common on all Bluegene systems:"
.TP
\fBAltMloaderImage\fR
Alternative MloaderImage.  This is an optional field only used for 
mulitple images on a system and should be followed by a Groups= with
the user groups allowed to use this image (i.e. Groups=da,jette) if 
Groups= is not stated then this image will be able to be used by all
groups. You can put as many alternative images as you want in the conf file.

.TP
\fBBasePartitionNodeCount\fR
The number of c\-nodes per base partition.
There is no default value and this must be specified. (For bgl systems this
is usually 512)

.TP
\fBBridgeAPILogFile\fR
Fully qualified pathname of a into which the Bridge API logs are 
to be written.
There is no default value.

.TP
\fBBridgeAPIVerbose\fR
Specify how verbose the Bridge API logs should be.
The default value is 0.
.RS
.TP
\fB0\fR: Log only error and warning messages
.TP
\fB1\fR: Log level 0 and information messasges
.TP
\fB2\fR: Log level 1 and basic debug messages
.TP
\fB3\fR: Log level 2 and more debug message
.TP
\fB4\fR: Log all messages
.RE

.TP
\fBDenyPassthrough\fR
Specify which dimsions you do not want to allow pass throughs.  Valid options are X, Y, Z or all.
example: If you don't want to allow passthroughs in the X and Y diminsions you would specify DenyPassthrough=X,Y

.TP
\fBLayoutMode\fR
Describes how SLURM should create bgblocks.
.RS
.TP
\fBSTATIC\fR: Create and use the defined non\-overlapping bgblocks.
.TP
\fBOVERLAP\fR: Create and use the defined bgblocks, which may overlap.
It is highly recommended that none of the bgblocks have any passthroughs 
in the X\-dimension. 
\fBUse this mode with extreme caution.\fR
.TP
\fBDYNAMIC\fR: Create and use bglblocks as needed for each job.
Bgblocks will not be defined in the bluegene.conf file.
Dynamic partitioning may introduce fragmentation of resources 
and starvation of larger jobs.
\fBUse this mode with caution.\fR
.RE

.TP
\fBMloaderImage\fR
MloaderImage used for creation of all bgblocks.
There is no default value and this must be specified.

.TP
\fBNodeCardNodeCount\fR
Number of c\-nodes per node card.
There is no default value and this must be specified. (For bgl systems this 
is usually 32)

.TP
\fBNumPsets\fR
The Numpsets used for creation of all bgblocks.  This value really means the
number of IOnodes on a base partition.  This number must be the smallest
number if you have a heterogeneous system.
There is no default value and this must be specified.  The typical settings
for bgl systems goes as follows... For IO rich systems 64 is the value that 
should be used to create small blocks.  For systems that are not IO rich, or 
you do not wish to create small blocks, 8 is usually the number to use.
For bgp IO rich systems 32 is the value that should be used to create small 
blocks since you can only have 2 ionodes per nodecard instead of 4 like on bgl.

.LP
Each bgblock is defined by the base partitions used to construct it.
Ordering is very important for laying out switch wires.  Please create
blocks with smap, and once done don't change the order of blocks created.
A bgblock is implicitly created containing all resources on the system.
Bgblocks must not overlap in static mode (except for implicitly 
created bgblock). This will be the case when smap is used to create 
a configuration file
All Nodes defined here must also be defined in the slurm.conf file.
Define only the numeric coordinates of the bgblocks here. The prefix
will be based upon the NodeName defined in slurm.conf

.TP
\fBBPs\fR
Define the XYZ coordinates of the bgblock end points.

.TP
\fBType\fR
Define the network connection type for the bgblock.
The default value is TORUS.
.RS
.TP
\fBMESH\fR: Communication occur over a mesh.
.TP
\fBSMALL\fR: The base partition is divided into more than one bgblock.
The administrator should define the number of single node cards and 
quarter base partition blocks using the options \fBNodeCards\fR and 
\fBQuarters\fR respectively. 
The total number of c\-nodes in defined blocks must not exceed
\fBBasePartitionNodeCnt\fR.
If not specified, the base partition will be divided into four 
blocks.
See example below.
.TP
\fBTORUS\fR: Communications occur over a torus (end\-points of network 
directly connect.
.RE

.SH "EXAMPLE"
.LP 
.br
##################################################################
.br
# bluegene.conf
.br
# build by smap on 03/06/2006
.br
##################################################################
.br
BridgeAPILogFile=/var/log/slurm/bridgeapi.log
.br
BridgeAPIVerbose=2
.br
BlrtsImage=/bgl/BlueLight/ppcfloor/bglsys/bin/rts_hw.rts
.br
LinuxImage=/bgl/BlueLight/ppcfloor/bglsys/bin/zImage.elf
.br
MloaderImage=/bgl/BlueLight/ppcfloor/bglsys/bin/mmcs\-mloader.rts
.br
RamDiskImage=/bgl/BlueLight/ppcfloor/bglsys/bin/ramdisk.elf
.br
BasePartitionNodeCnt=512
.br
NodeCardNodeCnt=32
.br
NumPsets=64	# An I/O rich environment
.br
LayoutMode=STATIC
.br
##################################################################
.br
# LEAVE AS COMMENT, Full\-system bglblock, implicitly created
.br
# BPs=[000x333] Type=TORUS        # 4x4x4 = 64 midplanes
.br
##################################################################
.br
BPs=[000x133] Type=TORUS          # 2x4x4 = 32
.br
BPs=[200x233] Type=TORUS          # 1x4x4 = 16
.br
BPs=[300x313] Type=TORUS          # 1x2x4 =  8
.br
BPs=[320x323] Type=TORUS          # 1x1x4 =  4
.br
BPs=[330x331] Type=TORUS          # 1x1x2 =  2
.br
BPs=[332x332] Type=TORUS          # 1x1x1 =  1
.br
BPs=[333x333] Type=SMALL NodeCards=4 Quarters=3 # 1/16 * 4 + 1/4 * 3

.SH "COPYING"
Copyright (C) 2006 The Regents of the University of California.
Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER).
CODE\-OCEC\-09\-009. All rights reserved.
.LP
This file is part of SLURM, a resource management program.
For details, see <https://computing.llnl.gov/linux/slurm/>.
.LP
SLURM is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option)
any later version.
.LP
SLURM is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more
details.
.SH "FILES"
/etc/bluegene.conf
.SH "SEE ALSO"
.LP
\fBslurm.conf\fR(5)