1
.\" OpenPBS (Portable Batch System) v2.3 Software License
3
.\" Copyright (c) 1999-2000 Veridian Information Solutions, Inc.
4
.\" All rights reserved.
6
.\" ---------------------------------------------------------------------------
7
.\" For a license to use or redistribute the OpenPBS software under conditions
8
.\" other than those described below, or to purchase support for this software,
9
.\" please contact Veridian Systems, PBS Products Department ("Licensor") at:
11
.\" www.OpenPBS.org +1 650 967-4675 sales@OpenPBS.org
12
.\" 877 902-4PBS (US toll-free)
13
.\" ---------------------------------------------------------------------------
15
.\" This license covers use of the OpenPBS v2.3 software (the "Software") at
16
.\" your site or location, and, for certain users, redistribution of the
17
.\" Software to other sites and locations. Use and redistribution of
18
.\" OpenPBS v2.3 in source and binary forms, with or without modification,
19
.\" are permitted provided that all of the following conditions are met.
20
.\" After December 31, 2001, only conditions 3-6 must be met:
22
.\" 1. Commercial and/or non-commercial use of the Software is permitted
23
.\" provided a current software registration is on file at www.OpenPBS.org.
24
.\" If use of this software contributes to a publication, product, or service
25
.\" proper attribution must be given; see www.OpenPBS.org/credit.html
27
.\" 2. Redistribution in any form is only permitted for non-commercial,
28
.\" non-profit purposes. There can be no charge for the Software or any
29
.\" software incorporating the Software. Further, there can be no
30
.\" expectation of revenue generated as a consequence of redistributing
33
.\" 3. Any Redistribution of source code must retain the above copyright notice
34
.\" and the acknowledgment contained in paragraph 6, this list of conditions
35
.\" and the disclaimer contained in paragraph 7.
37
.\" 4. Any Redistribution in binary form must reproduce the above copyright
38
.\" notice and the acknowledgment contained in paragraph 6, this list of
39
.\" conditions and the disclaimer contained in paragraph 7 in the
40
.\" documentation and/or other materials provided with the distribution.
42
.\" 5. Redistributions in any form must be accompanied by information on how to
43
.\" obtain complete source code for the OpenPBS software and any
44
.\" modifications and/or additions to the OpenPBS software. The source code
45
.\" must either be included in the distribution or be available for no more
46
.\" than the cost of distribution plus a nominal fee, and all modifications
47
.\" and additions to the Software must be freely redistributable by any party
48
.\" (including Licensor) without restriction.
50
.\" 6. All advertising materials mentioning features or use of the Software must
51
.\" display the following acknowledgment:
53
.\" "This product includes software developed by NASA Ames Research Center,
54
.\" Lawrence Livermore National Laboratory, and Veridian Information
56
.\" Visit www.OpenPBS.org for OpenPBS software support,
57
.\" products, and information."
59
.\" 7. DISCLAIMER OF WARRANTY
61
.\" THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. ANY EXPRESS
62
.\" OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
63
.\" OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT
64
.\" ARE EXPRESSLY DISCLAIMED.
66
.\" IN NO EVENT SHALL VERIDIAN CORPORATION, ITS AFFILIATED COMPANIES, OR THE
67
.\" U.S. GOVERNMENT OR ANY OF ITS AGENCIES BE LIABLE FOR ANY DIRECT OR INDIRECT,
68
.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
69
.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
70
.\" OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
71
.\" LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
72
.\" NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
73
.\" EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
75
.\" This license will be governed by the laws of the Commonwealth of Virginia,
76
.\" without reference to its choice of law rules.
78
.ie '\\n(.z'' .bp \\$1
82
.TH pbs_scheduler_basl 8B "" Local PBS
86
pbs_sched_basl \- pbs BASL scheduler
88
pbs_sched\ [\^-d home\^] [\^-L logfile\^] [\^-p print_file\^]
89
[\^-a alarm\^] [\^-S port\^] [\^-c configfile\^]
93
command starts the operation of a batch scheduler on the local host.
94
It runs in conjunction with the PBS server. It queries the
95
server about the state of PBS and communicates with
97
to get information about the status of running jobs, memory available etc.
98
It then makes decisions as to what jobs to run.
100
Typically, this command will be in a local boot file such as
103
pbs_sched must be executed with root permission.
106
Specifies the name of the PBS home directory, PBS_HOME.
107
If not specified, the value
108
of $PBS_SERVER_HOME as defined at compile time is used. Also see the -L
111
Specifies an absolute path name of the file to use as the log file.
112
If not specified, the scheduler will
113
open a file named for the current date in the PBS_HOME/sched_logs directory.
115
.IP "-p print_file" 5
116
This specifies the "print" file. Any output from the scheduler code which is
117
written to standard out or standard error will be written to this file.
118
If this option is not given, the file used will be
119
$PBS_HOME/sched_priv/sched_out. See the -d option.
121
This specifies the time in seconds to wait for a schedule run to finish. If a
122
scheduling iteration takes too long to finish, an alarm signal is sent, and
123
the scheduler is restarted. If a core file does not exist in the current
124
directory, abort() is called and a core file is generated. The default for
125
alarm is 180 seconds.
127
Specifies a port on which to talk to the server. This option is not
128
required. It merely overides the default PBS scheduler port.
129
.IP "-c configfile" 5
130
Specify a configuration file, see description below. If this is a relative
131
file name it will be relative to PBS_HOME/sched_priv, see the -d option.
132
If the -c option is not supplied, pbs_sched will not attempt to open a
133
configuration file. In BASL, this config file is almost always needed because
134
it is where the list of servers, nodes, and host resource queries are specified by the administrator.
137
This version of the scheduler requires knowledge of the BASL language. The
138
site must first write a function called
140
(and all functions supporting it) using BASL constructs, and then translate the
141
functions into C using the BASL compiler
143
which would also attach a
144
main program to the resulting code. This main program performs general
145
initialization and housekeeping chores such as setting up local socket to
146
communicate with the server running on the same machine, cd-ing to the priv
147
directory, opening log files, opening configuration file (if any), setting up
148
locks, forking the child to become a daemon, initializing a scheduling cycle
149
(i.e. get node attributes that are static in nature), setting up the signal
150
handlers, executing global initialization assignment statements specified by
151
the scheduler writer, and finally sitting on a loop waiting for a scheduling
152
command from the server. When the server sends the scheduler an appropriate
154
.Sc "SCH_SCHEDULE_NEW, SCH_SCHEDULE_TERM, SCH_SCHEDULE_TIME, SCH_SCHEDULE_RECYC, SCH_SCHEDULE_CMD, SCH_SCHEDULE_FIRST",
155
information about server(s), jobs, queues, and execution host(s) are obtained,
159
.Sh SCHEDULING LANGUAGE
160
The BAtch Scheduling Language (BASL) is a C-like procedural language. It
162
of constructs and predefined functions that facilitate dealing with
163
scheduling issues. Information about a PBS server, the queues that it owns,
164
jobs residing on each queue, and the computational nodes where jobs can be run,
165
are accessed via the BASL data types Server, Que, Job, CNode, Set Server,
166
Set Que, Set Job, and Set CNode.
169
simple sched_main() will cause the server to run all queued jobs on the
182
s = AllServersLocalHostGet(); // get local server
183
queues = ServerQueuesGet(s);
185
foreach( q in queues ) {
186
jobs = QueJobsGet(q);
187
foreach( j in jobs ) {
188
JobAction(j, SYNCRUN, NULLSTR);
195
For a more complete discussion of the Batch Scheduler Language, see
197
.Sh CONFIGURATION FILE
198
A configuration file may be specified with the -c option.
199
This file is used to specify the (1) hosts which are allowed to
200
connect to pbs_sched, (2) the list of server hosts for which the scheduler
201
writer wishes the system to periodically check for status, queues, and jobs
202
info, (3) list of execution hosts for which the scheduler writer wants the
203
system to periodically check for information like state, property, and so on,
204
and (4) various queries to send to each execution host.
205
.IP "(1) specifying client hosts:" 6
206
The hosts allowed to connect to pbs_sched are specified in the configuration
207
file in a manner identical to that used in pbs_mom. There is one line per
208
host using the syntax:
210
.Ty "$clienthost hostname"
216
are separated by white space.
217
Two host names are always allowed to connection to pbs_sched: "localhost"
218
and the name returned to pbs_sched by the system call gethostname(). These
219
names need not be specified in the configuration file.
220
.IP "(2) specifying list of servers:" 6
221
The list of servers is specified in a one host per line manner, using the
224
.Ty "$serverhost hostname port_number"
232
are separated by white space.
236
is 0, then the default PBS server port will be used.
238
Regardless of what has been specified in the file, the list of servers
239
will always include the local server - one running on the same host where
240
the scheduler is running.
242
Within the BASL code, access to data of the list of servers is done by calling
245
.I AllServersLocalHostGet()
246
which returns the local server on the list.
247
.IP "(3) specifying the list of execution hosts:" 6
248
The list of execution hosts (nodes), whose MOMs are to be queried from the
249
scheduler, is specified in a one host per line manner,
252
.Ty "$momhost hostname port_number"
259
are separated by white space.
263
is 0, then the default PBS MOM port will be used.
268
.I ServerNodesGet(AllServersLocalHostGet())
269
is available for getting the list of nodes known to the local system.
270
.IP "(4) specifying the list of host resources:"
271
For specifying the list of host resource queries to send to each execution
272
host's MOM, the following syntax is used:
274
.Ty "$node node_name CNode..Get host_resource
277
should be the same hostname string that was specified in a
281
value of "*" (wildcard) means to match any node.
283
Please consult section 9 of the PBS ERS (Resource Monitor/Resources) for a
284
list of possible values to
289
refers to the actual function name that is called from the
290
scheduler code to obtain the return values to host resource queries.
293
function names that can appear in the configuration file are:
298
================================
303
CNodeMemTotalGet[type]
304
CNodeNetworkBwGet[type]
305
CNodeSwapSpaceTotalGet[name]
306
CNodeDiskSpaceTotalGet[name]
307
CNodeDiskInBwGet[name]
308
CNodeDiskOutBwGet[name]
309
CNodeTapeSpaceTotalGet[name]
310
CNodeTapeInBwGet[name]
311
CNodeTapeOutBwGet[name]
312
CNodeSrfsSpaceTotalGet[name]
313
CNodeSrfsInBwGet[name]
314
CNodeSrfsOutBwGet[name]
317
================================
320
CNodeMemAvailGet[type]
321
CNodeSwapSpaceAvailGet[name]
322
CNodeSwapInBwGet[name]
323
CNodeSwapOutBwGet[name]
324
CNodeDiskSpaceReservedGet[name]
325
CNodeDiskSpaceAvailGet[name]
326
CNodeTapeSpaceAvailGet[name]
327
CNodeSrfsSpaceReservedGet[name]
328
CNodeSrfsSpaceAvailGet[name]
329
CNodeCpuPercentIdleGet
330
CNodeCpuPercentSysGet
331
CNodeCpuPercentUserGet
332
CNodeCpuPercentGuestGet
336
STATIC function names return values that are obtained only during the first
337
scheduling cycle, or when the scheduler is instructed to reconfig; whereas,
338
DYNAMIC function names return attribute values that are taken at every
339
subsequent scheduling cycle.
344
are arbitrarily defined. For example, you can choose to have
347
"$FASTDIR" for the CNodeSrfs* calls, and a sample configuration file entry
351
$node unicos8 CNodeSrfsSpaceAvailGet[$FASTDIR]
352
quota[type=ares_avail,dir=$FASTDIR]
355
So in a BASL code, if you call CNodeSrfsSpaceAvailGet(node, "$FASTDIR"), then
356
it will return the value to the query "quota[type=ares_avail,dir=$FASTDIR]"
357
(3rd parameter) as sent to the node's MOM.
359
By default, the scheduler has already internally defined the following
360
mappings, which can be overriden in the configuration file:
364
keyword node_name CNode..Get host_resource
365
======= ========= ================ =============
366
$node * CNodeOsGet arch
367
$node * CNodeLoadAveGet loadave
368
$node * CNodeIdletimeGet idletime
372
The above means that for all declared nodes (via $momhost), the host queries
377
will be sent to each node's MOM. The value to
379
is obtained internally by the system during the first scheduling cycle because
380
it falls under STATIC category, while values to
384
are taken at every scheduling iteration because they fall under the DYNAMIC
386
the return values is done by calling
387
.Ar "CNodeOsGet(node)",
388
.Ar "CNodeLoadAveGet(node)",
390
.Ar "CNodeIdletimeGet(node)",
392
The following are some sample $node arguments that you may put in the
397
node_name CNode..Get host res
398
================== ========================= ==========
399
<sunos4_nodename> CNodeIdletimeGet idletime
400
<sunos4_nodename> CNodeLoadAveGet loadave
401
<sunos4_nodename> CNodeMemTotalGet[real] physmem
402
<sunos4_nodename> CNodeMemTotalGet[virtual] totmem
403
<sunos4_nodename> CNodeMemAvailGet[virtual] availmem
405
<irix5_nodename> CNodeNumCpusGet ncpus
406
<irix5_nodename> CNodeMemTotalGet[real] physmem
407
<irix5_nodename> CNodeMemTotalGet[virtual] totmem
408
<irix5_nodename> CNodeIdletimeGet idletime
409
<irix5_nodename> CNodeLoadAveGet loadave
410
<irix5_nodename> CNodeMemAvailGet[virtual] availmem
412
<linux_nodename> CNodeNumCpusGet ncpus
413
<linux_nodename> CNodeMemTotalGet[real] physmem
414
<linux_nodename> CNodeMemTotalGet[virtual] totmem
415
<linux_nodename> CNodeIdletimeGet idletime
416
<linux_nodename> CNodeLoadAveGet loadave
417
<linux_nodename> CNodeMemAvailGet[virtual] availmem
419
<solaris5_nodename> CNodeIdletimeGet idletime
420
<solaris5_nodename> CNodeLoadAveGet loadave
421
<solaris5_nodename> CNodeNumCpusGet ncpus
422
<solaris5_nodename> CNodeMemTotalGet[real] physmem
424
<aix4_nodename> CNodeIdletimeGet idletime
425
<aix4_nodename> CNodeLoadAveGet loadave
426
<aix4_nodename> CNodeMemTotalGet[virtual] totmem
427
<aix4_nodename> CNodeMemAvailGet[virtual] availmem
429
<unicos8_nodename> CNodeIdletimeGet idletime
430
<unicos8_nodename> CNodeLoadAveGet loadave
431
<unicos8_nodename> CNodeNumCpusGet ncpus
432
<unicos8_nodename> CNodeMemTotalGet[real] physme
433
<unicos8_nodename> CNodeMemAvailGet[virtual] availmem
434
<unicos8_nodename> CNodeSwapSpaceTotalGet[primary] swaptotal
435
<unicos8_nodename> CNodeSwapSpaceAvailGet[primary] swapavail
436
<unicos8_nodename> CNodeSwapInBwGet[primary] swapinrate
437
<unicos8_nodename> CNodeSwapOutBwGet[primary] swapoutrate
438
<unicos8_nodename> CNodePercentIdleGet cpuidle
439
<unicos8_nodename> CNodePercentSysGet cpuunix
440
<unicos8_nodename> CNodePercentGuestGet cpuguest
441
<unicos8_nodename> CNodePercentUsrGet cpuuser
442
<unicos8_nodename> CNodeSrfsSpaceAvailGet[$FASTDIR] quota[type
446
<unicos8_nodename> CNodeSrfsSpaceAvailGet[$BIGDIR] quota[type
450
<unicos8_nodename> CNodeSrfsSpaceAvailGet[$WRKDIR] quota[type
454
<sp2_nodename> CNodeLoadAveGet loadave
458
Suppose you have an execution host that is of irix5 os type, then the
459
<irix5_node_name> entries will be consulted by the scheduler. The
460
initial scheduling cycle would involve sending the STATIC
465
to the execution host's MOM, and access to return values of the queries is done
467
.Ty "CNodeNumCpusGet(node)",
468
.Ty "CNodeMemTotalGet(node, ""real"")",
469
.Ty "CNodeMemTotalGet(node, ""virtual"")"
470
respectively, where node is the CNode representation of the execution host.
471
The subsequent scheduling cycles will only send DYNAMIC queries
476
and access to the return values of the queries is done via
477
.Ty "CNodeIdleTimeGet(node)",
478
.Ty "CNodeLoadAveGet(node)",
479
.Ty "CNodeMemAvailGet(node, ""virtual"")".
483
"Later" entries in the config file take precedence.
485
The configuration file must be "secure". It must be owned by a user id and
486
group id less than 10 and not be world writable.
488
On receipt of a SIGHUP signal, the scheduler will close and reopen
489
its log file and reread its configuration file (if any).
492
.IP $PBS_SERVER_HOME/sched_priv 10
493
the default directory for configuration files, typically
494
(/usr/spool/pbs)/sched_priv.
497
A C based scheduler will handle the following signals:
499
The server will close and reopen its log file and reread the config file
502
If the site supplied scheduling module exceeds the time limit, the Alarm
503
will cause the scheduler to attempt to core dump and restart itself.
504
.IP "SIGINT and SIGTERM"
505
Will result in an orderly shutdown of the scheduler.
507
All other signals have the default action installed.
509
Upon normal termination, an exit status of zero is returned.
511
basl2c(1B), pbs_sched_tcl(8B), pbs_server(8B), and pbs_mom(8B).
513
PBS Internal Design Specification
514
.\" turn off any extra indent left by the Sh macro
added
removed
doc/man8/pbs_sched_basl.8B
