1
.TH MPY 1 "2010 MARCH 21"
4
mpy \- Message Passing Yorick
26
is an interpreted language like Basic or Lisp, but far faster. See
28
(1) to learn more about it.
31
is a parallel version of
33
based on the Message Passing Interface (MPI). The exact syntax for launching a parallel job depends on your MPI environment. It may be necessary to launch a special daemon before calling
35
or an equivalent command.
38
The mpy package interfaces yorick to the MPI parallel programming
39
library. MPI stands for Message Passing Interface; the idea is to
40
connect multiple instances of yorick that communicate among themselves
41
via messages. Mpy can either perform simple, highly parallel tasks as
42
pure interpreted programs, or it can start and steer arbitrarily
43
complex compiled packages which are free to use the compiled MPI API.
44
The interpreted API is not intended to be an MPI wrapper; instead it
45
is stripped to the bare minimum.
47
This is version 2 of mpy (released in 2010); it is incompatible with
48
version 1 of mpy (released in the mid 1990s), because version 1 had
49
numerous design flaws making it very difficult to write programs free
50
of race conditions, and impossible to scale to millions of processors.
51
However, you can run most version 1 mpy programs under version 2 by
52
doing mp_include,"mpy1.i" before you mp_include any file defining an
53
mpy1 parallel task (that is before any file containg a call to
56
The MPI environment is not really specified by the standard; existing
57
environments are very crude, and strongly favor non-interactive batch
58
jobs. The number of processes is fixed before MPI begins; each
59
process has a rank, a number from 0 to one less than the number of
60
processes. You use the rank as an address to send messages, and the
61
process receiving the message can probe to see which ranks have sent
62
messages to it, and of course receive those messages.
64
A major problem in writing a message passing program is handling
65
events or messages arriving in an unplanned order. MPI guarantees
66
only that a sequence of messages send by rank A to rank B will arrive
67
in the order sent. There is no guarantee about the order of arrival
68
of those messages relative to messages sent to B from a third rank C.
69
In particular, suppose A sends a message to B, then A sends a message
70
to C (or even exchanges several messages with C) which results in C
71
sending a message to B. The message from C may arrive at B before the
72
message from A. An MPI program which does not allow for this
73
possibility has a bug called a "race condition". Race conditions may
74
be extremely subtle, especially when the number of processes is large.
76
The basic mpy interpreted interface consists of two variables:
77
mp_size = number of proccesses
78
mp_rank = rank of this process
80
mp_send, to, msg; // send msg to rank "to"
81
msg = mp_recv(from); // receive msg from rank "from"
82
ranks = mp_probe(block); // query senders of pending messages
83
mp_exec, string; // parse and execute string on every rank
85
You call mp_exec on rank 0 to start a parallel task. When the main
86
program thus created finishes, all ranks other than rank 0 return to
87
an idle loop, waiting for the next mp_exec. Rank 0 picks up the next
88
input line from stdin (that is, waits for input at its prompt in an
89
interactive session), or terminates all processes if no more input is
90
available in a batch session.
92
The mpy package modifies how yorick handles the #include parser
93
directive, and the include and require functions. Namely, if a
94
parallel task is running (that is, a function started by mp_exec),
95
these all become collective operations. That is, rank 0 reads the
96
entire file contents, and sends the contents to the other processes as
97
an MPI message (like mp_exec of the file contents). Every process
98
other than rank 0 is only running during parallel tasks; outside a
99
parallel task when only rank 0 is running (and all other ranks are
100
waiting for the next mp_exec), the #include directive and the include
101
and require functions return to their usual serial operation,
102
affecting only rank 0.
104
When mpy starts, it is in parallel mode, so that all the files yorick
105
includes when it starts (the files in Y_SITE/i0) are included as
106
collective operations. Without this feature, every yorick process
107
would attempt to open and read the startup include files, overloading
108
the file system before mpy ever gets started. Passing the contents of
109
these files as MPI messages is the only way to ensure there is enough
110
bandwidth for every process to read the contents of a single file.
112
The last file included at startup is either the file specified in the
113
\-batch option, or the custom.i file. To avoid problems with code in
114
custom.i which may not be safe for parallel execution, mpy does not
115
look for custom.i, but for custommp.i instead. The instructions in
116
the \-batch file or in custommp.i are executed in serial mode on rank 0
117
only. Similarly, mpy overrides the usual process_argv function, so
118
that \-i and other command line options are processed only on rank 0 in
119
serial mode. The intent in all these cases is to make the \-batch or
120
custommp.i or \-i include files execute only on rank 0, as if you had
121
typed them there interactively. You are free to call mp_exec from any
122
of these files to start parallel tasks, but the file itself is serial.
124
An additional command line option is added to the usual set:
127
includes somefile.i in parallel mode on all ranks (again, \-i other.i
128
includes other.i only on rank 0 in serial mode). If there are
129
multiple \-j options, the parallel includes happen in command line
130
order. If \-j and \-i options are mixed, however, all \-j includes
131
happen before any \-i includes.
133
As a side effect of the complexity of include functions in mpy, the
134
autoload feature is disabled; if your code actually triggers an
135
include by calling an autoloaded function, mpy will halt with an
136
error. You must explicitly load any functions necessary for a
137
parallel tasks using require function calls themselves inside a
140
The mp_send function can send any numeric yorick array (types char,
141
short, int, long, float, double, or complex), or a scalar string
142
value. The process of sending the message via MPI preserves only the
143
number of elements, so mp_recv produces only a scalar value or a 1D
144
array of values, no matter what dimensionality was passed to mp_send.
146
The mp_recv function requires you to specify the sender of the message
147
you mean to receive. It blocks until a message actually arrives from
148
that sender, queuing up any messages from other senders that may
149
arrive beforehand. The queued messages will be retrieved it the order
150
received when you call mp_recv for the matching sender. The queuing
151
feature makes it dramatically easier to avoid the simplest types of
152
race condition when you are write interpreted parallel programs.
154
The mp_probe function returns the list of all the senders of queued
155
messages (or nil if the queue is empty). Call mp_probe(0) to return
156
immediately, even if the queue is empty. Call mp_probe(1) to block if
157
the queue is empty, returning only when at least one message is
158
available for mp_recv. Call mp_probe(2) to block until a new message
159
arrives, even if some messages are currently available.
161
The mp_exec function uses a logarithmic fanout - rank 0 sends to F
162
processes, each of which sends to F more, and so on, until all
163
processes have the message. Once a process completes all its send
164
operations, it parses and executes the contents of the message. The
165
fanout algorithm reaches N processes in log to the base F of N steps.
166
The F processes rank 0 sends to are ranks 1, 2, 3, ..., F. In
167
general, the process with rank r sends to ranks r*F+1, r*F+2, ...,
168
r*F+F (when these are less than N-1 for N processes). This set is
169
called the "staff" of rank r. Ranks with r>0 receive the message from
170
rank (r\-1)/F, which is called the "boss" of r. The mp_exec call
171
interoperates with the mp_recv queue; in other words, messages from
172
a rank other than the boss during an mp_exec fanout will be queued for
173
later retrieval by mp_recv. (Without this feature, any parallel task
174
which used a message pattern other than logarithmic fanout would be
175
susceptible to race conditions.)
177
The logarithmic fanout and its inward equivalent are so useful that
178
mpy provides a couple of higher level functions that use the same
179
fanout pattern as mp_exec:
181
total = mp_handin(value);
183
To use mp_handout, rank 0 computes a msg, then all ranks call
184
mp_handout, which sends msg (an output on all ranks other than 0)
185
everywhere by the same fanout as mp_exec. To use mp_handin, every
186
process computes value, then calls mp_handin, which returns the sum of
187
their own value and all their staff, so that on rank 0 mp_handin
188
returns the sum of the values from every process.
190
You can call mp_handin as a function with no arguments to act as a
191
synchronization; when rank 0 continues after such a call, you know
192
that every other rank has reached that point. All parallel tasks
193
(anything started with mp_exec) must finish with a call to mp_handin,
194
or an equivalent guarantee that all processes have returned to an idle
195
state when the task finishes on rank 0.
197
You can retrieve or change the fanout parameter F using the mp_nfan
198
function. The default value is 16, which should be reasonable even
199
for very large numbers of processes.
201
One special parallel task is called mp_connect, which you can use to
202
feed interpreted command lines to any single non-0 rank, while all
203
other ranks sit idle. Rank 0 sits in a loop reading the keyboard and
204
sending the lines to the "connected" rank, which executes them, and
205
sends an acknowledgment back to rank 0. You run the mp_disconnect
206
function to complete the parallel task and drop back to rank 0.
208
Finally, a note about error recovery. In the event of an error during
209
a parallel task, mpy attempts to gracefully exit the mp_exec, so that
210
when rank 0 returns, all other ranks are known to be idle, ready for
211
the next mp_exec. This procedure will hang forever if any one of the
212
processes is in an infinite loop, or otherwise in a state where it
213
will never call mp_send, mp_recv, or mp_probe, because MPI provides no
214
means to send a signal that interrupts all processes. (This is one of
215
the ways in which the MPI environment is "crude".) The rank 0 process
216
is left with the rank of the first process that reported a fault, plus
217
a count of the number of processes that faulted for a reason other
218
than being sent a message that another rank had faulted. The first
219
faulting process can enter dbug mode via mp_connect; use mp_disconnect
220
or dbexit to drop back to serial mode on rank 0.
224
includes the Yorick source file
226
as mpy starts in parallel mode on all ranks. This is equivalent to
227
the mp_include function after mpy has started.
230
includes the Yorick source file
232
as mpy starts, in serial mode. This is equivalent to the #include
233
directive after mpy has started.
236
includes the Yorick source file
238
as mpy starts, in serial mode. Your customization file custommp.i, if any, is
241
and mpy is placed in batch mode. Use the help command on the batch
242
function (help, batch) to find out more about batch mode. In batch
243
mode, all errors are fatal; normally, mpy will halt execution and
244
wait for more input after an error.
248
David H. Munro, Lawrence Livermore National Laboratory
252
Mpy uses the same files as yorick, except that custom.i is replaced by
253
custommp.i (located in /etc/yorick/mpy/ on Debian based systems) and
254
the Y_SITE/i\-start/ directory is ignored.
added
removed
mpy/mpy.1
