1
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
4
<title>ARMCI - Aggregate Remote Memory Copy Interface</title>
5
<meta http-equiv="Content-Type"
6
content="text/html; charset=iso-8859-1">
7
<meta name="Description"
8
content="The homepage for the ARMCI portable remote memory copy library.">
10
content="remote memory copy, one-sided communication, 1-sided communication, MPI-2, Global Arrays, ACTS, parallel computing, pmodels">
13
style="background-image: url(file:///Q:/docs/parsoft/armci/images/main_bg.jpg);"
14
leftmargin="0" topmargin="0" marginwidth="0" marginheight="0">
15
<script type="text/javascript" language="javascript"
16
src="scripts/header.js"> </script>
17
<table width="100%" height="364" border="0" cellpadding="0"
22
<table width="750" height="600" border="0" cellpadding="0"
26
<td width="140" valign="top">
27
<table width="140" border="0" cellpadding="0"
31
<td width="122" height="40"><a href="#init"><font
32
size="-1">Initialization
33
& Termination</font></a></td>
36
<td width="122" height="40"><a href="#copyvec"><font
37
size="-1">Put/Get (vector)</font></a></td>
40
<td width="122" height="40"><a href="#copys"><font
41
size="-1">PutGet (strided)</font></a></td>
44
<td width="122" height="40"><a href="#copy"><font
46
(contiguous)</font></a></td>
49
<td width="122" height="40"><a href="#accv"><font
50
size="-1">Accumulate (vector)</font></a></td>
53
<td width="122" height="40"><a href="#accs"><font
55
(strided)</font></a></td>
58
<td width="122" height="40"><a href="#acc"><font
60
(contig.)</font></a></td>
63
<td width="122" height="40"><a href="#reg"><font
64
size="-1">Register Ops</font></a></td>
67
<td width="122" height="40"><a href="#compl"><font
68
size="-1">Fence / Wait /
69
Barrier</font></a></td>
72
<td width="122" height="40"><a href="#aggr"><font
73
size="-1">Aggregation</font></a></td>
76
<td width="122" height="40"><a href="#atomic"><font
77
size="-1">Atomic / Sync</font></a></td>
80
<td width="122" height="40"><a href="#mem"><font
81
size="-1">Memory Mgmt.</font></a></td>
84
<td width="122" height="40"><a href="#collect"><font
85
size="-1">Collective Ops</font></a></td>
88
<td style="vertical-align: top;"><small><a
89
href="#config">Configuration Info</a></small><br>
95
<td width="628" valign="top"><!-- ----------- TEXT STARTS HERE ------------- -->
97
<font face="Arial,Helvetica"><font color="#000099">
98
<h2>ARMCI - Programming Interfaces<br>
100
</font></font> <font face="Arial,Helvetica"><font size="+0"
101
color="#000099"> </font></font><small><font face="Arial,Helvetica"><font
102
size="-1"><big style="color: rgb(0, 0, 153);">ARMCI programming
104
explained below. This is an up-to-date document that containing all the
105
supporting ARMCI APIs. There is also a
107
describing the ARMCI design (API corresponding to release 1.0 only) is <a
108
href="publications/armci1-1.pdf">available in the
109
PDF</a><a href="publications/armci1-1.pdf">
110
format</a>. </big><br>
111
</font></font></small> <br>
112
<font face="Arial,Helvetica" style="font-weight: bold;"><font
113
size="+0" color="#000099">Header file</font></font><br>
114
<font face="Arial,Helvetica"><font size="+0" color="#000099">The
115
interfaces are prototyped in the "armci.h" header file.<br>
116
</font></font>
117
<h3><font face="Arial,Helvetica"><font size="+0"
118
color="#000099"><a name="init"></a>1 Initialization and Termination</font></font></h3>
119
<font face="Arial,Helvetica"><font size="+0" color="#000099">
121
<pre><font face="Arial,Helvetica"><font size="+0"
122
color="#000099">int <span style="font-weight: bold;">ARMCI_Init</span>()</font></font></pre>
123
<font face="Arial,Helvetica"><font size="+0" color="#000099">
125
<pre><font face="Arial,Helvetica"><font size="+0"
126
color="#000099">PURPOSE: Initializes the ARMCI. This function must be called before any ARMCI functions.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes). </font></font></pre>
127
<font face="Arial,Helvetica"><font size="+0" color="#000099">
129
<pre><font face="Arial,Helvetica"><font size="+0"
130
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_Finalize</span>()</font></font></pre>
131
<font face="Arial,Helvetica"><font size="+0" color="#000099">
133
<pre><font face="Arial,Helvetica"><font size="+0"
134
color="#000099">PURPOSE: Finalizes the ARMCI. This function must be called after using ARMCI functions.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
135
<font face="Arial,Helvetica"><font size="+0" color="#000099">
137
<pre><font face="Arial,Helvetica"><font size="+0"
138
color="#000099"><br>void <span style="font-weight: bold;">ARMCI_Cleanup</span>()</font></font></pre>
139
<pre><font face="Arial,Helvetica"><font size="+0"
140
color="#000099">PURPOSE: Releases any system resources (like System V shmem ids) that<br>ARMCI can be holding. It is intended to be used before terminating a program<br>(e.g., by calling MPI_Abort) in case of an error.</font></font></pre>
141
<font face="Arial,Helvetica"><font size="+0" color="#000099">
143
<pre><font face="Arial,Helvetica"><font size="+0"
144
color="#000099"><br>void <span style="font-weight: bold;">ARMCI_Error</span>(char *message, int code)</font></font></pre>
145
<font face="Arial,Helvetica"><font size="+0" color="#000099">Purpose:
146
Combines the functionality of ARMCI_Cleanup and MPI_Abort, and
147
it prints (to the stdout and stderr) a user specified message followed
148
by an integer code. </font></font>
149
<pre><font face="Arial,Helvetica"><font size="+0"
150
color="#000099">ARGUMENTS: <br>message - Message to print out<br>code - Error code<br></font></font></pre>
151
<font face="Arial,Helvetica"><font size="+0" color="#000099">
153
<hr style="width: 100%; height: 2px;"><font
154
face="Arial,Helvetica"><font size="+0" color="#000099"> </font></font>
155
<h3><font face="Arial,Helvetica"><font size="+0"
156
color="#000099"><a name="copyvec"></a>2 Copy operations using
157
the generalized I/O vector format</font></font></h3>
158
<font face="Arial,Helvetica"><font size="+0" color="#000099">
160
<pre><font face="Arial,Helvetica"><font size="+0"
161
color="#000099">int <span style="font-weight: bold;">ARMCI_PutV</span>(armci_giov_t *dsrc_arr, int arr_len, int proc)</font></font></pre>
162
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
163
Generalized I/O vector operation that transfers data
165
the local memory of calling process (source) to the memory of a remote
166
process (destination). </font></font>
167
<pre><font face="Arial,Helvetica"><font size="+0"
168
color="#000099">DATA TYPE:<br> typedef struct {<br> void **src_ptr_ar; - Source starting addresses of each data segment.<br> void **dst_ptr_ar; - Destination starting addresses of each data segment.<br> int bytes; - The length of each segment in bytes.<br> int ptr_ar_len; - Number of data segment.<br> }armci_giov_t;</font></font></pre>
169
<font face="Arial,Helvetica"><font size="+0" color="#000099">
171
<pre><font face="Arial,Helvetica"><font size="+0"
172
color="#000099">ARGUMENTS:<br> dsrc_arr - Array of data (type of armci_giov_t) to be put to remote process.<br> arr_len - Number of elements in the dsrc_arr.<br> proc - Remote process ID (destination).</font></font></pre>
173
<font face="Arial,Helvetica"><font size="+0" color="#000099">
175
<pre><font face="Arial,Helvetica"><font size="+0"
176
color="#000099">RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
177
<font face="Arial,Helvetica"><font size="+0" color="#000099">
179
<pre><font face="Arial,Helvetica"><font size="+0"
180
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbPutV</span>(armci_giov_t *dsrc_arr,int arr_len,int proc,armci_hdl_t* handle)</font></font></pre>
181
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
182
Generalized Non-Blocking I/O vector operation that transfers
184
from the local memory of the calling process (source) to the memory of
187
process (destination). </font></font>
188
<pre><font face="Arial,Helvetica"><font size="+0"
189
color="#000099">DATA TYPE:<br> typedef struct {<br> void **src_ptr_ar; - Source starting addresses of each data segment.<br> void **dst_ptr_ar; - Destination starting addresses of each data segment.<br> int bytes; - The length of each segment in bytes.<br> int ptr_ar_len; - Number of data segment.<br> }armci_giov_t;</font></font></pre>
190
<font face="Arial,Helvetica"><font size="+0" color="#000099">
192
<pre><font face="Arial,Helvetica"><font size="+0"
193
color="#000099">ARGUMENTS:<br> dsrc_arr - Array of data (type of armci_giov_t) to be put to remote process.<br> arr_len - Number of elements in the dsrc_arr.<br> proc - Remote process ID (destination).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.</font></font></pre>
194
<font face="Arial,Helvetica"><font size="+0" color="#000099">
196
<pre><font face="Arial,Helvetica"><font size="+0"
197
color="#000099">RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
198
<font face="Arial,Helvetica"><font size="+0" color="#000099">
200
<pre><font face="Arial,Helvetica"><font size="+0"
201
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_GetV</span>(armci_giov_t *dsrc_arr,int arr_len,int proc)</font></font></pre>
202
<font face="Arial,Helvetica"><font size="+0" color="#000099"> PURPOSE:
203
Generalized I/O vector blocking operation that
205
data from the remote process memory (source) to the calling process
207
memory (destination). </font></font>
208
<pre><font face="Arial,Helvetica"><font size="+0"
209
color="#000099">DATA TYPE:<br> typedef struct {<br> void **src_ptr_ar; - Source starting addresses of each data segment.<br> void **dst_ptr_ar; - Destination starting addresses of each data segment.<br> int bytes; - The length of each segment in bytes.<br> int ptr_ar_len; - Number of data segment.<br> }armci_giov_t;</font></font></pre>
210
<font face="Arial,Helvetica"><font size="+0" color="#000099">
212
<pre><font face="Arial,Helvetica"><font size="+0"
213
color="#000099">ARGUMENTS:<br> dsrc_arr - Array of data (type of armci_giov_t) to get from remote process.<br> arr_len - Number of elements in the dsrc_arr.<br> proc - Remote process ID (source).</font></font></pre>
214
<font face="Arial,Helvetica"><font size="+0" color="#000099">
216
<pre><font face="Arial,Helvetica"><font size="+0"
217
color="#000099">RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
218
<font face="Arial,Helvetica"><font size="+0" color="#000099">
220
<pre><font face="Arial,Helvetica"><font size="+0"
221
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbGetV</span>(armci_giov_t *dsrc_arr,int arr_len,int proc,armci_hdl_t* handle)</font></font></pre>
222
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
223
Generalized Non-blocking I/O vector operation that transfers
224
data from the remote process memory (source) to the callingprocess
225
local memory (destination). </font></font>
226
<pre><font face="Arial,Helvetica"><font size="+0"
227
color="#000099">DATA TYPE:<br> typedef struct {<br> void **src_ptr_ar; - Source starting addresses of each data segment.<br> void **dst_ptr_ar; - Destination starting addresses of each data segment.<br> int bytes; - The length of each segment in bytes.<br> int ptr_ar_len; - Number of data segment.<br> }armci_giov_t;<br>ARGUMENTS:<br><br> dsrc_arr - Array of data (type of armci_giov_t) to get from remote process.<br> arr_len - Number of elements in the dsrc_arr.<br> proc - Remote process ID (source).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
228
<font face="Arial,Helvetica"><font size="+0" color="#000099">_____________________________________________________________________________
231
<h3><font face="Arial,Helvetica"><font size="+0"
232
color="#000099"><a name="copys"></a>3 Copy operations using the
234
format</font></font></h3>
235
<font face="Arial,Helvetica"><font size="+0" color="#000099">
237
<pre><font face="Arial,Helvetica"><font size="+0"
238
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_PutS</span>(void* src_ptr, int src_stride_ar[], void* dst_ptr, int dst_stride_ar[], <br> int count[], int stride_levels, int proc)<br></font></font></pre>
239
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
240
Blocking strided operation that transfers data from the
241
local memory of calling process (source) to the memory of a remote
242
process (destination).<br>
244
<pre><font face="Arial,Helvetica"><font size="+0"
245
color="#000099">ARGUMENTS:<br> src_ptr - Source starting address of the data block to put.<br> src_stride_arr - Source array of stride distances in bytes.<br> dst_ptr - Destination starting address to put data.<br> dst_stride_ar - Destination array of stride distances in bytes.<br> count - Block size in each dimension. count[0] should be the<br> number of bytes of contiguous data in leading dimension.<br> stride_levels - The level of strides.<br> proc - Remote process ID (destination).<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
246
<font face="Arial,Helvetica"><font size="+0" color="#000099">
248
<pre><font face="Arial,Helvetica"><font size="+0"
249
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbPutS</span>(void* src_ptr, int src_stride_ar[], void* dst_ptr, int dst_stride_ar[], <br> int count[], int stride_levels, int proc,armci_hdl_t* handle)</font></font></pre>
250
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
251
Non-blocking strided operation that transfer data
253
the local memory of calling process (source) to the memory of a remote
254
process (destination). </font></font>
255
<pre><font face="Arial,Helvetica"><font size="+0"
256
color="#000099">ARGUMENTS:<br> src_ptr - Source starting address of the data block to put.<br> src_stride_arr - Source array of stride distances in bytes.<br> dst_ptr - Destination starting address to put data.<br> dst_stride_ar - Destination array of stride distances in bytes.<br> count - Block size in each dimension. count[0] should be the<br> number of bytes of contiguous data in leading dimension.<br> stride_levels - The level of strides.<br> proc - Remote process ID (destination).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
257
<font face="Arial,Helvetica"><font size="+0" color="#000099">
259
<pre><font face="Arial,Helvetica"><font size="+0"
260
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_GetS</span>(void *src_ptr c_ptr, int src_stride_ar[], void* dst_ptr,<br> int dst_stride_ar[], int count[], int stride_levels, int proc)</font></font></pre>
261
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
262
Blocking strided operation that transfers data
264
the remote process memory (source) to the calling process local
268
<pre><font face="Arial,Helvetica"><font size="+0"
269
color="#000099">ARGUMENTS:<br> src_ptr - Source starting address of the data block to get.<br> src_stride_arr - Source array of stride distances in bytes.<br> dst_ptr - Destination starting address to get data.<br> dst_stride_arr - Destination array of stride distances in bytes.<br> count - Block size in each dimension. count[0] should be the<br> number of bytes of contiguous data in leading dimension.<br> stride_levels - The level of strides.<br> proc - Remote process ID (source).<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
270
<font face="Arial,Helvetica"><font size="+0" color="#000099">
272
<pre><font face="Arial,Helvetica"><font size="+0"
273
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbGetS</span>(void *src_ptr c_ptr, int src_stride_ar[], void* dst_ptr, int dst_stride_ar[], <br> int count[], int stride_levels, int proc, armci_hdl_t* handle)</font></font></pre>
274
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
275
Non-blocking strided operation that transfers data
277
the remote process memory (source) to the calling process local
281
<pre><font face="Arial,Helvetica"><font size="+0"
282
color="#000099"> ARGUMENTS:<br> src_ptr - Source starting address of the data block to get.<br> src_stride_arr - Source array of stride distances in bytes.<br> dst_ptr - Destination starting address to get data.<br> dst_stride_arr - Destination array of stride distances in bytes.<br> count - Block size in each dimension. count[0] should be the<br> number of bytes of contiguous data in leading dimension.<br> stride_levels - The level of strides.<br> proc - Remote process ID (source).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
283
<font face="Arial,Helvetica"><font size="+0" color="#000099">_____________________________________________________________________________
286
<h3><font face="Arial,Helvetica"><font size="+0"
287
color="#000099"><a name="copy"></a>4 Copy operations for contiguous
288
data</font></font></h3>
289
<font face="Arial,Helvetica"><font size="+0" color="#000099">
291
<pre><font face="Arial,Helvetica"><font size="+0"
292
color="#000099">int <span style="font-weight: bold;">ARMCI_Put</span>(void* src, void* dst, int bytes, int proc)</font></font></pre>
293
<font face="Arial,Helvetica"><font size="+0" color="#000099">
294
<small>PURPOSE: Blocking transfer of contiguous data from
295
the local process
296
memory (source) to remote process memory (destination).</small> </font></font>
297
<pre><font face="Arial,Helvetica"><font size="+0"
298
color="#000099">ARGUMENTS:<br> src - Source starting address of the data block to put.<br> dst - Destination starting address to put data.<br> bytes - amount of data to transfer in bytes.<br> proc - Remote process ID (destination).<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
299
<font face="Arial,Helvetica"><font size="+0" color="#000099">
301
<pre><font face="Arial,Helvetica"><font size="+0"
302
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbPut</span>(void* src, void* dst, int bytes, int proc, armci_hdl_t* handle)</font></font></pre>
303
<font face="Arial,Helvetica"><font size="+0" color="#000099">
304
<small>PURPOSE: Non-blocking transfer of contiguous data
308
memory (source) to remote process memory (destination).</small> </font></font>
309
<pre><font face="Arial,Helvetica"><font size="+0"
310
color="#000099">ARGUMENTS:<br> src - Source starting address of the data block to put.<br> dst - Destination starting address to put data.<br> bytes - amount of data to transfer in bytes.<br> proc - Remote process ID (destination). <br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br><br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
311
<font face="Arial,Helvetica"><font size="+0" color="#000099">
313
<pre><font face="Arial,Helvetica"><font size="+0"
314
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_Get</span>(src, dst, bytes, proc)</font></font></pre>
315
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
316
Blocking transfer of contiguous data from the remote process
317
memory (source) to the calling process
318
memory (destination). </font></font>
319
<pre><font face="Arial,Helvetica"><font size="+0"
320
color="#000099">ARGUMENTS:<br> src - Source starting address of the data block to put.<br> dst - Destination starting address to put data.<br> bytes - amount of data to transfer in bytes.<br> proc - Remote process ID (destination).<br> <br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
321
<font face="Arial,Helvetica"><font size="+0" color="#000099">
323
<pre><font face="Arial,Helvetica"><font size="+0"
324
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbGet</span>(src, dst, bytes, proc, armci_hdl_t* handle)</font></font></pre>
325
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
326
Non-blocking transfer of contiguous
328
from the remote process memory (source) to the calling process
330
(destination). </font></font>
331
<pre><font face="Arial,Helvetica"><font size="+0"
332
color="#000099">ARGUMENTS:<br> src - Source starting address of the data block to put.<br> dst - Destination starting address to put data.<br> bytes - amount of data to transfer in bytes.<br> proc - Remote process ID (destination).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
333
<font face="Arial,Helvetica"><font size="+0" color="#000099">__________________________________________________________________________
335
<h3><font face="Arial,Helvetica"><font size="+0"
336
color="#000099"><a name="accv"></a>5 Accumulate operation using the
338
vector format</font></font></h3>
339
<font face="Arial,Helvetica"><font size="+0" color="#000099">Accumulate
340
operation performs atomic scaled reduction, i.e.,
341
remote += scale*local <br>
344
<pre><font face="Arial,Helvetica"><font size="+0"
345
color="#000099">int <span style="font-weight: bold;">ARMCI_AccV</span>(int datatype, void *scale, armci_giov_t *dsrc_arr, int arr_len, int proc, <br> armci_hdl_t* handle)<br><br>PURPOSE: Blocking generalized I/O vector operation that atomically updates the memory of<br> a remote process (destination).<br>DATA TYPE:<br> typedef struct {<br> void **src_ptr_ar; - Source starting addresses of each data segment.<br> void **dst_ptr_ar; - Destination starting addresses of each data segment.<br> int bytes; - The length of each segment in bytes.<br> int ptr_ar_len; - Number of data segment.<br> }armci_giov_t;<br><br>ARGUMENTS:<br> datatype - Supported data types are:<br> ARMCI_ACC_INT -> int, ARMCI_ACC_LNG -> long,<br> ARMCI_ACC_FLT -> float, ARMCI_ACC_DBL-> double,<br> ARMCI_ACC_CPL -> complex, ARMCI_ACC_DCPL -> double complex.<br> scale - Scale for the data (dest = dest + scale * src).<br> dsrc_arr - Array of data (type of armci_giov_t) to be accumulated to the remote process.<br> arr_len - Number of elements in the dsrc_arr.<br> proc - Remote process ID.<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
346
<font face="Arial,Helvetica"><font size="+0" color="#000099">
348
<pre><font face="Arial,Helvetica"><font size="+0"
349
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbAccV</span>(int datatype, void *scale, armci_giov_t *dsrc_arr, int arr_len, int proc, <br> armci_hdl_t* handle)</font></font></pre>
350
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
351
Non-blocking generalized I/O vector operation that
353
updates the memory of a remote process (destination). </font></font>
354
<pre><font face="Arial,Helvetica"><font size="+0"
355
color="#000099">DATA TYPE:<br> typedef struct {<br> void **src_ptr_ar; - Source starting addresses of each data segment.<br> void **dst_ptr_ar; - Destination starting addresses of each data segment.<br> int bytes; - The length of each segment in bytes.<br> int ptr_ar_len; - Number of data segment.<br> }armci_giov_t;<br>ARGUMENTS:<br> datatype - Supported data types are:<br> ARMCI_ACC_INT -> int, ARMCI_ACC_LNG -> long,<br> ARMCI_ACC_FLT -> float, ARMCI_ACC_DBL-> double,<br> ARMCI_ACC_CPL -> complex, ARMCI_ACC_DCPL -> double complex.</font></font></pre>
356
<font face="Arial,Helvetica"><font size="+0" color="#000099">
358
<pre><font face="Arial,Helvetica"><font size="+0"
359
color="#000099"> scale - Scale for the data (dest = dest + scale * src).<br> dsrc_arr - Array of data (type of armci_giov_t) to be accumulated to the remote process.<br> arr_len - Number of elements in the dsrc_arr.<br> proc - Remote process ID.<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
360
<font face="Arial,Helvetica"><font size="+0" color="#000099">_____________________________________________________________________________
362
<h3><font face="Arial,Helvetica"><font size="+0"
363
color="#000099"><a name="accs"></a>6 Accumulate operation using the
364
strided format</font></font></h3>
365
<font face="Arial,Helvetica"><font size="+0" color="#000099">
367
<pre><font face="Arial,Helvetica"><font size="+0"
368
color="#000099">int <span style="font-weight: bold;">ARMCI_AccS</span>(int datatype, void *scale, void* src_ptr,int src_stride_ar[], void* dst_ptr, <br> int dst_stride_ar[], int count[], int stride_levels, int proc)<br>PURPOSE: Blocking strided operation that atomicaly updates the memory of a remote process (destination).<br>ARGUMENTS:<br> datatype - Supported data types are:<br> ARMCI_ACC_INT -> int, ARMCI_ACC_LNG -> long,<br> ARMCI_ACC_FLT -> float, ARMCI_ACC_DBL-> double,<br> ARMCI_ACC_CPL -> complex, ARMCI_ACC_DCPL -> double complex.<br> scale - Scale for the data (dest = dest + scale * src).<br> src_ptr - Source starting address of the data block to put.<br> src_stride_arr - Source array of stride distances in bytes.<br> dst_ptr - Destination starting address to put data.<br> dst_stride_arr - Destination array stride distances in bytes.<br> count - Block size in each dimension. count[0] should be the<br> number of bytes of contiguous data in leading dimension.<br> stride_levels - The level of strides.<br> proc - Remote process ID (destination).<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
369
<font face="Arial,Helvetica"><font size="+0" color="#000099">
371
<pre><font face="Arial,Helvetica"><font size="+0"
372
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbAccS</span>(int datatype, void *scale, void* src_ptr,int src_stride_ar[], <br> void* dst_ptr, int dst_stride_ar[], int count[], int stride_levels, <br> int proc, armci_hdl_t* handle)</font></font></pre>
373
<font face="Arial,Helvetica"><font size="+0" color="#000099">
374
<small>PURPOSE: Non-blocking strided
376
that atomicaly updates the memory of a remote process (destination).</small>
378
<pre><font face="Arial,Helvetica"><font size="+0"
379
color="#000099">ARGUMENTS:<br> datatype - Supported data types are:<br> ARMCI_ACC_INT -> int, ARMCI_ACC_LNG -> long,<br> ARMCI_ACC_FLT -> float, ARMCI_ACC_DBL-> double,<br> ARMCI_ACC_CPL -> complex, ARMCI_ACC_DCPL -> double complex.<br> scale - Scale for the data (dest = dest + scale * src).<br> src_ptr - Source starting address of the data block to put.<br> src_stride_arr - Source array of stride distances in bytes.<br> dst_ptr - Destination starting address to put data.<br> dst_stride_arr - Destination array stride distances in bytes.<br> count - Block size in each dimension. count[0] should be the<br> number of bytes of contiguous data in leading dimension.<br> stride_levels - The level of strides.<br> proc - Remote process ID (destination).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).<br></font></font></pre>
380
<font face="Arial,Helvetica"><font size="+0" color="#000099">
382
<hr style="width: 100%; height: 2px;"><font
383
face="Arial,Helvetica"><font size="+0" color="#000099"> </font></font>
384
<h3><font face="Arial,Helvetica"><font size="+0"
385
color="#000099"><a name="acc"></a>7 Accumulate operation using
386
contiguous format<br>
388
<font face="Arial,Helvetica"><font size="+0" color="#000099">
390
<pre><font face="Arial,Helvetica"><font size="+0"
391
color="#000099">int <span style="font-weight: bold;">ARMCI_Acc</span>(int datatype, void *scale, void* src, void* dst, int bytes, int proc)<br>PURPOSE: Blocking operation that atomicaly updates the memory of a remote process (destination).<br><br>ARGUMENTS:<br> datatype - Supported data types are:<br> ARMCI_ACC_INT -> int, ARMCI_ACC_LNG -> long,<br> ARMCI_ACC_FLT -> float, ARMCI_ACC_DBL-> double,<br> ARMCI_ACC_CPL -> complex, ARMCI_ACC_DCPL -> double complex.<br> scale - Scale for the data (dest = dest + scale * src).<br> src - Source starting address of the data to transfer.<br> dst - Destination starting address to add incoming data.<br> bytes - amount of data to transfer in bytes.<br> proc - Remote process ID (destination).<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
392
<font face="Arial,Helvetica"><font size="+0" color="#000099">
394
<pre><font face="Arial,Helvetica"><font size="+0"
395
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbAcc</span>(int datatype, void *scale, void* src, void* dst, int bytes, int proc, <br> armci_hdl_t* handle)<br><br>PURPOSE: Bon-blocking operation that atomicaly updates the memory of a remote process (destination).<br>ARGUMENTS:<br> datatype - Supported data types are:<br> ARMCI_ACC_INT -> int, ARMCI_ACC_LNG -> long,<br> ARMCI_ACC_FLT -> float, ARMCI_ACC_DBL-> double,<br> ARMCI_ACC_CPL -> complex, ARMCI_ACC_DCPL -> double complex.<br> scale - Scale for the data (dest = dest + scale * src).<br> src - Source starting address of the data to transfer.<br> dst - Destination starting address to add incoming data.<br> bytes - amount of data to transfer in bytes.<br> proc - Remote process ID (destination).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).<br></font></font></pre>
396
<font face="Arial,Helvetica"><font size="+0" color="#000099"> _____________________________________________________________________________
398
<h3><font face="Arial,Helvetica"><font size="+0"
399
color="#000099"><a name="reg"></a>8 Register Originated Data Transfer
400
Operations</font></font></h3>
401
<font face="Arial,Helvetica"><font size="+0" color="#000099">
403
<pre><font face="Arial,Helvetica"><font size="+0"
404
color="#000099">int <span style="font-weight: bold;">ARMCI_PutValueXXX</span>(DATATYPE src, void* dst, int proc)</font></font></pre>
405
<font face="Arial,Helvetica"><font size="+0" color="#000099">
406
<small>PURPOSE: Transfer of a value stored in a
407
register of a local
409
to remote process memory (destination). XXX can be
410
"Long"/"Int"/"Double"/"Float"</small> </font></font>
411
<pre><font face="Arial,Helvetica"><font size="+0"
412
color="#000099">ARGUMENTS:<br>DATA TYPE - long, int, float, double according to XXX in the function name<br> src - Value in a register to put.<br> dst - Destination starting address to put data.<br> proc - Remote process ID (destination).<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
413
<font face="Arial,Helvetica"><font size="+0" color="#000099">
415
<pre><font face="Arial,Helvetica"><font size="+0"
416
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_NbPutValueXXX</span>(DATATYPE src, void* dst, int proc, armci_hdl_t* handle)</font></font></pre>
417
<font face="Arial,Helvetica"><font size="+0" color="#000099">
418
<small>PURPOSE: Non-blocking transfer of a value stored in
421
process to remote process memory (destination).
422
XXX can be "Long"/"Int"/"Double"/"Float"</small> </font></font>
423
<pre><font face="Arial,Helvetica"><font size="+0"
424
color="#000099">ARGUMENTS:<br>DATA TYPE: - long, int, float, double according to XXX in the function name<br> src - Value in a register to put.<br> dst - Destination starting address to put data.<br> proc - Remote process ID (destination).<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> Passing of a NULL value for this arg makes this function do an implicit <br> handle non-blocking transfer.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
425
<font face="Arial,Helvetica"><font size="+0" color="#000099">
427
<pre><font face="Arial,Helvetica"><font size="+0"
428
color="#000099"><br>DATATYPE <span style="font-weight: bold;">ARMCI_GetValueXXX</span>(void *src, int proc)</font></font></pre>
429
<font face="Arial,Helvetica"><font size="+0" color="#000099">
430
<small>PURPOSE: Transfer of a value stored in a
431
register of a process
432
(source).
433
XXX can be "Long"/"Int"/"Double"/"Float"</small> </font></font>
434
<pre><font face="Arial,Helvetica"><font size="+0"
435
color="#000099">ARGUMENTS:<br> DATATYPE - long, int, float, double according to XXX in the function name<br> src - Source starting address.<br> proc - Remote process ID (source).<br>RETURN VALUE:<br> the value (of type DATATYPE) is returned.</font></font></pre>
436
<font face="Arial,Helvetica"><font size="+0" color="#000099">_____________________________________________________________________________
438
<h3><font face="Arial,Helvetica"><font size="+0"
439
color="#000099"><a name="compl"></a>9 Completion of Outstanding
440
Operations</font></font></h3>
441
<font face="Arial,Helvetica"><font size="+0" color="#000099">
443
<pre><font face="Arial,Helvetica"><font size="+0"
444
color="#000099">int <span style="font-weight: bold;">ARMCI_Fence</span>(int proc)</font></font></pre>
445
<font face="Arial,Helvetica"><font size="+0" color="#000099">
446
<small>PURPOSE: Blocks the calling process until all
450
issued to the specified remote process complete at the destination.</small>
452
<pre><font face="Arial,Helvetica"><font size="+0"
453
color="#000099">ARGUMENTS:<br> proc - Remote process ID.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
454
<font face="Arial,Helvetica"><font size="+0" color="#000099">
456
<pre><font face="Arial,Helvetica"><font size="+0"
457
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_AllFence</span>()</font></font></pre>
458
<font face="Arial,Helvetica"><font size="+0" color="#000099">
459
<small>PURPOSE: Blocks the calling process until all the
461
put or accumulate<br>
462
operations complete remotely regardless of the destination processor.</small>
464
<p><font face="Arial,Helvetica"><font size="+0"
465
color="#000099"><small>ARGUMENTS: none </small><br>
467
<font face="Arial,Helvetica"><font size="+0" color="#000099">
469
<pre><font face="Arial,Helvetica"><font size="+0"
470
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_Wait</span>(armci_hdl_t* handle)</font></font></pre>
471
<font face="Arial,Helvetica"><font size="+0" color="#000099">
472
<small>PURPOSE: A common function to be used to wait
475
operations with explicit handle.</small><br>
477
<pre><font face="Arial,Helvetica"><font size="+0"
478
color="#000099">ARGUMENTS:<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> A value of NULL for the pointer is erroneous. A value other then NULL would <br> make this routine act as a wait for an explicit non-blocking request with <br> the handle pointed to by reqid.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
479
<font face="Arial,Helvetica"><font size="+0" color="#000099">
481
<pre><font face="Arial,Helvetica"><font size="+0"
482
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_Test</span>(armci_hdl_t* handle)</font></font></pre>
483
<font face="Arial,Helvetica"><font size="+0" color="#000099">PURPOSE:
484
A function to be used as check completion status of
486
ARMCI operations with explicit handle. </font></font>
487
<pre><font face="Arial,Helvetica"><font size="+0"
488
color="#000099">ARGUMENTS:<br> handle - Pointer to a desciptor associated with a particular non-blocking transfer. <br> A value of NULL for the pointer is erroneous. A value other then NULL would <br> make this routine act as a wait for an explicit non-blocking request with <br> the handle pointed to by reqid.<br>RETURN VALUE:<br> zero - Completed<br> 1 - In progress<br> other value - Error code (described in the release notes).</font></font></pre>
489
<font face="Arial,Helvetica"><font size="+0" color="#000099">
491
<pre><font face="Arial,Helvetica"><font size="+0"
492
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_WaitProc</span>(int proc)</font></font></pre>
493
<font face="Arial,Helvetica"><font size="+0" color="#000099">
494
<small>PURPOSE: Wait for all outstanding non-blocking
495
operations with implicit
496
handles to a particular process to finish.</small> </font></font>
497
<pre><font face="Arial,Helvetica"><font size="+0"
498
color="#000099">ARGUMENTS:<br> proc - proc for which all the outstanding non-blocking operations have to be completed.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
499
<font face="Arial,Helvetica"><font size="+0" color="#000099">
501
<pre><font face="Arial,Helvetica"><font size="+0"
502
color="#000099"><br>int<span style="font-weight: bold;"> ARMCI_WaitAll</span>()</font></font></pre>
503
<font face="Arial,Helvetica"><font size="+0" color="#000099">
504
<small>PURPOSE: Wait for all outstanding non-blocking
505
operations with implicit
506
handles to finish.</small> </font></font>
507
<pre><font face="Arial,Helvetica"><font size="+0"
508
color="#000099">ARGUMENTS: none<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
509
<font face="Arial,Helvetica"><font size="+0" color="#000099">
511
<pre><font face="Arial,Helvetica"><font size="+0"
512
color="#000099"><br>int<span style="font-weight: bold;"> ARMCI_Barrier</span>()<br><br>PURPOSE: Synchronize processors and memory. This operation combines functionality of <br> MPI_Barrier and ARMCI_AllFence.<br>ARGUMENTS: none<br>RETURN VALUE: none</font></font></pre>
513
<font face="Arial,Helvetica"><font size="+0" color="#000099">_____________________________________________________________________________
515
<h3><font face="Arial,Helvetica"><font size="+0"
516
color="#000099"><a name="aggr"></a>10 Aggregated Data Transfer
517
Operations</font></font></h3>
518
<font face="Arial,Helvetica"><font size="+0" color="#000099">
519
<span style="font-weight: bold;">ARMCI_SET_AGGREGATE_HANDLE</span>
521
handle) </font></font>
522
<pre><font face="Arial,Helvetica"><font size="+0"
523
color="#000099">handle - Pointer to a desciptor associated with a particular non-blocking transfer.</font></font></pre>
524
<font face="Arial,Helvetica"><font size="+0" color="#000099">
525
<small>PURPOSE: Mark a handle as aggregate. This will allow
528
operations that use that particular handle and process them as a single
529
operation. In the initial implementation only contiguous puts or gets
531
use aggregate handle. Specifying the same handle for a mix of put anmd
532
get calls is not allowed i.e., only multiple put or only multiple get
534
can use the same handle.</small><br>
537
<p><span style="font-weight: bold;"><font
538
face="Arial,Helvetica"><font size="+0" color="#000099">ARMCI_UNSET_AGGREGATE_HANDLE</font></font></span>
539
<font face="Arial,Helvetica"><font size="+0" color="#000099">(armci_hdl_t*
540
handle) </font></font></p>
541
<font face="Arial,Helvetica"><font size="+0" color="#000099">
543
<pre><font face="Arial,Helvetica"><font size="+0"
544
color="#000099">handle - Pointer to a desciptor associated with a particular non-blocking transfer.</font></font></pre>
545
<font face="Arial,Helvetica"><font size="+0" color="#000099">
546
<small>PURPOSE: Clears a handle that has been marked as
547
aggregate</small>. <br>
549
<hr style="width: 100%; height: 2px;"><font
550
face="Arial,Helvetica"><font size="+0" color="#000099"> </font></font>
551
<h3><span style="font-weight: bold;"><font
552
face="Arial,Helvetica"><font size="+0" color="#000099"><a name="atomic"></a>11
556
</font></font></span></h3>
557
<font face="Arial,Helvetica"><font size="+0" color="#000099">
559
<pre><font face="Arial,Helvetica"><font size="+0"
560
color="#000099">int <span style="font-weight: bold;">ARMCI_Rmw</span>(int op, void *ploc, void *prem, int value, proc)</font></font></pre>
561
<font face="Arial,Helvetica"><font size="+0" color="#000099">
562
<small>PURPOSE: Combines atomically the specified
563
integer value with the
564
corresponding integer value (int or long) at the remote memory location
565
and returns the original value found at that location.</small> </font></font>
566
<pre><font face="Arial,Helvetica"><font size="+0"
567
color="#000099">ARGUMENTS: <br> op - Available operations are:<br> ARMCI_FETCH_AND_ADD -> int<br> ARMCI_FETCH_AND_ADD_LONG -> long<br> ARMCI_SWAP -> int<br> ARMCI_SWAP_LONG ->long<br> ploc - Pointer to the local memory.<br> prem - Pointer to the remote memory.<br> value - Value to be added to the remote memory.<br> proc - Remote process ID.</font></font></pre>
568
<font face="Arial,Helvetica"><font size="+0" color="#000099">
569
<span style="font-weight: bold;"><br>
570
</span> </font></font>
571
<pre><font face="Arial,Helvetica"><font size="+0"
572
color="#000099">int <span style="font-weight: bold;">ARMCI_Create_mutexes</span>(int count)</font></font></pre>
573
<font face="Arial,Helvetica"><font size="+0" color="#000099">
574
<small>PURPOSE: Collective operation to create sets of
575
mutexes on individual
577
Each process specifies the number of mutexes associated with that <br>
578
process. The total number of mutexes allocate will be a sum of the <br>
579
values specified on each process.</small> </font></font>
580
<pre><font face="Arial,Helvetica"><font size="+0"
581
color="#000099">ARGUMENTS:<br> count - number of mutexes allocated on calling process<br> count=0 means that no mutexes will be associated with that process.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
582
<font face="Arial,Helvetica"><font size="+0" color="#000099">
584
<pre><font face="Arial,Helvetica"><font size="+0"
585
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_Destroy_mutexes</span>(void)</font></font></pre>
586
<font face="Arial,Helvetica"><font size="+0" color="#000099">
587
<small>PURPOSE: Collective operation to destroy mutex sets
589
ARMCI_Create_mutexes.</small> </font></font>
590
<pre><font face="Arial,Helvetica"><font size="+0"
591
color="#000099">ARGUMENTS: none<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).</font></font></pre>
592
<font face="Arial,Helvetica"><font size="+0" color="#000099">
594
<pre><font face="Arial,Helvetica"><font size="+0"
595
color="#000099"><br>void <span style="font-weight: bold;">ARMCI_Lock</span>(int mutex, int proc)</font></font></pre>
596
<font face="Arial,Helvetica"><font size="+0" color="#000099">
597
<small>PURPOSE: Acquire the specified mutex on the
600
of the calling process.</small><br>
602
<pre><font face="Arial,Helvetica"><font size="+0"
603
color="#000099">ARGUMENTS:<br> mutex - Mutex number (0..count-1)<br> proc - Remote process ID</font></font></pre>
604
<font face="Arial,Helvetica"><font size="+0" color="#000099">
606
<pre><font face="Arial,Helvetica"><font size="+0"
607
color="#000099"><br>void <span style="font-weight: bold;">ARMCI_Unlock</span>(int mutex, int proc)</font></font></pre>
608
<font face="Arial,Helvetica"><font size="+0" color="#000099">
609
<small>PURPOSE: Releas the specified mutex on the specified
611
of the calling process. The mutex must have been acquired with
612
ARMCI_Lock.</small> </font></font>
613
<pre><span style="font-family: arial,helvetica;"><font
614
face="Arial,Helvetica"><font size="+0" color="#000099">ARGUMENTS:</font></font></span><font
615
face="Arial,Helvetica"><font size="+0" color="#000099"><br> mutex - Mutex number (0..count-1)<br> proc - Remote process ID<br></font></font></pre>
616
<font face="Arial,Helvetica"><font size="+0" color="#000099">
618
<hr style="width: 100%; height: 2px;"><font
619
face="Arial,Helvetica"><font size="+0" color="#000099"> </font></font>
620
<h3><font face="Arial,Helvetica"><font size="+0"
621
color="#000099"><a name="mem"></a>12 Memory Management<br>
623
<font face="Arial,Helvetica"><font size="+0" color="#000099">
625
<pre><font face="Arial,Helvetica"><font size="+0"
626
color="#000099">int <span style="font-weight: bold;">ARMCI_Malloc</span>(void* ptr[], armci_size_t bytes)</font></font></pre>
627
<font face="Arial,Helvetica"><font size="+0" color="#000099">
628
<small>PURPOSE: Collective operation to allocate memory
631
of ARMCI copy operations.</small><br>
633
<pre><font face="Arial,Helvetica"><font size="+0"
634
color="#000099">ARGUMENTS:<br> ptr - Pointer array. Each pointer points to the allocated memory of one process.<br>bytes - The size of allocated memory in bytes.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).<br><br></font></font></pre>
635
<font face="Arial,Helvetica"><font size="+0" color="#000099">
637
<pre><font face="Arial,Helvetica"><font size="+0"
638
color="#000099">void* <span style="font-weight: bold;">ARMCI_Malloc_local</span>(armci_size_t bytes)</font></font></pre>
639
<font face="Arial,Helvetica"><font size="+0" color="#000099">
640
<small>PURPOSE: Operation (noncollective) to allocate local
642
memory can only be accessed locally. However, using this memory in
643
ARMCI operations can improve performance on some systems. For example,
644
on Myrinet or Infiniband, the memory is registered and therefore
645
suitable for the native RDMA communication.</small><br>
647
<pre><font face="Arial,Helvetica"><font size="+0"
648
color="#000099">ARGUMENTS:<br>bytes - The size of allocated memory in bytes.<br>RETURN VALUE:<br> NULL pointer - Failure.<br> other value - address of newly allocated memory.<br><br></font></font></pre>
649
<font face="Arial,Helvetica"><font size="+0" color="#000099">
651
<pre><font face="Arial,Helvetica"><font size="+0"
652
color="#000099">int <span style="font-weight: bold;">ARMCI_Free</span>(void *address)</font></font></pre>
653
<font face="Arial,Helvetica"><font size="+0" color="#000099">
654
<small>PURPOSE: Collective operation to free memory which
656
ARMCI_Malloc.</small> </font></font>
657
<pre><font face="Arial,Helvetica"><font size="+0"
658
color="#000099">ARGUMENTS:<br> address - pointer to the allocated memory.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).<br></font></font></pre>
659
<font face="Arial,Helvetica"><font size="+0" color="#000099">
661
<pre><font face="Arial,Helvetica"><font size="+0"
662
color="#000099"><br>int <span style="font-weight: bold;">ARMCI_Free_local</span>(void *address)</font></font></pre>
663
<font face="Arial,Helvetica"><font size="+0" color="#000099">
664
<small>PURPOSE: Non-collective operation to free memory
665
which was allocated by
666
ARMCI_Malloc_local.</small> </font></font>
667
<pre><font face="Arial,Helvetica"><font size="+0"
668
color="#000099">ARGUMENTS:<br> address - pointer to the allocated memory.<br>RETURN VALUE:<br> zero - Successful.<br> other value - Error code (described in the release notes).<br></font></font></pre>
669
<font face="Arial,Helvetica"><font size="+0" color="#000099">
671
<hr style="width: 100%; height: 2px;"><font
672
face="Arial,Helvetica"><font size="+0" color="#000099"> </font></font>
673
<h3><font face="Arial,Helvetica"><font size="+0"
674
color="#000099"><a name="collect"></a>13 Collective Operations</font></font></h3>
675
<font face="Arial,Helvetica"><font size="+0" color="#000099">These
676
operations can be used as an alternative to the collective
677
operations in a message-passing library ARMCI is running with. The
678
programmer can use either. </font></font>
679
<pre><font face="Arial,Helvetica"><font size="+0"
680
color="#000099"><br>void <span style="font-weight: bold;">armci_msg_brdcst</span>(void* buffer, int len, int root)</font></font></pre>
681
<font face="Arial,Helvetica"><font size="+0" color="#000099">
683
<pre><font face="Arial,Helvetica"><font size="+0"
684
color="#000099">PURPOSE: broadcast data from process "root" to everybody else.<br>ARGUMENTS:<br> buffer - data to broadcast/receive<br> len - size of the data<br> root - rank of the sending process<br>RETURN VALUE: none</font></font></pre>
685
<font face="Arial,Helvetica"><font size="+0" color="#000099">
687
<pre><font face="Arial,Helvetica"><font size="+0"
688
color="#000099"><br>void <span style="font-weight: bold;">armci_msg_gop2</span>(void *x, int n, int type, char* op)</font></font></pre>
689
<font face="Arial,Helvetica"><font size="+0" color="#000099">
691
<pre><font face="Arial,Helvetica"><font size="+0"
692
color="#000099">PURPOSE: allreduce operation for int, long, float, double corresponding to XXX is "i"/"l","f","d" <br>ARGUMENTS:<br> x - data <br> n - number of elements<br> type - data type, one of: ARMCI_INT/ARMCI_LONG/ARMCI_FLOAT/ARMCI_DOUBLE<br> op - operator, one of: "+","-","*","min","max","abs"<br>RETURN VALUE: none</font></font></pre>
693
<font face="Arial,Helvetica"><font size="+0" color="#000099">
695
<pre><font face="Arial,Helvetica"><font size="+0"
696
color="#000099"><br>void <span style="font-weight: bold;">armci_msg_barrier</span>(void)<br>PURPOSE: synchronize all processors<br>ARGUMENTS: none<br>RETURN VALUE: none</font></font></pre>
697
<font face="Arial,Helvetica"><font size="+0" color="#000099">
699
<pre><font face="Arial,Helvetica"><font size="+0"
700
color="#000099"><br>void <span style="font-weight: bold;">armci_msg_reduce</span>(void *x, int n, char* op, int type, int root)<br>PURPOSE: reduce operation<br>ARGUMENTS:<br> x - data <br> len - size of the data<br> type - data type, one of: ARMCI_INT/ARMCI_LONG/ARMCI_FLOAT/ARMCI_DOUBLE<br> op - operator, one of: "+","-","*","min","max","abs"<br>RETURN VALUE: none<br></font></font></pre>
701
<font face="Arial,Helvetica"><font size="+0" color="#000099">
703
<hr style="width: 100%; height: 2px;">
704
<h3><font face="Arial,Helvetica"><font size="+0"
706
<h3><a name="config"></a>14 System Configuration</h3>
708
<font face="Arial,Helvetica"><font size="+0" color="#000099">These
709
operations can be used to determine configuration of the system the
710
application is running on.<br>
711
The system configuration is described in terms of locality domains. For
712
example on clusters with SMP nodes, SMP node is one of two locality
713
domains for a process. The ARMCI header file predefines
714
ARMCI_DOMAIN_SMP for querying configuration information on clusters
715
composed of computer nodes with shared memory. <br>
716
</font></font><font face="Arial,Helvetica"><font size="+0"
717
color="#000099"> </font></font>
718
<pre><font face="Arial,Helvetica"><font size="+0"
719
color="#000099"><br>int <span style="font-weight: bold;">armci_domain_nprocs</span>(armci_domain_t domain, int id)</font></font></pre>
720
<font face="Arial,Helvetica"><font size="+0" color="#000099">
722
<pre><font face="Arial,Helvetica"><font size="+0"
723
color="#000099">PURPOSE: return number of processes/tasks in locality domain represented by id. <br>ARGUMENTS:<br> domain - domain name <br> id - identifier of a node within the locality domain, value < 0 means my node<br>RETURN VALUE:<br> < 0 - error<br> other value - number of processes/tasks (0, ..., armci_domain_count(domain)-1)<br><br></font></font><pre
724
style="font-family: monospace;"><font face="Arial,Helvetica"><font
725
size="+0" color="#000099"><font size="+0" color="#000099">int <span
726
style="font-weight: bold;">armci_domain_count</span>(armci_domain_t domain)<br></font></font></font><pre><font
727
face="Arial,Helvetica"><font size="+0" color="#000099"><font size="+0"
728
color="#000099"><font><font><font face="Arial,Helvetica"><font
729
size="+0" color="#000099">PURPOSE: return number of nodes in specified locality domain. <br>ARGUMENTS:<br> domain - domain name <br>RETURN VALUE:<br> < 0 - error<br> other value - number of nodes<br><br></font></font></font></font></font></font></font></pre><pre><font
730
face="Arial,Helvetica"><font size="+0" color="#000099"><font size="+0"
731
color="#000099"><font><font><font face="Arial,Helvetica"><font
732
size="+0" color="#000099">int <span style="font-weight: bold;">armci_domain_id</span>(armci_domain_t domain, int glob_proc_id)<br><br>PURPOSE: return ID of locality domain of specified process<br>ARGUMENTS:<br> domain - domain name <br> id - process/task id<br>RETURN VALUE<br> < 0 - error<br> other value - process/task i<span
733
style="font-family: monospace;">d<br><br></span></font></font></font></font></font></font></font></pre><span
734
style="font-family: arial; color: rgb(0, 0, 153);">int </span><span
735
style="font-weight: bold; font-family: arial; color: rgb(0, 0, 153);">armci_domain_glob_proc_id</span><span
736
style="font-family: arial; color: rgb(0, 0, 153);">(armci_domain_t domain, int id, int loc_proc_id)</span></pre></pre>
737
<font face="Arial,Helvetica" style="font-family: arial;"
738
size="-1"><font color="#000099"><font color="#000099"><font><font><font
739
color="#000099"><font><font><font color="#000099">PURPOSE: Returns
740
global process/task id based on its id in a given locality domain node </font></font></font></font></font></font></font></font></font><br
741
style="font-family: arial;">
742
<font size="-1" style="font-family: arial;"> </font><font
743
face="Arial,Helvetica" style="font-family: arial;" size="-1"><font
744
color="#000099"><font color="#000099"><font><font><font color="#000099"><font><font><font
745
color="#000099">ARGUMENTS:</font></font></font></font></font></font></font></font></font><br
746
style="font-family: arial;">
747
<font size="-1" style="font-family: arial;"> </font><font
748
face="Arial,Helvetica" style="font-family: arial;" size="-1"><font
749
color="#000099"><font color="#000099"><font><font><font color="#000099"><font><font><font
750
color="#000099"> domain - domain name</font></font></font></font></font></font></font></font></font><br
751
style="font-family: arial;">
752
<font size="-1" style="font-family: arial;"> </font><font
753
face="Arial,Helvetica" style="font-family: arial;" size="-1"><font
754
color="#000099"><font color="#000099"><font><font><font color="#000099"><font><font><font
755
color="#000099"> id - identifier of a node within the locality
756
domain, value < 0 means my node</font></font></font></font></font></font></font></font></font><br
757
style="font-family: arial;">
758
<font size="-1" style="font-family: arial;"> </font><font
759
face="Arial,Helvetica" style="font-family: arial;" size="-1"><font
760
color="#000099"><font color="#000099"><font><font><font color="#000099"><font><font><font
761
color="#000099">RETURN VALUE:</font></font></font></font></font></font></font></font></font><br
762
style="font-family: arial;">
763
<font size="-1" style="font-family: arial;"> </font><font
764
face="Arial,Helvetica" style="font-family: arial;" size="-1"><font
765
color="#000099"><font color="#000099"><font><font><font color="#000099"><font><font><font
766
color="#000099"> < 0 - error</font></font></font></font></font></font></font></font></font><br
767
style="font-family: arial;">
768
<font size="-1" style="font-family: arial;"> </font><font
769
face="Arial,Helvetica" style="font-family: arial;" size="-1"><font
770
color="#000099"><font color="#000099"><font><font><font color="#000099"><font><font><font
771
color="#000099"> other value - process/task id</font></font></font></font></font></font></font></font></font>
772
<pre style="font-family: arial; color: rgb(0, 0, 153);"><pre
773
style="font-family: arial;"><font size="-1">int <span
774
style="font-weight: bold;">armci_domain_my_id</span>(armci_domain_t domain)</font></pre><font
775
size="-1">PURPOSE: Returns id node in specified domain the calling process/task belongs to <br>ARGUMENTS:<br> domain - domain name<br> id - identifier of a node within the locality domain, value < 0 means my node<br>RETURN VALUE: id of domain</font></pre>
776
<font face="Arial,Helvetica"><font size="+0" color="#000099">
777
</font></font><font face="Arial,Helvetica"><font size="+0"
779
<hr style="width: 100%; height: 2px;"> </font></font><!-- ----------- TEXT ENDS HERE ------------- -->
added
removed
src/tools/ga-5-1/doc/armci/www/armci_doc.htm
