1
NetPIPE Network Protocol Independent Performance Evaluator, Release 2.4
2
Copyright 1997, 1998, 1999 Iowa State University Research Foundation, Inc.
4
$Id: README,v 1.13 1999/12/16 14:21:48 ghelmer Exp $
6
This program is free software; you can redistribute it and/or modify
7
it under the terms of the GNU General Public License as published by
8
the Free Software Foundation. You should have received a copy of the
9
GNU General Public License along with this program; if not, write to the
10
Free Software Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
12
The URL for this document:
14
ftp://ftp.scl.ameslab.gov/pub/netpipe/README
19
The NetPIPE implementation in C can be found at:
21
ftp://ftp.scl.ameslab.gov/pub/netpipe/netpipe-2.4.tar.gz
23
The source code for NetPIPE 2.4 is provided as a gzipped tar archive,
24
which can be uncompressed with "gunzip netpipe-2.4.tar.gz" (or "gzip
25
-d netpipe-2.4.tar.gz"), and then extracted from the uncompressed
26
archive with the command "tar xvf netpipe-2.4.tar". If you do not
27
have the gzip program, it can be obtained as:
29
ftp://prep.ai.mit.edu/pub/gnu/gzip-1.2.4.tar
31
Similarly, the NetPIPE implementation in Java can be found at:
33
ftp://ftp.scl.ameslab.gov/pub/netpipe-Java-1.0.tar.gz
35
The instructions that follow apply to the C implementation of
41
NetPIPE is a protocol independent performance tool that encapsulates
42
the best of ttcp and netperf and visually represents the network
43
performance under a variety of conditions. By taking the end-to-end
44
application view of a network, NetPIPE clearly shows the overhead
45
associated with different protocol layers. Netpipe answers such
46
questions as: how soon will a given data block of size k arrive at its
47
destination? Which network and protocol will transmit size k blocks
48
the fastest? What is a given network's effective maximum throughput
49
and saturation level? Does there exist a block size k for which the
50
throughput is maximized? How much communication overhead is due to the
51
network communication protocol layer(s)? How quickly will a small (< 1
52
kbyte) control message arrive, and which network and protocol are best
55
For a paper fully describing NetPIPE and sample investigation of
56
network performance issues using NetPIPE, see
57
http://www.scl.ameslab.gov/netpipe/paper/full.html.
62
NetPIPE is provided with protocol-specific shims for TCP (using the
63
Berkeley sockets interface), MPI, and PVM. If you do not have MPI or
64
PVM, don't worry; TCP is the typical shim used. It should be easy to
65
write new interfaces for other protocols based on the examples shown
66
by the TCP, MPI and PVM interfaces.
68
NetPIPE requires an ANSI C compiler.
70
Review the provided Makefile and change any necessary settings, such
71
as the CFLAGS compiler flags, required extra libraries, and MPI or PVM
72
library & include file pathnames if you have these communication
73
libraries. If you want to turn on getrusage calls to get CPU time
74
required for communication, add "-DHAVE_GETRUSAGE" to the CFLAGS line
77
Compile NetPIPE with the desired communication interface by using the
78
command "make TCP", "make MPI", or "make PVM" as appropriate,
79
corresponding to the executable files NPtcp, NPmpi, or NPpvm
82
Consult the appropriate section below for details on running NetPIPE
83
over TCP, MPI, or PVM, and the following section on interpreting the
89
For TCP, run a NetPIPE receiver on one computer by issuing the command
90
"NPtcp -r". Run a NetPIPE sender on another computer by issuing the
91
command "NPtcp -t -h <receiver's address> -o <output file> -P" and any
92
other options as appropriate (each option affects only the process on
93
which it is specified -- options are not negotiated between the
94
transmitter and the receiver):
96
-A: specify buffers alignment e.g. "-A 4096"
98
-a: asynchronous receive (a.k.a. preposted receive)
99
This option currently has no effect on TCP
101
-b: specify send and receive TCP buffer sizes e.g. "-b 32768"
103
-h: specify hostname of receiver e.g. "-h mumblehost"
105
-i: specify increment step size e.g. "-i 64"
106
Default is exponential increment calculated at runtime
108
-l: lower bound (start value for block size) e.g. "-l 1"
110
-O: specify buffer offset e.g. "-O 127"
112
-o: specify output filename e.g. "-o output.txt"
114
-P: print real-time results on stdout
116
-p: specify port e.g. "-p 5150"
118
-s: stream option (default mode is "ping pong")
119
If this option is used, it must be specified on both
120
the sending and receiving processes
122
-u: upper bound (stop value for block size) e.g. "-u 1048576"
127
For MPI, how you run NPmpi may depend on the MPI implementation you
128
are using. Assuming you are using the "p4" device (for a cluster of
129
individual systems interconnected using TCP/IP) in the Argonne MPICH
130
implementation, you could run NPmpi one of two ways.
132
If your system's default machine file begins with the two
133
names of the systems you want to test, use "mpirun -np 2
134
NPmpi", followed by any of the NetPIPE options listed below.
136
Otherwise, create a file that contains the host names of the
137
two systems you want to include in the test, one host name on
138
each line (assume the file is named "machines.p4"). Then, use
139
the command "mpirun -machinefile machines.p4 -np 2 NPmpi",
140
followed by any of the NetPIPE options listed below.
142
To find out how to run NPmpi using any other implementation of MPI,
143
please consult the implementation's documentation.
145
The NetPIPE options for MPI are:
147
-A: specify buffers alignment e.g. "-A 4096"
149
-a: asynchronous receive (a.k.a. preposted receive)
150
May not have any effect, depending on your MPI
153
-i: specify increment step size e.g. "-i 64"
154
Default is exponential increment calculated at runtime
156
-l: lower bound (start value for block size) e.g. "-l 1"
158
-O: specify buffer offset e.g. "-O 127"
160
-o: specify output filename e.g. "-o output.txt"
162
-P: print real-time results on stdout
164
-s: stream option (default mode is "ping pong")
165
If this option is used, it must be specified on both
166
the sending and receiving processes
168
-u: upper bound (stop value for block size) e.g. "-u 1048576"
173
First, start PVM with the command "pvm" on one machine and a second
174
machine with the PVM command "add <othermachine>", where
175
<othermachine> is the name of the other computer to include in the
176
test. Exit the PVM command line interface. Start the receiver
177
process on one of the machines with the command "NPpvm -r". Finally,
178
start the transmitter process on the other machine with the command
179
"NPpvm -t -o <output file> -P" and any other options as appropriate
180
(each option affects only the process on which it is specified --
181
options are not negotiated between the transmitter and the receiver):
183
-A: specify buffers alignment e.g. "-A 4096"
185
-a: asynchronous receive (a.k.a. preposted receive)
186
This option has no effect on PVM
188
-i: specify increment step size e.g. "-i 64"
189
Default is exponential increment calculated at runtime
191
-l: lower bound (start value for block size) e.g. "-l 1"
193
-O: specify buffer offset e.g. "-O 127"
195
-o: specify output filename e.g. "-o output.txt"
197
-P: print real-time results on stdout
199
-s: stream option (default mode is "ping pong")
200
If this option is used, it must be specified on both
201
the sending and receiving processes
203
-u: upper bound (stop value for block size) e.g. "-u 1048576"
206
Interpreting the Results
207
------------------------
209
NetPIPE's output file contains five columns: time to transfer the block,
210
bits per second, bits in block, bytes in block, and variance. These
211
columns may be graphed to represent and compare the network's
212
performance. For example, the "network signature" graph can be
213
created by graphing the time column versus the bits per second column
214
(see the NetPIPE report at the URL above for the details why this
215
graph is important and how to interpret it). The more traditional
216
"throughput versus block size" graph can be created by
217
graphing the bytes column versus the bits per second column.
219
See http://www.scl.ameslab.gov/Projects/ClusterCookbook/nprun.html for
220
a detailed tutorial on running NetPIPE and graphing the results.
225
NetPIPE is currently maintained by Guy Helmer. Email
226
"ghelmer@scl.ameslab.gov" or call 515-294-9469 for help or
232
version 2.4 (12/16/99)
233
* Add getrusage calls to get CPU time used by communication if
234
HAVE_GETRUSAGE is defined (be aware that no studies have been
235
conducted to test the accuracy of results across different systems)
236
* Use "unsigned int" instead of "unsigned long" to communicate 32-bit
237
integers in TCP.c (this solves interoperability problems between
238
Compaq/DEC Alphas and most other systems)
239
* Add dummy "echo" commands after TCP, MPI, and PVM targets in the
240
Makefile. Some implementations of make(1) (such as those found
241
in Linux distributions) interpret the targets with no following
242
statements as a rule to do something silly like
243
"cc -O -o TCP TCP.c" after the dependency is satisfied.
245
version 2.3 (9/24/98)
246
* Add PVM interface contributed by Clark E. Dorman <dorman@s3i.com>
248
* Revamp README file with instructions for NPmpi and NPpvm, and
249
clarify some instructions for NPtcp
251
version 2.2 (8/21/98):
252
* Carefully check all return values from write(2) and read(2)
253
system calls in TCP.c. Handle short reads properly. Make the Sync()
254
function transmit and receive a useful string which can be
255
checked for validity.
257
* Correct the overloading of SendTime() and RecvTime() functions
258
by breaking out SendRepeat() and RecvRepeat() as separate
261
* Handle systems whose accept(2) system call does not carry socket
262
options over from the listening socket. In particular, set the
263
TCP_NODELAY flag and socket buffers on an accepted socket.