« back to all changes in this revision
Viewing changes to doc/gawktexi.in
- browse files at revision 502
- compare with another revision
- compare with revision 28
- download diff
- download tarball
- view history from revision 502
- Committer: Juergen Kahrs
- Date: 2014-06-24 11:18:33 UTC
- mfrom: (408.2.336)
- Revision ID: git-v1:f1245d04a9f076773c60499b468f44ed9c91b59b
Merge remote-tracking branch 'origin/master' into cmake
- files added:
- README_d/README.gcc-3
- vms/vax
- files removed:
- awklib/eg/data/BBS-list
- files modified:
- .gitignore
added
removed
doc/gawktexi.in
14
14
* awk: (gawk)Invoking gawk. Text scanning and processing.
15
15
@end direntry
16
16
17
@ifset FOR_PRINT
18
@tex
19
\gdef\xrefprintnodename#1{``#1''}
20
@end tex
21
@end ifset
22
23
@ifclear FOR_PRINT
24
@c With early 2014 texinfo.tex, restore PDF links and colors
25
@tex
26
\gdef\linkcolor{0.5 0.09 0.12} % Dark Red
27
\gdef\urlcolor{0.5 0.09 0.12} % Also
28
\global\urefurlonlylinktrue
29
@end tex
30
@end ifclear
31
32
@ifnotdocbook
33
@set BULLET @bullet{}
34
@set MINUS @minus{}
35
@end ifnotdocbook
36
37
@ifdocbook
38
@set BULLET
39
@set MINUS
40
@end ifdocbook
41
17
42
@set xref-automatic-section-title
18
43
19
44
@c The following information should be updated here only!
21
46
@c applies to and all the info about who's publishing this edition
22
47
23
48
@c These apply across the board.
24
@set UPDATE-MONTH May, 2013
49
@set UPDATE-MONTH June, 2014
25
50
@set VERSION 4.1
26
@set PATCHLEVEL 0
27
28
@set FSF
51
@set PATCHLEVEL 1
29
52
30
53
@set TITLE GAWK: Effective AWK Programming
31
54
@set SUBTITLE A User's Guide for GNU Awk
39
62
@set SUBSECTION subsection
40
63
@set DARKCORNER @inmargin{@image{lflashlight,1cm}, @image{rflashlight,1cm}}
41
64
@set COMMONEXT (c.e.)
65
@set PAGE page
42
66
@end iftex
43
67
@ifinfo
44
68
@set DOCUMENT Info file
48
72
@set SUBSECTION node
49
73
@set DARKCORNER (d.c.)
50
74
@set COMMONEXT (c.e.)
75
@set PAGE screen
51
76
@end ifinfo
52
77
@ifhtml
53
78
@set DOCUMENT Web page
57
82
@set SUBSECTION subsection
58
83
@set DARKCORNER (d.c.)
59
84
@set COMMONEXT (c.e.)
85
@set PAGE screen
60
86
@end ifhtml
61
87
@ifdocbook
62
88
@set DOCUMENT book
66
92
@set SUBSECTION subsection
67
93
@set DARKCORNER (d.c.)
68
94
@set COMMONEXT (c.e.)
95
@set PAGE page
69
96
@end ifdocbook
70
97
@ifxml
71
98
@set DOCUMENT book
75
102
@set SUBSECTION subsection
76
103
@set DARKCORNER (d.c.)
77
104
@set COMMONEXT (c.e.)
105
@set PAGE page
78
106
@end ifxml
79
107
@ifplaintext
80
108
@set DOCUMENT book
84
112
@set SUBSECTION subsection
85
113
@set DARKCORNER (d.c.)
86
114
@set COMMONEXT (c.e.)
115
@set PAGE page
87
116
@end ifplaintext
88
117
118
@ifdocbook
119
@c empty on purpose
120
@set PART1
121
@set PART2
122
@set PART3
123
@set PART4
124
@end ifdocbook
125
126
@ifnotdocbook
127
@set PART1 Part I:@*
128
@set PART2 Part II:@*
129
@set PART3 Part III:@*
130
@set PART4 Part IV:@*
131
@end ifnotdocbook
132
89
133
@c some special symbols
90
134
@iftex
91
135
@set LEQ @math{@leq}
92
136
@set PI @math{@pi}
93
137
@end iftex
138
@ifdocbook
139
@set LEQ @inlineraw{docbook, ≤}
140
@set PI @inlineraw{docbook, &pgr;}
141
@end ifdocbook
94
142
@ifnottex
143
@ifnotdocbook
95
144
@set LEQ <=
96
145
@set PI @i{pi}
146
@end ifnotdocbook
97
147
@end ifnottex
98
148
99
149
@ifnottex
150
@ifnotdocbook
100
151
@macro ii{text}
101
152
@i{\text\}
102
153
@end macro
154
@end ifnotdocbook
103
155
@end ifnottex
104
156
157
@ifdocbook
158
@macro ii{text}
159
@inlineraw{docbook,<lineannotation>\text\</lineannotation>}
160
@end macro
161
@end ifdocbook
162
163
@ifclear FOR_PRINT
164
@set FN file name
165
@set FFN File Name
166
@set DF data file
167
@set DDF Data File
168
@set PVERSION version
169
@end ifclear
170
@ifset FOR_PRINT
171
@set FN filename
172
@set FFN Filename
173
@set DF datafile
174
@set DDF Datafile
175
@set PVERSION Version
176
@end ifset
177
105
178
@c For HTML, spell out email addresses, to avoid problems with
106
179
@c address harvesters for spammers.
107
180
@ifhtml
115
188
@end macro
116
189
@end ifnothtml
117
190
191
@c Indexing macros
192
@ifinfo
193
194
@macro cindexawkfunc{name}
195
@cindex @code{\name\}
196
@end macro
197
198
@macro cindexgawkfunc{name}
199
@cindex @code{\name\}
200
@end macro
201
202
@end ifinfo
203
204
@ifnotinfo
205
206
@macro cindexawkfunc{name}
207
@cindex @code{\name\()} function
208
@end macro
209
210
@macro cindexgawkfunc{name}
211
@cindex @code{\name\()} function (@command{gawk})
212
@end macro
213
@end ifnotinfo
214
118
215
@ignore
119
216
Some comments on the layout for TeX.
120
1. Use at least texinfo.tex 2000-09-06.09
121
2. I have done A LOT of work to make this look good. There are `@page' commands
122
and use of `@group ... @end group' in a number of places. If you muck
123
with anything, it's your responsibility not to break the layout.
217
1. Use at least texinfo.tex 2014-01-30.15
218
2. When using @docbook, if the last line is part of a paragraph, end
219
it with a space and @c so that the lines won't run together. This is a
220
quirk of the language / makeinfo, and isn't going to change.
124
221
@end ignore
125
222
126
223
@c merge the function and variable indexes into the concept index
136
233
@syncodeindex fn cp
137
234
@syncodeindex vr cp
138
235
@end ifxml
236
@ifdocbook
237
@synindex fn cp
238
@synindex vr cp
239
@end ifdocbook
139
240
140
241
@c If "finalout" is commented out, the printed output will show
141
242
@c black boxes that mark lines that are too long. Thus, it is
147
248
@end iftex
148
249
149
250
@copying
150
Copyright @copyright{} 1989, 1991, 1992, 1993, 1996, 1997, 1998, 1999,
151
2000, 2001, 2002, 2003, 2004, 2005, 2007, 2009, 2010, 2011, 2012, 2013
152
Free Software Foundation, Inc.
251
@docbook
252
<para>
253
“To boldly go where no man has gone before” is a
254
Registered Trademark of Paramount Pictures Corporation.</para>
255
256
<para>Published by:</para>
257
258
<literallayout class="normal">Free Software Foundation
259
51 Franklin Street, Fifth Floor
260
Boston, MA 02110-1301 USA
261
Phone: +1-617-542-5942
262
Fax: +1-617-542-2652
263
Email: <email>gnu@@gnu.org</email>
264
URL: <ulink url="http://www.gnu.org">http://www.gnu.org/</ulink></literallayout>
265
266
<literallayout class="normal">Copyright © 1989, 1991, 1992, 1993, 1996–2005, 2007, 2009–2014
267
Free Software Foundation, Inc.
268
All Rights Reserved.</literallayout>
269
@end docbook
270
271
@ifnotdocbook
272
Copyright @copyright{} 1989, 1991, 1992, 1993, 1996--2005, 2007, 2009--2014 @*
273
Free Software Foundation, Inc.
274
@end ifnotdocbook
153
275
@sp 2
154
276
155
277
This is Edition @value{EDITION} of @cite{@value{TITLE}: @value{SUBTITLE}},
197
319
@subtitle @value{UPDATE-MONTH}
198
320
@author Arnold D. Robbins
199
321
322
@ifnotdocbook
200
323
@c Include the Distribution inside the titlepage environment so
201
324
@c that headings are turned off. Headings on and off do not work.
202
325
221
344
ISBN 1-882114-28-0 @*
222
345
@sp 2
223
346
@insertcopying
347
@end ifnotdocbook
224
348
@end titlepage
225
349
226
350
@c Thanks to Bob Chassell for directions on doing dedications.
229
353
@page
230
354
@w{ }
231
355
@sp 9
232
@center @i{To Miriam, for making me complete.}
233
@sp 1
234
@center @i{To Chana, for the joy you bring us.}
235
@sp 1
236
@center @i{To Rivka, for the exponential increase.}
237
@sp 1
238
@center @i{To Nachum, for the added dimension.}
239
@sp 1
240
@center @i{To Malka, for the new beginning.}
356
@center @i{To my parents, for their love, and for the wonderful example they set for me.}
357
@sp 1
358
@center @i{To my wife Miriam, for making me complete.
359
Thank you for building your life together with me.}
360
@sp 1
361
@center @i{To our children Chana, Rivka, Nachum and Malka, for enrichening our lives in innumerable ways.}
362
@sp 1
241
363
@w{ }
242
364
@page
243
365
@w{ }
245
367
@headings on
246
368
@end iftex
247
369
370
@docbook
371
<dedication>
372
<para>To my parents, for their love, and for the wonderful
373
example they set for me.</para>
374
<para>To my wife Miriam, for making me complete.
375
Thank you for building your life together with me.</para>
376
<para>To our children Chana, Rivka, Nachum and Malka,
377
for enrichening our lives in innumerable ways.</para>
378
</dedication>
379
@end docbook
380
248
381
@iftex
249
382
@headings off
250
383
@evenheading @thispage@ @ @ @strong{@value{TITLE}} @| @|
253
386
254
387
@ifnottex
255
388
@ifnotxml
389
@ifnotdocbook
256
390
@node Top
257
391
@top General Introduction
258
392
@c Preface node should come right after the Top
264
398
265
399
@insertcopying
266
400
401
@end ifnotdocbook
267
402
@end ifnotxml
268
403
@end ifnottex
269
404
331
466
includes command-line syntax.
332
467
* One-shot:: Running a short throwaway
333
468
@command{awk} program.
334
* Read Terminal:: Using no input files (input from
335
terminal instead).
469
* Read Terminal:: Using no input files (input from the
470
keyboard instead).
336
471
* Long:: Putting permanent @command{awk}
337
472
programs in files.
338
473
* Executable Scripts:: Making self-contained @command{awk}
354
489
* Other Features:: Other Features of @command{awk}.
355
490
* When:: When to use @command{gawk} and when to
356
491
use other things.
492
* Intro Summary:: Summary of the introduction.
357
493
* Command Line:: How to run @command{awk}.
358
494
* Options:: Command-line options and their
359
495
meanings.
375
511
program.
376
512
* Obsolete:: Obsolete Options and/or features.
377
513
* Undocumented:: Undocumented Options and Features.
514
* Invoking Summary:: Invocation summary.
378
515
* Regexp Usage:: How to Use Regular Expressions.
379
516
* Escape Sequences:: How to write nonprinting characters.
380
517
* Regexp Operators:: Regular Expression Operators.
383
520
* Case-sensitivity:: How to do case-insensitive matching.
384
521
* Leftmost Longest:: How much text matches.
385
522
* Computed Regexps:: Using Dynamic Regexps.
523
* Regexp Summary:: Regular expressions summary.
386
524
* Records:: Controlling how data is split into
387
525
records.
526
* awk split records:: How standard @command{awk} splits
527
records.
528
* gawk split records:: How @command{gawk} splits records.
388
529
* Fields:: An introduction to fields.
389
530
* Nonconstant Fields:: Nonconstant Field Numbers.
390
531
* Changing Fields:: Changing the Contents of a Field.
396
537
field.
397
538
* Command Line Field Separator:: Setting @code{FS} from the
398
539
command-line.
540
* Full Line Fields:: Making the full line be a single
541
field.
399
542
* Field Splitting Summary:: Some final points and a summary table.
400
543
* Constant Size:: Reading constant width data.
401
544
* Splitting By Content:: Defining Fields By Content
421
564
* Read Timeout:: Reading input with a timeout.
422
565
* Command line directories:: What happens if you put a directory on
423
566
the command line.
567
* Input Summary:: Input summary.
568
* Input Exercises:: Exercises.
424
569
* Print:: The @code{print} statement.
425
570
* Print Examples:: Simple examples of @code{print}
426
571
statements.
444
589
* Special Caveats:: Things to watch out for.
445
590
* Close Files And Pipes:: Closing Input and Output Files and
446
591
Pipes.
592
* Output Summary:: Output summary.
593
* Output exercises:: Exercises.
447
594
* Values:: Constants, Variables, and Regular
448
595
Expressions.
449
596
* Constants:: String, numeric and regexp constants.
459
606
This is an advanced method of input.
460
607
* Conversion:: The conversion of strings to numbers
461
608
and vice versa.
609
* Strings And Numbers:: How @command{awk} Converts Between
610
Strings And Numbers.
611
* Locale influences conversions:: How the locale may affect conversions.
462
612
* All Operators:: @command{gawk}'s operators.
463
613
* Arithmetic Ops:: Arithmetic operations (@samp{+},
464
614
@samp{-}, etc.)
486
636
* Function Calls:: A function call is an expression.
487
637
* Precedence:: How various operators nest.
488
638
* Locales:: How the locale affects things.
639
* Expressions Summary:: Expressions summary.
489
640
* Pattern Overview:: What goes into a pattern.
490
641
* Regexp Patterns:: Using regexps as patterns.
491
642
* Expression Patterns:: Any expression can be used as a
532
683
gives you information.
533
684
* ARGC and ARGV:: Ways to use @code{ARGC} and
534
685
@code{ARGV}.
686
* Pattern Action Summary:: Patterns and Actions summary.
535
687
* Array Basics:: The basics of arrays.
536
688
* Array Intro:: Introduction to Arrays
537
689
* Reference to Elements:: How to examine one element of an
554
706
@command{awk}.
555
707
* Multiscanning:: Scanning multidimensional arrays.
556
708
* Arrays of Arrays:: True multidimensional arrays.
709
* Arrays Summary:: Summary of arrays.
557
710
* Built-in:: Summarizes the built-in functions.
558
711
* Calling Built-in:: How to call built-in functions.
559
712
* Numeric Functions:: Functions that work with numbers,
588
741
runtime.
589
742
* Indirect Calls:: Choosing the function to call at
590
743
runtime.
744
* Functions Summary:: Summary of functions.
591
745
* Library Names:: How to best name private global
592
746
variables in library functions.
593
747
* General Functions:: Functions that are of general use.
622
776
* Group Functions:: Functions for getting group
623
777
information.
624
778
* Walking Arrays:: A function to walk arrays of arrays.
779
* Library Functions Summary:: Summary of library functions.
780
* Library exercises:: Exercises.
625
781
* Running Examples:: How to run these examples.
626
782
* Clones:: Clones of common utilities.
627
783
* Cut Program:: The @command{cut} utility.
651
807
* Anagram Program:: Finding anagrams from a dictionary.
652
808
* Signature Program:: People do amazing things with too much
653
809
time on their hands.
810
* Programs Summary:: Summary of programs.
811
* Programs Exercises:: Exercises.
654
812
* Nondecimal Data:: Allowing nondecimal input data.
655
813
* Array Sorting:: Facilities for controlling array
656
814
traversal and sorting arrays.
662
820
* TCP/IP Networking:: Using @command{gawk} for network
663
821
programming.
664
822
* Profiling:: Profiling your @command{awk} programs.
823
* Advanced Features Summary:: Summary of advanced features.
665
824
* I18N and L10N:: Internationalization and Localization.
666
* Explaining gettext:: How GNU @code{gettext} works.
825
* Explaining gettext:: How GNU @command{gettext} works.
667
826
* Programmer i18n:: Features for the programmer.
668
827
* Translator i18n:: Features for the translator.
669
828
* String Extraction:: Extracting marked strings.
673
832
* I18N Example:: A simple i18n example.
674
833
* Gawk I18N:: @command{gawk} is also
675
834
internationalized.
835
* I18N Summary:: Summary of I18N stuff.
676
836
* Debugging:: Introduction to @command{gawk}
677
837
debugger.
678
838
* Debugging Concepts:: Debugging in General.
691
851
* Miscellaneous Debugger Commands:: Miscellaneous Commands.
692
852
* Readline Support:: Readline support.
693
853
* Limitations:: Limitations and future plans.
694
* General Arithmetic:: An introduction to computer
695
arithmetic.
696
* Floating Point Issues:: Stuff to know about floating-point
697
numbers.
698
* String Conversion Precision:: The String Value Can Lie.
699
* Unexpected Results:: Floating Point Numbers Are Not
700
Abstract Numbers.
701
* POSIX Floating Point Problems:: Standards Versus Existing Practice.
702
* Integer Programming:: Effective integer programming.
703
* Floating-point Programming:: Effective Floating-point Programming.
704
* Floating-point Representation:: Binary floating-point representation.
705
* Floating-point Context:: Floating-point context.
706
* Rounding Mode:: Floating-point rounding mode.
707
* Gawk and MPFR:: How @command{gawk} provides
708
arbitrary-precision arithmetic.
709
* Arbitrary Precision Floats:: Arbitrary Precision Floating-point
710
Arithmetic with @command{gawk}.
711
* Setting Precision:: Setting the working precision.
712
* Setting Rounding Mode:: Setting the rounding mode.
713
* Floating-point Constants:: Representing floating-point constants.
714
* Changing Precision:: Changing the precision of a number.
715
* Exact Arithmetic:: Exact arithmetic with floating-point
716
numbers.
854
* Debugging Summary:: Debugging summary.
855
* Computer Arithmetic:: A quick intro to computer math.
856
* Math Definitions:: Defining terms used.
857
* MPFR features:: The MPFR features in @command{gawk}.
858
* FP Math Caution:: Things to know.
859
* Inexactness of computations:: Floating point math is not exact.
860
* Inexact representation:: Numbers are not exactly represented.
861
* Comparing FP Values:: How to compare floating point values.
862
* Errors accumulate:: Errors get bigger as they go.
863
* Getting Accuracy:: Getting more accuracy takes some work.
864
* Try To Round:: Add digits and round.
865
* Setting precision:: How to set the precision.
866
* Setting the rounding mode:: How to set the rounding mode.
717
867
* Arbitrary Precision Integers:: Arbitrary Precision Integer Arithmetic
718
868
with @command{gawk}.
869
* POSIX Floating Point Problems:: Standards Versus Existing Practice.
870
* Floating point summary:: Summary of floating point discussion.
719
871
* Extension Intro:: What is an extension.
720
872
* Plugin License:: A note about licensing.
721
873
* Extension Mechanism Outline:: An outline of how it works.
723
875
* Extension API Functions Introduction:: Introduction to the API functions.
724
876
* General Data Types:: The data types.
725
877
* Requesting Values:: How to get a value.
878
* Memory Allocation Functions:: Functions for allocating memory.
726
879
* Constructor Functions:: Functions for creating values.
727
880
* Registration Functions:: Functions to register things with
728
881
@command{gawk}.
776
929
* Extension Sample Time:: An interface to @code{gettimeofday()}
777
930
and @code{sleep()}.
778
931
* gawkextlib:: The @code{gawkextlib} project.
932
* Extension summary:: Extension summary.
933
* Extension Exercises:: Exercises.
779
934
* V7/SVR3.1:: The major changes between V7 and
780
935
System V Release 3.1.
781
936
* SVR4:: Minor changes between System V
785
940
version of @command{awk}.
786
941
* POSIX/GNU:: The extensions in @command{gawk} not
787
942
in POSIX @command{awk}.
943
* Feature History:: The history of the features in
944
@command{gawk}.
788
945
* Common Extensions:: Common Extensions Summary.
789
946
* Ranges and Locales:: How locales used to affect regexp
790
947
ranges.
791
948
* Contributors:: The major contributors to
792
949
@command{gawk}.
950
* History summary:: History summary.
793
951
* Gawk Distribution:: What is in the @command{gawk}
794
952
distribution.
795
953
* Getting:: How to get the distribution.
817
975
* VMS Installation:: Installing @command{gawk} on VMS.
818
976
* VMS Compilation:: How to compile @command{gawk} under
819
977
VMS.
978
* VMS Dynamic Extensions:: Compiling @command{gawk} dynamic
979
extensions on VMS.
820
980
* VMS Installation Details:: How to install @command{gawk} under
821
981
VMS.
822
982
* VMS Running:: How to run @command{gawk} under VMS.
983
* VMS GNV:: The VMS GNV Project.
823
984
* VMS Old Gawk:: An old version comes with some VMS
824
985
systems.
825
986
* Bugs:: Reporting Problems and Bugs.
826
987
* Other Versions:: Other freely available @command{awk}
827
988
implementations.
989
* Installation summary:: Summary of installation.
828
990
* Compatibility Mode:: How to disable certain @command{gawk}
829
991
extensions.
830
992
* Additions:: Making Additions To @command{gawk}.
833
995
@command{gawk}.
834
996
* New Ports:: Porting @command{gawk} to a new
835
997
operating system.
836
* Derived Files:: Why derived files are kept in the
837
@command{git} repository.
998
* Derived Files:: Why derived files are kept in the Git
999
repository.
838
1000
* Future Extensions:: New features that may be implemented
839
1001
one day.
840
1002
* Implementation Limitations:: Some limitations of the
845
1007
* Extension Other Design Decisions:: Some other design decisions.
846
1008
* Extension Future Growth:: Some room for future growth.
847
1009
* Old Extension Mechanism:: Some compatibility for old extensions.
1010
* Notes summary:: Summary of implementation notes.
848
1011
* Basic High Level:: The high level view.
849
1012
* Basic Data Typing:: A very quick intro to data types.
850
1013
@end detailmenu
852
1015
853
1016
@c dedication for Info file
854
1017
@ifinfo
855
@center To Miriam, for making me complete.
856
@sp 1
857
@center To Chana, for the joy you bring us.
858
@sp 1
859
@center To Rivka, for the exponential increase.
860
@sp 1
861
@center To Nachum, for the added dimension.
862
@sp 1
863
@center To Malka, for the new beginning.
1018
To my parents, for their love, and for the wonderful
1019
example they set for me.
1020
@sp 1
1021
To my wife Miriam, for making me complete.
1022
Thank you for building your life together with me.
1023
@sp 1
1024
To our children Chana, Rivka, Nachum and Malka,
1025
for enrichening our lives in innumerable ways.
864
1026
@end ifinfo
865
1027
866
1028
@summarycontents
869
1031
@node Foreword
870
1032
@unnumbered Foreword
871
1033
1034
@c This bit is post-processed by a script which turns the chapter
1035
@c tag into a preface tag, and moves this stuff to before the title.
1036
@c Bleah.
1037
@docbook
1038
<prefaceinfo>
1039
<author>
1040
<firstname>Michael</firstname>
1041
<surname>Brennan</surname>
1042
<!-- can't put mawk into command tags. sigh. -->
1043
<affiliation><jobtitle>Author of mawk</jobtitle></affiliation>
1044
</author>
1045
<date>March, 2001</date>
1046
</prefaceinfo>
1047
@end docbook
1048
872
1049
Arnold Robbins and I are good friends. We were introduced
873
1050
@c 11 years ago
874
1051
in 1990
953
1130
The new @command{pgawk} (profiling @command{gawk}), produces
954
1131
program execution counts.
955
1132
I recently experimented with an algorithm that for
956
@math{n} lines of input, exhibited
1133
@ifnotdocbook
1134
@math{n}
1135
@end ifnotdocbook
1136
@ifdocbook
1137
@i{n}
1138
@end ifdocbook
1139
lines of input, exhibited
957
1140
@tex
958
1141
$\sim\! Cn^2$
959
1142
@end tex
960
1143
@ifnottex
1144
@ifnotdocbook
961
1145
~ C n^2
1146
@end ifnotdocbook
962
1147
@end ifnottex
1148
@docbook
1149
<emphasis>∼ Cn<superscript>2</superscript></emphasis> @c
1150
@end docbook
963
1151
performance, while
964
1152
theory predicted
965
1153
@tex
966
1154
$\sim\! Cn\log n$
967
1155
@end tex
968
1156
@ifnottex
1157
@ifnotdocbook
969
1158
~ C n log n
1159
@end ifnotdocbook
970
1160
@end ifnottex
1161
@docbook
1162
<emphasis>∼ Cn log n</emphasis> @c
1163
@end docbook
971
1164
behavior. A few minutes poring
972
1165
over the @file{awkprof.out} profile pinpointed the problem to
973
1166
a single line of code. @command{pgawk} is a welcome addition to
977
1170
using AWK programs, and developing @command{gawk}, into this book. If you use
978
1171
AWK or want to learn how, then read this book.
979
1172
1173
@ifnotdocbook
1174
@cindex Brennan, Michael
980
1175
@display
981
1176
Michael Brennan
982
1177
Author of @command{mawk}
983
1178
March, 2001
984
1179
@end display
1180
@end ifnotdocbook
985
1181
986
1182
@node Preface
987
1183
@unnumbered Preface
990
1186
@c
991
1187
@c 12/2000: Chuck wants the preface & intro combined.
992
1188
1189
@c This bit is post-processed by a script which turns the chapter
1190
@c tag into a preface tag, and moves this stuff to before the title.
1191
@c Bleah.
1192
@docbook
1193
<prefaceinfo>
1194
<author>
1195
<firstname>Arnold</firstname>
1196
<surname>Robbins</surname>
1197
<affiliation><jobtitle>Nof Ayalon</jobtitle></affiliation>
1198
<affiliation><jobtitle>ISRAEL</jobtitle></affiliation>
1199
</author>
1200
<date>June, 2014</date>
1201
</prefaceinfo>
1202
@end docbook
1203
993
1204
Several kinds of tasks occur repeatedly
994
1205
when working with text files.
995
1206
You might want to extract certain lines and discard the rest.
1001
1212
The @command{awk} utility interprets a special-purpose programming language
1002
1213
that makes it easy to handle simple data-reformatting jobs.
1003
1214
1215
@cindex Brian Kernighan's @command{awk}
1004
1216
The GNU implementation of @command{awk} is called @command{gawk}; if you
1005
1217
invoke it with the proper options or environment variables
1006
1218
(@pxref{Options}), it is fully
1007
1219
compatible with
1008
the POSIX@footnote{The 2008 POSIX standard is online at
1009
@url{http://www.opengroup.org/onlinepubs/9699919799/}.}
1220
the POSIX@footnote{The 2008 POSIX standard is accessable online at
1221
@w{@url{http://www.opengroup.org/onlinepubs/9699919799/}.}}
1010
1222
specification of the @command{awk} language
1011
1223
and with the Unix version of @command{awk} maintained
1012
1224
by Brian Kernighan.
1023
1235
@cindex @command{awk}, uses for
1024
1236
Using @command{awk} allows you to:
1025
1237
1026
@itemize @bullet
1238
@itemize @value{BULLET}
1027
1239
@item
1028
1240
Manage small, personal databases
1029
1241
1048
1260
@command{gawk}
1049
1261
provides facilities that make it easy to:
1050
1262
1051
@itemize @bullet
1263
@itemize @value{BULLET}
1052
1264
@item
1053
1265
Extract bits and pieces of data for processing
1054
1266
1057
1269
1058
1270
@item
1059
1271
Perform simple network communications
1272
1273
@item
1274
Profile and debug @command{awk} programs.
1275
1276
@item
1277
Extend the language with functions written in C or C++.
1060
1278
@end itemize
1061
1279
1062
1280
This @value{DOCUMENT} teaches you about the @command{awk} language and
1072
1290
different computing environments. This @value{DOCUMENT}, while describing
1073
1291
the @command{awk} language in general, also describes the particular
1074
1292
implementation of @command{awk} called @command{gawk} (which stands for
1075
``GNU awk''). @command{gawk} runs on a broad range of Unix systems,
1293
``GNU @command{awk}''). @command{gawk} runs on a broad range of Unix systems,
1076
1294
ranging from Intel@registeredsymbol{}-architecture PC-based computers
1077
up through large-scale systems,
1078
such as Crays. @command{gawk} has also been ported to Mac OS X,
1079
Microsoft Windows (all versions) and OS/2 PCs,
1080
and VMS.
1295
up through large-scale systems.
1296
@command{gawk} has also been ported to Mac OS X,
1297
Microsoft Windows
1298
@ifset FOR_PRINT
1299
(all versions),
1300
@end ifset
1301
@ifclear FOR_PRINT
1302
(all versions) and OS/2 PCs,
1303
@end ifclear
1304
and OpenVMS.
1081
1305
(Some other, obsolete systems to which @command{gawk} was once ported
1082
1306
are no longer supported and the code for those systems
1083
1307
has been removed.)
1151
1375
@cite{TCP/IP Internetworking with @command{gawk}}
1152
1376
(a separate document, available as part of the @command{gawk} distribution).
1153
1377
His code finally became part of the main @command{gawk} distribution
1154
with @command{gawk} version 3.1.
1378
with @command{gawk} @value{PVERSION} 3.1.
1155
1379
1156
1380
John Haque rewrote the @command{gawk} internals, in the process providing
1157
1381
an @command{awk}-level debugger. This version became available as
1158
@command{gawk} version 4.0, in 2011.
1382
@command{gawk} @value{PVERSION} 4.0, in 2011.
1159
1383
1160
1384
@xref{Contributors},
1161
1385
for a complete list of those who made important contributions to @command{gawk}.
1170
1394
is often referred to as ``new @command{awk}'' (@command{nawk}).
1171
1395
1172
1396
@cindex @command{awk}, versions of
1173
Because of this, there are systems with multiple
1174
versions of @command{awk}.
1175
Some systems have an @command{awk} utility that implements the
1176
original version of the @command{awk} language and a @command{nawk} utility
1177
for the new version.
1178
Others have an @command{oawk} version for the ``old @command{awk}''
1179
language and plain @command{awk} for the new one. Still others only
1180
have one version, which is usually the new one.@footnote{Often, these systems
1181
use @command{gawk} for their @command{awk} implementation!}
1182
1183
1397
@cindex @command{nawk} utility
1184
1398
@cindex @command{oawk} utility
1185
All in all, this makes it difficult for you to know which version of
1186
@command{awk} you should run when writing your programs. The best advice
1187
we can give here is to check your local documentation. Look for @command{awk},
1188
@command{oawk}, and @command{nawk}, as well as for @command{gawk}.
1189
It is likely that you already
1190
have some version of new @command{awk} on your system, which is what
1191
you should use when running your programs. (Of course, if you're reading
1192
this @value{DOCUMENT}, chances are good that you have @command{gawk}!)
1399
For some time after new @command{awk} was introduced, there were
1400
systems with multiple versions of @command{awk}. Some systems had
1401
an @command{awk} utility that implemented the original version of the
1402
@command{awk} language and a @command{nawk} utility for the new version.
1403
Others had an @command{oawk} version for the ``old @command{awk}''
1404
language and plain @command{awk} for the new one. Still others only
1405
had one version, which is usually the new one.
1406
1407
Today, only Solaris systems still use an old @command{awk} for the
1408
default @command{awk} utility. (A more modern @command{awk} lives in
1409
@file{/usr/xpg6/bin} on these systems.) All other modern systems use
1410
some version of new @command{awk}.@footnote{Many of these systems use
1411
@command{gawk} for their @command{awk} implementation!}
1412
1413
It is likely that you already have some version of new @command{awk} on
1414
your system, which is what you should use when running your programs.
1415
(Of course, if you're reading this @value{DOCUMENT}, chances are good
1416
that you have @command{gawk}!)
1193
1417
1194
1418
Throughout this @value{DOCUMENT}, whenever we refer to a language feature
1195
1419
that should be available in any complete implementation of POSIX @command{awk},
1207
1431
This @value{DOCUMENT} explains
1208
1432
both how to write programs in the @command{awk} language and how to
1209
1433
run the @command{awk} utility.
1210
The term @dfn{@command{awk} program} refers to a program written by you in
1434
The term ``@command{awk} program'' refers to a program written by you in
1211
1435
the @command{awk} programming language.
1212
1436
1213
1437
@cindex @command{gawk}, @command{awk} and
1217
1441
as defined in the POSIX standard. It does so in the context of the
1218
1442
@command{gawk} implementation. While doing so, it also
1219
1443
attempts to describe important differences between @command{gawk}
1220
and other @command{awk} implementations.@footnote{All such differences
1444
and other @command{awk}
1445
@ifclear FOR_PRINT
1446
implementations.@footnote{All such differences
1221
1447
appear in the index under the
1222
1448
entry ``differences in @command{awk} and @command{gawk}.''}
1449
@end ifclear
1450
@ifset FOR_PRINT
1451
implementations.
1452
@end ifset
1223
1453
Finally, any @command{gawk} features that are not in
1224
1454
the POSIX standard for @command{awk} are noted.
1225
1455
1227
1457
This @value{DOCUMENT} has the difficult task of being both a tutorial and a reference.
1228
1458
If you are a novice, feel free to skip over details that seem too complex.
1229
1459
You should also ignore the many cross-references; they are for the
1230
expert user and for the online Info and HTML versions of the document.
1460
expert user and for the online Info and HTML versions of the @value{DOCUMENT}.
1231
1461
@end ifnotinfo
1232
1462
1233
1463
There are sidebars
1251
1481
1252
1482
This @value{DOCUMENT} is split into several parts, as follows:
1253
1483
1484
@c FULLXREF ON
1485
1254
1486
Part I describes the @command{awk} language and @command{gawk} program in detail.
1255
1487
It starts with the basics, and continues through all of the features of @command{awk}.
1256
1488
It contains the following chapters:
1334
1566
@ref{Dynamic Extensions}, describes how to add new variables and
1335
1567
functions to @command{gawk} by writing extensions in C or C++.
1336
1568
1569
@ifclear FOR_PRINT
1337
1570
Part IV provides the appendices, the Glossary, and two licenses that cover
1338
1571
the @command{gawk} source code and this @value{DOCUMENT}, respectively.
1339
1572
It contains the following appendices:
1573
@end ifclear
1574
@ifset FOR_PRINT
1575
Part IV provides the following appendices:
1576
@end ifset
1340
1577
1341
1578
@ref{Language History},
1342
1579
describes how the @command{awk} language has evolved since
1351
1588
in @command{gawk} and where to get other freely
1352
1589
available @command{awk} implementations.
1353
1590
1591
@ifset FOR_PRINT
1592
The version of this @value{DOCUMENT} distributed with @command{gawk}
1593
contains additional appendices and other end material.
1594
To save space, we have omitted them from the
1595
printed edition. You may find them online, as follows:
1596
1597
@uref{http://www.gnu.org/software/gawk/manual/html_node/Notes.html,
1598
The appendix on implementation notes}
1599
describes how to disable @command{gawk}'s extensions, as
1600
well as how to contribute new code to @command{gawk},
1601
and some possible future directions for @command{gawk} development.
1602
1603
@uref{http://www.gnu.org/software/gawk/manual/html_node/Basic-Concepts.html,
1604
The appendix on basic concepts}
1605
provides some very cursory background material for those who
1606
are completely unfamiliar with computer programming.
1607
1608
@uref{http://www.gnu.org/software/gawk/manual/html_node/Glossary.html,
1609
The Glossary}
1610
defines most, if not all, the significant terms used
1611
throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with,
1612
try looking them up here.
1613
1614
@uref{http://www.gnu.org/software/gawk/manual/html_node/Copying.html, The GNU GPL} and
1615
@uref{http://www.gnu.org/software/gawk/manual/html_node/GNU-Free-Documentation-License.html, the GNU FDL}
1616
are the licenses that cover the @command{gawk} source code
1617
and this @value{DOCUMENT}, respectively.
1618
@end ifset
1619
1620
@ifclear FOR_PRINT
1354
1621
@ref{Notes},
1355
1622
describes how to disable @command{gawk}'s extensions, as
1356
1623
well as how to contribute new code to @command{gawk},
1361
1628
are completely unfamiliar with computer programming.
1362
1629
1363
1630
The @ref{Glossary}, defines most, if not all, the significant terms used
1364
throughout the book. If you find terms that you aren't familiar with,
1631
throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with,
1365
1632
try looking them up here.
1366
1633
1367
1634
@ref{Copying}, and
1368
1635
@ref{GNU Free Documentation License},
1369
1636
present the licenses that cover the @command{gawk} source code
1370
1637
and this @value{DOCUMENT}, respectively.
1638
@end ifclear
1639
1640
@c FULLXREF OFF
1371
1641
1372
1642
@node Conventions
1373
1643
@unnumberedsec Typographical Conventions
1409
1679
strongly, it is done @strong{like this}. The first occurrence of
1410
1680
a new term is usually its @dfn{definition} and appears in the same
1411
1681
font as the previous occurrence of ``definition'' in this sentence.
1412
Finally, file names are indicated like this: @file{/path/to/ourfile}.
1682
Finally, @value{FN}s are indicated like this: @file{/path/to/ourfile}.
1413
1683
@end ifnotinfo
1414
1684
1415
1685
Characters that you type at the keyboard look @kbd{like this}. In particular,
1441
1711
@ifnottex
1442
1712
``(d.c.)''.
1443
1713
@end ifnottex
1714
@ifclear FOR_PRINT
1444
1715
They also appear in the index under the heading ``dark corner.''
1716
@end ifclear
1445
1717
1446
As noted by the opening quote, though, any
1447
coverage of dark corners
1448
is, by definition, incomplete.
1718
As noted by the opening quote, though, any coverage of dark corners is,
1719
by definition, incomplete.
1449
1720
1450
1721
Extensions to the standard @command{awk} language that are supported by
1451
1722
more than one @command{awk} implementation are marked
1723
@ifclear FOR_PRINT
1452
1724
``@value{COMMONEXT},'' and listed in the index under ``common extensions''
1453
1725
and ``extensions, common.''
1726
@end ifclear
1727
@ifset FOR_PRINT
1728
``@value{COMMONEXT}.''
1729
@end ifset
1454
1730
1455
1731
@node Manual History
1456
1732
@unnumberedsec The GNU Project and This Book
1473
1749
computing environment.
1474
1750
The FSF uses the ``GNU General Public License'' (GPL) to ensure that
1475
1751
their software's
1476
source code is always available to the end user. A
1477
copy of the GPL is included
1752
source code is always available to the end user.
1753
@ifclear FOR_PRINT
1754
A copy of the GPL is included
1478
1755
@ifnotinfo
1479
1756
in this @value{DOCUMENT}
1480
1757
@end ifnotinfo
1481
1758
for your reference
1482
1759
(@pxref{Copying}).
1760
@end ifclear
1483
1761
The GPL applies to the C language source code for @command{gawk}.
1484
1762
To find out more about the FSF and the GNU Project online,
1485
1763
see @uref{http://www.gnu.org, the GNU Project's home page}.
1502
1780
system for Intel@registeredsymbol{},
1503
1781
Power Architecture,
1504
1782
Sun SPARC, IBM S/390, and other
1783
@ifclear FOR_PRINT
1505
1784
systems.@footnote{The terminology ``GNU/Linux'' is explained
1506
1785
in the @ref{Glossary}.}
1786
@end ifclear
1787
@ifset FOR_PRINT
1788
systems.
1789
@end ifset
1507
1790
Many GNU/Linux distributions are
1508
1791
available for download from the Internet.
1509
1792
1523
1806
information in it is free to anyone. The machine-readable
1524
1807
source code for the @value{DOCUMENT} comes with @command{gawk}; anyone
1525
1808
may take this @value{DOCUMENT} to a copying machine and make as many
1526
copies as they like. (Take a moment to check the Free Documentation
1809
copies as they like.
1810
@ifclear FOR_PRINT
1811
(Take a moment to check the Free Documentation
1527
1812
License in @ref{GNU Free Documentation License}.)
1813
@end ifclear
1528
1814
@end ifnotinfo
1529
1815
1530
@ignore
1531
@cindex Close, Diane
1532
The @value{DOCUMENT} itself has gone through several previous,
1533
preliminary editions.
1534
Paul Rubin wrote the very first draft of @cite{The GAWK Manual};
1535
it was around 40 pages in size.
1536
Diane Close and Richard Stallman improved it, yielding the
1537
version which I started working with in the fall of 1988.
1538
It was around 90 pages long and barely described the original, ``old''
1539
version of @command{awk}. After substantial revision, the first version of
1540
the @cite{The GAWK Manual} to be released was Edition 0.11 Beta in
1541
October of 1989. The manual then underwent more substantial revision
1542
for Edition 0.13 of December 1991.
1543
David Trueman, Pat Rankin and Michal Jaegermann contributed sections
1544
of the manual for Edition 0.13.
1545
That edition was published by the
1546
FSF as a bound book early in 1992. Since then there were several
1547
minor revisions, notably Edition 0.14 of November 1992 that was published
1548
by the FSF in January of 1993 and Edition 0.16 of August 1993.
1549
1550
Edition 1.0 of @cite{GAWK: The GNU Awk User's Guide} represented a significant re-working
1551
of @cite{The GAWK Manual}, with much additional material.
1552
The FSF and I agreed that I was now the primary author.
1553
@c I also felt that the manual needed a more descriptive title.
1554
1555
In January 1996, SSC published Edition 1.0 under the title @cite{Effective AWK Programming}.
1556
In February 1997, they published Edition 1.0.3 which had minor changes
1557
as a ``second edition.''
1558
In 1999, the FSF published this same version as Edition 2
1559
of @cite{GAWK: The GNU Awk User's Guide}.
1560
1561
Edition @value{EDITION} maintains the basic structure of Edition 1.0,
1562
but with significant additional material, reflecting the host of new features
1563
in @command{gawk} version @value{VERSION}.
1564
Of particular note is
1565
@ref{Array Sorting},
1566
@ref{Bitwise Functions},
1567
@ref{Internationalization},
1568
@ref{Advanced Features},
1569
and
1570
@ref{Dynamic Extensions}.
1571
@end ignore
1572
1573
1816
@cindex Close, Diane
1574
1817
The @value{DOCUMENT} itself has gone through a number of previous editions.
1575
1818
Paul Rubin wrote the very first draft of @cite{The GAWK Manual};
1585
1828
In 1996, Edition 1.0 was released with @command{gawk} 3.0.0.
1586
1829
The FSF published the first two editions under
1587
1830
the title @cite{The GNU Awk User's Guide}.
1831
@ifset FOR_PRINT
1832
SSC published two editions of the @value{DOCUMENT} under the
1833
title @cite{Effective awk Programming}, and in O'Reilly published
1834
the third edition in 2001.
1835
@end ifset
1588
1836
1589
1837
This edition maintains the basic structure of the previous editions.
1590
For Edition 4.0, the content has been thoroughly reviewed
1838
For FSF edition 4.0, the content has been thoroughly reviewed
1591
1839
and updated. All references to @command{gawk} versions prior to 4.0 have been
1592
1840
removed.
1593
1841
Of significant note for this edition was @ref{Debugger}.
1594
1842
1595
For edition @value{EDITION}, the content has been reorganized into parts,
1843
For FSF edition
1844
@ifclear FOR_PRINT
1845
@value{EDITION},
1846
@end ifclear
1847
@ifset FOR_PRINT
1848
@value{EDITION}
1849
(the fourth edition as published by O'Reilly),
1850
@end ifset
1851
the content has been reorganized into parts,
1596
1852
and the major new additions are @ref{Arbitrary Precision Arithmetic},
1597
1853
and @ref{Dynamic Extensions}.
1598
1854
1599
@cite{@value{TITLE}} will undoubtedly continue to evolve.
1600
An electronic version
1601
comes with the @command{gawk} distribution from the FSF.
1602
If you find an error in this @value{DOCUMENT}, please report it!
1603
@xref{Bugs}, for information on submitting
1604
problem reports electronically.
1605
1855
This @value{DOCUMENT} will undoubtedly continue to evolve. An electronic
1856
version comes with the @command{gawk} distribution from the FSF. If you
1857
find an error in this @value{DOCUMENT}, please report it! @xref{Bugs},
1858
for information on submitting problem reports electronically.
1859
1860
@ifset FOR_PRINT
1861
@c fakenode --- for prepinfo
1862
@unnumberedsec How to Stay Current
1863
1864
It may be you have a version of @command{gawk} which is newer than the
1865
one described in this @value{DOCUMENT}. To find out what has changed,
1866
you should first look at the @file{NEWS} file in the @command{gawk}
1867
distribution, which provides a high level summary of what changed in
1868
each release.
1869
1870
You can then look at the @uref{http://www.gnu.org/software/gawk/manual/,
1871
online version} of this @value{DOCUMENT} to read about any new features.
1872
@end ifset
1873
1874
@ifclear FOR_PRINT
1606
1875
@node How To Contribute
1607
1876
@unnumberedsec How to Contribute
1608
1877
1619
1888
contributed code: the archive did not grow and the domain went unused
1620
1889
for several years.
1621
1890
1622
Fortunately, late in 2008, a volunteer took on the task of setting up
1891
Late in 2008, a volunteer took on the task of setting up
1623
1892
an @command{awk}-related web site---@uref{http://awk.info}---and did a very
1624
1893
nice job.
1625
1894
1628
1897
of the world, please see @uref{http://awk.info/?contribute} for how to
1629
1898
contribute it to the web site.
1630
1899
1900
As of this writing, this website is in search of a maintainer; please
1901
contact me if you are interested.
1902
1631
1903
@ignore
1632
1904
Other links:
1633
1905
1634
1906
http://www.reddit.com/r/linux/comments/dtect/composing_music_in_awk/
1635
1907
@end ignore
1908
@end ifclear
1636
1909
1637
1910
@node Acknowledgments
1638
1911
@unnumberedsec Acknowledgments
1723
1996
3.1 release of @command{gawk}.
1724
1997
@end quotation
1725
1998
1726
@cindex Beebe, Nelson
1999
@cindex Beebe, Nelson H.F.@:
1727
2000
@cindex Buening, Andreas
1728
2001
@cindex Collado, Manuel
1729
2002
@cindex Colombo, Antonio
1740
2013
@cindex Rankin, Pat
1741
2014
@cindex Schorr, Andrew
1742
2015
@cindex Vinschen, Corinna
1743
@cindex Wallin, Anders
1744
2016
@cindex Zaretskii, Eli
1745
2017
1746
2018
Dr.@: Nelson Beebe,
1760
2032
Pat Rankin,
1761
2033
Andrew Schorr,
1762
2034
Corinna Vinschen,
1763
Anders Wallin,
1764
2035
and Eli Zaretskii
1765
2036
(in alphabetical order)
1766
2037
make up the current
1772
2043
Notable code and documentation contributions were made by
1773
2044
a number of people. @xref{Contributors}, for the full list.
1774
2045
2046
Thanks to Patrice Dumas for the new @command{makeinfo} program.
2047
Thanks to Karl Berry who continues to work to keep
2048
the Texinfo markup language sane.
2049
1775
2050
@cindex Kernighan, Brian
1776
2051
I would like to thank Brian Kernighan for invaluable assistance during the
1777
2052
testing and debugging of @command{gawk}, and for ongoing
1791
2066
Finally, I also must acknowledge my gratitude to G-d, for the many opportunities
1792
2067
He has sent my way, as well as for the gifts He has given me with which to
1793
2068
take advantage of those opportunities.
2069
@iftex
1794
2070
@sp 2
1795
2071
@noindent
1796
2072
Arnold Robbins @*
1797
2073
Nof Ayalon @*
1798
2074
ISRAEL @*
1799
May, 2013
1800
1801
@iftex
1802
@part Part I:@* The @command{awk} Language
2075
May, 2014
1803
2076
@end iftex
1804
2077
1805
@ignore
2078
@ifnotinfo
2079
@part @value{PART1}The @command{awk} Language
2080
@end ifnotinfo
2081
1806
2082
@ifdocbook
1807
@part Part I:@* The @command{awk} Language
1808
1809
Part I describes the @command{awk} language and @command{gawk} program in detail.
1810
It starts with the basics, and continues through all of the features of @command{awk}
1811
and @command{gawk}. It contains the following chapters:
1812
1813
@itemize @bullet
2083
2084
Part I describes the @command{awk} language and @command{gawk} program
2085
in detail. It starts with the basics, and continues through all of
2086
the features of @command{awk}. Included also are many, but not all,
2087
of the features of @command{gawk}. This part contains the
2088
following chapters:
2089
2090
@itemize @value{BULLET}
1814
2091
@item
1815
2092
@ref{Getting Started}.
1816
2093
1839
2116
@ref{Functions}.
1840
2117
@end itemize
1841
2118
@end ifdocbook
1842
@end ignore
1843
2119
1844
2120
@node Getting Started
1845
2121
@chapter Getting Started with @command{awk}
1879
2155
upon finding the pattern.
1880
2156
1881
2157
Syntactically, a rule consists of a pattern followed by an action. The
1882
action is enclosed in curly braces to separate it from the pattern.
2158
action is enclosed in braces to separate it from the pattern.
1883
2159
Newlines usually separate rules. Therefore, an @command{awk}
1884
2160
program looks like this:
1885
2161
1903
2179
* Other Features:: Other Features of @command{awk}.
1904
2180
* When:: When to use @command{gawk} and when to use
1905
2181
other things.
2182
* Intro Summary:: Summary of the introduction.
1906
2183
@end menu
1907
2184
1908
2185
@node Running gawk
1931
2208
@menu
1932
2209
* One-shot:: Running a short throwaway @command{awk}
1933
2210
program.
1934
* Read Terminal:: Using no input files (input from terminal
2211
* Read Terminal:: Using no input files (input from the keyboard
1935
2212
instead).
1936
2213
* Long:: Putting permanent @command{awk} programs in
1937
2214
files.
1995
2272
1996
2273
@noindent
1997
2274
@command{awk} applies the @var{program} to the @dfn{standard input},
1998
which usually means whatever you type on the terminal. This continues
2275
which usually means whatever you type on the keyboard. This continues
1999
2276
until you indicate end-of-file by typing @kbd{Ctrl-d}.
2277
@ifset FOR_PRINT
2278
(On other operating systems, the end-of-file character may be different.)
2279
@end ifset
2280
@ifclear FOR_PRINT
2000
2281
(On other operating systems, the end-of-file character may be different.
2001
2282
For example, on OS/2, it is @kbd{Ctrl-z}.)
2283
@end ifclear
2002
2284
2003
2285
@cindex files, input, See input files
2004
2286
@cindex input files, running @command{awk} without
2018
2300
@print{} Don't Panic!
2019
2301
@end example
2020
2302
2021
@cindex quoting
2022
@cindex double quote (@code{"})
2023
@cindex @code{"} (double quote)
2024
@cindex @code{\} (backslash)
2025
@cindex backslash (@code{\})
2303
@cindex shell quoting, double quote
2304
@cindex double quote (@code{"}) in shell commands
2305
@cindex @code{"} (double quote) in shell commands
2306
@cindex @code{\} (backslash) in shell commands
2307
@cindex backslash (@code{\}) in shell commands
2026
2308
This program does not read any input. The @samp{\} before each of the
2027
2309
inner double quotes is necessary because of the shell's quoting
2028
2310
rules---in particular because it mixes both single quotes and
2061
2343
awk -f @var{source-file} @var{input-file1} @var{input-file2} @dots{}
2062
2344
@end example
2063
2345
2064
@cindex @code{-f} option
2065
@cindex command line, options
2066
@cindex options, command-line
2346
@cindex @option{-f} option
2347
@cindex command line, option @option{-f}
2067
2348
The @option{-f} instructs the @command{awk} utility to get the @command{awk} program
2068
from the file @var{source-file}. Any file name can be used for
2349
from the file @var{source-file}. Any @value{FN} can be used for
2069
2350
@var{source-file}. For example, you could put the program:
2070
2351
2071
2352
@example
2086
2367
awk "BEGIN @{ print \"Don't Panic!\" @}"
2087
2368
@end example
2088
2369
2089
@cindex quoting
2370
@cindex quoting in @command{gawk} command lines
2090
2371
@noindent
2091
2372
This was explained earlier
2092
2373
(@pxref{Read Terminal}).
2093
Note that you don't usually need single quotes around the file name that you
2094
specify with @option{-f}, because most file names don't contain any of the shell's
2374
Note that you don't usually need single quotes around the @value{FN} that you
2375
specify with @option{-f}, because most @value{FN}s don't contain any of the shell's
2095
2376
special characters. Notice that in @file{advice}, the @command{awk}
2096
2377
program did not have single quotes around it. The quotes are only needed
2097
2378
for programs that are provided on the @command{awk} command line.
2098
2379
2099
2380
@c STARTOFRANGE sq1x
2100
@cindex single quote (@code{'})
2381
@cindex single quote (@code{'}) in @command{gawk} command lines
2101
2382
@c STARTOFRANGE qs2x
2102
@cindex @code{'} (single quote)
2383
@cindex @code{'} (single quote) in @command{gawk} command lines
2103
2384
If you want to clearly identify your @command{awk} program files as such,
2104
you can add the extension @file{.awk} to the file name. This doesn't
2385
you can add the extension @file{.awk} to the @value{FN}. This doesn't
2105
2386
affect the execution of the @command{awk} program but it does make
2106
2387
``housekeeping'' easier.
2107
2388
2128
2409
After making this file executable (with the @command{chmod} utility),
2129
2410
simply type @samp{advice}
2130
2411
at the shell and the system arranges to run @command{awk}@footnote{The
2131
line beginning with @samp{#!} lists the full file name of an interpreter
2412
line beginning with @samp{#!} lists the full @value{FN} of an interpreter
2132
2413
to run and an optional initial command-line argument to pass to that
2133
2414
interpreter. The operating system then runs the interpreter with the given
2134
2415
argument and the full argument list of the executed program. The first argument
2135
in the list is the full file name of the @command{awk} program.
2416
in the list is the full @value{FN} of the @command{awk} program.
2136
2417
The rest of the
2137
argument list contains either options to @command{awk}, or data files,
2418
argument list contains either options to @command{awk}, or @value{DF}s,
2138
2419
or both. Note that on many systems @command{awk} may be found in
2139
2420
@file{/usr/bin} instead of in @file{/bin}. Caveat Emptor.} as if you had
2140
2421
typed @samp{awk -f advice}:
2209
2490
comment is to help you or another person understand the program
2210
2491
when reading it at a later time.
2211
2492
2212
@cindex quoting
2493
@cindex quoting, for small awk programs
2213
2494
@cindex single quote (@code{'}), vs.@: apostrophe
2214
2495
@cindex @code{'} (single quote), vs.@: apostrophe
2215
2496
@quotation CAUTION
2225
2506
For example, look at the following:
2226
2507
2227
2508
@example
2228
$ @kbd{awk '@{ print "hello" @} # let's be cute'}
2509
$ @kbd{awk 'BEGIN @{ print "hello" @} # let's be cute'}
2229
2510
>
2230
2511
@end example
2231
2512
2250
2531
2251
2532
@node Quoting
2252
2533
@subsection Shell-Quoting Issues
2253
@cindex quoting, rules for
2534
@cindex shell quoting, rules for
2254
2535
2255
2536
@menu
2256
2537
* DOS Quoting:: Quoting in Windows Batch Files.
2273
2554
POSIX-compliant, Bourne-style shells (such as Bash, the GNU Bourne-Again
2274
2555
Shell). If you use the C shell, you're on your own.
2275
2556
2276
@itemize @bullet
2557
Before diving into the rules, we introduce a concept that appears
2558
throughout this @value{DOCUMENT}, which is that of the @dfn{null},
2559
or empty, string.
2560
2561
The null string is character data that has no value.
2562
In other words, it is empty. It is written in @command{awk} programs
2563
like this: @code{""}. In the shell, it can be written using single
2564
or double quotes: @code{""} or @code{''}. While the null string has
2565
no characters in it, it does exist. Consider this command:
2566
2567
@example
2568
$ @kbd{echo ""}
2569
@end example
2570
2571
@noindent
2572
Here, the @command{echo} utility receives a single argument, even
2573
though that argument has no characters in it. In the rest of this
2574
@value{DOCUMENT}, we use the terms @dfn{null string} and @dfn{empty string}
2575
interchangeably. Now, on to the quoting rules.
2576
2577
2578
@itemize @value{BULLET}
2277
2579
@item
2278
2580
Quoted items can be concatenated with nonquoted items as well as with other
2279
2581
quoted items. The shell turns everything into one argument for
2285
2587
character on to the command.
2286
2588
2287
2589
@item
2288
@cindex @code{\} (backslash)
2289
@cindex backslash (@code{\})
2290
@cindex single quote (@code{'})
2291
@cindex @code{'} (single quote)
2590
@cindex @code{\} (backslash), in shell commands
2591
@cindex backslash (@code{\}), in shell commands
2592
@cindex single quote (@code{'}), in shell commands
2593
@cindex @code{'} (single quote), in shell commands
2292
2594
Single quotes protect everything between the opening and closing quotes.
2293
2595
The shell does no interpretation of the quoted text, passing it on verbatim
2294
2596
to the command.
2298
2600
for an example of what happens if you try.
2299
2601
2300
2602
@item
2301
@cindex double quote (@code{"})
2302
@cindex @code{"} (double quote)
2603
@cindex double quote (@code{"}), in shell commands
2604
@cindex @code{"} (double quote), in shell commands
2303
2605
Double quotes protect most things between the opening and closing quotes.
2304
2606
The shell does at least variable and command substitution on the quoted text.
2305
2607
Different shells may do additional kinds of processing on double-quoted text.
2336
2638
@end example
2337
2639
2338
2640
@noindent
2339
@cindex null strings, quoting and
2641
@cindex null strings in @command{gawk} arguments, quoting and
2340
2642
Don't use this:
2341
2643
2342
2644
@example
2345
2647
2346
2648
@noindent
2347
2649
In the second case, @command{awk} will attempt to use the text of the program
2348
as the value of @code{FS}, and the first file name as the text of the program!
2650
as the value of @code{FS}, and the first @value{FN} as the text of the program!
2349
2651
This results in syntax errors at best, and confusing behavior at worst.
2350
2652
@end itemize
2351
2653
2352
@cindex quoting, tricks for
2654
@cindex quoting in @command{gawk} command lines, tricks for
2353
2655
Mixing single and double quotes is difficult. You have to resort
2354
2656
to shell quoting tricks, like this:
2355
2657
2448
2750
POSIX shell, the following issue arises often enough for many users that
2449
2751
it is worth addressing.
2450
2752
2753
@cindex Brink, Jeroen
2451
2754
The ``shells'' on Microsoft Windows systems use the double-quote
2452
2755
character for quoting, and make it difficult or impossible to include an
2453
2756
escaped double-quote character in a command-line script.
2460
2763
2461
2764
2462
2765
@node Sample Data Files
2463
@section Data Files for the Examples
2464
@c For gawk >= 4.0, update these data files. No-one has such slow modems!
2766
@section @value{DDF}s for the Examples
2465
2767
2466
2768
@cindex input files, examples
2467
@cindex @code{BBS-list} file
2769
@cindex @code{mail-list} file
2468
2770
Many of the examples in this @value{DOCUMENT} take their input from two sample
2469
data files. The first, @file{BBS-list}, represents a list of
2470
computer bulletin board systems together with information about those systems.
2471
The second data file, called @file{inventory-shipped}, contains
2771
@value{DF}s. The first, @file{mail-list}, represents a list of peoples' names
2772
together with their email addresses and information about those people.
2773
The second @value{DF}, called @file{inventory-shipped}, contains
2472
2774
information about monthly shipments. In both files,
2473
2775
each line is considered to be one @dfn{record}.
2474
2776
2475
In the data file @file{BBS-list}, each record contains the name of a computer
2476
bulletin board, its phone number, the board's baud rate(s), and a code for
2477
the number of hours it is operational. An @samp{A} in the last column
2478
means the board operates 24 hours a day. A @samp{B} in the last
2479
column means the board only operates on evening and weekend hours.
2480
A @samp{C} means the board operates only on weekends:
2777
In the @value{DF} @file{mail-list}, each record contains the name of a person,
2778
his/her phone number, his/her email-address, and a code for their relationship
2779
with the author of the list. An @samp{A} in the last column
2780
means that the person is an acquaintance. An @samp{F} in the last
2781
column means that the person is a friend.
2782
An @samp{R} means that the person is a relative:
2481
2783
2482
@c 2e: Update the baud rates to reflect today's faster modems
2483
2784
@example
2484
2785
@c system if test ! -d eg ; then mkdir eg ; fi
2485
2786
@c system if test ! -d eg/lib ; then mkdir eg/lib ; fi
2486
2787
@c system if test ! -d eg/data ; then mkdir eg/data ; fi
2487
2788
@c system if test ! -d eg/prog ; then mkdir eg/prog ; fi
2488
2789
@c system if test ! -d eg/misc ; then mkdir eg/misc ; fi
2489
@c file eg/data/BBS-list
2490
aardvark 555-5553 1200/300 B
2491
alpo-net 555-3412 2400/1200/300 A
2492
barfly 555-7685 1200/300 A
2493
bites 555-1675 2400/1200/300 A
2494
camelot 555-0542 300 C
2495
core 555-2912 1200/300 C
2496
fooey 555-1234 2400/1200/300 B
2497
foot 555-6699 1200/300 B
2498
macfoo 555-6480 1200/300 A
2499
sdace 555-3430 2400/1200/300 A
2500
sabafoo 555-2127 1200/300 C
2790
@c file eg/data/mail-list
2791
Amelia 555-5553 amelia.zodiacusque@@gmail.com F
2792
Anthony 555-3412 anthony.asserturo@@hotmail.com A
2793
Becky 555-7685 becky.algebrarum@@gmail.com A
2794
Bill 555-1675 bill.drowning@@hotmail.com A
2795
Broderick 555-0542 broderick.aliquotiens@@yahoo.com R
2796
Camilla 555-2912 camilla.infusarum@@skynet.be R
2797
Fabius 555-1234 fabius.undevicesimus@@ucb.edu F
2798
Julie 555-6699 julie.perscrutabor@@skeeve.com F
2799
Martin 555-6480 martin.codicibus@@hotmail.com A
2800
Samuel 555-3430 samuel.lanceolis@@shu.edu A
2801
Jean-Paul 555-2127 jeanpaul.campanorum@@nyu.edu R
2501
2802
@c endfile
2502
2803
@end example
2503
2804
2504
2805
@cindex @code{inventory-shipped} file
2505
The data file @file{inventory-shipped} represents
2806
The @value{DF} @file{inventory-shipped} represents
2506
2807
information about shipments during the year.
2507
2808
Each record contains the month, the number
2508
2809
of green crates shipped, the number of red boxes shipped, the number of
2532
2833
@c endfile
2533
2834
@end example
2534
2835
2535
@ifinfo
2536
If you are reading this in GNU Emacs using Info, you can copy the regions
2537
of text showing these sample files into your own test files. This way you
2538
can try out the examples shown in the remainder of this document. You do
2539
this by using the command @kbd{M-x write-region} to copy text from the Info
2540
file into a file for use with @command{awk}
2541
(@xref{Misc File Ops, , Miscellaneous File Operations, emacs, GNU Emacs Manual},
2542
for more information). Using this information, create your own
2543
@file{BBS-list} and @file{inventory-shipped} files and practice what you
2544
learn in this @value{DOCUMENT}.
2545
2546
@cindex Texinfo
2547
If you are using the stand-alone version of Info,
2548
see @ref{Extract Program},
2549
for an @command{awk} program that extracts these data files from
2550
@file{gawk.texi}, the (generated) Texinfo source file for this Info file.
2551
@end ifinfo
2836
The sample files are included in the @command{gawk} distribution,
2837
in the directory @file{awklib/eg/data}.
2552
2838
2553
2839
@node Very Simple
2554
2840
@section Some Simple Examples
2555
2841
2556
2842
The following command runs a simple @command{awk} program that searches the
2557
input file @file{BBS-list} for the character string @samp{foo} (a
2843
input file @file{mail-list} for the character string @samp{li} (a
2558
2844
grouping of characters is usually called a @dfn{string};
2559
2845
the term @dfn{string} is based on similar usage in English, such
2560
2846
as ``a string of pearls,'' or ``a string of cars in a train''):
2561
2847
2562
2848
@example
2563
awk '/foo/ @{ print $0 @}' BBS-list
2849
awk '/li/ @{ print $0 @}' mail-list
2564
2850
@end example
2565
2851
2566
2852
@noindent
2567
When lines containing @samp{foo} are found, they are printed because
2853
When lines containing @samp{li} are found, they are printed because
2568
2854
@w{@samp{print $0}} means print the current line. (Just @samp{print} by
2569
2855
itself means the same thing, so we could have written that
2570
2856
instead.)
2571
2857
2572
You will notice that slashes (@samp{/}) surround the string @samp{foo}
2573
in the @command{awk} program. The slashes indicate that @samp{foo}
2858
You will notice that slashes (@samp{/}) surround the string @samp{li}
2859
in the @command{awk} program. The slashes indicate that @samp{li}
2574
2860
is the pattern to search for. This type of pattern is called a
2575
2861
@dfn{regular expression}, which is covered in more detail later
2576
2862
(@pxref{Regexp}).
2582
2868
Here is what this program prints:
2583
2869
2584
2870
@example
2585
$ @kbd{awk '/foo/ @{ print $0 @}' BBS-list}
2586
@print{} fooey 555-1234 2400/1200/300 B
2587
@print{} foot 555-6699 1200/300 B
2588
@print{} macfoo 555-6480 1200/300 A
2589
@print{} sabafoo 555-2127 1200/300 C
2871
$ @kbd{awk '/li/ @{ print $0 @}' mail-list}
2872
@print{} Amelia 555-5553 amelia.zodiacusque@@gmail.com F
2873
@print{} Broderick 555-0542 broderick.aliquotiens@@yahoo.com R
2874
@print{} Julie 555-6699 julie.perscrutabor@@skeeve.com F
2875
@print{} Samuel 555-3430 samuel.lanceolis@@shu.edu A
2590
2876
@end example
2591
2877
2592
2878
@cindex actions, default
2597
2883
action is to print all lines that match the pattern.
2598
2884
2599
2885
@cindex actions, empty
2600
Thus, we could leave out the action (the @code{print} statement and the curly
2886
Thus, we could leave out the action (the @code{print} statement and the
2601
2887
braces) in the previous example and the result would be the same:
2602
@command{awk} prints all lines matching the pattern @samp{foo}. By comparison,
2603
omitting the @code{print} statement but retaining the curly braces makes an
2888
@command{awk} prints all lines matching the pattern @samp{li}. By comparison,
2889
omitting the @code{print} statement but retaining the braces makes an
2604
2890
empty action that does nothing (i.e., no lines are printed).
2605
2891
2606
2892
@cindex @command{awk} programs, one-line examples
2609
2895
programs contain constructs that haven't been covered yet. (The description
2610
2896
of the program will give you a good idea of what is going on, but please
2611
2897
read the rest of the @value{DOCUMENT} to become an @command{awk} expert!)
2612
Most of the examples use a data file named @file{data}. This is just a
2898
Most of the examples use a @value{DF} named @file{data}. This is just a
2613
2899
placeholder; if you use these programs yourself, substitute
2614
your own file names for @file{data}.
2900
your own @value{FN}s for @file{data}.
2615
2901
For future reference, note that there is often more than
2616
2902
one way to do things in @command{awk}. At some point, you may want
2617
2903
to look back at these examples and see if
2618
2904
you can come up with different ways to do the same things shown here:
2619
2905
2620
@itemize @bullet
2906
@itemize @value{BULLET}
2621
2907
@item
2622
2908
Print the length of the longest input line:
2623
2909
2634
2920
@end example
2635
2921
2636
2922
The sole rule has a relational expression as its pattern and it has no
2637
action---so the default action, printing the record, is used.
2923
action---so it uses the default action, printing the record.
2638
2924
2639
2925
@cindex @command{expand} utility
2640
2926
@item
2701
2987
@end example
2702
2988
2703
2989
@item
2704
Print the even-numbered lines in the data file:
2990
Print the even-numbered lines in the @value{DF}:
2705
2991
2706
2992
@example
2707
2993
awk 'NR % 2 == 0' data
2717
3003
2718
3004
The @command{awk} utility reads the input files one line at a
2719
3005
time. For each line, @command{awk} tries the patterns of each of the rules.
2720
If several patterns match, then several actions are run in the order in
3006
If several patterns match, then several actions execute in the order in
2721
3007
which they appear in the @command{awk} program. If no patterns match, then
2722
no actions are run.
3008
no actions run.
2723
3009
2724
3010
After processing all the rules that match the line (and perhaps there are none),
2725
3011
@command{awk} reads the next line. (However,
2743
3029
@samp{12} @emph{or} the string @samp{21}. If a line contains both
2744
3030
strings, it is printed twice, once by each rule.
2745
3031
2746
This is what happens if we run this program on our two sample data files,
2747
@file{BBS-list} and @file{inventory-shipped}:
3032
This is what happens if we run this program on our two sample @value{DF}s,
3033
@file{mail-list} and @file{inventory-shipped}:
2748
3034
2749
3035
@example
2750
3036
$ @kbd{awk '/12/ @{ print $0 @}}
2751
> @kbd{/21/ @{ print $0 @}' BBS-list inventory-shipped}
2752
@print{} aardvark 555-5553 1200/300 B
2753
@print{} alpo-net 555-3412 2400/1200/300 A
2754
@print{} barfly 555-7685 1200/300 A
2755
@print{} bites 555-1675 2400/1200/300 A
2756
@print{} core 555-2912 1200/300 C
2757
@print{} fooey 555-1234 2400/1200/300 B
2758
@print{} foot 555-6699 1200/300 B
2759
@print{} macfoo 555-6480 1200/300 A
2760
@print{} sdace 555-3430 2400/1200/300 A
2761
@print{} sabafoo 555-2127 1200/300 C
2762
@print{} sabafoo 555-2127 1200/300 C
3037
> @kbd{/21/ @{ print $0 @}' mail-list inventory-shipped}
3038
@print{} Anthony 555-3412 anthony.asserturo@@hotmail.com A
3039
@print{} Camilla 555-2912 camilla.infusarum@@skynet.be R
3040
@print{} Fabius 555-1234 fabius.undevicesimus@@ucb.edu F
3041
@print{} Jean-Paul 555-2127 jeanpaul.campanorum@@nyu.edu R
3042
@print{} Jean-Paul 555-2127 jeanpaul.campanorum@@nyu.edu R
2763
3043
@print{} Jan 21 36 64 620
2764
3044
@print{} Apr 21 70 74 514
2765
3045
@end example
2766
3046
2767
3047
@noindent
2768
Note how the line beginning with @samp{sabafoo}
2769
in @file{BBS-list} was printed twice, once for each rule.
3048
Note how the line beginning with @samp{Jean-Paul}
3049
in @file{mail-list} was printed twice, once for each rule.
2770
3050
2771
3051
@node More Complex
2772
3052
@section A More Complex Example
2809
3089
The fifth field contains the size of the file in bytes. The
2810
3090
sixth, seventh, and eighth fields contain the month, day, and time,
2811
3091
respectively, that the file was last modified. Finally, the ninth field
2812
contains the file name.@footnote{The @samp{LC_ALL=C} is
3092
contains the @value{FN}.@footnote{The @samp{LC_ALL=C} is
2813
3093
needed to produce this traditional-style output from @command{ls}.}
2814
3094
2815
3095
@c @cindex automatic initialization
2817
3097
The @samp{$6 == "Nov"} in our @command{awk} program is an expression that
2818
3098
tests whether the sixth field of the output from @w{@samp{ls -l}}
2819
3099
matches the string @samp{Nov}. Each time a line has the string
2820
@samp{Nov} for its sixth field, the action @samp{sum += $5} is
2821
performed. This adds the fifth field (the file's size) to the variable
3100
@samp{Nov} for its sixth field, @command{awk} performs the action
3101
@samp{sum += $5}. This adds the fifth field (the file's size) to the variable
2822
3102
@code{sum}. As a result, when @command{awk} has finished reading all the
2823
3103
input lines, @code{sum} is the total of the sizes of the files whose
2824
3104
lines matched the pattern. (This works because @command{awk} variables
2845
3125
2846
3126
@example
2847
3127
awk '/12/ @{ print $0 @}
2848
/21/ @{ print $0 @}' BBS-list inventory-shipped
3128
/21/ @{ print $0 @}' mail-list inventory-shipped
2849
3129
@end example
2850
3130
2851
3131
@cindex @command{gawk}, newlines in
2885
3165
@command{gawk} places no limit on the
2886
3166
length of a line, so backslash continuation is never strictly necessary;
2887
3167
it just makes programs more readable. For this same reason, as well as
2888
for clarity, we have kept most statements short in the sample programs
3168
for clarity, we have kept most statements short in the programs
2889
3169
presented throughout the @value{DOCUMENT}. Backslash continuation is
2890
3170
most useful when your @command{awk} program is in a separate source file
2891
3171
instead of entered from the command line. You should also note that
2950
3230
> BEGIN rule
2951
3231
> @}'
2952
3232
@error{} gawk: cmd. line:2: BEGIN rule
2953
@error{} gawk: cmd. line:2: ^ parse error
3233
@error{} gawk: cmd. line:2: ^ syntax error
2954
3234
@end example
2955
3235
2956
3236
@noindent
2960
3240
@code{BEGIN} is noted as a syntax error.
2961
3241
2962
3242
@cindex statements, multiple
2963
@cindex @code{;} (semicolon)
2964
@cindex semicolon (@code{;})
3243
@cindex @code{;} (semicolon), separating statements in actions
3244
@cindex semicolon (@code{;}), separating statements in actions
2965
3245
When @command{awk} statements within one rule are short, you might want to put
2966
3246
more than one of them on a line. This is accomplished by separating the statements
2967
3247
with a semicolon (@samp{;}).
3021
3301
can avoid the (usually lengthy) compilation part of the typical
3022
3302
edit-compile-test-debug cycle of software development.
3023
3303
3304
@cindex Brian Kernighan's @command{awk}
3024
3305
Complex programs have been written in @command{awk}, including a complete
3025
retargetable assembler for eight-bit microprocessors (@pxref{Glossary}, for
3026
more information), and a microcode assembler for a special-purpose Prolog
3306
retargetable assembler for
3307
@ifclear FOR_PRINT
3308
eight-bit microprocessors (@pxref{Glossary}, for more information),
3309
@end ifclear
3310
@ifset FOR_PRINT
3311
eight-bit microprocessors,
3312
@end ifset
3313
and a microcode assembler for a special-purpose Prolog
3027
3314
computer.
3028
3315
While the original @command{awk}'s capabilities were strained by tasks
3029
3316
of such complexity, modern versions are more capable. Even Brian Kernighan's
3033
3320
@cindex @command{awk} programs, complex
3034
3321
If you find yourself writing @command{awk} scripts of more than, say, a few
3035
3322
hundred lines, you might consider using a different programming
3036
language. Emacs Lisp is a good choice if you need sophisticated string
3037
or pattern matching capabilities. The shell is also good at string and
3323
language.
3324
The shell is good at string and
3038
3325
pattern matching; in addition, it allows powerful use of the system
3039
3326
utilities. More conventional languages, such as C, C++, and Java, offer
3040
3327
better facilities for system programming and for managing the complexity
3041
of large programs. Programs in these languages may require more lines
3328
of large programs.
3329
Python offers a nice balance between high-level ease of programming and
3330
access to system facilities.
3331
Programs in these languages may require more lines
3042
3332
of source code than the equivalent @command{awk} programs, but they are
3043
3333
easier to maintain and usually run more efficiently.
3044
3334
3335
@node Intro Summary
3336
@section Summary
3337
3338
@itemize @value{BULLET}
3339
@item
3340
Programs in @command{awk} consist of @var{pattern}-@var{action} pairs.
3341
3342
@item
3343
Use either
3344
@samp{awk '@var{program}' @var{files}}
3345
or
3346
@samp{awk -f @var{program-file} @var{files}}
3347
to run @command{awk}.
3348
3349
@item
3350
You may use the special @samp{#!} header line to create @command{awk}
3351
programs that are directly executable.
3352
3353
@item
3354
Comments in @command{awk} programs start with @samp{#} and continue to
3355
the end of the same line.
3356
3357
@item
3358
Be aware of quoting issues when writing @command{awk} programs as
3359
part of a larger shell script (or MS-Windows batch file).
3360
3361
@item
3362
You may use backslash continuation to continue a source line.
3363
Lines are automatically continued after
3364
a comma, open brace, question mark, colon,
3365
@samp{||}, @samp{&&}, @code{do} and @code{else}.
3366
@end itemize
3367
3045
3368
@node Invoking Gawk
3046
3369
@chapter Running @command{awk} and @command{gawk}
3047
3370
3048
This @value{CHAPTER} covers how to run awk, both POSIX-standard
3371
This @value{CHAPTER} covers how to run @command{awk}, both POSIX-standard
3049
3372
and @command{gawk}-specific command-line options, and what
3050
3373
@command{awk} and
3051
3374
@command{gawk} do with non-option arguments.
3070
3393
* Loading Shared Libraries:: Loading shared libraries into your program.
3071
3394
* Obsolete:: Obsolete Options and/or features.
3072
3395
* Undocumented:: Undocumented Options and Features.
3396
* Invoking Summary:: Invocation summary.
3073
3397
@end menu
3074
3398
3075
3399
@node Command Line
3083
3407
one or more program files. Here are templates for both of them; items
3084
3408
enclosed in [@dots{}] in these templates are optional:
3085
3409
3086
@example
3087
awk @r{[@var{options}]} -f progfile @r{[@code{--}]} @var{file} @dots{}
3088
awk @r{[@var{options}]} @r{[@code{--}]} '@var{program}' @var{file} @dots{}
3089
@end example
3410
@display
3411
@command{awk} [@var{options}] @option{-f} @var{progfile} [@option{--}] @var{file} @dots{}
3412
@command{awk} [@var{options}] [@option{--}] @code{'@var{program}'} @var{file} @dots{}
3413
@end display
3090
3414
3091
3415
@cindex GNU long options
3092
3416
@cindex long options
3102
3426
awk '' datafile1 datafile2
3103
3427
@end example
3104
3428
3105
@cindex @code{--lint} option
3429
@cindex @option{--lint} option
3106
3430
@noindent
3107
3431
Doing so makes little sense, though; @command{awk} exits
3108
3432
silently when given an empty program.
3142
3466
@table @code
3143
3467
@item -F @var{fs}
3144
3468
@itemx --field-separator @var{fs}
3145
@cindex @code{-F} option
3146
@cindex @code{--field-separator} option
3469
@cindex @option{-F} option
3470
@cindex @option{--field-separator} option
3147
3471
@cindex @code{FS} variable, @code{--field-separator} option and
3148
3472
Set the @code{FS} variable to @var{fs}
3149
3473
(@pxref{Field Separators}).
3150
3474
3151
3475
@item -f @var{source-file}
3152
3476
@itemx --file @var{source-file}
3153
@cindex @code{-f} option
3154
@cindex @code{--file} option
3477
@cindex @option{-f} option
3478
@cindex @option{--file} option
3155
3479
@cindex @command{awk} programs, location of
3156
3480
Read @command{awk} program source from @var{source-file}
3157
3481
instead of in the first non-option argument.
3158
3482
This option may be given multiple times; the @command{awk}
3159
program consists of the concatenation the contents of
3483
program consists of the concatenation of the contents of
3160
3484
each specified @var{source-file}.
3161
3485
3162
@item -i @var{source-file}
3163
@itemx --include @var{source-file}
3164
@cindex @code{-i} option
3165
@cindex @code{--include} option
3166
@cindex @command{awk} programs, location of
3167
Read @command{awk} source library from @var{source-file}. This option is
3168
completely equivalent to using the @samp{@@include} directive inside
3169
your program. This option is very
3170
similar to the @option{-f} option, but there are two important differences.
3171
First, when @option{-i} is used, the program source will not be loaded if it has
3172
been previously loaded, whereas the @option{-f} will always load the file.
3173
Second, because this option is intended to be used with code libraries,
3174
@command{gawk} does not recognize such files as constituting main program
3175
input. Thus, after processing an @option{-i} argument, @command{gawk} still expects to
3176
find the main source code via the @option{-f} option or on the command-line.
3177
3178
3486
@item -v @var{var}=@var{val}
3179
3487
@itemx --assign @var{var}=@var{val}
3180
@cindex @code{-v} option
3181
@cindex @code{--assign} option
3488
@cindex @option{-v} option
3489
@cindex @option{--assign} option
3182
3490
@cindex variables, setting
3183
3491
Set the variable @var{var} to the value @var{val} @emph{before}
3184
3492
execution of the program begins. Such variable values are available
3199
3507
@end quotation
3200
3508
3201
3509
@item -W @var{gawk-opt}
3202
@cindex @code{-W} option
3510
@cindex @option{-W} option
3203
3511
Provide an implementation-specific option.
3204
3512
This is the POSIX convention for providing implementation-specific options.
3205
3513
These options
3218
3526
3219
3527
@cindex @code{-} (hyphen), filenames beginning with
3220
3528
@cindex hyphen (@code{-}), filenames beginning with
3221
This is useful if you have file names that start with @samp{-},
3222
or in shell scripts, if you have file names that will be specified
3529
This is useful if you have @value{FN}s that start with @samp{-},
3530
or in shell scripts, if you have @value{FN}s that will be specified
3223
3531
by the user that could start with @samp{-}.
3224
3532
It is also useful for passing options on to the @command{awk}
3225
3533
program; see @ref{Getopt Function}.
3229
3537
3230
3538
The following list describes @command{gawk}-specific options:
3231
3539
3232
@table @code
3233
@item -b
3234
@itemx --characters-as-bytes
3235
@cindex @code{-b} option
3236
@cindex @code{--characters-as-bytes} option
3540
@c Have to use @asis here to get docbook to come out right.
3541
@table @asis
3542
@item @option{-b}
3543
@itemx @option{--characters-as-bytes}
3544
@cindex @option{-b} option
3545
@cindex @option{--characters-as-bytes} option
3237
3546
Cause @command{gawk} to treat all input data as single-byte characters.
3238
3547
In addition, all output written with @code{print} or @code{printf}
3239
3548
are treated as single-byte characters.
3240
3549
3241
3550
Normally, @command{gawk} follows the POSIX standard and attempts to process
3242
its input data according to the current locale. This can often involve
3551
its input data according to the current locale (@pxref{Locales}). This can often involve
3243
3552
converting multibyte characters into wide characters (internally), and
3244
3553
can lead to problems or confusion if the input data does not contain valid
3245
3554
multibyte characters. This option is an easy way to tell @command{gawk}:
3246
3555
``hands off my data!''.
3247
3556
3248
@item -c
3249
@itemx --traditional
3250
@cindex @code{--c} option
3251
@cindex @code{--traditional} option
3557
@item @option{-c}
3558
@itemx @option{--traditional}
3559
@cindex @option{-c} option
3560
@cindex @option{--traditional} option
3252
3561
@cindex compatibility mode (@command{gawk}), specifying
3253
3562
Specify @dfn{compatibility mode}, in which the GNU extensions to
3254
3563
the @command{awk} language are disabled, so that @command{gawk} behaves just
3255
3564
like Brian Kernighan's version @command{awk}.
3256
3565
@xref{POSIX/GNU},
3257
which summarizes the extensions. Also see
3566
which summarizes the extensions.
3567
@ifclear FOR_PRINT
3568
Also see
3258
3569
@ref{Compatibility Mode}.
3570
@end ifclear
3259
3571
3260
@item -C
3261
@itemx --copyright
3262
@cindex @code{-C} option
3263
@cindex @code{--copyright} option
3572
@item @option{-C}
3573
@itemx @option{--copyright}
3574
@cindex @option{-C} option
3575
@cindex @option{--copyright} option
3264
3576
@cindex GPL (General Public License), printing
3265
3577
Print the short version of the General Public License and then exit.
3266
3578
3267
@item -d@r{[}@var{file}@r{]}
3268
@itemx --dump-variables@r{[}=@var{file}@r{]}
3269
@cindex @code{-d} option
3270
@cindex @code{--dump-variables} option
3271
@cindex @code{awkvars.out} file
3272
@cindex files, @code{awkvars.out}
3579
@item @option{-d}[@var{file}]
3580
@itemx @option{--dump-variables}[@code{=}@var{file}]
3581
@cindex @option{-d} option
3582
@cindex @option{--dump-variables} option
3583
@cindex dump all variables of a program
3584
@cindex @file{awkvars.out} file
3585
@cindex files, @file{awkvars.out}
3273
3586
@cindex variables, global, printing list of
3274
3587
Print a sorted list of global variables, their types, and final values
3275
3588
to @var{file}. If no @var{file} is provided, print this
3286
3599
(This is a particularly easy mistake to make with simple variable
3287
3600
names like @code{i}, @code{j}, etc.)
3288
3601
3289
@item -D@r{[}@var{file}@r{]}
3290
@itemx --debug=@r{[}@var{file}@r{]}
3291
@cindex @code{-D} option
3292
@cindex @code{--debug} option
3602
@item @option{-D}[@var{file}]
3603
@itemx @option{--debug}[@code{=}@var{file}]
3604
@cindex @option{-D} option
3605
@cindex @option{--debug} option
3293
3606
@cindex @command{awk} debugging, enabling
3294
3607
Enable debugging of @command{awk} programs
3295
3608
(@pxref{Debugging}).
3296
By default, the debugger reads commands interactively from the terminal.
3609
By default, the debugger reads commands interactively from the keyboard.
3297
3610
The optional @var{file} argument allows you to specify a file with a list
3298
3611
of commands for the debugger to execute non-interactively.
3299
3612
No space is allowed between the @option{-D} and @var{file}, if
3300
3613
@var{file} is supplied.
3301
3614
3302
@item -e @var{program-text}
3303
@itemx --source @var{program-text}
3304
@cindex @code{-e} option
3305
@cindex @code{--source} option
3615
@item @option{-e} @var{program-text}
3616
@itemx @option{--source} @var{program-text}
3617
@cindex @option{-e} option
3618
@cindex @option{--source} option
3306
3619
@cindex source code, mixing
3307
3620
Provide program source code in the @var{program-text}.
3308
3621
This option allows you to mix source code in files with source
3311
3624
when you have library functions that you want to use from your command-line
3312
3625
programs (@pxref{AWKPATH Variable}).
3313
3626
3314
@item -E @var{file}
3315
@itemx --exec @var{file}
3316
@cindex @code{-E} option
3317
@cindex @code{--exec} option
3627
@item @option{-E} @var{file}
3628
@itemx @option{--exec} @var{file}
3629
@cindex @option{-E} option
3630
@cindex @option{--exec} option
3318
3631
@cindex @command{awk} programs, location of
3319
3632
@cindex CGI, @command{awk} scripts for
3320
3633
Similar to @option{-f}, read @command{awk} program text from @var{file}.
3321
3634
There are two differences from @option{-f}:
3322
3635
3323
@itemize @bullet
3636
@itemize @value{BULLET}
3324
3637
@item
3325
3638
This option terminates option processing; anything
3326
3639
else on the command line is passed on directly to the @command{awk} program.
3342
3655
@var{awk program here @dots{}}
3343
3656
@end example
3344
3657
3345
@item -g
3346
@itemx --gen-pot
3347
@cindex @code{-g} option
3348
@cindex @code{--gen-pot} option
3658
@item @option{-g}
3659
@itemx @option{--gen-pot}
3660
@cindex @option{-g} option
3661
@cindex @option{--gen-pot} option
3349
3662
@cindex portable object files, generating
3350
3663
@cindex files, portable object, generating
3351
3664
Analyze the source program and
3352
generate a GNU @code{gettext} Portable Object Template file on standard
3665
generate a GNU @command{gettext} Portable Object Template file on standard
3353
3666
output for all string constants that have been marked for translation.
3354
3667
@xref{Internationalization},
3355
3668
for information about this option.
3356
3669
3357
@item -h
3358
@itemx --help
3359
@cindex @code{-h} option
3360
@cindex @code{--help} option
3670
@item @option{-h}
3671
@itemx @option{--help}
3672
@cindex @option{-h} option
3673
@cindex @option{--help} option
3361
3674
@cindex GNU long options, printing list of
3362
3675
@cindex options, printing list of
3363
3676
@cindex printing, list of options
3364
3677
Print a ``usage'' message summarizing the short and long style options
3365
3678
that @command{gawk} accepts and then exit.
3366
3679
3367
@item -l @var{lib}
3368
@itemx --load @var{lib}
3369
@cindex @code{-l} option
3370
@cindex @code{--load} option
3371
@cindex loading, library
3372
Load a shared library @var{lib}. This searches for the library using the @env{AWKLIBPATH}
3680
@item @option{-i} @var{source-file}
3681
@itemx @option{--include} @var{source-file}
3682
@cindex @option{-i} option
3683
@cindex @option{--include} option
3684
@cindex @command{awk} programs, location of
3685
Read @command{awk} source library from @var{source-file}. This option
3686
is completely equivalent to using the @code{@@include} directive inside
3687
your program. This option is very similar to the @option{-f} option,
3688
but there are two important differences. First, when @option{-i} is
3689
used, the program source is not loaded if it has been previously
3690
loaded, whereas with @option{-f}, @command{gawk} always loads the file.
3691
Second, because this option is intended to be used with code libraries,
3692
@command{gawk} does not recognize such files as constituting main program
3693
input. Thus, after processing an @option{-i} argument, @command{gawk}
3694
still expects to find the main source code via the @option{-f} option
3695
or on the command-line.
3696
3697
@item @option{-l} @var{ext}
3698
@itemx @option{--load} @var{ext}
3699
@cindex @option{-l} option
3700
@cindex @option{--load} option
3701
@cindex loading, extensions
3702
Load a dynamic extension named @var{ext}. Extensions
3703
are stored as system shared libraries.
3704
This option searches for the library using the @env{AWKLIBPATH}
3373
3705
environment variable. The correct library suffix for your platform will be
3374
supplied by default, so it need not be specified in the library name.
3375
The library initialization routine should be named @code{dl_load()}.
3376
An alternative is to use the @samp{@@load} keyword inside the program to load
3377
a shared library.
3706
supplied by default, so it need not be specified in the extension name.
3707
The extension initialization routine should be named @code{dl_load()}.
3708
An alternative is to use the @code{@@load} keyword inside the program to load
3709
a shared library. This feature is described in detail in @ref{Dynamic Extensions}.
3378
3710
3379
@item -L @r{[}value@r{]}
3380
@itemx --lint@r{[}=value@r{]}
3381
@cindex @code{-l} option
3382
@cindex @code{--lint} option
3711
@item @option{-L}[@var{value}]
3712
@itemx @option{--lint}[@code{=}@var{value}]
3713
@cindex @option{-l} option
3714
@cindex @option{--lint} option
3383
3715
@cindex lint checking, issuing warnings
3384
3716
@cindex warnings, issuing
3385
3717
Warn about constructs that are dubious or nonportable to
3386
3718
other @command{awk} implementations.
3719
No space is allowed between the @option{-D} and @var{value}, if
3720
@var{value} is supplied.
3387
3721
Some warnings are issued when @command{gawk} first reads your program. Others
3388
3722
are issued at runtime, as your program executes.
3389
3723
With an optional argument of @samp{fatal},
3399
3733
care to search for all occurrences of each inappropriate construct. As
3400
3734
@command{awk} programs are usually short, doing so is not burdensome.
3401
3735
3402
@item -M
3403
@itemx --bignum
3404
@cindex @code{-M} option
3405
@cindex @code{--bignum} option
3736
@item @option{-M}
3737
@itemx @option{--bignum}
3738
@cindex @option{-M} option
3739
@cindex @option{--bignum} option
3406
3740
Force arbitrary precision arithmetic on numbers. This option has no effect
3407
3741
if @command{gawk} is not compiled to use the GNU MPFR and MP libraries
3408
3742
(@pxref{Arbitrary Precision Arithmetic}).
3409
3743
3410
@item -n
3411
@itemx --non-decimal-data
3412
@cindex @code{-n} option
3413
@cindex @code{--non-decimal-data} option
3744
@item @option{-n}
3745
@itemx @option{--non-decimal-data}
3746
@cindex @option{-n} option
3747
@cindex @option{--non-decimal-data} option
3414
3748
@cindex hexadecimal values@comma{} enabling interpretation of
3415
3749
@cindex octal values@comma{} enabling interpretation of
3416
3750
@cindex troubleshooting, @code{--non-decimal-data} option
3423
3757
Use with care.
3424
3758
@end quotation
3425
3759
3426
@item -N
3427
@itemx --use-lc-numeric
3428
@cindex @code{-N} option
3429
@cindex @code{--use-lc-numeric} option
3760
@item @option{-N}
3761
@itemx @option{--use-lc-numeric}
3762
@cindex @option{-N} option
3763
@cindex @option{--use-lc-numeric} option
3430
3764
Force the use of the locale's decimal point character
3431
3765
when parsing numeric input data (@pxref{Locales}).
3432
3766
3433
@item -o@r{[}@var{file}@r{]}
3434
@itemx --pretty-print@r{[}=@var{file}@r{]}
3435
@cindex @code{-o} option
3436
@cindex @code{--pretty-print} option
3767
@item @option{-o}[@var{file}]
3768
@itemx @option{--pretty-print}[@code{=}@var{file}]
3769
@cindex @option{-o} option
3770
@cindex @option{--pretty-print} option
3437
3771
Enable pretty-printing of @command{awk} programs.
3438
By default, output program is created in a file named @file{awkprof.out}.
3772
By default, output program is created in a file named @file{awkprof.out}
3773
(@pxref{Profiling}).
3439
3774
The optional @var{file} argument allows you to specify a different
3440
file name for the output.
3775
@value{FN} for the output.
3441
3776
No space is allowed between the @option{-o} and @var{file}, if
3442
3777
@var{file} is supplied.
3443
3778
3444
@item -O
3445
@itemx --optimize
3446
@cindex @code{--optimize} option
3447
@cindex @code{-O} option
3779
@quotation NOTE
3780
Due to the way @command{gawk} has evolved, with this option
3781
your program is still executed. This will change in the
3782
next major release such that @command{gawk} will only
3783
pretty-print the program and not run it.
3784
@end quotation
3785
3786
@item @option{-O}
3787
@itemx @option{--optimize}
3788
@cindex @option{--optimize} option
3789
@cindex @option{-O} option
3448
3790
Enable some optimizations on the internal representation of the program.
3449
At the moment this includes just simple constant folding. The @command{gawk}
3450
maintainer hopes to add more optimizations over time.
3791
At the moment this includes just simple constant folding.
3451
3792
3452
@item -p@r{[}@var{file}@r{]}
3453
@itemx --profile@r{[}=@var{file}@r{]}
3454
@cindex @code{-p} option
3455
@cindex @code{--profile} option
3793
@item @option{-p}[@var{file}]
3794
@itemx @option{--profile}[@code{=}@var{file}]
3795
@cindex @option{-p} option
3796
@cindex @option{--profile} option
3456
3797
@cindex @command{awk} profiling, enabling
3457
3798
Enable profiling of @command{awk} programs
3458
3799
(@pxref{Profiling}).
3459
3800
By default, profiles are created in a file named @file{awkprof.out}.
3460
3801
The optional @var{file} argument allows you to specify a different
3461
file name for the profile file.
3802
@value{FN} for the profile file.
3462
3803
No space is allowed between the @option{-p} and @var{file}, if
3463
3804
@var{file} is supplied.
3464
3805
3465
3806
The profile contains execution counts for each statement in the program
3466
3807
in the left margin, and function call counts for each function.
3467
3808
3468
@item -P
3469
@itemx --posix
3470
@cindex @code{-P} option
3471
@cindex @code{--posix} option
3809
@item @option{-P}
3810
@itemx @option{--posix}
3811
@cindex @option{-P} option
3812
@cindex @option{--posix} option
3472
3813
@cindex POSIX mode
3473
3814
@cindex @command{gawk}, extensions@comma{} disabling
3474
3815
Operate in strict POSIX mode. This disables all @command{gawk}
3480
3821
the following additional
3481
3822
restrictions apply:
3482
3823
3483
@itemize @bullet
3824
@itemize @value{BULLET}
3484
3825
3485
3826
@cindex newlines
3486
3827
@cindex whitespace, newlines as
3509
3850
3510
3851
@c @cindex automatic warnings
3511
3852
@c @cindex warnings, automatic
3512
@cindex @code{--traditional} option, @code{--posix} option and
3513
@cindex @code{--posix} option, @code{--traditional} option and
3853
@cindex @option{--traditional} option, @code{--posix} option and
3854
@cindex @option{--posix} option, @code{--traditional} option and
3514
3855
If you supply both @option{--traditional} and @option{--posix} on the
3515
3856
command line, @option{--posix} takes precedence. @command{gawk}
3516
also issues a warning if both options are supplied.
3857
issues a warning if both options are supplied.
3517
3858
3518
@item -r
3519
@itemx --re-interval
3520
@cindex @code{-r} option
3521
@cindex @code{--re-interval} option
3859
@item @option{-r}
3860
@itemx @option{--re-interval}
3861
@cindex @option{-r} option
3862
@cindex @option{--re-interval} option
3522
3863
@cindex regular expressions, interval expressions and
3523
3864
Allow interval expressions
3524
3865
(@pxref{Regexp Operators})
3525
3866
in regexps.
3526
3867
This is now @command{gawk}'s default behavior.
3527
3868
Nevertheless, this option remains both for backward compatibility,
3528
and for use in combination with the @option{--traditional} option.
3869
and for use in combination with @option{--traditional}.
3529
3870
3530
@item -S
3531
@itemx --sandbox
3532
@cindex @code{-S} option
3533
@cindex @code{--sandbox} option
3871
@item @option{-S}
3872
@itemx @option{--sandbox}
3873
@cindex @option{-S} option
3874
@cindex @option{--sandbox} option
3534
3875
@cindex sandbox mode
3535
3876
Disable the @code{system()} function,
3536
3877
input redirections with @code{getline},
3538
3879
and dynamic extensions.
3539
3880
This is particularly useful when you want to run @command{awk} scripts
3540
3881
from questionable sources and need to make sure the scripts
3541
can't access your system (other than the specified input data file).
3882
can't access your system (other than the specified input @value{DF}).
3542
3883
3543
@item -t
3544
@itemx --lint-old
3545
@cindex @code{--L} option
3546
@cindex @code{--lint-old} option
3884
@item @option{-t}
3885
@itemx @option{--lint-old}
3886
@cindex @option{-L} option
3887
@cindex @option{--lint-old} option
3547
3888
Warn about constructs that are not available in the original version of
3548
3889
@command{awk} from Version 7 Unix
3549
3890
(@pxref{V7/SVR3.1}).
3550
3891
3551
@item -V
3552
@itemx --version
3553
@cindex @code{-V} option
3554
@cindex @code{--version} option
3892
@item @option{-V}
3893
@itemx @option{--version}
3894
@cindex @option{-V} option
3895
@cindex @option{--version} option
3555
3896
@cindex @command{gawk}, versions of, information about@comma{} printing
3556
3897
Print version information for this particular copy of @command{gawk}.
3557
3898
This allows you to determine if your copy of @command{gawk} is up to date
3565
3906
any other options are flagged as invalid with a warning message but
3566
3907
are otherwise ignored.
3567
3908
3568
@cindex @code{-F} option, @code{-Ft} sets @code{FS} to TAB
3909
@cindex @option{-F} option, @option{-Ft} sets @code{FS} to TAB
3569
3910
In compatibility mode, as a special case, if the value of @var{fs} supplied
3570
3911
to the @option{-F} option is @samp{t}, then @code{FS} is set to the TAB
3571
3912
character (@code{"\t"}). This is true only for @option{--traditional} and not
3572
3913
for @option{--posix}
3573
3914
(@pxref{Field Separators}).
3574
3915
3575
@cindex @code{-f} option, multiple uses
3916
@cindex @option{-f} option, multiple uses
3576
3917
The @option{-f} option may be used more than once on the command line.
3577
3918
If it is, @command{awk} reads its program source from all of the named files, as
3578
3919
if they had been concatenated together into one big file. This is
3584
3925
function names must be unique.)
3585
3926
3586
3927
With standard @command{awk}, library functions can still be used, even
3587
if the program is entered at the terminal,
3928
if the program is entered at the keyboard,
3588
3929
by specifying @samp{-f /dev/tty}. After typing your program,
3589
3930
type @kbd{Ctrl-d} (the end-of-file character) to terminate it.
3590
3931
(You may also use @samp{-f -} to read program source from the standard
3591
3932
input but then you will not be able to also use the standard input as a
3592
3933
source of data.)
3593
3934
3594
Because it is clumsy using the standard @command{awk} mechanisms to mix source
3595
file and command-line @command{awk} programs, @command{gawk} provides the
3596
@option{--source} option. This does not require you to pre-empt the standard
3597
input for your source code; it allows you to easily mix command-line
3598
and library source code
3599
(@pxref{AWKPATH Variable}).
3600
The @option{--source} option may also be used multiple times on the command line.
3935
Because it is clumsy using the standard @command{awk} mechanisms to mix
3936
source file and command-line @command{awk} programs, @command{gawk}
3937
provides the @option{--source} option. This does not require you to
3938
pre-empt the standard input for your source code; it allows you to easily
3939
mix command-line and library source code (@pxref{AWKPATH Variable}).
3940
As with @option{-f}, the @option{--source} and @option{--include}
3941
options may also be used multiple times on the command line.
3601
3942
3602
@cindex @code{--source} option
3943
@cindex @option{--source} option
3603
3944
If no @option{-f} or @option{--source} option is specified, then @command{gawk}
3604
3945
uses the first non-option command-line argument as the text of the
3605
3946
program source code.
3609
3950
@cindex POSIX mode
3610
3951
If the environment variable @env{POSIXLY_CORRECT} exists,
3611
3952
then @command{gawk} behaves in strict POSIX mode, exactly as if
3612
you had supplied the @option{--posix} command-line option.
3953
you had supplied @option{--posix}.
3613
3954
Many GNU programs look for this environment variable to suppress
3614
3955
extensions that conflict with POSIX, but @command{gawk} behaves
3615
3956
differently: it suppresses all extensions, even those that do not
3658
3999
3659
4000
@cindex @command{gawk}, @code{ARGIND} variable in
3660
4001
@cindex @code{ARGIND} variable, command-line arguments
4002
@cindex @code{ARGV} array, indexing into
3661
4003
@cindex @code{ARGC}/@code{ARGV} variables, command-line arguments
3662
4004
All these arguments are made available to your @command{awk} program in the
3663
4005
@code{ARGV} array (@pxref{Built-in Variables}). Command-line options
3668
4010
current element.
3669
4011
3670
4012
@cindex input files, variable assignments and
3671
The distinction between file name arguments and variable-assignment
4013
@cindex variable assignments and input files
4014
The distinction between @value{FN} arguments and variable-assignment
3672
4015
arguments is made when @command{awk} is about to open the next input file.
3673
At that point in execution, it checks the file name to see whether
4016
At that point in execution, it checks the @value{FN} to see whether
3674
4017
it is really a variable assignment; if so, @command{awk} sets the variable
3675
4018
instead of reading a file.
3676
4019
3686
4029
sequences (@pxref{Escape Sequences}).
3687
4030
@value{DARKCORNER}
3688
4031
3689
In some earlier implementations of @command{awk}, when a variable assignment
3690
occurred before any file names, the assignment would happen @emph{before}
4032
In some very early implementations of @command{awk}, when a variable assignment
4033
occurred before any @value{FN}s, the assignment would happen @emph{before}
3691
4034
the @code{BEGIN} rule was executed. @command{awk}'s behavior was thus
3692
4035
inconsistent; some command-line assignments were available inside the
3693
4036
@code{BEGIN} rule, while others were not. Unfortunately,
3698
4041
3699
4042
The variable assignment feature is most useful for assigning to variables
3700
4043
such as @code{RS}, @code{OFS}, and @code{ORS}, which control input and
3701
output formats before scanning the data files. It is also useful for
3702
controlling state if multiple passes are needed over a data file. For
4044
output formats, before scanning the @value{DF}s. It is also useful for
4045
controlling state if multiple passes are needed over a @value{DF}. For
3703
4046
example:
3704
4047
3705
4048
@cindex files, multiple passes over
3735
4078
files with @code{getline} (@pxref{Getline/File}).
3736
4079
3737
4080
In addition, @command{gawk} allows you to specify the special
3738
file name @file{/dev/stdin}, both on the command line and
4081
@value{FN} @file{/dev/stdin}, both on the command line and
3739
4082
with @code{getline}.
3740
4083
Some other versions of @command{awk} also support this, but it
3741
4084
is not standard.
3742
4085
(Some operating systems provide a @file{/dev/stdin} file
3743
in the file system, however, @command{gawk} always processes
3744
this file name itself.)
4086
in the file system; however, @command{gawk} always processes
4087
this @value{FN} itself.)
3745
4088
3746
4089
@node Environment Variables
3747
4090
@section The Environment Variables @command{gawk} Uses
4091
@cindex environment variables used by @command{gawk}
3748
4092
3749
4093
A number of environment variables influence how @command{gawk}
3750
4094
behaves.
3760
4104
@node AWKPATH Variable
3761
4105
@subsection The @env{AWKPATH} Environment Variable
3762
4106
@cindex @env{AWKPATH} environment variable
3763
@cindex directories, searching
3764
@cindex search paths
4107
@cindex directories, searching for source files
3765
4108
@cindex search paths, for source files
3766
4109
@cindex differences in @command{awk} and @command{gawk}, @code{AWKPATH} environment variable
3767
4110
@ifinfo
3771
4114
In most @command{awk}
3772
4115
implementations, you must supply a precise path name for each program
3773
4116
file, unless the file is in the current directory.
3774
But in @command{gawk}, if the file name supplied to the @option{-f}
4117
But in @command{gawk}, if the @value{FN} supplied to the @option{-f}
3775
4118
or @option{-i} options
3776
does not contain a @samp{/}, then @command{gawk} searches a list of
4119
does not contain a directory separator @samp{/}, then @command{gawk} searches a list of
3777
4120
directories (called the @dfn{search path}), one by one, looking for a
3778
4121
file with the specified name.
3779
4122
3780
4123
The search path is a string consisting of directory names
3781
separated by colons. @command{gawk} gets its search path from the
4124
separated by colons@footnote{Semicolons on MS-Windows and MS-DOS.}. @command{gawk} gets its search path from the
3782
4125
@env{AWKPATH} environment variable. If that variable does not exist,
3783
4126
@command{gawk} uses a default path,
3784
4127
@samp{.:/usr/local/share/awk}.@footnote{Your version of @command{gawk}
3788
4131
@command{gawk} was configured. You probably don't need to worry about this,
3789
4132
though.}
3790
4133
3791
The search path feature is particularly useful for building libraries
4134
The search path feature is particularly helpful for building libraries
3792
4135
of useful @command{awk} functions. The library files can be placed in a
3793
4136
standard directory in the default path and then specified on
3794
the command line with a short file name. Otherwise, the full file name
4137
the command line with a short @value{FN}. Otherwise, the full @value{FN}
3795
4138
would have to be typed for each file.
3796
4139
3797
4140
By using the @option{-i} option, or the @option{--source} and @option{-f} options, your command-line
3802
4145
@xref{Options}.
3803
4146
3804
4147
If the source code is not found after the initial search, the path is searched
3805
again after adding the default @samp{.awk} suffix to the filename.
4148
again after adding the default @samp{.awk} suffix to the @value{FN}.
3806
4149
3807
4150
@quotation NOTE
4151
@c 4/2014:
4152
@c using @samp{.} to get quotes, since @file{} no longer supplies them.
3808
4153
To include
3809
4154
the current directory in the path, either place
3810
@file{.} explicitly in the path or write a null entry in the
4155
@samp{.} explicitly in the path or write a null entry in the
3811
4156
path. (A null entry is indicated by starting or ending the path with a
3812
colon or by placing two colons next to each other (@samp{::}).)
4157
colon or by placing two colons next to each other [@samp{::}].)
3813
4158
This path search mechanism is similar
3814
4159
to the shell's.
3815
@c someday, @cite{The Bourne Again Shell}....
4160
(See @uref{http://www.gnu.org/software/bash/manual/,
4161
@cite{The Bourne-Again SHell manual}.})
3816
4162
3817
4163
However, @command{gawk} always looks in the current directory @emph{before}
3818
4164
searching @env{AWKPATH}, so there is no real reason to include
3824
4170
If @env{AWKPATH} is not defined in the
3825
4171
environment, @command{gawk} places its default search path into
3826
4172
@code{ENVIRON["AWKPATH"]}. This makes it easy to determine
3827
the actual search path that @command{gawk} will use
4173
the actual search path that @command{gawk} used
3828
4174
from within an @command{awk} program.
3829
4175
3830
4176
While you can change @code{ENVIRON["AWKPATH"]} within your @command{awk}
3836
4182
@node AWKLIBPATH Variable
3837
4183
@subsection The @env{AWKLIBPATH} Environment Variable
3838
4184
@cindex @env{AWKLIBPATH} environment variable
3839
@cindex directories, searching
3840
@cindex search paths
3841
@cindex search paths, for shared libraries
4185
@cindex directories, searching for loadable extensions
4186
@cindex search paths, for loadable extensions
3842
4187
@cindex differences in @command{awk} and @command{gawk}, @code{AWKLIBPATH} environment variable
3843
4188
3844
4189
The @env{AWKLIBPATH} environment variable is similar to the @env{AWKPATH}
3845
variable, but it is used to search for shared libraries specified
3846
with the @option{-l} option rather than for source files. If the library
3847
is not found, the path is searched again after adding the appropriate
3848
shared library suffix for the platform. For example, on GNU/Linux systems,
3849
the suffix @samp{.so} is used.
3850
The search path specified is also used for libraries loaded via the
3851
@samp{@@load} keyword (@pxref{Loading Shared Libraries}).
4190
variable, but it is used to search for loadable extensions (stored as
4191
system shared libraries) specified with the @option{-l} option rather
4192
than for source files. If the extension is not found, the path is
4193
searched again after adding the appropriate shared library suffix for
4194
the platform. For example, on GNU/Linux systems, the suffix @samp{.so}
4195
is used. The search path specified is also used for extensions loaded
4196
via the @code{@@load} keyword (@pxref{Loading Shared Libraries}).
3852
4197
3853
4198
@node Other Environment Variables
3854
4199
@subsection Other Environment Variables
3864
4209
@xref{Options}.
3865
4210
3866
4211
@item GAWK_SOCK_RETRIES
3867
Controls the number of time @command{gawk} will attempt to
4212
Controls the number of times @command{gawk} attempts to
3868
4213
retry a two-way TCP/IP (socket) connection before giving up.
3869
4214
@xref{TCP/IP Networking}.
3870
4215
3885
4230
They are subject to change. The variables are:
3886
4231
3887
4232
@table @env
4233
@item AWKBUFSIZE
4234
This variable only affects @command{gawk} on POSIX-compliant systems.
4235
With a value of @samp{exact}, @command{gawk} uses the size of each input
4236
file as the size of the memory buffer to allocate for I/O. Otherwise,
4237
the value should be a number, and @command{gawk} uses that number as
4238
the size of the buffer to allocate. (When this variable is not set,
4239
@command{gawk} uses the smaller of the file's size and the ``default''
4240
blocksize, which is usually the file systems I/O blocksize.)
4241
3888
4242
@item AWK_HASH
3889
4243
If this variable exists with a value of @samp{gst}, @command{gawk}
3890
will switch to using the hash function from GNU Smalltalk for
4244
switches to using the hash function from GNU Smalltalk for
3891
4245
managing arrays.
3892
4246
This function may be marginally faster than the standard function.
3893
4247
3912
4266
supposed to be differences, but occasionally theory and practice don't
3913
4267
coordinate with each other.)
3914
4268
4269
@item GAWK_NO_PP_RUN
4270
If this variable exists, then when invoked with the @option{--pretty-print}
4271
option, @command{gawk} skips running the program. This variable will
4272
not survive into the next major release.
4273
3915
4274
@item GAWK_STACKSIZE
3916
4275
This specifies the amount by which @command{gawk} should grow its
3917
4276
internal evaluation stack, when needed.
3956
4315
3957
4316
This @value{SECTION} describes a feature that is specific to @command{gawk}.
3958
4317
3959
The @samp{@@include} keyword can be used to read external @command{awk} source
4318
The @code{@@include} keyword can be used to read external @command{awk} source
3960
4319
files. This gives you the ability to split large @command{awk} source files
3961
4320
into smaller, more manageable pieces, and also lets you reuse common @command{awk}
3962
4321
code from various @command{awk} scripts. In other words, you can group
3963
4322
together @command{awk} functions, used to carry out specific tasks,
3964
4323
into external files. These files can be used just like function libraries,
3965
using the @samp{@@include} keyword in conjunction with the @env{AWKPATH}
4324
using the @code{@@include} keyword in conjunction with the @env{AWKPATH}
3966
4325
environment variable. Note that source files may also be included
3967
4326
using the @option{-i} option.
3968
4327
3996
4355
@end example
3997
4356
3998
4357
@code{gawk} runs the @file{test2} script which includes @file{test1}
3999
using the @samp{@@include}
4358
using the @code{@@include}
4000
4359
keyword. So, to include external @command{awk} source files you just
4001
use @samp{@@include} followed by the name of the file to be included,
4360
use @code{@@include} followed by the name of the file to be included,
4002
4361
enclosed in double quotes.
4003
4362
4004
4363
@quotation NOTE
4005
Keep in mind that this is a language construct and the file name cannot
4006
be a string variable, but rather just a literal string in double quotes.
4364
Keep in mind that this is a language construct and the @value{FN} cannot
4365
be a string variable, but rather just a literal string constant in double quotes.
4007
4366
@end quotation
4008
4367
4009
4368
The files to be included may be nested; e.g., given a third
4027
4386
@print{} This is file test3.
4028
4387
@end example
4029
4388
4030
The file name can, of course, be a pathname. For example:
4389
The @value{FN} can, of course, be a pathname. For example:
4031
4390
4032
4391
@example
4033
4392
@@include "../io_funcs"
4042
4401
4043
4402
@noindent
4044
4403
are valid. The @code{AWKPATH} environment variable can be of great
4045
value when using @samp{@@include}. The same rules for the use
4404
value when using @code{@@include}. The same rules for the use
4046
4405
of the @code{AWKPATH} variable in command-line file searches
4047
4406
(@pxref{AWKPATH Variable}) apply to
4048
@samp{@@include} also.
4407
@code{@@include} also.
4049
4408
4050
4409
This is very helpful in constructing @command{gawk} function libraries.
4051
4410
If you have a large script with useful, general purpose @command{awk}
4052
4411
functions, you can break it down into library files and put those files
4053
4412
in a special directory. You can then include those ``libraries,'' using
4054
4413
either the full pathnames of the files, or by setting the @code{AWKPATH}
4055
environment variable accordingly and then using @samp{@@include} with
4414
environment variable accordingly and then using @code{@@include} with
4056
4415
just the file part of the full pathname. Of course you can have more
4057
4416
than one directory to keep library files; the more complex the working
4058
4417
environment is, the more directories you may need to organize the files
4059
4418
to be included.
4060
4419
4061
4420
Given the ability to specify multiple @option{-f} options, the
4062
@samp{@@include} mechanism is not strictly necessary.
4063
However, the @samp{@@include} keyword
4421
@code{@@include} mechanism is not strictly necessary.
4422
However, the @code{@@include} keyword
4064
4423
can help you in constructing self-contained @command{gawk} programs,
4065
4424
thus reducing the need for writing complex and tedious command lines.
4066
In particular, @samp{@@include} is very useful for writing CGI scripts
4425
In particular, @code{@@include} is very useful for writing CGI scripts
4067
4426
to be run from web pages.
4068
4427
4069
4428
As mentioned in @ref{AWKPATH Variable}, the current directory is always
4070
4429
searched first for source files, before searching in @env{AWKPATH},
4071
and this also applies to files named with @samp{@@include}.
4430
and this also applies to files named with @code{@@include}.
4072
4431
4073
4432
@node Loading Shared Libraries
4074
@section Loading Shared Libraries Into Your Program
4433
@section Loading Dynamic Extensions Into Your Program
4075
4434
4076
4435
This @value{SECTION} describes a feature that is specific to @command{gawk}.
4077
4436
4078
The @samp{@@load} keyword can be used to read external @command{awk} shared
4079
libraries. This allows you to link in compiled code that may offer superior
4437
The @code{@@load} keyword can be used to read external @command{awk} extensions
4438
(stored as system shared libraries).
4439
This allows you to link in compiled code that may offer superior
4080
4440
performance and/or give you access to extended capabilities not supported
4081
4441
by the @command{awk} language. The @env{AWKLIBPATH} variable is used to
4082
search for the shared library. Using @samp{@@load} is completely equivalent
4442
search for the extension. Using @code{@@load} is completely equivalent
4083
4443
to using the @option{-l} command-line option.
4084
4444
4085
If the shared library is not initially found in @env{AWKLIBPATH}, another
4445
If the extension is not initially found in @env{AWKLIBPATH}, another
4086
4446
search is conducted after appending the platform's default shared library
4087
suffix to the filename. For example, on GNU/Linux systems, the suffix
4447
suffix to the @value{FN}. For example, on GNU/Linux systems, the suffix
4088
4448
@samp{.so} is used.
4089
4449
4090
4450
@example
4102
4462
4103
4463
@noindent
4104
4464
For command-line usage, the @option{-l} option is more convenient,
4105
but @samp{@@load} is useful for embedding inside an @command{awk} source file
4106
that requires access to a shared library.
4465
but @code{@@load} is useful for embedding inside an @command{awk} source file
4466
that requires access to an extension.
4107
4467
4108
4468
@ref{Dynamic Extensions}, describes how to write extensions (in C or C++)
4109
that can be loaded with either @samp{@@load} or the @option{-l} option.
4469
that can be loaded with either @code{@@load} or the @option{-l} option.
4110
4470
4111
4471
@node Obsolete
4112
4472
@section Obsolete Options and/or Features
4113
4473
4114
@cindex features, advanced, See advanced features
4474
@c update this section for each release!
4475
4115
4476
@cindex options, deprecated
4116
4477
@cindex features, deprecated
4117
4478
@cindex obsolete features
4120
4481
current version or that are still supported but deprecated (meaning that
4121
4482
they will @emph{not} be in the next release).
4122
4483
4123
@c update this section for each release!
4124
4125
@cindex @code{PROCINFO} array
4126
4484
The process-related special files @file{/dev/pid}, @file{/dev/ppid},
4127
4485
@file{/dev/pgrpid}, and @file{/dev/user} were deprecated in @command{gawk}
4128
3.1, but still worked. As of version 4.0, they are no longer
4486
3.1, but still worked. As of @value{PVERSION} 4.0, they are no longer
4129
4487
interpreted specially by @command{gawk}. (Use @code{PROCINFO} instead;
4130
4488
see @ref{Auto-set}.)
4131
4489
4148
4506
@author Obi-Wan
4149
4507
@end quotation
4150
4508
4509
@cindex shells, sea
4151
4510
This @value{SECTION} intentionally left
4152
4511
blank.
4153
4512
4160
4519
@table @code
4161
4520
@item -W nostalgia
4162
4521
@itemx --nostalgia
4163
Print the message @code{"awk: bailing out near line 1"} and dump core.
4522
Print the message @samp{awk: bailing out near line 1} and dump core.
4164
4523
This option was inspired by the common behavior of very early versions of
4165
4524
Unix @command{awk} and by a t--shirt.
4166
4525
The message is @emph{not} subject to translation in non-English locales.
4204
4563
4205
4564
@end ignore
4206
4565
4566
@node Invoking Summary
4567
@section Summary
4568
4569
@itemize @value{BULLET}
4570
@item
4571
Use either
4572
@samp{awk '@var{program}' @var{files}}
4573
or
4574
@samp{awk -f @var{program-file} @var{files}}
4575
to run @command{awk}.
4576
4577
@item
4578
The three standard @command{awk} options are @option{-f}, @option{-F}
4579
and @option{-v}. @command{gawk} supplies these and many others, as well
4580
as corresponding GNU-style long options.
4581
4582
@item
4583
Non-option command-line arguments are usually treated as @value{FN}s,
4584
unless they have the form @samp{@var{var}=@var{value}}, in which case
4585
they are taken as variable assignments to be performed at that point
4586
in processing the input.
4587
4588
@item
4589
All non-option command-line arguments, excluding the program text,
4590
are placed in the @code{ARGV} array. Adjusting @code{ARGC} and @code{ARGV}
4591
affects how @command{awk} processes input.
4592
4593
@item
4594
You can use a single minus sign (@samp{-}) to refer to standard input
4595
on the command line.
4596
4597
@item
4598
@command{gawk} pays attention to a number of environment variables.
4599
@env{AWKPATH}, @env{AWKLIBPATH}, and @env{POSIXLY_CORRECT} are the
4600
most important ones.
4601
4602
@item
4603
@command{gawk}'s exit status conveys information to the program
4604
that invoked it. Use the @code{exit} statement from within
4605
an @command{awk} program to set the exit status.
4606
4607
@item
4608
@command{gawk} allows you to include other @command{awk} source files into
4609
your program using the @code{@@include} statement and/or the @option{-i}
4610
and @option{-f} command-line options.
4611
4612
@item
4613
@command{gawk} allows you to load additional functions written in C
4614
or C++ using the @code{@@load} statement and/or the @option{-l} option.
4615
(This advanced feature is described later on in @ref{Dynamic Extensions}.)
4616
@end itemize
4617
4207
4618
@node Regexp
4208
4619
@chapter Regular Expressions
4209
@cindex regexp, See regular expressions
4620
@cindex regexp
4210
4621
@c STARTOFRANGE regexp
4211
4622
@cindex regular expressions
4212
4623
4215
4626
Because regular expressions are such a fundamental part of @command{awk}
4216
4627
programming, their format and use deserve a separate @value{CHAPTER}.
4217
4628
4218
@cindex forward slash (@code{/})
4219
@cindex @code{/} (forward slash)
4629
@cindex forward slash (@code{/}) to enclose regular expressions
4630
@cindex @code{/} (forward slash) to enclose regular expressions
4220
4631
A regular expression enclosed in slashes (@samp{/})
4221
4632
is an @command{awk} pattern that matches every input record whose text
4222
4633
belongs to that set.
4242
4653
* Case-sensitivity:: How to do case-insensitive matching.
4243
4654
* Leftmost Longest:: How much text matches.
4244
4655
* Computed Regexps:: Using Dynamic Regexps.
4656
* Regexp Summary:: Regular expressions summary.
4245
4657
@end menu
4246
4658
4247
4659
@node Regexp Usage
4252
4664
slashes. Then the regular expression is tested against the
4253
4665
entire text of each record. (Normally, it only needs
4254
4666
to match some part of the text in order to succeed.) For example, the
4255
following prints the second field of each record that contains the string
4256
@samp{foo} anywhere in it:
4667
following prints the second field of each record where the string
4668
@samp{li} appears anywhere in the record:
4257
4669
4258
4670
@example
4259
$ @kbd{awk '/foo/ @{ print $2 @}' BBS-list}
4260
@print{} 555-1234
4671
$ @kbd{awk '/li/ @{ print $2 @}' mail-list}
4672
@print{} 555-5553
4673
@print{} 555-0542
4261
4674
@print{} 555-6699
4262
@print{} 555-6480
4263
@print{} 555-2127
4675
@print{} 555-3430
4264
4676
@end example
4265
4677
4266
4678
@cindex regular expressions, operators
4272
4684
@cindex @code{!} (exclamation point), @code{!~} operator
4273
4685
@cindex exclamation point (@code{!}), @code{!~} operator
4274
4686
@c @cindex operators, @code{!~}
4275
@cindex @code{if} statement
4276
@cindex @code{while} statement
4277
@cindex @code{do}-@code{while} statement
4687
@cindex @code{if} statement, use of regexps in
4688
@cindex @code{while} statement, use of regexps in
4689
@cindex @code{do}-@code{while} statement, use of regexps in
4278
4690
@c @cindex statements, @code{if}
4279
4691
@c @cindex statements, @code{while}
4280
4692
@c @cindex statements, @code{do}
4333
4745
@end example
4334
4746
4335
4747
@cindex regexp constants
4748
@cindex constant regexps
4336
4749
@cindex regular expressions, constants, See regexp constants
4337
4750
When a regexp is enclosed in slashes, such as @code{/foo/}, we call it
4338
4751
a @dfn{regexp constant}, much like @code{5.27} is a numeric constant and
4341
4754
@node Escape Sequences
4342
4755
@section Escape Sequences
4343
4756
4344
@cindex escape sequences
4757
@cindex escape sequences, in strings
4345
4758
@cindex backslash (@code{\}), in escape sequences
4346
4759
@cindex @code{\} (backslash), in escape sequences
4347
4760
Some characters cannot be included literally in string constants
4382
4795
@cindex backslash (@code{\}), @code{\a} escape sequence
4383
4796
@item \a
4384
4797
The ``alert'' character, @kbd{Ctrl-g}, ASCII code 7 (BEL).
4385
(This usually makes some sort of audible noise.)
4798
(This often makes some sort of audible noise.)
4386
4799
4387
4800
@cindex @code{\} (backslash), @code{\b} escape sequence
4388
4801
@cindex backslash (@code{\}), @code{\b} escape sequence
4476
4889
4477
4890
To summarize:
4478
4891
4479
@itemize @bullet
4892
@itemize @value{BULLET}
4480
4893
@item
4481
4894
The escape sequences in the table above are always processed first,
4482
4895
for both string constants and regexp constants. This happens very early,
4506
4919
4507
4920
@c @cindex automatic warnings
4508
4921
@c @cindex warnings, automatic
4922
@cindex Brian Kernighan's @command{awk}
4509
4923
@table @asis
4510
4924
@item Strip the backslash out
4511
4925
This is what Brian Kernighan's @command{awk} and @command{gawk} both do.
4519
4933
4520
4934
@cindex @command{gawk}, escape sequences
4521
4935
@cindex Unix @command{awk}, backslashes in escape sequences
4936
@cindex @command{mawk} utility
4522
4937
@item Leave the backslash alone
4523
4938
Some other @command{awk} implementations do this.
4524
4939
In such implementations, typing @code{"a\qc"} is the same as typing
4550
4965
@section Regular Expression Operators
4551
4966
@c STARTOFRANGE regexpo
4552
4967
@cindex regular expressions, operators
4968
@cindex metacharacters in regular expressions
4553
4969
4554
4970
You can combine regular expressions with special characters,
4555
4971
called @dfn{regular expression operators} or @dfn{metacharacters}, to
4567
4983
Here is a list of metacharacters. All characters that are not escape
4568
4984
sequences and that are not listed in the table stand for themselves:
4569
4985
4570
@table @code
4571
@cindex backslash (@code{\})
4572
@cindex @code{\} (backslash)
4573
@item \
4986
@c Use @asis so the docbook comes out ok. Sigh.
4987
@table @asis
4988
@cindex backslash (@code{\}), regexp operator
4989
@cindex @code{\} (backslash), regexp operator
4990
@item @code{\}
4574
4991
This is used to suppress the special meaning of a character when
4575
4992
matching. For example, @samp{\$}
4576
4993
matches the character @samp{$}.
4579
4996
@cindex Texinfo, chapter beginnings in files
4580
4997
@cindex @code{^} (caret), regexp operator
4581
4998
@cindex caret (@code{^}), regexp operator
4582
@item ^
4999
@item @code{^}
4583
5000
This matches the beginning of a string. For example, @samp{^@@chapter}
4584
5001
matches @samp{@@chapter} at the beginning of a string and can be used
4585
5002
to identify chapter beginnings in Texinfo source files.
4587
5004
match only at the beginning of the string.
4588
5005
4589
5006
It is important to realize that @samp{^} does not match the beginning of
4590
a line embedded in a string.
5007
a line (the point right after a @samp{\n} newline character) embedded in a string.
4591
5008
The condition is not true in the following example:
4592
5009
4593
5010
@example
4594
5011
if ("line1\nLINE 2" ~ /^L/) @dots{}
4595
5012
@end example
4596
5013
4597
@cindex @code{$} (dollar sign)
4598
@cindex dollar sign (@code{$})
4599
@item $
5014
@cindex @code{$} (dollar sign), regexp operator
5015
@cindex dollar sign (@code{$}), regexp operator
5016
@item @code{$}
4600
5017
This is similar to @samp{^}, but it matches only at the end of a string.
4601
5018
For example, @samp{p$}
4602
5019
matches a record that ends with a @samp{p}. The @samp{$} is an anchor
4603
and does not match the end of a line embedded in a string.
5020
and does not match the end of a line
5021
(the point right before a @samp{\n} newline character)
5022
embedded in a string.
4604
5023
The condition in the following example is not true:
4605
5024
4606
5025
@example
4607
5026
if ("line1\nLINE 2" ~ /1$/) @dots{}
4608
5027
@end example
4609
5028
4610
@cindex @code{.} (period)
4611
@cindex period (@code{.})
4612
@item . @r{(period)}
5029
@cindex @code{.} (period), regexp operator
5030
@cindex period (@code{.}), regexp operator
5031
@item @code{.} (period)
4613
5032
This matches any single character,
4614
5033
@emph{including} the newline character. For example, @samp{.P}
4615
5034
matches any single character followed by a @samp{P} in a string. Using
4624
5043
Otherwise, @sc{nul} is just another character. Other versions of @command{awk}
4625
5044
may not be able to match the @sc{nul} character.
4626
5045
4627
@cindex @code{[]} (square brackets)
4628
@cindex square brackets (@code{[]})
5046
@cindex @code{[]} (square brackets), regexp operator
5047
@cindex square brackets (@code{[]}), regexp operator
4629
5048
@cindex bracket expressions
4630
5049
@cindex character sets, See Also bracket expressions
4631
5050
@cindex character lists, See bracket expressions
4632
@item [@dots{}]
5051
@cindex character classes, See bracket expressions
5052
@item @code{[}@dots{}@code{]}
4633
5053
This is called a @dfn{bracket expression}.@footnote{In other literature,
4634
5054
you may see a bracket expression referred to as either a
4635
5055
@dfn{character set}, a @dfn{character class}, or a @dfn{character list}.}
4641
5061
@ref{Bracket Expressions}.
4642
5062
4643
5063
@cindex bracket expressions, complemented
4644
@item [^ @dots{}]
5064
@item @code{[^}@dots{}@code{]}
4645
5065
This is a @dfn{complemented bracket expression}. The first character after
4646
5066
the @samp{[} @emph{must} be a @samp{^}. It matches any characters
4647
5067
@emph{except} those in the square brackets. For example, @samp{[^awk]}
4650
5070
4651
5071
@cindex @code{|} (vertical bar)
4652
5072
@cindex vertical bar (@code{|})
4653
@item |
5073
@item @code{|}
4654
5074
This is the @dfn{alternation operator} and it is used to specify
4655
5075
alternatives.
4656
5076
The @samp{|} has the lowest precedence of all the regular
4661
5081
4662
5082
The alternation applies to the largest possible regexps on either side.
4663
5083
4664
@cindex @code{()} (parentheses)
4665
@cindex parentheses @code{()}
4666
@item (@dots{})
5084
@cindex @code{()} (parentheses), regexp operator
5085
@cindex parentheses @code{()}, regexp operator
5086
@item @code{(}@dots{}@code{)}
4667
5087
Parentheses are used for grouping in regular expressions, as in
4668
5088
arithmetic. They can be used to concatenate regular expressions
4669
5089
containing the alternation operator, @samp{|}. For example,
4674
5094
4675
5095
@cindex @code{*} (asterisk), @code{*} operator, as regexp operator
4676
5096
@cindex asterisk (@code{*}), @code{*} operator, as regexp operator
4677
@item *
5097
@item @code{*}
4678
5098
This symbol means that the preceding regular expression should be
4679
5099
repeated as many times as necessary to find a match. For example, @samp{ph*}
4680
5100
applies the @samp{*} symbol to the preceding @samp{h} and looks for matches
4690
5110
Notice the escaping of the parentheses by preceding them
4691
5111
with backslashes.
4692
5112
4693
@cindex @code{+} (plus sign)
4694
@cindex plus sign (@code{+})
4695
@item +
5113
@cindex @code{+} (plus sign), regexp operator
5114
@cindex plus sign (@code{+}), regexp operator
5115
@item @code{+}
4696
5116
This symbol is similar to @samp{*}, except that the preceding expression must be
4697
5117
matched at least once. This means that @samp{wh+y}
4698
5118
would match @samp{why} and @samp{whhy}, but not @samp{wy}, whereas
4699
@samp{wh*y} would match all three of these strings.
5119
@samp{wh*y} would match all three.
4700
5120
The following is a simpler
4701
5121
way of writing the last @samp{*} example:
4702
5122
4704
5124
awk '/\(c[ad]+r x\)/ @{ print @}' sample
4705
5125
@end example
4706
5126
4707
@cindex @code{?} (question mark) regexp operator
4708
@cindex question mark (@code{?}) regexp operator
4709
@item ?
5127
@cindex @code{?} (question mark), regexp operator
5128
@cindex question mark (@code{?}), regexp operator
5129
@item @code{?}
4710
5130
This symbol is similar to @samp{*}, except that the preceding expression can be
4711
5131
matched either once or not at all. For example, @samp{fe?d}
4712
5132
matches @samp{fed} and @samp{fd}, but nothing else.
4713
5133
4714
@cindex interval expressions
4715
@item @{@var{n}@}
4716
@itemx @{@var{n},@}
4717
@itemx @{@var{n},@var{m}@}
5134
@cindex interval expressions, regexp operator
5135
@item @code{@{}@var{n}@code{@}}
5136
@itemx @code{@{}@var{n}@code{,@}}
5137
@itemx @code{@{}@var{n}@code{,}@var{m}@code{@}}
4718
5138
One or two numbers inside braces denote an @dfn{interval expression}.
4719
5139
If there is one number in the braces, the preceding regexp is repeated
4720
5140
@var{n} times.
4745
5165
@command{gawk} did @emph{not} match interval expressions
4746
5166
in regexps.
4747
5167
4748
However, beginning with version 4.0,
5168
However, beginning with @value{PVERSION} 4.0,
4749
5169
@command{gawk} does match interval expressions by default.
4750
5170
This is because compatibility with POSIX has become more
4751
5171
important to most @command{gawk} users than compatibility with
4788
5208
@cindex bracket expressions
4789
5209
@cindex bracket expressions, range expressions
4790
5210
@cindex range expressions (regexps)
5211
@cindex character lists in regular expression
4791
5212
4792
5213
As mentioned earlier, a bracket expression matches any character amongst
4793
5214
those listed between the opening and closing square brackets.
4889
5310
@item Collating symbols
4890
5311
Multicharacter collating elements enclosed between
4891
5312
@samp{[.} and @samp{.]}. For example, if @samp{ch} is a collating element,
4892
then @code{[[.ch.]]} is a regexp that matches this collating element, whereas
4893
@code{[ch]} is a regexp that matches either @samp{c} or @samp{h}.
5313
then @samp{[[.ch.]]} is a regexp that matches this collating element, whereas
5314
@samp{[ch]} is a regexp that matches either @samp{c} or @samp{h}.
4894
5315
4895
5316
@cindex bracket expressions, equivalence classes
4896
5317
@item Equivalence classes
4898
5319
characters that are equal. The name is enclosed between
4899
5320
@samp{[=} and @samp{=]}.
4900
5321
For example, the name @samp{e} might be used to represent all of
4901
``e,'' ``@`e,'' and ``@'e.'' In this case, @code{[[=e=]]} is a regexp
5322
``e,'' ``@`e,'' and ``@'e.'' In this case, @samp{[[=e=]]} is a regexp
4902
5323
that matches any of @samp{e}, @samp{@'e}, or @samp{@`e}.
4903
5324
@end table
4904
5325
4942
5363
@item \s
4943
5364
Matches any whitespace character.
4944
5365
Think of it as shorthand for
4945
@w{@code{[[:space:]]}}.
5366
@w{@samp{[[:space:]]}}.
4946
5367
4947
5368
@c @cindex operators, @code{\S} (@command{gawk})
4948
5369
@cindex backslash (@code{\}), @code{\S} operator (@command{gawk})
4950
5371
@item \S
4951
5372
Matches any character that is not whitespace.
4952
5373
Think of it as shorthand for
4953
@w{@code{[^[:space:]]}}.
5374
@w{@samp{[^[:space:]]}}.
4954
5375
4955
5376
@c @cindex operators, @code{\w} (@command{gawk})
4956
5377
@cindex backslash (@code{\}), @code{\w} operator (@command{gawk})
4958
5379
@item \w
4959
5380
Matches any word-constituent character---that is, it matches any
4960
5381
letter, digit, or underscore. Think of it as shorthand for
4961
@w{@code{[[:alnum:]_]}}.
5382
@w{@samp{[[:alnum:]_]}}.
4962
5383
4963
5384
@c @cindex operators, @code{\W} (@command{gawk})
4964
5385
@cindex backslash (@code{\}), @code{\W} operator (@command{gawk})
4966
5387
@item \W
4967
5388
Matches any character that is not word-constituent.
4968
5389
Think of it as shorthand for
4969
@w{@code{[^[:alnum:]_]}}.
5390
@w{@samp{[^[:alnum:]_]}}.
4970
5391
4971
5392
@c @cindex operators, @code{\<} (@command{gawk})
4972
5393
@cindex backslash (@code{\}), @code{\<} operator (@command{gawk})
5027
5448
end of a buffer (string).
5028
5449
@end table
5029
5450
5030
@cindex @code{^} (caret)
5031
@cindex caret (@code{^})
5032
@cindex @code{?} (question mark) regexp operator
5033
@cindex question mark (@code{?}) regexp operator
5451
@cindex @code{^} (caret), regexp operator
5452
@cindex caret (@code{^}), regexp operator
5453
@cindex @code{?} (question mark), regexp operator
5454
@cindex question mark (@code{?}), regexp operator
5034
5455
Because @samp{^} and @samp{$} always work in terms of the beginning
5035
5456
and end of strings, these operators don't add any new capabilities
5036
5457
for @command{awk}. They are provided for compatibility with other
5047
5468
method of using @samp{\y} for the GNU @samp{\b} appears to be the
5048
5469
lesser of two evils.
5049
5470
5050
@c NOTE!!! Keep this in sync with the same table in the summary appendix!
5051
@c
5052
@c Should really do this with file inclusion.
5053
5471
@cindex regular expressions, @command{gawk}, command-line options
5054
@cindex @command{gawk}, command-line options
5472
@cindex @command{gawk}, command-line options, and regular expressions
5055
5473
The various command-line options
5056
5474
(@pxref{Options})
5057
5475
control how @command{gawk} interprets characters in regexps:
5065
5483
GNU regexp operators.
5066
5484
@end ifnotinfo
5067
5485
@ifnottex
5486
@ifnotdocbook
5068
5487
GNU regexp operators described
5069
5488
in @ref{Regexp Operators}.
5489
@end ifnotdocbook
5070
5490
@end ifnottex
5071
5491
5072
5492
@item @code{--posix}
5074
5494
(e.g., @samp{\w} matches a literal @samp{w}). Interval expressions
5075
5495
are allowed.
5076
5496
5497
@cindex Brian Kernighan's @command{awk}
5077
5498
@item @code{--traditional}
5078
5499
Traditional Unix @command{awk} regexps are matched. The GNU operators
5079
5500
are not special, and interval expressions are not available.
5080
The POSIX character classes (@code{[[:alnum:]]}, etc.) are supported,
5501
The POSIX character classes (@samp{[[:alnum:]]}, etc.) are supported,
5081
5502
as Brian Kernighan's @command{awk} does support them.
5082
5503
Characters described by octal and hexadecimal escape sequences are
5083
5504
treated literally, even if they represent regexp metacharacters.
5129
5550
@cindex tilde (@code{~}), @code{~} operator
5130
5551
@cindex @code{!} (exclamation point), @code{!~} operator
5131
5552
@cindex exclamation point (@code{!}), @code{!~} operator
5132
@cindex @code{IGNORECASE} variable
5553
@cindex @code{IGNORECASE} variable, with @code{~} and @code{!~} operators
5133
5554
@cindex @command{gawk}, @code{IGNORECASE} variable in
5134
5555
@c @cindex variables, @code{IGNORECASE}
5135
5556
Another method, specific to @command{gawk}, is to set the variable
5136
5557
@code{IGNORECASE} to a nonzero value (@pxref{Built-in Variables}).
5137
5558
When @code{IGNORECASE} is not zero, @emph{all} regexp and string
5138
operations ignore case. Changing the value of
5139
@code{IGNORECASE} dynamically controls the case-sensitivity of the
5140
program as it runs. Case is significant by default because
5141
@code{IGNORECASE} (like most variables) is initialized to zero:
5559
operations ignore case.
5560
5561
Changing the value of @code{IGNORECASE} dynamically controls the
5562
case-sensitivity of the program as it runs. Case is significant by
5563
default because @code{IGNORECASE} (like most variables) is initialized
5564
to zero:
5142
5565
5143
5566
@example
5144
5567
x = "aB"
5168
5591
Setting @code{IGNORECASE} from the command line is a way to make
5169
5592
a program case-insensitive without having to edit it.
5170
5593
5171
Both regexp and string comparison
5172
operations are affected by @code{IGNORECASE}.
5173
5174
5594
@c @cindex ISO 8859-1
5175
5595
@c @cindex ISO Latin-1
5176
5596
In multibyte locales,
5248
5668
be any expression. The expression is evaluated and converted to a string
5249
5669
if necessary; the contents of the string are then used as the
5250
5670
regexp. A regexp computed in this way is called a @dfn{dynamic
5251
regexp}:
5671
regexp} or a @dfn{computed regexp}:
5252
5672
5253
5673
@example
5254
5674
BEGIN @{ digits_regexp = "[[:digit:]]+" @}
5294
5714
regular expressions, which should you use? The answer is ``regexp
5295
5715
constants,'' for several reasons:
5296
5716
5297
@itemize @bullet
5717
@itemize @value{BULLET}
5298
5718
@item
5299
5719
String constants are more complicated to write and
5300
5720
more difficult to read. Using regexp constants makes your programs
5317
5737
@cindex regular expressions, dynamic, with embedded newlines
5318
5738
@cindex newlines, in dynamic regexps
5319
5739
5320
Some commercial versions of @command{awk} do not allow the newline
5740
Some versions of @command{awk} do not allow the newline
5321
5741
character to be used inside a bracket expression for a dynamic regexp:
5322
5742
5323
5743
@example
5344
5764
@end sidebar
5345
5765
@c ENDOFRANGE dregexp
5346
5766
@c ENDOFRANGE regexpd
5767
5768
@node Regexp Summary
5769
@section Summary
5770
5771
@itemize @value{BULLET}
5772
@item
5773
Regular expressions describe sets of strings to be matched.
5774
In @command{awk}, regular expression constants are written enclosed
5775
between slashes: @code{/}@dots{}@code{/}.
5776
5777
@item
5778
Regexp constants may be used by standalone in patterns and
5779
in conditional expressions, or as part of matching expressions
5780
using the @samp{~} and @samp{!~} operators.
5781
5782
@item
5783
Escape sequences let you represent non-printable characters and
5784
also let you represent regexp metacharacters as literal characters
5785
to be matched.
5786
5787
@item
5788
Regexp operators provide grouping, alternation and repetition.
5789
5790
@item
5791
Bracket expressions give you a shorthand for specifying sets
5792
of characters that can match at a particular point in a regexp.
5793
Within bracket expressions, POSIX character classes let you specify
5794
certain groups of characters in a locale-independent fashion.
5795
5796
@item
5797
@command{gawk}'s @code{IGNORECASE} variable lets you control the
5798
case sensitivity of regexp matching. In other @command{awk}
5799
versions, use @code{tolower()} or @code{toupper()}.
5800
5801
@item
5802
Regular expressions match the leftmost longest text in the string being
5803
matched. This matters for cases where you need to know the extent of
5804
the match, such as for text substitution and when the record separator
5805
is a regexp.
5806
5807
@item
5808
Matching expressions may use dynamic regexps; that is string values
5809
treated as regular expressions.
5810
5811
@end itemize
5812
5347
5813
@c ENDOFRANGE regexp
5348
5814
5349
5815
@node Reading Files
5350
5816
@chapter Reading Input Files
5351
5817
5352
5818
@c STARTOFRANGE infir
5819
@cindex reading input files
5353
5820
@cindex input files, reading
5354
5821
@cindex input files
5355
5822
@cindex @code{FILENAME} variable
5392
5859
* Read Timeout:: Reading input with a timeout.
5393
5860
* Command line directories:: What happens if you put a directory on the
5394
5861
command line.
5862
* Input Summary:: Input summary.
5863
* Input Exercises:: Exercises.
5395
5864
@end menu
5396
5865
5397
5866
@node Records
5411
5880
from the current input file. This value is stored in a
5412
5881
built-in variable called @code{FNR}. It is reset to zero when a new
5413
5882
file is started. Another built-in variable, @code{NR}, records the total
5414
number of input records read so far from all data files. It starts at zero,
5883
number of input records read so far from all @value{DF}s. It starts at zero,
5415
5884
but is never automatically reset to zero.
5416
5885
5886
@menu
5887
* awk split records:: How standard @command{awk} splits records.
5888
* gawk split records:: How @command{gawk} splits records.
5889
@end menu
5890
5891
@node awk split records
5892
@subsection Record Splitting With Standard @command{awk}
5893
5417
5894
@cindex separators, for records
5418
5895
@cindex record separators
5419
5896
Records are separated by a character called the @dfn{record separator}.
5436
5913
(@pxref{BEGIN/END}).
5437
5914
For example:
5438
5915
5439
@cindex @code{BEGIN} pattern
5440
5916
@example
5441
awk 'BEGIN @{ RS = "/" @}
5442
@{ print $0 @}' BBS-list
5917
awk 'BEGIN @{ RS = "u" @}
5918
@{ print $0 @}' mail-list
5443
5919
@end example
5444
5920
5445
5921
@noindent
5446
changes the value of @code{RS} to @code{"/"}, before reading any input.
5447
This is a string whose first character is a slash; as a result, records
5448
are separated by slashes. Then the input file is read, and the second
5922
changes the value of @code{RS} to @samp{u}, before reading any input.
5923
This is a string whose first character is the letter ``u;'' as a result, records
5924
are separated by the letter ``u.'' Then the input file is read, and the second
5449
5925
rule in the @command{awk} program (the action with no pattern) prints each
5450
5926
record. Because each @code{print} statement adds a newline at the end of
5451
5927
its output, this @command{awk} program copies the input
5452
with each slash changed to a newline. Here are the results of running
5453
the program on @file{BBS-list}:
5928
with each @samp{u} changed to a newline. Here are the results of running
5929
the program on @file{mail-list}:
5454
5930
5455
5931
@example
5456
$ @kbd{awk 'BEGIN @{ RS = "/" @}}
5457
> @kbd{@{ print $0 @}' BBS-list}
5458
@print{} aardvark 555-5553 1200
5459
@print{} 300 B
5460
@print{} alpo-net 555-3412 2400
5461
@print{} 1200
5462
@print{} 300 A
5463
@print{} barfly 555-7685 1200
5464
@print{} 300 A
5465
@print{} bites 555-1675 2400
5466
@print{} 1200
5467
@print{} 300 A
5468
@print{} camelot 555-0542 300 C
5469
@print{} core 555-2912 1200
5470
@print{} 300 C
5471
@print{} fooey 555-1234 2400
5472
@print{} 1200
5473
@print{} 300 B
5474
@print{} foot 555-6699 1200
5475
@print{} 300 B
5476
@print{} macfoo 555-6480 1200
5477
@print{} 300 A
5478
@print{} sdace 555-3430 2400
5479
@print{} 1200
5480
@print{} 300 A
5481
@print{} sabafoo 555-2127 1200
5482
@print{} 300 C
5483
@print{}
5932
$ @kbd{awk 'BEGIN @{ RS = "u" @}}
5933
> @kbd{@{ print $0 @}' mail-list}
5934
@print{} Amelia 555-5553 amelia.zodiac
5935
@print{} sq
5936
@print{} e@@gmail.com F
5937
@print{} Anthony 555-3412 anthony.assert
5938
@print{} ro@@hotmail.com A
5939
@print{} Becky 555-7685 becky.algebrar
5940
@print{} m@@gmail.com A
5941
@print{} Bill 555-1675 bill.drowning@@hotmail.com A
5942
@print{} Broderick 555-0542 broderick.aliq
5943
@print{} otiens@@yahoo.com R
5944
@print{} Camilla 555-2912 camilla.inf
5945
@print{} sar
5946
@print{} m@@skynet.be R
5947
@print{} Fabi
5948
@print{} s 555-1234 fabi
5949
@print{} s.
5950
@print{} ndevicesim
5951
@print{} s@@
5952
@print{} cb.ed
5953
@print{} F
5954
@print{} J
5955
@print{} lie 555-6699 j
5956
@print{} lie.perscr
5957
@print{} tabor@@skeeve.com F
5958
@print{} Martin 555-6480 martin.codicib
5959
@print{} s@@hotmail.com A
5960
@print{} Sam
5961
@print{} el 555-3430 sam
5962
@print{} el.lanceolis@@sh
5963
@print{} .ed
5964
@print{} A
5965
@print{} Jean-Pa
5966
@print{} l 555-2127 jeanpa
5967
@print{} l.campanor
5968
@print{} m@@ny
5969
@print{} .ed
5970
@print{} R
5971
@print{}
5484
5972
@end example
5485
5973
5486
5974
@noindent
5487
Note that the entry for the @samp{camelot} BBS is not split.
5488
In the original data file
5975
Note that the entry for the name @samp{Bill} is not split.
5976
In the original @value{DF}
5489
5977
(@pxref{Sample Data Files}),
5490
5978
the line looks like this:
5491
5979
5492
5980
@example
5493
camelot 555-0542 300 C
5981
Bill 555-1675 bill.drowning@@hotmail.com A
5494
5982
@end example
5495
5983
5496
5984
@noindent
5497
It has one baud rate only, so there are no slashes in the record,
5498
unlike the others which have two or more baud rates.
5499
In fact, this record is treated as part of the record
5500
for the @samp{core} BBS; the newline separating them in the output
5501
is the original newline in the data file, not the one added by
5985
It contains no @samp{u} so there is no reason to split the record,
5986
unlike the others which have one or more occurrences of the @samp{u}.
5987
In fact, this record is treated as part of the previous record;
5988
the newline separating them in the output
5989
is the original newline in the @value{DF}, not the one added by
5502
5990
@command{awk} when it printed the record!
5503
5991
5504
5992
@cindex record separators, changing
5508
5996
(@pxref{Other Arguments}):
5509
5997
5510
5998
@example
5511
awk '@{ print $0 @}' RS="/" BBS-list
5999
awk '@{ print $0 @}' RS="u" mail-list
5512
6000
@end example
5513
6001
5514
6002
@noindent
5515
This sets @code{RS} to @samp{/} before processing @file{BBS-list}.
6003
This sets @code{RS} to @samp{u} before processing @file{mail-list}.
5516
6004
5517
Using an unusual character such as @samp{/} for the record separator
5518
produces correct behavior in the vast majority of cases.
6005
Using an alphabetic character such as @samp{u} for the record separator
6006
is highly likely to produce strange results.
6007
Using an unusual character such as @samp{/} is more likely to
6008
produce correct behavior in the majority of cases, but there
6009
are no guarantees. The moral is: Know Your Data.
5519
6010
5520
6011
There is one unusual case, that occurs when @command{gawk} is
5521
6012
being fully POSIX-compliant (@pxref{Options}).
5537
6028
even if the last character in the file is not the character in @code{RS}.
5538
6029
@value{DARKCORNER}
5539
6030
6031
@cindex empty strings
5540
6032
@cindex null strings
5541
6033
@cindex strings, empty, See null strings
5542
6034
The empty string @code{""} (a string without any characters)
5562
6054
sets the variable @code{RT} to the text in the input that matched
5563
6055
@code{RS}.
5564
6056
6057
@node gawk split records
6058
@subsection Record Splitting With @command{gawk}
6059
5565
6060
@cindex common extensions, @code{RS} as a regexp
5566
6061
@cindex extensions, common@comma{} @code{RS} as a regexp
5567
6062
When using @command{gawk},
5635
6130
5636
6131
@sidebar @code{RS = "\0"} Is Not Portable
5637
6132
@cindex portability, data files as single record
5638
There are times when you might want to treat an entire data file as a
6133
There are times when you might want to treat an entire @value{DF} as a
5639
6134
single record. The only way to make this happen is to give @code{RS}
5640
6135
a value that you know doesn't occur in the input file. This is hard
5641
6136
to do in a general way, such that a program always works for arbitrary
5642
6137
input files.
5643
@c can you say `understatement' boys and girls?
5644
6138
5645
6139
You might think that for text files, the @sc{nul} character, which
5646
6140
consists of a character with all bits equal to zero, is a good
5653
6147
@cindex differences in @command{awk} and @command{gawk}, strings, storing
5654
6148
@command{gawk} in fact accepts this, and uses the @sc{nul}
5655
6149
character for the record separator.
6150
This works for certain special files, such as @file{/proc/environ} on
6151
GNU/Linux systems, where the @sc{nul} character is in fact the record separator.
5656
6152
However, this usage is @emph{not} portable
5657
to other @command{awk} implementations.
6153
to most other @command{awk} implementations.
5658
6154
5659
6155
@cindex dark corner, strings, storing
5660
All other @command{awk} implementations@footnote{At least that we know
6156
Almost all other @command{awk} implementations@footnote{At least that we know
5661
6157
about.} store strings internally as C-style strings. C strings use the
5662
6158
@sc{nul} character as the string terminator. In effect, this means that
5663
6159
@samp{RS = "\0"} is the same as @samp{RS = ""}.
5664
6160
@value{DARKCORNER}
5665
6161
6162
It happens that recent versions of @command{mawk} can use the @sc{nul}
6163
character as a record separator. However, this is a special case:
6164
@command{mawk} does not allow embedded @sc{nul} characters in strings.
6165
5666
6166
@cindex records, treating files as
5667
@cindex files, as single records
5668
The best way to treat a whole file as a single record is to
5669
simply read the file in, one record at a time, concatenating each
5670
record onto the end of the previous ones.
6167
@cindex treating files, as single records
6168
@xref{Readfile Function}, for an interesting, portable way to read
6169
whole files. If you are using @command{gawk}, see @ref{Extension Sample
6170
Readfile}, for another option.
5671
6171
@end sidebar
5672
6172
@c ENDOFRANGE inspl
5673
6173
@c ENDOFRANGE recspl
5703
6203
@cindex @code{$} (dollar sign), @code{$} field operator
5704
6204
@cindex dollar sign (@code{$}), @code{$} field operator
5705
6205
@cindex field operators@comma{} dollar sign as
5706
A dollar-sign (@samp{$}) is used
6206
You use a dollar-sign (@samp{$})
5707
6207
to refer to a field in an @command{awk} program,
5708
6208
followed by the number of the field you want. Thus, @code{$1}
5709
6209
refers to the first field, @code{$2} to the second, and so on.
5734
6234
the empty string. (If used in a numeric operation, you get zero.)
5735
6235
5736
6236
The use of @code{$0}, which looks like a reference to the ``zero-th'' field, is
5737
a special case: it represents the whole input record
6237
a special case: it represents the whole input record. Use it
5738
6238
when you are not interested in specific fields.
5739
6239
Here are some more examples:
5740
6240
5741
6241
@example
5742
$ @kbd{awk '$1 ~ /foo/ @{ print $0 @}' BBS-list}
5743
@print{} fooey 555-1234 2400/1200/300 B
5744
@print{} foot 555-6699 1200/300 B
5745
@print{} macfoo 555-6480 1200/300 A
5746
@print{} sabafoo 555-2127 1200/300 C
6242
$ @kbd{awk '$1 ~ /li/ @{ print $0 @}' mail-list}
6243
@print{} Amelia 555-5553 amelia.zodiacusque@@gmail.com F
6244
@print{} Julie 555-6699 julie.perscrutabor@@skeeve.com F
5747
6245
@end example
5748
6246
5749
6247
@noindent
5750
This example prints each record in the file @file{BBS-list} whose first
5751
field contains the string @samp{foo}. The operator @samp{~} is called a
6248
This example prints each record in the file @file{mail-list} whose first
6249
field contains the string @samp{li}. The operator @samp{~} is called a
5752
6250
@dfn{matching operator}
5753
6251
(@pxref{Regexp Usage});
5754
6252
it tests whether a string (here, the field @code{$1}) matches a given regular
5755
6253
expression.
5756
6254
5757
6255
By contrast, the following example
5758
looks for @samp{foo} in @emph{the entire record} and prints the first
6256
looks for @samp{li} in @emph{the entire record} and prints the first
5759
6257
field and the last field for each matching input record:
5760
6258
5761
6259
@example
5762
$ @kbd{awk '/foo/ @{ print $1, $NF @}' BBS-list}
5763
@print{} fooey B
5764
@print{} foot B
5765
@print{} macfoo A
5766
@print{} sabafoo C
6260
$ @kbd{awk '/li/ @{ print $1, $NF @}' mail-list}
6261
@print{} Amelia F
6262
@print{} Broderick R
6263
@print{} Julie F
6264
@print{} Samuel A
5767
6265
@end example
5768
6266
@c ENDOFRANGE fiex
5769
6267
5772
6270
@cindex fields, numbers
5773
6271
@cindex field numbers
5774
6272
5775
The number of a field does not need to be a constant. Any expression in
6273
A field number need not be a constant. Any expression in
5776
6274
the @command{awk} language can be used after a @samp{$} to refer to a
5777
6275
field. The value of the expression specifies the field number. If the
5778
6276
value is a string, rather than a number, it is converted to a number.
5791
6289
Here is another example of using expressions as field numbers:
5792
6290
5793
6291
@example
5794
awk '@{ print $(2*2) @}' BBS-list
6292
awk '@{ print $(2*2) @}' mail-list
5795
6293
@end example
5796
6294
5797
6295
@command{awk} evaluates the expression @samp{(2*2)} and uses
5799
6297
represents multiplication, so the expression @samp{2*2} evaluates to four.
5800
6298
The parentheses are used so that the multiplication is done before the
5801
6299
@samp{$} operation; they are necessary whenever there is a binary
5802
operator in the field-number expression. This example, then, prints the
5803
hours of operation (the fourth field) for every line of the file
5804
@file{BBS-list}. (All of the @command{awk} operators are listed, in
6300
operator@footnote{A @dfn{binary operator}, such as @samp{*} for
6301
multiplication, is one that takes two operands. The distinction
6302
is required, since @command{awk} also has unary (one-operand)
6303
and ternary (three-operand) operators.}
6304
in the field-number expression. This example, then, prints the
6305
type of relationship (the fourth field) for every line of the file
6306
@file{mail-list}. (All of the @command{awk} operators are listed, in
5805
6307
order of decreasing precedence, in
5806
6308
@ref{Precedence}.)
5807
6309
5849
6351
(Someone in the warehouse made a consistent mistake while inventorying
5850
6352
the red boxes.)
5851
6353
5852
For this to work, the text in field @code{$3} must make sense
6354
For this to work, the text in @code{$3} must make sense
5853
6355
as a number; the string of characters must be converted to a number
5854
6356
for the computer to do arithmetic on it. The number resulting
5855
6357
from the subtraction is converted back to a string of characters that
5940
6442
@end example
5941
6443
5942
6444
@noindent
5943
The field is still there; it just has an empty value, denoted by
6445
The field is still there; it just has an empty value, delimited by
5944
6446
the two colons between @samp{a} and @samp{c}.
5945
6447
This example shows what happens if you create a new field:
5946
6448
6024
6526
* Regexp Field Splitting:: Using regexps as the field separator.
6025
6527
* Single Character Fields:: Making each character a separate field.
6026
6528
* Command Line Field Separator:: Setting @code{FS} from the command-line.
6529
* Full Line Fields:: Making the full line be a single field.
6027
6530
* Field Splitting Summary:: Some final points and a summary table.
6028
6531
@end menu
6029
6532
6191
6694
@cindex null strings
6192
6695
@cindex strings, null
6193
6696
@cindex empty strings, See null strings
6194
In this case, the first field is @dfn{null} or empty.
6697
In this case, the first field is null, or empty.
6195
6698
6196
6699
The stripping of leading and trailing whitespace also comes into
6197
6700
play whenever @code{$0} is recomputed. For instance, study this pipeline:
6211
6714
Finally, the last @code{print} statement prints the new @code{$0}.
6212
6715
6213
6716
@cindex @code{FS}, containing @code{^}
6214
@cindex @code{^}, in @code{FS}
6717
@cindex @code{^} (caret), in @code{FS}
6215
6718
@cindex dark corner, @code{^}, in @code{FS}
6216
6719
There is an additional subtlety to be aware of when using regular expressions
6217
6720
for field splitting.
6222
6725
should not rely on any specific behavior in your programs.
6223
6726
@value{DARKCORNER}
6224
6727
6728
@cindex Brian Kernighan's @command{awk}
6225
6729
As a point of information, Brian Kernighan's @command{awk} allows @samp{^}
6226
6730
to match only at the beginning of the record. @command{gawk}
6227
6731
also works this way. For example:
6265
6769
@end example
6266
6770
6267
6771
@cindex dark corner, @code{FS} as null string
6268
@cindex FS variable, as null string
6772
@cindex @code{FS} variable, as null string
6269
6773
Traditionally, the behavior of @code{FS} equal to @code{""} was not defined.
6270
6774
In this case, most versions of Unix @command{awk} simply treat the entire record
6271
6775
as only having one field.
6277
6781
6278
6782
@node Command Line Field Separator
6279
6783
@subsection Setting @code{FS} from the Command Line
6280
@cindex @code{-F} option
6281
@cindex options, command-line
6282
@cindex command line, options
6283
@cindex field separators, on command line
6784
@cindex @option{-F} option, command line
6785
@cindex field separator, on command line
6284
6786
@cindex command line, @code{FS} on@comma{} setting
6285
6787
@cindex @code{FS} variable, setting from command line
6286
6788
6330
6832
not @samp{t}s. Use @samp{-v FS="t"} or @samp{-F"[t]"} on the command line
6331
6833
if you really do want to separate your fields with @samp{t}s.
6332
6834
6333
As an example, let's use an @command{awk} program file called @file{baud.awk}
6334
that contains the pattern @code{/300/} and the action @samp{print $1}:
6835
As an example, let's use an @command{awk} program file called @file{edu.awk}
6836
that contains the pattern @code{/edu/} and the action @samp{print $1}:
6335
6837
6336
6838
@example
6337
/300/ @{ print $1 @}
6839
/edu/ @{ print $1 @}
6338
6840
@end example
6339
6841
6340
6842
Let's also set @code{FS} to be the @samp{-} character and run the
6341
program on the file @file{BBS-list}. The following command prints a
6342
list of the names of the bulletin boards that operate at 300 baud and
6843
program on the file @file{mail-list}. The following command prints a
6844
list of the names of the people that work at or attend a university, and
6343
6845
the first three digits of their phone numbers:
6344
6846
6345
@c tweaked to make the tex output look better in @smallbook
6346
6847
@example
6347
$ @kbd{awk -F- -f baud.awk BBS-list}
6348
@print{} aardvark 555
6349
@print{} alpo
6350
@print{} barfly 555
6351
@print{} bites 555
6352
@print{} camelot 555
6353
@print{} core 555
6354
@print{} fooey 555
6355
@print{} foot 555
6356
@print{} macfoo 555
6357
@print{} sdace 555
6358
@print{} sabafoo 555
6848
$ @kbd{awk -F- -f edu.awk mail-list}
6849
@print{} Fabius 555
6850
@print{} Samuel 555
6851
@print{} Jean
6359
6852
@end example
6360
6853
6361
6854
@noindent
6362
Note the second line of output. The second line
6855
Note the third line of output. The third line
6363
6856
in the original file looked like this:
6364
6857
6365
6858
@example
6366
alpo-net 555-3412 2400/1200/300 A
6859
Jean-Paul 555-2127 jeanpaul.campanorum@@nyu.edu R
6367
6860
@end example
6368
6861
6369
The @samp{-} as part of the system's name was used as the field
6862
The @samp{-} as part of the person's name was used as the field
6370
6863
separator, instead of the @samp{-} in the phone number that was
6371
6864
originally intended. This demonstrates why you have to be careful in
6372
6865
choosing your field and record separators.
6373
6866
6374
6867
@cindex Unix @command{awk}, password files@comma{} field separators and
6375
Perhaps the most common use of a single character as the field
6376
separator occurs when processing the Unix system password file.
6377
On many Unix systems, each user has a separate entry in the system password
6378
file, one line per user. The information in these lines is separated
6379
by colons. The first field is the user's login name and the second is
6380
the user's (encrypted or shadow) password. A password file entry might look
6381
like this:
6868
Perhaps the most common use of a single character as the field separator
6869
occurs when processing the Unix system password file. On many Unix
6870
systems, each user has a separate entry in the system password file, one
6871
line per user. The information in these lines is separated by colons.
6872
The first field is the user's login name and the second is the user's
6873
encrypted or shadow password. (A shadow password is indicated by the
6874
presence of a single @samp{x} in the second field.) A password file
6875
entry might look like this:
6382
6876
6383
6877
@cindex Robbins, Arnold
6384
6878
@example
6385
arnold:xyzzy:2076:10:Arnold Robbins:/home/arnold:/bin/bash
6879
arnold:x:2076:10:Arnold Robbins:/home/arnold:/bin/bash
6386
6880
@end example
6387
6881
6388
6882
The following program searches the system password file and prints
6389
the entries for users who have no password:
6390
6391
@example
6392
awk -F: '$2 == ""' /etc/passwd
6393
@end example
6883
the entries for users whose full name is not indicated:
6884
6885
@example
6886
awk -F: '$5 == ""' /etc/passwd
6887
@end example
6888
6889
@node Full Line Fields
6890
@subsection Making The Full Line Be A Single Field
6891
6892
Occasionally, it's useful to treat the whole input line as a
6893
single field. This can be done easily and portably simply by
6894
setting @code{FS} to @code{"\n"} (a newline).@footnote{Thanks to
6895
Andrew Schorr for this tip.}
6896
6897
@example
6898
awk -F'\n' '@var{program}' @var{files @dots{}}
6899
@end example
6900
6901
@noindent
6902
When you do this, @code{$1} is the same as @code{$0}.
6394
6903
6395
6904
@node Field Splitting Summary
6396
6905
@subsection Field-Splitting Summary
6432
6941
@sidebar Changing @code{FS} Does Not Affect the Fields
6433
6942
6434
6943
@cindex POSIX @command{awk}, field separators and
6435
@cindex field separators, POSIX and
6944
@cindex field separator, POSIX and
6436
6945
According to the POSIX standard, @command{awk} is supposed to behave
6437
6946
as if each record is split into fields at the time it is read.
6438
6947
In particular, this means that if you change the value of @code{FS}
6502
7011
@node Constant Size
6503
7012
@section Reading Fixed-Width Data
6504
7013
6505
@ifnotinfo
6506
7014
@quotation NOTE
6507
7015
This @value{SECTION} discusses an advanced
6508
7016
feature of @command{gawk}. If you are a novice @command{awk} user,
6509
7017
you might want to skip it on the first reading.
6510
7018
@end quotation
6511
@end ifnotinfo
6512
6513
@ifinfo
6514
(This @value{SECTION} discusses an advanced feature of @command{awk}.
6515
If you are a novice @command{awk} user, you might want to skip it on
6516
the first reading.)
6517
@end ifinfo
6518
7019
6519
7020
@cindex data, fixed-width
6520
7021
@cindex fixed-width data
6612
7113
to simplify reading the data. (Of course, getting @command{gawk} to run on
6613
7114
a system with card readers is another story!)
6614
7115
6615
@ignore
6616
Exercise: Write a ballot card reading program
6617
@end ignore
6618
6619
7116
@cindex @command{gawk}, splitting fields and
6620
7117
Assigning a value to @code{FS} causes @command{gawk} to use
6621
7118
@code{FS} for field splitting again. Use @samp{FS = FS} to make this happen,
6632
7129
else if (PROCINFO["FS"] == "FIELDWIDTHS")
6633
7130
@var{fixed-width field splitting} @dots{}
6634
7131
else
6635
@var{content-based field splitting} @dots{} (see next @value{SECTION})
7132
@var{content-based field splitting} @dots{} @ii{(see next @value{SECTION})}
6636
7133
@end example
6637
7134
6638
7135
This information is useful when writing a function
6644
7141
@node Splitting By Content
6645
7142
@section Defining Fields By Content
6646
7143
6647
@ifnotinfo
6648
7144
@quotation NOTE
6649
7145
This @value{SECTION} discusses an advanced
6650
7146
feature of @command{gawk}. If you are a novice @command{awk} user,
6651
7147
you might want to skip it on the first reading.
6652
7148
@end quotation
6653
@end ifnotinfo
6654
6655
@ifinfo
6656
(This @value{SECTION} discusses an advanced feature of @command{awk}.
6657
If you are a novice @command{awk} user, you might want to skip it on
6658
the first reading.)
6659
@end ifinfo
6660
7149
6661
7150
@cindex advanced features, specifying field content
6662
7151
Normally, when using @code{FS}, @command{gawk} defines the fields as the
6754
7243
Since there is no formal specification for CSV data, there isn't much
6755
7244
more to be done;
6756
7245
the @code{FPAT} mechanism provides an elegant solution for the majority
6757
of cases, and the @command{gawk} maintainer is satisfied with that.
7246
of cases, and the @command{gawk} developers are satisfied with that.
6758
7247
@end quotation
6759
7248
6760
7249
As written, the regexp used for @code{FPAT} requires that each field
6771
7260
@node Multiple Line
6772
7261
@section Multiple-Line Records
6773
7262
7263
@cindex multiple-line records
6774
7264
@c STARTOFRANGE recm
6775
7265
@cindex records, multiline
6776
7266
@c STARTOFRANGE imr
6815
7305
appear in a row, they are considered one record separator.
6816
7306
6817
7307
@cindex dark corner, multiline records
6818
There is an important difference between @samp{RS = ""} and
7308
However, there is an important difference between @samp{RS = ""} and
6819
7309
@samp{RS = "\n\n+"}. In the first case, leading newlines in the input
6820
data file are ignored, and if a file ends without extra blank lines
7310
@value{DF} are ignored, and if a file ends without extra blank lines
6821
7311
after the last record, the final newline is removed from the record.
6822
7312
In the second case, this special processing is not done.
6823
7313
@value{DARKCORNER}
6824
7314
6825
@cindex field separators, in multiline records
7315
@cindex field separator, in multiline records
7316
@cindex @code{FS}, in multiline records
6826
7317
Now that the input is separated into records, the second step is to
6827
7318
separate the fields in the record. One way to do this is to divide each
6828
7319
of the lines into fields in the normal manner. This happens by default
6852
7343
put each field on a separate line: to do this, just set the
6853
7344
variable @code{FS} to the string @code{"\n"}. (This single
6854
7345
character separator matches a single newline.)
6855
A practical example of a data file organized this way might be a mailing
7346
A practical example of a @value{DF} organized this way might be a mailing
6856
7347
list, where each entry is separated by blank lines. Consider a mailing
6857
7348
list in a file named @file{addresses}, which looks like this:
6858
7349
6917
7408
@table @code
6918
7409
@item RS == "\n"
6919
7410
Records are separated by the newline character (@samp{\n}). In effect,
6920
every line in the data file is a separate record, including blank lines.
7411
every line in the @value{DF} is a separate record, including blank lines.
6921
7412
This is the default.
6922
7413
6923
7414
@item RS == @var{any single character}
6953
7444
6954
7445
@c STARTOFRANGE getl
6955
7446
@cindex @code{getline} command, explicit input with
7447
@c STARTOFRANGE inex
6956
7448
@cindex input, explicit
6957
7449
So far we have been getting our input data from @command{awk}'s main
6958
input stream---either the standard input (usually your terminal, sometimes
7450
input stream---either the standard input (usually your keyboard, sometimes
6959
7451
the output from another program) or from the
6960
7452
files specified on the command line. The @command{awk} language has a
6961
7453
special built-in command called @code{getline} that
6966
7458
The examples that follow the explanation of the @code{getline} command
6967
7459
include material that has not been covered yet. Therefore, come back
6968
7460
and study the @code{getline} command @emph{after} you have reviewed the
6969
rest of this @value{DOCUMENT} and have a good knowledge of how @command{awk} works.
7461
rest of
7462
@ifinfo
7463
this @value{DOCUMENT}
7464
@end ifinfo
7465
@ifhtml
7466
this @value{DOCUMENT}
7467
@end ifhtml
7468
@ifnotinfo
7469
@ifnothtml
7470
Parts I and II
7471
@end ifnothtml
7472
@end ifnotinfo
7473
and have a good knowledge of how @command{awk} works.
6970
7474
6971
7475
@cindex @command{gawk}, @code{ERRNO} variable in
6972
@cindex @code{ERRNO} variable
7476
@cindex @code{ERRNO} variable, with @command{getline} command
6973
7477
@cindex differences in @command{awk} and @command{gawk}, @code{getline} command
6974
7478
@cindex @code{getline} command, return values
6975
@cindex @code{--sandbox} option, input redirection with @command{getline}
7479
@cindex @option{--sandbox} option, input redirection with @code{getline}
6976
7480
6977
7481
The @code{getline} command returns one if it finds a record and zero if
6978
7482
it encounters the end of the file. If there is some error in getting
7045
7549
expression. (This program has a subtle problem---it does not work if one
7046
7550
comment ends and another begins on the same line.)
7047
7551
7048
@ignore
7049
Exercise,
7050
write a program that does handle multiple comments on the line.
7051
@end ignore
7052
7053
7552
This form of the @code{getline} command sets @code{NF},
7054
7553
@code{NR}, @code{FNR}, @code{RT}, and the value of @code{$0}.
7055
7554
7065
7564
7066
7565
@node Getline/Variable
7067
7566
@subsection Using @code{getline} into a Variable
7567
@cindex @code{getline} into a variable
7068
7568
@cindex variables, @code{getline} command into@comma{} using
7069
7569
7070
7570
You can use @samp{getline @var{var}} to read the next record from
7116
7616
@node Getline/File
7117
7617
@subsection Using @code{getline} from a File
7118
7618
7619
@cindex @code{getline} from a file
7119
7620
@cindex input redirection
7120
7621
@cindex redirection of input
7121
7622
@cindex @code{<} (left angle bracket), @code{<} operator (I/O)
7123
7624
@cindex operators, input/output
7124
7625
Use @samp{getline < @var{file}} to read the next record from @var{file}.
7125
7626
Here @var{file} is a string-valued expression that
7126
specifies the file name. @samp{< @var{file}} is called a @dfn{redirection}
7627
specifies the @value{FN}. @samp{< @var{file}} is called a @dfn{redirection}
7127
7628
because it directs input to come from a different place.
7128
7629
For example, the following
7129
7630
program reads its input record from the file @file{secondary.input} when it
7151
7652
According to POSIX, @samp{getline < @var{expression}} is ambiguous if
7152
7653
@var{expression} contains unparenthesized operators other than
7153
7654
@samp{$}; for example, @samp{getline < dir "/" file} is ambiguous
7154
because the concatenation operator is not parenthesized. You should
7155
write it as @samp{getline < (dir "/" file)} if you want your program
7156
to be portable to all @command{awk} implementations.
7655
because the concatenation operator (not discussed yet; @pxref{Concatenation})
7656
is not parenthesized. You should write it as @samp{getline < (dir "/" file)} if
7657
you want your program to be portable to all @command{awk} implementations.
7157
7658
7158
7659
@node Getline/Variable/File
7159
7660
@subsection Using @code{getline} into a Variable from a File
7164
7665
@var{file}, and put it in the variable @var{var}. As above, @var{file}
7165
7666
is a string-valued expression that specifies the file from which to read.
7166
7667
7167
@cindex @command{gawk}, @code{RT} variable in
7168
@cindex @code{RT} variable
7169
7668
In this version of @code{getline}, none of the built-in variables are
7170
7669
changed and the record is not split into fields. The only variable
7171
7670
changed is @var{var}.@footnote{This is not quite true. @code{RT} could
7188
7687
7189
7688
Note here how the name of the extra input file is not built into
7190
7689
the program; it is taken directly from the data, specifically from the second field on
7191
the @samp{@@include} line.
7690
the @code{@@include} line.
7192
7691
7193
@cindex @code{close()} function
7194
7692
The @code{close()} function is called to ensure that if two identical
7195
@samp{@@include} lines appear in the input, the entire specified file is
7693
@code{@@include} lines appear in the input, the entire specified file is
7196
7694
included twice.
7197
7695
@xref{Close Files And Pipes}.
7198
7696
7199
7697
One deficiency of this program is that it does not process nested
7200
@samp{@@include} statements
7201
(i.e., @samp{@@include} statements in included files)
7698
@code{@@include} statements
7699
(i.e., @code{@@include} statements in included files)
7202
7700
the way a true macro preprocessor would.
7203
7701
@xref{Igawk Program}, for a program
7204
that does handle nested @samp{@@include} statements.
7702
that does handle nested @code{@@include} statements.
7205
7703
7206
7704
@node Getline/Pipe
7207
7705
@subsection Using @code{getline} from a Pipe
7208
7706
7209
7707
@c From private email, dated October 2, 1988. Used by permission, March 2013.
7708
@cindex Kernighan, Brian
7210
7709
@quotation
7211
7710
@i{Omniscience has much to recommend it.
7212
7711
Failing that, attention to details would be useful.}
7216
7715
@cindex @code{|} (vertical bar), @code{|} operator (I/O)
7217
7716
@cindex vertical bar (@code{|}), @code{|} operator (I/O)
7218
7717
@cindex input pipeline
7219
@cindex pipes, input
7718
@cindex pipe, input
7220
7719
@cindex operators, input/output
7221
7720
The output of a command can also be piped into @code{getline}, using
7222
7721
@samp{@var{command} | getline}. In
7240
7739
@end example
7241
7740
7242
7741
@noindent
7243
@cindex @code{close()} function
7244
7742
The @code{close()} function is called to ensure that if two identical
7245
7743
@samp{@@execute} lines appear in the input, the command is run for
7246
7744
each one.
7247
7745
@ifnottex
7746
@ifnotdocbook
7248
7747
@xref{Close Files And Pipes}.
7748
@end ifnotdocbook
7249
7749
@end ifnottex
7250
@c Exercise!!
7251
7750
@c This example is unrealistic, since you could just use system
7252
7751
Given the input:
7253
7752
7294
7793
write it as @samp{(@w{"echo "} "date") | getline} if you want your program
7295
7794
to be portable to all @command{awk} implementations.
7296
7795
7796
@cindex Brian Kernighan's @command{awk}
7797
@cindex @command{mawk} utility
7297
7798
@quotation NOTE
7298
7799
Unfortunately, @command{gawk} has not been consistent in its treatment
7299
7800
of a construct like @samp{@w{"echo "} "date" | getline}.
7405
7906
Here are some miscellaneous points about @code{getline} that
7406
7907
you should bear in mind:
7407
7908
7408
@itemize @bullet
7909
@itemize @value{BULLET}
7409
7910
@item
7410
7911
When @code{getline} changes the value of @code{$0} and @code{NF},
7411
7912
@command{awk} does @emph{not} automatically jump to the start of the
7417
7918
@cindex @command{awk}, implementations, limits
7418
7919
@cindex @command{gawk}, implementation issues, limits
7419
7920
@item
7420
Many @command{awk} implementations limit the number of pipelines that an @command{awk}
7921
Some very old @command{awk} implementations limit the number of pipelines that an @command{awk}
7421
7922
program may have open to just one. In @command{gawk}, there is no such limit.
7422
7923
You can open as many pipelines (and coprocesses) as the underlying operating
7423
7924
system permits.
7430
7931
@item
7431
7932
An interesting side effect occurs if you use @code{getline} without a
7432
7933
redirection inside a @code{BEGIN} rule. Because an unredirected @code{getline}
7433
reads from the command-line data files, the first @code{getline} command
7934
reads from the command-line @value{DF}s, the first @code{getline} command
7434
7935
causes @command{awk} to set the value of @code{FILENAME}. Normally,
7435
7936
@code{FILENAME} does not have a value inside @code{BEGIN} rules, because you
7436
have not yet started to process the command-line data files.
7937
have not yet started to process the command-line @value{DF}s.
7437
7938
@value{DARKCORNER}
7438
7939
(@xref{BEGIN/END},
7439
7940
also @pxref{Auto-set}.)
7456
7957
@command{awk} to start reading a new input file.
7457
7958
7458
7959
@item
7960
@cindex Moore, Duncan
7459
7961
If the variable being assigned is an expression with side effects,
7460
7962
different versions of @command{awk} behave differently upon encountering
7461
7963
end-of-file. Some versions don't evaluate the expression; many versions
7480
7982
7481
7983
@command{gawk} treats @code{getline} like a function call, and evaluates
7482
7984
the expression @samp{a[++c]} before attempting to read from @file{f}.
7483
Other versions of @command{awk} only evaluate the expression once they
7985
However, some versions of @command{awk} only evaluate the expression once they
7484
7986
know that there is a string value to be assigned. Caveat Emptor.
7485
7987
@end itemize
7486
7988
7516
8018
@section Reading Input With A Timeout
7517
8019
@cindex timeout, reading input
7518
8020
7519
You may specify a timeout in milliseconds for reading input from a terminal,
7520
pipe or two-way communication including, TCP/IP sockets. This can be done
8021
@cindex differences in @command{awk} and @command{gawk}, read timeouts
8022
This @value{SECTION} describes a feature that is specific to @command{gawk}.
8023
8024
You may specify a timeout in milliseconds for reading input from the keyboard,
8025
a pipe, or two-way communication, including TCP/IP sockets. This can be done
7521
8026
on a per input, command or connection basis, by setting a special element
7522
in the @code{PROCINFO} array:
8027
in the @code{PROCINFO} (@pxref{Auto-set}) array:
7523
8028
7524
8029
@example
7525
8030
PROCINFO["input_name", "READ_TIMEOUT"] = @var{timeout in milliseconds}
7539
8044
print ERRNO
7540
8045
@end example
7541
8046
7542
Here is how to read interactively from the terminal@footnote{This assumes
7543
that standard input is the keyboard} without waiting
8047
Here is how to read interactively from the user@footnote{This assumes
8048
that standard input is the keyboard.} without waiting
7544
8049
for more than five seconds:
7545
8050
7546
8051
@example
7549
8054
print $0
7550
8055
@end example
7551
8056
7552
@command{gawk} will terminate the read operation if input does not
7553
arrive after waiting for the timeout period, return failure
7554
and set the @code{ERRNO} variable to an appropriate string value.
8057
@command{gawk} terminates the read operation if input does not
8058
arrive after waiting for the timeout period, returns failure
8059
and sets the @code{ERRNO} variable to an appropriate string value.
7555
8060
A negative or zero value for the timeout is the same as specifying
7556
8061
no timeout at all.
7557
8062
7558
A timeout can also be set for reading from the terminal in the implicit
8063
A timeout can also be set for reading from the keyboard in the implicit
7559
8064
loop that reads input records and matches them against patterns,
7560
8065
like so:
7561
8066
7618
8123
7619
8124
@node Command line directories
7620
8125
@section Directories On The Command Line
8126
@cindex differences in @command{awk} and @command{gawk}, command line directories
7621
8127
@cindex directories, command line
7622
8128
@cindex command line, directories on
7623
8129
7624
8130
According to the POSIX standard, files named on the @command{awk}
7625
command line must be text files. It is a fatal error if they are not.
8131
command line must be text files; it is a fatal error if they are not.
7626
8132
Most versions of @command{awk} treat a directory on the command line as
7627
8133
a fatal error.
7628
8134
7629
8135
By default, @command{gawk} produces a warning for a directory on the
7630
command line, but otherwise ignores it. If either of the @option{--posix}
8136
command line, but otherwise ignores it. This makes it easier to use
8137
shell wildcards with your @command{awk} program:
8138
8139
@example
8140
$ @kbd{gawk -f whizprog.awk *} @ii{Directories could kill this progam}
8141
@end example
8142
8143
If either of the @option{--posix}
7631
8144
or @option{--traditional} options is given, then @command{gawk} reverts
7632
8145
to treating a directory on the command line as a fatal error.
7633
8146
8147
@xref{Extension Sample Readdir}, for a way to treat directories
8148
as usable data from an @command{awk} program.
8149
8150
@node Input Summary
8151
@section Summary
8152
8153
@itemize @value{BULLET}
8154
@item
8155
Input is split into records based on the value of @code{RS}.
8156
The possibilities are as follows:
8157
8158
@multitable @columnfractions .25 .35 .40
8159
@headitem Value of @code{RS} @tab Records are split on @tab @command{awk} / @command{gawk}
8160
@item Any single character @tab That character @tab @command{awk}
8161
@item The empty string (@code{""}) @tab Runs of two or more newlines @tab @command{awk}
8162
@item A regexp @tab Text that matches the regexp @tab @command{gawk}
8163
@end multitable
8164
8165
@item
8166
@command{gawk} sets @code{RT} to the text matched by @code{RS}.
8167
8168
@item
8169
After splitting the input into records, @command{awk} further splits
8170
the record into individual fields, named @code{$1}, @code{$2} and so
8171
on. @code{$0} is the whole record, and @code{NF} indicates how many
8172
fields there are. The default way to split fields is between whitespace
8173
characters.
8174
8175
@item
8176
Fields may be referenced using a variable, as in @samp{$NF}. Fields
8177
may also be assigned values, which causes the value of @code{$0} to be
8178
recomputed when it is later referenced. Assigning to a field with a number
8179
greater than @code{NF} creates the field and rebuilds the record, using
8180
@code{OFS} to separate the fields. Incrementing @code{NF} does the same
8181
thing. Decrementing @code{NF} throws away fields and rebuilds the record.
8182
8183
@item
8184
Field splitting is more complicated than record splitting.
8185
8186
@multitable @columnfractions .40 .40 .20
8187
@headitem Field separator value @tab Fields are split @dots{} @tab @command{awk} / @command{gawk}
8188
@item @code{FS == " "} @tab On runs of whitespace @tab @command{awk}
8189
@item @code{FS == @var{any single character}} @tab On that character @tab @command{awk}
8190
@item @code{FS == @var{regexp}} @tab On text matching the regexp @tab @command{awk}
8191
@item @code{FS == ""} @tab Each individual character is a separate field @tab @command{gawk}
8192
@item @code{FIELDWIDTHS == @var{list of columns}} @tab Based on character position @tab @command{gawk}
8193
@item @code{FPAT == @var{regexp}} @tab On text around text matching the regexp @tab @command{gawk}
8194
@end multitable
8195
8196
Using @samp{FS = "\n"} causes the entire record to be a single field
8197
(assuming that newlines separate records).
8198
8199
@item
8200
@code{FS} may be set from the command line using the @option{-F} option.
8201
This can also be done using command-line variable assignment.
8202
8203
@item
8204
@code{PROCINFO["FS"]} can be used to see how fields are being split.
8205
8206
@item
8207
Use @code{getline} in its various forms to read additional records,
8208
from the default input stream, from a file, or from a pipe or co-process.
8209
8210
@item
8211
Use @code{PROCINFO[@var{file}, "READ_TIMEOUT"]} to cause reads to timeout
8212
for @var{file}.
8213
8214
@item
8215
Directories on the command line are fatal for standard @command{awk};
8216
@command{gawk} ignores them if not in POSIX mode.
8217
8218
@end itemize
8219
8220
@node Input Exercises
8221
@section Exercises
8222
8223
@enumerate
8224
@item
8225
Using the @code{FIELDWIDTHS} variable (@pxref{Constant Size}),
8226
write a program to read election data, where each record represents
8227
one voter's votes. Come up with a way to define which columns are
8228
associated with each ballot item, and print the total votes,
8229
including abstentions, for each item.
8230
8231
@item
8232
@ref{Plain Getline}, presented a program to remove C-style
8233
comments (@samp{/* @dots{} */}) from the input. That program
8234
does not work if one comment ends on one line and another one
8235
starts later on the same line.
8236
Write a program that does handle multiple comments on the line.
8237
8238
@end enumerate
8239
7634
8240
@node Printing
7635
8241
@chapter Printing Output
7636
8242
7655
8261
@cindex @code{printf} statement
7656
8262
Besides basic and formatted printing, this @value{CHAPTER}
7657
8263
also covers I/O redirections to files and pipes, introduces
7658
the special file names that @command{gawk} processes internally,
8264
the special @value{FN}s that @command{gawk} processes internally,
7659
8265
and discusses the @code{close()} built-in function.
7660
8266
7661
8267
@menu
7670
8276
@command{gawk} allows access to inherited file
7671
8277
descriptors.
7672
8278
* Close Files And Pipes:: Closing Input and Output Files and Pipes.
8279
* Output Summary:: Output summary.
8280
* Output exercises:: Exercises.
7673
8281
@end menu
7674
8282
7675
8283
@node Print
7676
8284
@section The @code{print} Statement
7677
8285
7678
8286
The @code{print} statement is used for producing output with simple, standardized
7679
formatting. Specify only the strings or numbers to print, in a
8287
formatting. You specify only the strings or numbers to print, in a
7680
8288
list separated by commas. They are output, separated by single spaces,
7681
8289
followed by a newline. The statement looks like this:
7682
8290
7759
8367
To someone unfamiliar with the @file{inventory-shipped} file, neither
7760
8368
example's output makes much sense. A heading line at the beginning
7761
8369
would make it clearer. Let's add some headings to our table of months
7762
(@code{$1}) and green crates shipped (@code{$2}). We do this using the
7763
@code{BEGIN} pattern
7764
(@pxref{BEGIN/END})
7765
so that the headings are only printed once:
8370
(@code{$1}) and green crates shipped (@code{$2}). We do this using
8371
a @code{BEGIN} rule (@pxref{BEGIN/END}) so that the headings are only
8372
printed once:
7766
8373
7767
8374
@example
7768
8375
awk 'BEGIN @{ print "Month Crates"
7848
8455
record, separated by a semicolon, with a blank line added after each
7849
8456
newline:
7850
8457
7851
@ignore
7852
Exercise,
7853
Rewrite the
7854
@example
7855
awk 'BEGIN @{ print "Month Crates"
7856
print "----- ------" @}
7857
@{ print $1, " ", $2 @}' inventory-shipped
7858
@end example
7859
program by using a new value of @code{OFS}.
7860
@end ignore
7861
8458
7862
8459
@example
7863
8460
$ @kbd{awk 'BEGIN @{ OFS = ";"; ORS = "\n\n" @}}
7864
> @kbd{@{ print $1, $2 @}' BBS-list}
7865
@print{} aardvark;555-5553
7866
@print{}
7867
@print{} alpo-net;555-3412
7868
@print{}
7869
@print{} barfly;555-7685
7870
@dots{}
8461
> @kbd{@{ print $1, $2 @}' mail-list}
8462
@print{} Amelia;555-5553
8463
@print{}
8464
@print{} Anthony;555-3412
8465
@print{}
8466
@print{} Becky;555-7685
8467
@print{}
8468
@print{} Bill;555-1675
8469
@print{}
8470
@print{} Broderick;555-0542
8471
@print{}
8472
@print{} Camilla;555-2912
8473
@print{}
8474
@print{} Fabius;555-1234
8475
@print{}
8476
@print{} Julie;555-6699
8477
@print{}
8478
@print{} Martin;555-6480
8479
@print{}
8480
@print{} Samuel;555-3430
8481
@print{}
8482
@print{} Jean-Paul;555-2127
8483
@print{}
7871
8484
@end example
7872
8485
7873
8486
If the value of @code{ORS} does not contain a newline, the program's output
7889
8502
more fully in
7890
8503
@ref{Control Letters}.
7891
8504
7892
@cindex @code{sprintf()} function
8505
@cindexawkfunc{sprintf}
7893
8506
@cindex @code{OFMT} variable
7894
8507
@cindex output, format specifier@comma{} @code{OFMT}
7895
8508
The built-in variable @code{OFMT} contains the default format specification
7955
8568
relational operator; otherwise, it can be confused with an output redirection
7956
8569
(@pxref{Redirection}).
7957
8570
7958
@cindex format strings
8571
@cindex format specifiers
7959
8572
The difference between @code{printf} and @code{print} is the @var{format}
7960
8573
argument. This is an expression whose value is taken as a string; it
7961
8574
specifies how to output each of the other arguments. It is called the
7998
8611
optional @dfn{modifiers} that control @emph{how} to print the value, such as
7999
8612
the field width. Here is a list of the format-control letters:
8000
8613
8001
@table @code
8002
@item %c
8614
@c @asis for docbook to come out right
8615
@table @asis
8616
@item @code{%c}
8003
8617
Print a number as an ASCII character; thus, @samp{printf "%c",
8004
8618
65} outputs the letter @samp{A}. The output for a string value is
8005
8619
the first character of the string.
8007
8621
@cindex dark corner, format-control characters
8008
8622
@cindex @command{gawk}, format-control characters
8009
8623
@quotation NOTE
8010
@ignore
8011
The @samp{%c} format does @emph{not} handle values outside the range
8012
0--255. On most systems, values from 0--127 are within the range of
8013
ASCII and will yield an ASCII character. Values in the range 128--255
8014
may format as characters in some extended character set, or they may not.
8015
System 390 (IBM architecture mainframe) systems use 8-bit characters,
8016
and thus values from 0--255 yield the corresponding EBCDIC character.
8017
Any value above 255 is treated as modulo 255; i.e., the lowest eight bits
8018
of the value are used. The locale and character set are always ignored.
8019
@end ignore
8020
8624
The POSIX standard says the first character of a string is printed.
8021
8625
In locales with multibyte characters, @command{gawk} attempts to
8022
8626
convert the leading bytes of the string into a valid wide character
8031
8635
@end quotation
8032
8636
8033
8637
8034
@item %d@r{,} %i
8638
@item @code{%d}, @code{%i}
8035
8639
Print a decimal integer.
8036
8640
The two control letters are equivalent.
8037
8641
(The @samp{%i} specification is for compatibility with ISO C.)
8038
8642
8039
@item %e@r{,} %E
8643
@item @code{%e}, @code{%E}
8040
8644
Print a number in scientific (exponential) notation;
8041
8645
for example:
8042
8646
8051
8655
discussed in the next @value{SUBSECTION}.)
8052
8656
@samp{%E} uses @samp{E} instead of @samp{e} in the output.
8053
8657
8054
@item %f
8658
@item @code{%f}
8055
8659
Print a number in floating-point notation.
8056
8660
For example:
8057
8661
8071
8675
@samp{-inf} or @samp{-infinity},
8072
8676
and positive infinity as
8073
8677
@samp{inf} and @samp{infinity}.
8074
The special ``not a number'' value formats as @samp{-nan} or @samp{nan}.
8678
The special ``not a number'' value formats as @samp{-nan} or @samp{nan}
8679
(@pxref{Math Definitions}).
8075
8680
8076
@item %F
8681
@item @code{%F}
8077
8682
Like @samp{%f} but the infinity and ``not a number'' values are spelled
8078
8683
using uppercase letters.
8079
8684
8080
8685
The @samp{%F} format is a POSIX extension to ISO C; not all systems
8081
8686
support it. On those that don't, @command{gawk} uses @samp{%f} instead.
8082
8687
8083
@item %g@r{,} %G
8688
@item @code{%g}, @code{%G}
8084
8689
Print a number in either scientific notation or in floating-point
8085
8690
notation, whichever uses fewer characters; if the result is printed in
8086
8691
scientific notation, @samp{%G} uses @samp{E} instead of @samp{e}.
8087
8692
8088
@item %o
8693
@item @code{%o}
8089
8694
Print an unsigned octal integer
8090
8695
(@pxref{Nondecimal-numbers}).
8091
8696
8092
@item %s
8697
@item @code{%s}
8093
8698
Print a string.
8094
8699
8095
@item %u
8700
@item @code{%u}
8096
8701
Print an unsigned decimal integer.
8097
8702
(This format is of marginal use, because all numbers in @command{awk}
8098
8703
are floating-point; it is provided primarily for compatibility with C.)
8099
8704
8100
@item %x@r{,} %X
8705
@item @code{%x}, @code{%X}
8101
8706
Print an unsigned hexadecimal integer;
8102
8707
@samp{%X} uses the letters @samp{A} through @samp{F}
8103
8708
instead of @samp{a} through @samp{f}
8104
8709
(@pxref{Nondecimal-numbers}).
8105
8710
8106
@item %%
8711
@item @code{%%}
8107
8712
Print a single @samp{%}.
8108
8713
This does not consume an
8109
8714
argument and it ignores any modifiers.
8138
8743
@table @code
8139
8744
@cindex differences in @command{awk} and @command{gawk}, @code{print}/@code{printf} statements
8140
8745
@cindex @code{printf} statement, positional specifiers
8141
@c the command does NOT start a secondary
8746
@c the code{} does NOT start a secondary
8142
8747
@cindex positional specifiers, @code{printf} statement
8143
8748
@item @var{N}$
8144
8749
An integer constant followed by a @samp{$} is a @dfn{positional specifier}.
8214
8819
$ @kbd{cat thousands.awk} @ii{Show source program}
8215
8820
@print{} BEGIN @{ printf "%'d\n", 1234567 @}
8216
8821
$ @kbd{LC_ALL=C gawk -f thousands.awk}
8217
@print{} 1234567 @ii{Results in "C" locale}
8822
@print{} 1234567 @ii{Results in} "C" @ii{locale}
8218
8823
$ @kbd{LC_ALL=en_US.UTF-8 gawk -f thousands.awk}
8219
8824
@print{} 1,234,567 @ii{Results in US English UTF locale}
8220
8825
@end example
8324
8929
@c @cindex lint checks
8325
8930
@cindex troubleshooting, fatal errors, @code{printf} format strings
8326
8931
@cindex POSIX @command{awk}, @code{printf} format strings and
8327
C programmers may be used to supplying additional
8328
@samp{l}, @samp{L}, and @samp{h}
8329
modifiers in @code{printf} format strings. These are not valid in @command{awk}.
8330
Most @command{awk} implementations silently ignore them.
8331
If @option{--lint} is provided on the command line
8332
(@pxref{Options}),
8333
@command{gawk} warns about their use. If @option{--posix} is supplied,
8334
their use is a fatal error.
8932
C programmers may be used to supplying additional modifiers (@samp{h},
8933
@samp{j}, @samp{l}, @samp{L}, @samp{t}, and @samp{z}) in @code{printf}
8934
format strings. These are not valid in @command{awk}. Most @command{awk}
8935
implementations silently ignore them. If @option{--lint} is provided
8936
on the command line (@pxref{Options}), @command{gawk} warns about their
8937
use. If @option{--posix} is supplied, their use is a fatal error.
8335
8938
@c ENDOFRANGE pfm
8336
8939
8337
8940
@node Printf Examples
8341
8944
how to use @code{printf} to make an aligned table:
8342
8945
8343
8946
@example
8344
awk '@{ printf "%-10s %s\n", $1, $2 @}' BBS-list
8947
awk '@{ printf "%-10s %s\n", $1, $2 @}' mail-list
8345
8948
@end example
8346
8949
8347
8950
@noindent
8348
8951
This command
8349
prints the names of the bulletin boards (@code{$1}) in the file
8350
@file{BBS-list} as a string of 10 characters that are left-justified. It also
8952
prints the names of the people (@code{$1}) in the file
8953
@file{mail-list} as a string of 10 characters that are left-justified. It also
8351
8954
prints the phone numbers (@code{$2}) next on the line. This
8352
8955
produces an aligned two-column table of names and phone numbers,
8353
8956
as shown here:
8354
8957
8355
8958
@example
8356
$ @kbd{awk '@{ printf "%-10s %s\n", $1, $2 @}' BBS-list}
8357
@print{} aardvark 555-5553
8358
@print{} alpo-net 555-3412
8359
@print{} barfly 555-7685
8360
@print{} bites 555-1675
8361
@print{} camelot 555-0542
8362
@print{} core 555-2912
8363
@print{} fooey 555-1234
8364
@print{} foot 555-6699
8365
@print{} macfoo 555-6480
8366
@print{} sdace 555-3430
8367
@print{} sabafoo 555-2127
8959
$ @kbd{awk '@{ printf "%-10s %s\n", $1, $2 @}' mail-list}
8960
@print{} Amelia 555-5553
8961
@print{} Anthony 555-3412
8962
@print{} Becky 555-7685
8963
@print{} Bill 555-1675
8964
@print{} Broderick 555-0542
8965
@print{} Camilla 555-2912
8966
@print{} Fabius 555-1234
8967
@print{} Julie 555-6699
8968
@print{} Martin 555-6480
8969
@print{} Samuel 555-3430
8970
@print{} Jean-Paul 555-2127
8368
8971
@end example
8369
8972
8370
8973
In this case, the phone numbers had to be printed as strings because
8377
8980
after them.
8378
8981
8379
8982
The table could be made to look even nicer by adding headings to the
8380
tops of the columns. This is done using the @code{BEGIN} pattern
8983
tops of the columns. This is done using a @code{BEGIN} rule
8381
8984
(@pxref{BEGIN/END})
8382
8985
so that the headers are only printed once, at the beginning of
8383
8986
the @command{awk} program:
8385
8988
@example
8386
8989
awk 'BEGIN @{ print "Name Number"
8387
8990
print "---- ------" @}
8388
@{ printf "%-10s %s\n", $1, $2 @}' BBS-list
8991
@{ printf "%-10s %s\n", $1, $2 @}' mail-list
8389
8992
@end example
8390
8993
8391
8994
The above example mixes @code{print} and @code{printf} statements in
8395
8998
@example
8396
8999
awk 'BEGIN @{ printf "%-10s %s\n", "Name", "Number"
8397
9000
printf "%-10s %s\n", "----", "------" @}
8398
@{ printf "%-10s %s\n", $1, $2 @}' BBS-list
9001
@{ printf "%-10s %s\n", $1, $2 @}' mail-list
8399
9002
@end example
8400
9003
8401
9004
@noindent
8410
9013
awk 'BEGIN @{ format = "%-10s %s\n"
8411
9014
printf format, "Name", "Number"
8412
9015
printf format, "----", "------" @}
8413
@{ printf format, $1, $2 @}' BBS-list
9016
@{ printf format, $1, $2 @}' mail-list
8414
9017
@end example
8415
9018
8416
@c !!! exercise
8417
At this point, it would be a worthwhile exercise to use the
8418
@code{printf} statement to line up the headings and table data for the
8419
@file{inventory-shipped} example that was covered earlier in the @value{SECTION}
8420
on the @code{print} statement
8421
(@pxref{Print}).
8422
9019
@c ENDOFRANGE printfs
8423
9020
8424
9021
@node Redirection
8425
9022
@section Redirecting Output of @code{print} and @code{printf}
8426
9023
9024
@c STARTOFRANGE outre
8427
9025
@cindex output redirection
9026
@c STARTOFRANGE reout
8428
9027
@cindex redirection of output
8429
@cindex @code{--sandbox} option, output redirection with @code{print}, @code{printf}
9028
@cindex @option{--sandbox} option, output redirection with @code{print}, @code{printf}
8430
9029
So far, the output from @code{print} and @code{printf} has gone
8431
9030
to the standard
8432
9031
output, usually the screen. Both @code{print} and @code{printf} can
8443
9042
commands, except that they are written inside the @command{awk} program.
8444
9043
8445
9044
@c the commas here are part of the see also
8446
@cindex @code{print} statement, See Also redirection, of output
8447
@cindex @code{printf} statement, See Also redirection, of output
9045
@cindex @code{print} statement, See Also redirection@comma{} of output
9046
@cindex @code{printf} statement, See Also redirection@comma{} of output
8448
9047
There are four forms of output redirection: output to a file, output
8449
9048
appended to a file, output through a pipe to another command, and output
8450
to a coprocess. They are all shown for the @code{print} statement,
9049
to a coprocess. We show them all for the @code{print} statement,
8451
9050
but they work identically for @code{printf}:
8452
9051
8453
9052
@table @code
8456
9055
@cindex operators, input/output
8457
9056
@item print @var{items} > @var{output-file}
8458
9057
This redirection prints the items into the output file named
8459
@var{output-file}. The file name @var{output-file} can be any
9058
@var{output-file}. The @value{FN} @var{output-file} can be any
8460
9059
expression. Its value is changed to a string and then used as a
8461
file name (@pxref{Expressions}).
9060
@value{FN} (@pxref{Expressions}).
8462
9061
8463
9062
When this type of redirection is used, the @var{output-file} is erased
8464
9063
before the first output is written to it. Subsequent writes to the same
8465
9064
@var{output-file} do not erase @var{output-file}, but append to it.
8466
9065
(This is different from how you use redirections in shell scripts.)
8467
9066
If @var{output-file} does not exist, it is created. For example, here
8468
is how an @command{awk} program can write a list of BBS names to one
9067
is how an @command{awk} program can write a list of peoples' names to one
8469
9068
file named @file{name-list}, and a list of phone numbers to another file
8470
9069
named @file{phone-list}:
8471
9070
8472
9071
@example
8473
9072
$ @kbd{awk '@{ print $2 > "phone-list"}
8474
> @kbd{print $1 > "name-list" @}' BBS-list}
9073
> @kbd{print $1 > "name-list" @}' mail-list}
8475
9074
$ @kbd{cat phone-list}
8476
9075
@print{} 555-5553
8477
9076
@print{} 555-3412
8478
9077
@dots{}
8479
9078
$ @kbd{cat name-list}
8480
@print{} aardvark
8481
@print{} alpo-net
9079
@print{} Amelia
9080
@print{} Anthony
8482
9081
@dots{}
8483
9082
@end example
8484
9083
8496
9095
If @var{output-file} does not exist, then it is created.
8497
9096
8498
9097
@cindex @code{|} (vertical bar), @code{|} operator (I/O)
8499
@cindex pipes, output
9098
@cindex pipe, output
8500
9099
@cindex output, pipes
8501
9100
@item print @var{items} | @var{command}
8502
9101
It is possible to send output to another program through a pipe
8507
9106
The redirection argument @var{command} is actually an @command{awk}
8508
9107
expression. Its value is converted to a string whose contents give
8509
9108
the shell command to be run. For example, the following produces two
8510
files, one unsorted list of BBS names, and one list sorted in reverse
9109
files, one unsorted list of peoples' names, and one list sorted in reverse
8511
9110
alphabetical order:
8512
9111
8513
9112
@ignore
8520
9119
@example
8521
9120
awk '@{ print $1 > "names.unsorted"
8522
9121
command = "sort -r > names.sorted"
8523
print $1 | command @}' BBS-list
9122
print $1 | command @}' mail-list
8524
9123
@end example
8525
9124
8526
9125
The unsorted list is written with an ordinary redirection, while
8552
9151
a @var{file} or @var{command}---it is not necessary to always
8553
9152
use a string constant. Using a variable is generally a good idea,
8554
9153
because (if you mean to refer to that same file or command)
8555
@command{awk} requires that the string value be spelled identically
9154
@command{awk} requires that the string value be written identically
8556
9155
every time.
8557
9156
8558
9157
@cindex coprocesses
8611
9210
many
8612
9211
@end ifnotinfo
8613
9212
@ifnottex
9213
@ifnotdocbook
8614
9214
Many
9215
@end ifnotdocbook
8615
9216
@end ifnottex
8616
9217
older
8617
9218
@command{awk} implementations limit the number of pipelines that an @command{awk}
8624
9225
8625
9226
A particularly powerful way to use redirection is to build command lines
8626
9227
and pipe them into the shell, @command{sh}. For example, suppose you
8627
have a list of files brought over from a system where all the file names
9228
have a list of files brought over from a system where all the @value{FN}s
8628
9229
are stored in uppercase, and you wish to rename them to have names in
8629
9230
all lowercase. The following program is both simple and efficient:
8630
9231
8646
9247
@c ENDOFRANGE reout
8647
9248
8648
9249
@node Special Files
8649
@section Special File Names in @command{gawk}
9250
@section Special @value{FFN} in @command{gawk}
8650
9251
@c STARTOFRANGE gfn
8651
9252
@cindex @command{gawk}, file names in
8652
9253
8653
@command{gawk} provides a number of special file names that it interprets
8654
internally. These file names provide access to standard file descriptors
9254
@command{gawk} provides a number of special @value{FN}s that it interprets
9255
internally. These @value{FN}s provide access to standard file descriptors
8655
9256
and TCP/IP networking.
8656
9257
8657
9258
@menu
8715
9316
terminal at all.
8716
9317
Then opening @file{/dev/tty} fails.
8717
9318
8718
@command{gawk} provides special file names for accessing the three standard
8719
streams. @value{COMMONEXT}. It also provides syntax for accessing
8720
any other inherited open files. If the file name matches
9319
@command{gawk} provides special @value{FN}s for accessing the three standard
9320
streams. @value{COMMONEXT} It also provides syntax for accessing
9321
any other inherited open files. If the @value{FN} matches
8721
9322
one of these special names when @command{gawk} redirects input or output,
8722
then it directly uses the stream that the file name stands for.
8723
These special file names work for all operating systems that @command{gawk}
9323
then it directly uses the stream that the @value{FN} stands for.
9324
These special @value{FN}s work for all operating systems that @command{gawk}
8724
9325
has been ported to, not just those that are POSIX-compliant:
8725
9326
8726
9327
@cindex common extensions, @code{/dev/stdin} special file
8730
9331
@cindex extensions, common@comma{} @code{/dev/stdout} special file
8731
9332
@cindex extensions, common@comma{} @code{/dev/stderr} special file
8732
9333
@cindex file names, standard streams in @command{gawk}
8733
@cindex @code{/dev/@dots{}} special files (@command{gawk})
9334
@cindex @code{/dev/@dots{}} special files
8734
9335
@cindex files, @code{/dev/@dots{}} special files
8735
@cindex @code{/dev/fd/@var{N}} special files
9336
@cindex @code{/dev/fd/@var{N}} special files (@command{gawk})
8736
9337
@table @file
8737
9338
@item /dev/stdin
8738
9339
The standard input (file descriptor 0).
8750
9351
@command{gawk} is invoked, only descriptors 0, 1, and 2 are available.
8751
9352
@end table
8752
9353
8753
The file names @file{/dev/stdin}, @file{/dev/stdout}, and @file{/dev/stderr}
9354
The @value{FN}s @file{/dev/stdin}, @file{/dev/stdout}, and @file{/dev/stderr}
8754
9355
are aliases for @file{/dev/fd/0}, @file{/dev/fd/1}, and @file{/dev/fd/2},
8755
9356
respectively. However, they are more self-explanatory.
8756
9357
The proper way to write an error message in a @command{gawk} program
8761
9362
@end example
8762
9363
8763
9364
@cindex troubleshooting, quotes with file names
8764
Note the use of quotes around the file name.
9365
Note the use of quotes around the @value{FN}.
8765
9366
Like any other redirection, the value must be a string.
8766
9367
It is a common error to omit the quotes, which leads
8767
9368
to confusing results.
8768
@c Exercise: What does it do? :-)
8769
9369
8770
Finally, using the @code{close()} function on a file name of the
9370
Finally, using the @code{close()} function on a @value{FN} of the
8771
9371
form @code{"/dev/fd/@var{N}"}, for file descriptor numbers
8772
9372
above two, does actually close the given file descriptor.
8773
9373
8783
9383
@command{gawk} programs
8784
9384
can open a two-way
8785
9385
TCP/IP connection, acting as either a client or a server.
8786
This is done using a special file name of the form:
9386
This is done using a special @value{FN} of the form:
8787
9387
8788
9388
@example
8789
9389
@file{/@var{net-type}/@var{protocol}/@var{local-port}/@var{remote-host}/@var{remote-port}}
8793
9393
The @var{protocol} is one of @samp{tcp} or @samp{udp},
8794
9394
and the other fields represent the other essential pieces of information
8795
9395
for making a networking connection.
8796
These file names are used with the @samp{|&} operator for communicating
9396
These @value{FN}s are used with the @samp{|&} operator for communicating
8797
9397
with a coprocess
8798
9398
(@pxref{Two-way I/O}).
8799
9399
This is an advanced feature, mentioned here only for completeness.
8801
9401
@ref{TCP/IP Networking}.
8802
9402
8803
9403
@node Special Caveats
8804
@subsection Special File Name Caveats
9404
@subsection Special @value{FFN} Caveats
8805
9405
8806
9406
Here is a list of things to bear in mind when using the
8807
special file names that @command{gawk} provides:
9407
special @value{FN}s that @command{gawk} provides:
8808
9408
8809
@itemize @bullet
9409
@itemize @value{BULLET}
8810
9410
@cindex compatibility mode (@command{gawk}), file names
8811
9411
@cindex file names, in compatibility mode
8812
9412
@item
8813
Recognition of these special file names is disabled if @command{gawk} is in
9413
Recognition of these special @value{FN}s is disabled if @command{gawk} is in
8814
9414
compatibility mode (@pxref{Options}).
8815
9415
8816
9416
@item
8817
9417
@command{gawk} @emph{always}
8818
interprets these special file names.
9418
interprets these special @value{FN}s.
8819
9419
For example, using @samp{/dev/fd/4}
8820
9420
for output actually writes on file descriptor 4, and not on a new
8821
9421
file descriptor that is @code{dup()}'ed from file descriptor 4. Most of
8833
9433
@c STARTOFRANGE ofc
8834
9434
@cindex output, files@comma{} closing
8835
9435
@c STARTOFRANGE pc
8836
@cindex pipes, closing
9436
@cindex pipe, closing
8837
9437
@c STARTOFRANGE cc
8838
9438
@cindex coprocesses, closing
8839
9439
@cindex @code{getline} command, coprocesses@comma{} using from
8840
9440
8841
If the same file name or the same shell command is used with @code{getline}
9441
If the same @value{FN} or the same shell command is used with @code{getline}
8842
9442
more than once during the execution of an @command{awk} program
8843
9443
(@pxref{Getline}),
8844
9444
the file is opened (or the command is executed) the first time only.
8847
9447
another record is read from it, and so on.
8848
9448
8849
9449
Similarly, when a file or pipe is opened for output, @command{awk} remembers
8850
the file name or command associated with it, and subsequent
9450
the @value{FN} or command associated with it, and subsequent
8851
9451
writes to the same file or command are appended to the previous writes.
8852
9452
The file or pipe stays open until @command{awk} exits.
8853
9453
8854
@cindex @code{close()} function
9454
@cindexawkfunc{close}
8855
9455
This implies that special steps are necessary in order to read the same
8856
9456
file again from the beginning, or to rerun a shell command (rather than
8857
9457
reading more output from the same command). The @code{close()} function
8889
9489
file or command, reopens the file or reruns the command.
8890
9490
Because the expression that you use to close a file or pipeline must
8891
9491
exactly match the expression used to open the file or run the command,
8892
it is good practice to use a variable to store the file name or command.
9492
it is good practice to use a variable to store the @value{FN} or command.
8893
9493
The previous example becomes the following:
8894
9494
8895
9495
@example
8903
9503
This helps avoid hard-to-find typographical errors in your @command{awk}
8904
9504
programs. Here are some of the reasons for closing an output file:
8905
9505
8906
@itemize @bullet
9506
@itemize @value{BULLET}
8907
9507
@item
8908
9508
To write a file and read it back later on in the same @command{awk}
8909
9509
program. Close the file after writing it, then
8936
9536
8937
9537
@cindex differences in @command{awk} and @command{gawk}, @code{close()} function
8938
9538
@cindex portability, @code{close()} function and
9539
@cindex @code{close()} function, portability
8939
9540
If you use more files than the system allows you to have open,
8940
9541
@command{gawk} attempts to multiplex the available open files among
8941
your data files. @command{gawk}'s ability to do this depends upon the
9542
your @value{DF}s. @command{gawk}'s ability to do this depends upon the
8942
9543
facilities of your operating system, so it may not always work. It is
8943
9544
therefore both good practice and good portability advice to always
8944
9545
use @code{close()} on your files when you are done with them.
8971
9572
is not closed and released until @code{close()} is called or
8972
9573
@command{awk} exits.
8973
9574
8974
@code{close()} will silently do nothing if given an argument that
9575
@code{close()} silently does nothing if given an argument that
8975
9576
does not represent a file, pipe or coprocess that was opened with
8976
a redirection.
9577
a redirection. In such a case, it returns a negative value,
9578
indicating an error. In addition, @command{gawk} sets @code{ERRNO}
9579
to a string indicating the error.
8977
9580
8978
Note also that @samp{close(FILENAME)} has no
8979
``magic'' effects on the implicit loop that reads through the
8980
files named on the command line. It is, more likely, a close
8981
of a file that was never opened, so @command{awk} silently
8982
does nothing.
9581
Note also that @samp{close(FILENAME)} has no ``magic'' effects on the
9582
implicit loop that reads through the files named on the command line.
9583
It is, more likely, a close of a file that was never opened with a
9584
redirection, so @command{awk} silently does nothing.
8983
9585
8984
9586
@cindex @code{|} (vertical bar), @code{|&} operator (I/O), pipes@comma{} closing
8985
9587
When using the @samp{|&} operator to communicate with a coprocess,
9003
9605
@cindex differences in @command{awk} and @command{gawk}, @code{close()} function
9004
9606
@cindex Unix @command{awk}, @code{close()} function and
9005
9607
9006
In many versions of Unix @command{awk}, the @code{close()} function
9608
In many older versions of Unix @command{awk}, the @code{close()} function
9007
9609
is actually a statement. It is a syntax error to try and use the return
9008
9610
value from @code{close()}:
9009
9611
@value{DARKCORNER}
9015
9617
@end example
9016
9618
9017
9619
@cindex @command{gawk}, @code{ERRNO} variable in
9018
@cindex @code{ERRNO} variable
9620
@cindex @code{ERRNO} variable, with @command{close()} function
9019
9621
@command{gawk} treats @code{close()} as a function.
9020
9622
The return value is @minus{}1 if the argument names something
9021
9623
that was never opened with a redirection, or if there is
9048
9650
@c ENDOFRANGE ofc
9049
9651
@c ENDOFRANGE pc
9050
9652
@c ENDOFRANGE cc
9653
9654
@node Output Summary
9655
@section Summary
9656
9657
@itemize @value{BULLET}
9658
@item
9659
The @code{print} statement prints comma-separated expressions. Each
9660
expression is separated by the value of @code{OFS} and terminated by
9661
the value of @code{ORS}. @code{OFMT} provides the conversion format
9662
for numeric values for the @code{print} statement.
9663
9664
@item
9665
The @code{printf} statement provides finer-grained control over output,
9666
with format control letters for different data types and various flags
9667
that modify the behavior of the format control letters.
9668
9669
@item
9670
Output from both @code{print} and @code{printf} may be redirected to
9671
files, pipes, and co-processes.
9672
9673
@item
9674
@command{gawk} provides special file names for access to standard input,
9675
output and error, and for network communications.
9676
9677
@item
9678
Use @code{close()} to close open file, pipe and co-process redirections.
9679
For co-processes, it is possible to close only one direction of the
9680
communications.
9681
9682
@end itemize
9683
9684
@node Output exercises
9685
@section Exercises
9686
9687
@enumerate
9688
@item
9689
Rewrite the program:
9690
9691
@example
9692
awk 'BEGIN @{ print "Month Crates"
9693
print "----- ------" @}
9694
@{ print $1, " ", $2 @}' inventory-shipped
9695
@end example
9696
9697
@noindent
9698
from @ref{Output Separators}, by using a new value of @code{OFS}.
9699
9700
@item
9701
Use the @code{printf} statement to line up the headings and table data
9702
for the @file{inventory-shipped} example that was covered in @ref{Print}.
9703
9704
@item
9705
What happens if you forget the double quotes when redirecting
9706
output, as follows:
9707
9708
@example
9709
BEGIN @{ print "Serious error detected!" > /dev/stderr @}
9710
@end example
9711
9712
@end enumerate
9713
9051
9714
@c ENDOFRANGE prnt
9052
9715
9053
9716
@node Expressions
9074
9737
* Function Calls:: A function call is an expression.
9075
9738
* Precedence:: How various operators nest.
9076
9739
* Locales:: How the locale affects things.
9740
* Expressions Summary:: Expressions summary.
9077
9741
@end menu
9078
9742
9079
9743
@node Values
9093
9757
9094
9758
@node Constants
9095
9759
@subsection Constant Expressions
9760
9761
@c STARTOFRANGE cnst
9096
9762
@cindex constants, types of
9097
9763
9098
9764
The simplest type of expression is the @dfn{constant}, which always has
9112
9778
@node Scalar Constants
9113
9779
@subsubsection Numeric and String Constants
9114
9780
9115
@cindex numeric, constants
9781
@cindex constants, numeric
9782
@cindex numeric constants
9116
9783
A @dfn{numeric constant} stands for a number. This number can be an
9117
9784
integer, a decimal fraction, or a number in scientific (exponential)
9118
9785
notation.@footnote{The internal representation of all numbers,
9119
including integers, uses double precision
9120
floating-point numbers.
9121
On most modern systems, these are in IEEE 754 standard format.}
9786
including integers, uses double precision floating-point numbers.
9787
On most modern systems, these are in IEEE 754 standard format.
9788
@xref{Arbitrary Precision Arithmetic}, for much more information.}
9122
9789
Here are some examples of numeric constants that all
9123
9790
have the same value:
9124
9791
9138
9805
9139
9806
@noindent
9140
9807
@cindex differences in @command{awk} and @command{gawk}, strings
9141
@cindex strings, length of
9808
@cindex strings, length limitations
9142
9809
represents the string whose contents are @samp{parrot}. Strings in
9143
9810
@command{gawk} can be of any length, and they can contain any of the possible
9144
9811
eight-bit ASCII characters including ASCII @sc{nul} (character code zero).
9325
9992
9326
9993
@cindex differences in @command{awk} and @command{gawk}, regexp constants
9327
9994
@cindex dark corner, regexp constants, as arguments to user-defined functions
9328
@cindex @code{gensub()} function (@command{gawk})
9329
@cindex @code{sub()} function
9330
@cindex @code{gsub()} function
9995
@cindexgawkfunc{gensub}
9996
@cindexawkfunc{sub}
9997
@cindexawkfunc{gsub}
9331
9998
Constant regular expressions are also used as the first argument for
9332
9999
the @code{gensub()}, @code{sub()}, and @code{gsub()} functions, as the
9333
10000
second argument of the @code{match()} function,
9334
and as the third argument of the @code{patsplit()} function
10001
and as the third argument of the @code{split()} and @code{patsplit()} functions
9335
10002
(@pxref{String Functions}).
9336
10003
Modern implementations of @command{awk}, including @command{gawk}, allow
9337
10004
the third argument of @code{split()} to be a regexp constant, but some
9438
10105
@var{variable}=@var{text}
9439
10106
@end example
9440
10107
9441
@cindex @code{-v} option
10108
@cindex @option{-v} option
9442
10109
@noindent
9443
10110
With it, a variable is set either at the beginning of the
9444
10111
@command{awk} run or in between input files.
9452
10119
@noindent
9453
10120
the variable is set at the very beginning, even before the
9454
10121
@code{BEGIN} rules execute. The @option{-v} option and its assignment
9455
must precede all the file name arguments, as well as the program text.
10122
must precede all the @value{FN} arguments, as well as the program text.
9456
10123
(@xref{Options}, for more information about
9457
10124
the @option{-v} option.)
9458
10125
Otherwise, the variable assignment is performed at a time determined by
9460
10127
preceding input file argument. For example:
9461
10128
9462
10129
@example
9463
awk '@{ print $n @}' n=4 inventory-shipped n=2 BBS-list
10130
awk '@{ print $n @}' n=4 inventory-shipped n=2 mail-list
9464
10131
@end example
9465
10132
9466
10133
@noindent
9469
10136
equal to four. This causes the fourth field to be printed in lines from
9470
10137
@file{inventory-shipped}. After the first file has finished,
9471
10138
but before the second file is started, @code{n} is set to two, so that the
9472
second field is printed in lines from @file{BBS-list}:
10139
second field is printed in lines from @file{mail-list}:
9473
10140
9474
10141
@example
9475
$ @kbd{awk '@{ print $n @}' n=4 inventory-shipped n=2 BBS-list}
10142
$ @kbd{awk '@{ print $n @}' n=4 inventory-shipped n=2 mail-list}
9476
10143
@print{} 15
9477
10144
@print{} 24
9478
10145
@dots{}
9493
10160
@node Conversion
9494
10161
@subsection Conversion of Strings and Numbers
9495
10162
10163
Number to string and string to number conversion are generally
10164
straightforward. There can be subtleties to be aware of;
10165
this @value{SECTION} discusses this important facet of @command{awk}.
10166
10167
@menu
10168
* Strings And Numbers:: How @command{awk} Converts Between Strings And
10169
Numbers.
10170
* Locale influences conversions:: How the locale may affect conversions.
10171
@end menu
10172
10173
@node Strings And Numbers
10174
@subsubsection How @command{awk} Converts Between Strings And Numbers
10175
9496
10176
@cindex converting, strings to numbers
9497
10177
@cindex strings, converting
9498
10178
@cindex numbers, converting
9562
10242
@code{b} has the value @code{"12"}, not @code{"12.00"}.
9563
10243
@value{DARKCORNER}
9564
10244
10245
@sidebar Pre-POSIX @command{awk} Used @code{OFMT} For String Conversion
9565
10246
@cindex POSIX @command{awk}, @code{OFMT} variable and
9566
10247
@cindex @code{OFMT} variable
9567
10248
@cindex portability, new @command{awk} vs.@: old @command{awk}
9573
10254
conversion from the semantics of printing. Both @code{CONVFMT} and
9574
10255
@code{OFMT} have the same default value: @code{"%.6g"}. In the vast majority
9575
10256
of cases, old @command{awk} programs do not change their behavior.
9576
However, these semantics for @code{OFMT} are something to keep in mind if you must
9577
port your new-style program to older implementations of @command{awk}.
9578
We recommend
9579
that instead of changing your programs, just port @command{gawk} itself.
9580
@xref{Print},
9581
for more information on the @code{print} statement.
9582
9583
And, once again, where you are can matter when it comes to converting
9584
between numbers and strings. In @ref{Locales}, we mentioned that
9585
the local character set and language (the locale) can affect how
9586
@command{gawk} matches characters. The locale also affects numeric
9587
formats. In particular, for @command{awk} programs, it affects the
9588
decimal point character. The @code{"C"} locale, and most English-language
9589
locales, use the period character (@samp{.}) as the decimal point.
9590
However, many (if not most) European and non-English locales use the comma
9591
(@samp{,}) as the decimal point character.
10257
@xref{Print}, for more information on the @code{print} statement.
10258
@end sidebar
10259
10260
@node Locale influences conversions
10261
@subsubsection Locales Can Influence Conversion
10262
10263
Where you are can matter when it comes to converting between numbers and
10264
strings. The local character set and language---the @dfn{locale}---can
10265
affect numeric formats. In particular, for @command{awk} programs,
10266
it affects the decimal point character and the thousands-separator
10267
character. The @code{"C"} locale, and most English-language locales,
10268
use the period character (@samp{.}) as the decimal point and don't
10269
have a thousands separator. However, many (if not most) European and
10270
non-English locales use the comma (@samp{,}) as the decimal point
10271
character. European locales often use either a space or a period as
10272
the thousands separator, if they have one.
9592
10273
9593
10274
@cindex dark corner, locale's decimal point character
9594
10275
The POSIX standard says that @command{awk} always uses the period as the decimal
9595
point when reading the @command{awk} program source code, and for command-line
9596
variable assignments (@pxref{Other Arguments}).
9597
However, when interpreting input data, for @code{print} and @code{printf} output,
9598
and for number to string conversion, the local decimal point character is used.
9599
@value{DARKCORNER}.
9600
Here are some examples indicating the difference in behavior,
9601
on a GNU/Linux system:
10276
point when reading the @command{awk} program source code, and for
10277
command-line variable assignments (@pxref{Other Arguments}). However,
10278
when interpreting input data, for @code{print} and @code{printf} output,
10279
and for number to string conversion, the local decimal point character
10280
is used. @value{DARKCORNER} In all cases, numbers in source code and
10281
in input data cannot have a thousands separator. Here are some examples
10282
indicating the difference in behavior, on a GNU/Linux system:
9602
10283
9603
10284
@example
9604
10285
$ @kbd{export POSIXLY_CORRECT=1} @ii{Force POSIX behavior}
9613
10294
@end example
9614
10295
9615
10296
@noindent
9616
The @samp{en_DK.utf-8} locale is for English in Denmark, where the comma acts as
10297
The @code{en_DK.utf-8} locale is for English in Denmark, where the comma acts as
9617
10298
the decimal point separator. In the normal @code{"C"} locale, @command{gawk}
9618
10299
treats @samp{4,321} as @samp{4}, while in the Danish locale, it's treated
9619
10300
as the full number, 4.321.
9760
10441
@end example
9761
10442
9762
10443
One possibly undesirable effect of this definition of remainder is that
9763
@code{@var{x} % @var{y}} is negative if @var{x} is negative. Thus:
10444
@samp{@var{x} % @var{y}} is negative if @var{x} is negative. Thus:
9764
10445
9765
10446
@example
9766
10447
-17 % 8 = -1
9768
10449
9769
10450
In other @command{awk} implementations, the signedness of the remainder
9770
10451
may be machine-dependent.
9771
@c !!! what does posix say?
10452
@c FIXME !!! what does posix say?
9772
10453
9773
10454
@cindex portability, @code{**} operator and
9774
10455
@cindex @code{*} (asterisk), @code{**} operator
9795
10476
writing expressions next to one another, with no operator. For example:
9796
10477
9797
10478
@example
9798
$ @kbd{awk '@{ print "Field number one: " $1 @}' BBS-list}
9799
@print{} Field number one: aardvark
9800
@print{} Field number one: alpo-net
10479
$ @kbd{awk '@{ print "Field number one: " $1 @}' mail-list}
10480
@print{} Field number one: Amelia
10481
@print{} Field number one: Anthony
9801
10482
@dots{}
9802
10483
@end example
9803
10484
9805
10486
runs together. For example:
9806
10487
9807
10488
@example
9808
$ @kbd{awk '@{ print "Field number one:" $1 @}' BBS-list}
9809
@print{} Field number one:aardvark
9810
@print{} Field number one:alpo-net
10489
$ @kbd{awk '@{ print "Field number one:" $1 @}' mail-list}
10490
@print{} Field number one:Amelia
10491
@print{} Field number one:Anthony
9811
10492
@dots{}
9812
10493
@end example
9813
10494
9824
10505
print "something meaningful" > file name
9825
10506
@end example
9826
10507
10508
@cindex Brian Kernighan's @command{awk}
10509
@cindex @command{mawk} utility
9827
10510
@noindent
9828
10511
This produces a syntax error with some versions of Unix
9829
10512
@command{awk}.@footnote{It happens that Brian Kernighan's
9852
10535
@end example
9853
10536
9854
10537
@noindent
9855
It is not defined whether the assignment to @code{a} happens
10538
It is not defined whether the second assignment to @code{a} happens
9856
10539
before or after the value of @code{a} is retrieved for producing the
9857
10540
concatenated value. The result could be either @samp{don't panic},
9858
10541
or @samp{panic panic}.
9974
10657
9975
10658
@cindex variables, types of
9976
10659
It is important to note that variables do @emph{not} have permanent types.
9977
A variable's type is simply the type of whatever value it happens
9978
to hold at the moment. In the following program fragment, the variable
10660
A variable's type is simply the type of whatever value was last assigned
10661
to it. In the following program fragment, the variable
9979
10662
@code{foo} has a numeric value at first, and a string value later on:
9980
10663
9981
10664
@example
10076
10759
and see @ref{Numeric Functions}, for more information).
10077
10760
This example illustrates an important fact about assignment
10078
10761
operators: the lefthand expression is only evaluated @emph{once}.
10762
10079
10763
It is up to the implementation as to which expression is evaluated
10080
10764
first, the lefthand or the righthand.
10081
10765
Consider this example:
10108
10792
@caption{Arithmetic Assignment Operators}
10109
10793
@multitable @columnfractions .30 .70
10110
10794
@headitem Operator @tab Effect
10111
@item @var{lvalue} @code{+=} @var{increment} @tab Adds @var{increment} to the value of @var{lvalue}.
10112
@item @var{lvalue} @code{-=} @var{decrement} @tab Subtracts @var{decrement} from the value of @var{lvalue}.
10113
@item @var{lvalue} @code{*=} @var{coefficient} @tab Multiplies the value of @var{lvalue} by @var{coefficient}.
10114
@item @var{lvalue} @code{/=} @var{divisor} @tab Divides the value of @var{lvalue} by @var{divisor}.
10115
@item @var{lvalue} @code{%=} @var{modulus} @tab Sets @var{lvalue} to its remainder by @var{modulus}.
10795
@item @var{lvalue} @code{+=} @var{increment} @tab Add @var{increment} to the value of @var{lvalue}.
10796
@item @var{lvalue} @code{-=} @var{decrement} @tab Subtract @var{decrement} from the value of @var{lvalue}.
10797
@item @var{lvalue} @code{*=} @var{coefficient} @tab Multiply the value of @var{lvalue} by @var{coefficient}.
10798
@item @var{lvalue} @code{/=} @var{divisor} @tab Divide the value of @var{lvalue} by @var{divisor}.
10799
@item @var{lvalue} @code{%=} @var{modulus} @tab Set @var{lvalue} to its remainder by @var{modulus}.
10116
10800
@cindex common extensions, @code{**=} operator
10117
10801
@cindex extensions, common@comma{} @code{**=} operator
10118
10802
@cindex @command{awk} language, POSIX version
10119
10803
@cindex POSIX @command{awk}
10120
10804
@item @var{lvalue} @code{^=} @var{power} @tab
10121
@item @var{lvalue} @code{**=} @var{power} @tab Raises @var{lvalue} to the power @var{power}. @value{COMMONEXT}
10805
@item @var{lvalue} @code{**=} @var{power} @tab Raise @var{lvalue} to the power @var{power}. @value{COMMONEXT}
10122
10806
@end multitable
10123
10807
@end float
10124
10808
10163
10847
awk '/[=]=/' /dev/null
10164
10848
@end example
10165
10849
10166
@command{gawk} does not have this problem,
10167
nor do the other
10168
freely available versions described in
10169
@ref{Other Versions}.
10850
@command{gawk} does not have this problem; Brian Kernighan's @command{awk}
10851
and @command{mawk} also do not (@pxref{Other Versions}).
10170
10852
@end sidebar
10171
10853
@c ENDOFRANGE exas
10172
10854
@c ENDOFRANGE opas
10190
10872
@cindex side effects, decrement/increment operators
10191
10873
The operator used for adding one is written @samp{++}. It can be used to increment
10192
10874
a variable either before or after taking its value.
10193
To pre-increment a variable @code{v}, write @samp{++v}. This adds
10875
To @dfn{pre-increment} a variable @code{v}, write @samp{++v}. This adds
10194
10876
one to the value of @code{v}---that new value is also the value of the
10195
expression. (The assignment expression @samp{v += 1} is completely
10196
equivalent.)
10197
Writing the @samp{++} after the variable specifies post-increment. This
10877
expression. (The assignment expression @samp{v += 1} is completely equivalent.)
10878
Writing the @samp{++} after the variable specifies @dfn{post-increment}. This
10198
10879
increments the variable value just the same; the difference is that the
10199
10880
value of the increment expression itself is the variable's @emph{old}
10200
10881
value. Thus, if @code{foo} has the value four, then the expression @samp{foo++}
10206
10887
+= 1) - 1}. It is not perfectly equivalent because all numbers in
10207
10888
@command{awk} are floating-point---in floating-point, @samp{foo + 1 - 1} does
10208
10889
not necessarily equal @code{foo}. But the difference is minute as
10209
long as you stick to numbers that are fairly small (less than 10e12).
10890
long as you stick to numbers that are fairly small (less than
10891
@iftex
10892
@math{10^12}).
10893
@end iftex
10894
@ifnottex
10895
@ifnotdocbook
10896
10e12).
10897
@end ifnotdocbook
10898
@end ifnottex
10899
@docbook
10900
10<superscript>12</superscript>). @c
10901
@end docbook
10210
10902
10211
10903
@cindex @code{$} (dollar sign), incrementing fields and arrays
10212
10904
@cindex dollar sign (@code{$}), incrementing fields and arrays
10215
10907
and a variable increment at the same time. The parentheses are necessary
10216
10908
because of the precedence of the field reference operator @samp{$}.)
10217
10909
10910
@c STARTOFRANGE deop
10218
10911
@cindex decrement operators
10219
10912
The decrement operator @samp{--} works just like @samp{++}, except that
10220
10913
it subtracts one instead of adding it. As with @samp{++}, it can be used before
10393
11086
for determining the type of a variable.
10394
11087
The type of the variable is important because the types of two variables
10395
11088
determine how they are compared.
11089
10396
11090
The various versions of the POSIX standard did not get the rules
10397
11091
quite right for several editions. Fortunately, as of at least the
10398
11092
2008 standard (and possibly earlier), the standard has been fixed,
10400
11094
followed these rules for many years,
10401
11095
and it is gratifying that the POSIX standard is also now correct.}
10402
11096
10403
@itemize @bullet
11097
@itemize @value{BULLET}
10404
11098
@item
10405
11099
A numeric constant or the result of a numeric operation has the @var{numeric}
10406
11100
attribute.
10486
11180
}}}
10487
11181
@end tex
10488
11182
@ifnottex
11183
@ifnotdocbook
10489
11184
@display
10490
11185
+----------------------------------------------
10491
11186
| STRING NUMERIC STRNUM
10498
11193
STRNUM | string numeric numeric
10499
11194
--------+----------------------------------------------
10500
11195
@end display
11196
@end ifnotdocbook
10501
11197
@end ifnottex
11198
@docbook
11199
<informaltable>
11200
<tgroup cols="4">
11201
<colspec colname="1" align="left"/>
11202
<colspec colname="2" align="left"/>
11203
<colspec colname="3" align="left"/>
11204
<colspec colname="4" align="left"/>
11205
<thead>
11206
<row>
11207
<entry/>
11208
<entry>STRING</entry>
11209
<entry>NUMERIC</entry>
11210
<entry>STRNUM</entry>
11211
</row>
11212
</thead>
11213
11214
<tbody>
11215
<row>
11216
<entry><emphasis role="bold">STRING</emphasis></entry>
11217
<entry>string</entry>
11218
<entry>string</entry>
11219
<entry>string</entry>
11220
</row>
11221
11222
<row>
11223
<entry><emphasis role="bold">NUMERIC</emphasis></entry>
11224
<entry>string</entry>
11225
<entry>numeric</entry>
11226
<entry>numeric</entry>
11227
</row>
11228
11229
<row>
11230
<entry><emphasis role="bold">STRNUM</emphasis></entry>
11231
<entry>string</entry>
11232
<entry>numeric</entry>
11233
<entry>numeric</entry>
11234
</row>
11235
11236
</tbody>
11237
</tgroup>
11238
</informaltable>
11239
11240
@end docbook
10502
11241
10503
11242
The basic idea is that user input that looks numeric---and @emph{only}
10504
11243
user input---should be treated as numeric, even though it is actually
10517
11256
and so is first and foremost of @var{string} type; input strings
10518
11257
that look numeric are additionally given the @var{strnum} attribute.
10519
11258
Thus, the six-character input string @w{@samp{ +3.14}} receives the
10520
@var{strnum} attribute. In contrast, the eight-character literal
10521
@w{@code{" +3.14"}} appearing in program text is a string constant.
11259
@var{strnum} attribute. In contrast, the eight characters
11260
@w{@code{" +3.14"}} appearing in program text comprise a string constant.
10522
11261
The following examples print @samp{1} when the comparison between
10523
11262
the two different constants is true, @samp{0} otherwise:
10524
11263
10633
11372
string comparison (true)
10634
11373
10635
11374
@item a = 2; b = " +2"
10636
@item a == b
11375
@itemx a == b
10637
11376
string comparison (false)
10638
11377
@end table
10639
11378
10679
11418
@cindex @code{!} (exclamation point), @code{!~} operator
10680
11419
@cindex exclamation point (@code{!}), @code{!~} operator
10681
11420
The righthand operand of the @samp{~} and @samp{!~} operators may be
10682
either a regexp constant (@code{/@dots{}/}) or an ordinary
11421
either a regexp constant (@code{/}@dots{}@code{/}) or an ordinary
10683
11422
expression. In the latter case, the value of the expression as a string is used as a
10684
11423
dynamic regexp (@pxref{Regexp Usage}; also
10685
11424
@pxref{Computed Regexps}).
10704
11443
@subsubsection String Comparison With POSIX Rules
10705
11444
10706
11445
The POSIX standard says that string comparison is performed based
10707
on the locale's collating order. This is usually very different
11446
on the locale's @dfn{collating order}. This is the order in which
11447
characters sort, as defined by the locale (for more discussion,
11448
@pxref{Ranges and Locales}). This order is usually very different
10708
11449
from the results obtained when doing straight character-by-character
10709
11450
comparison.@footnote{Technically, string comparison is supposed
10710
11451
to behave the same way as if the strings are compared with the C
10712
11453
10713
11454
Because this behavior differs considerably from existing practice,
10714
11455
@command{gawk} only implements it when in POSIX mode (@pxref{Options}).
10715
Here is an example to illustrate the difference, in an @samp{en_US.UTF-8}
11456
Here is an example to illustrate the difference, in an @code{en_US.UTF-8}
10716
11457
locale:
10717
11458
10718
11459
@example
10767
11508
@item @var{boolean1} && @var{boolean2}
10768
11509
True if both @var{boolean1} and @var{boolean2} are true. For example,
10769
11510
the following statement prints the current input record if it contains
10770
both @samp{2400} and @samp{foo}:
11511
both @samp{edu} and @samp{li}:
10771
11512
10772
11513
@example
10773
if ($0 ~ /2400/ && $0 ~ /foo/) print
11514
if ($0 ~ /edu/ && $0 ~ /li/) print
10774
11515
@end example
10775
11516
10776
11517
@cindex side effects, Boolean operators
10783
11524
@item @var{boolean1} || @var{boolean2}
10784
11525
True if at least one of @var{boolean1} or @var{boolean2} is true.
10785
11526
For example, the following statement prints all records in the input
10786
that contain @emph{either} @samp{2400} or
10787
@samp{foo} or both:
11527
that contain @emph{either} @samp{edu} or
11528
@samp{li} or both:
10788
11529
10789
11530
@example
10790
if ($0 ~ /2400/ || $0 ~ /foo/) print
11531
if ($0 ~ /edu/ || $0 ~ /li/) print
10791
11532
@end example
10792
11533
10793
11534
The subexpression @var{boolean2} is evaluated only if @var{boolean1}
10928
11669
of either character does not work without using backslash continuation
10929
11670
(@pxref{Statements/Lines}).
10930
11671
If @option{--posix} is specified
10931
(@pxref{Options}), then this extension is disabled.
11672
(@pxref{Options}), this extension is disabled.
10932
11673
10933
11674
@node Function Calls
10934
11675
@section Function Calls
10947
11688
functions for use in your program.
10948
11689
@xref{User-defined},
10949
11690
for instructions on how to do this.
11691
Finally, @command{gawk} lets you write functions in C or C++
11692
that may be called from your program: see @ref{Dynamic Extensions}.
10950
11693
10951
11694
@cindex arguments, in function calls
10952
11695
The way to use a function is with a @dfn{function call} expression,
10988
11731
use a reasonable default value.
10989
11732
@xref{Built-in}, for full details. If arguments
10990
11733
are omitted in calls to user-defined functions, then those arguments are
10991
treated as local variables and initialized to the empty string
11734
treated as local variables. Such local variables act like the
11735
empty string if referenced where a string value is required,
11736
and like zero if referenced where a numeric value is required
10992
11737
(@pxref{User-defined}).
10993
11738
10994
11739
As an advanced feature, @command{gawk} provides indirect function calls,
10997
11742
this feature until later; see @ref{Indirect Calls}.
10998
11743
10999
11744
@cindex side effects, function calls
11000
Like every other expression, the function call has a value, which is
11001
computed by the function based on the arguments you give it. In this
11002
example, the value of @samp{sqrt(@var{argument})} is the square root of
11003
@var{argument}.
11004
The following program reads numbers, one number per line, and prints the
11005
square root of each one:
11745
Like every other expression, the function call has a value, often
11746
called the @dfn{return value}, which is computed by the function
11747
based on the arguments you give it. In this example, the return value
11748
of @samp{sqrt(@var{argument})} is the square root of @var{argument}.
11749
The following program reads numbers, one number per line, and prints
11750
the square root of each one:
11006
11751
11007
11752
@example
11008
11753
$ @kbd{awk '@{ print "The square root of", $1, "is", sqrt($1) @}'}
11090
11835
This table presents @command{awk}'s operators, in order of highest
11091
11836
to lowest precedence:
11092
11837
11093
@c use @code in the items, looks better in TeX w/o all the quotes
11094
@table @code
11095
@item (@dots{})
11838
@c @asis for docbook to come out right
11839
@table @asis
11840
@item @code{(}@dots{}@code{)}
11096
11841
Grouping.
11097
11842
11098
11843
@cindex @code{$} (dollar sign), @code{$} field operator
11099
11844
@cindex dollar sign (@code{$}), @code{$} field operator
11100
@item $
11845
@item @code{$}
11101
11846
Field reference.
11102
11847
11103
11848
@cindex @code{+} (plus sign), @code{++} operator
11104
11849
@cindex plus sign (@code{+}), @code{++} operator
11105
11850
@cindex @code{-} (hyphen), @code{--} operator
11106
11851
@cindex hyphen (@code{-}), @code{--} operator
11107
@item ++ --
11852
@item @code{++ --}
11108
11853
Increment, decrement.
11109
11854
11110
11855
@cindex @code{^} (caret), @code{^} operator
11111
11856
@cindex caret (@code{^}), @code{^} operator
11112
11857
@cindex @code{*} (asterisk), @code{**} operator
11113
11858
@cindex asterisk (@code{*}), @code{**} operator
11114
@item ^ **
11859
@item @code{^ **}
11115
11860
Exponentiation. These operators group right-to-left.
11116
11861
11117
11862
@cindex @code{+} (plus sign), @code{+} operator
11120
11865
@cindex hyphen (@code{-}), @code{-} operator
11121
11866
@cindex @code{!} (exclamation point), @code{!} operator
11122
11867
@cindex exclamation point (@code{!}), @code{!} operator
11123
@item + - !
11868
@item @code{+ - !}
11124
11869
Unary plus, minus, logical ``not.''
11125
11870
11126
11871
@cindex @code{*} (asterisk), @code{*} operator, as multiplication operator
11129
11874
@cindex forward slash (@code{/}), @code{/} operator
11130
11875
@cindex @code{%} (percent sign), @code{%} operator
11131
11876
@cindex percent sign (@code{%}), @code{%} operator
11132
@item * / %
11877
@item @code{* / %}
11133
11878
Multiplication, division, remainder.
11134
11879
11135
11880
@cindex @code{+} (plus sign), @code{+} operator
11136
11881
@cindex plus sign (@code{+}), @code{+} operator
11137
11882
@cindex @code{-} (hyphen), @code{-} operator
11138
11883
@cindex hyphen (@code{-}), @code{-} operator
11139
@item + -
11884
@item @code{+ -}
11140
11885
Addition, subtraction.
11141
11886
11142
@item @r{String Concatenation}
11887
@item String Concatenation
11143
11888
There is no special symbol for concatenation.
11144
11889
The operands are simply written side by side
11145
11890
(@pxref{Concatenation}).
11165
11910
@cindex @code{|} (vertical bar), @code{|&} operator (I/O)
11166
11911
@cindex vertical bar (@code{|}), @code{|&} operator (I/O)
11167
11912
@cindex operators, input/output
11168
@item < <= == != > >= >> | |&
11913
@item @code{< <= == != > >= >> | |&}
11169
11914
Relational and redirection.
11170
11915
The relational operators and the redirections have the same precedence
11171
11916
level. Characters such as @samp{>} serve both as relationals and as
11186
11931
@cindex tilde (@code{~}), @code{~} operator
11187
11932
@cindex @code{!} (exclamation point), @code{!~} operator
11188
11933
@cindex exclamation point (@code{!}), @code{!~} operator
11189
@item ~ !~
11934
@item @code{~ !~}
11190
11935
Matching, nonmatching.
11191
11936
11192
11937
@cindex @code{in} operator
11193
@item in
11938
@item @code{in}
11194
11939
Array membership.
11195
11940
11196
11941
@cindex @code{&} (ampersand), @code{&&} operator
11197
11942
@cindex ampersand (@code{&}), @code{&&} operator
11198
@item &&
11943
@item @code{&&}
11199
11944
Logical ``and''.
11200
11945
11201
11946
@cindex @code{|} (vertical bar), @code{||} operator
11202
11947
@cindex vertical bar (@code{|}), @code{||} operator
11203
@item ||
11948
@item @code{||}
11204
11949
Logical ``or''.
11205
11950
11206
11951
@cindex @code{?} (question mark), @code{?:} operator
11207
11952
@cindex question mark (@code{?}), @code{?:} operator
11208
@item ?:
11953
@item @code{?:}
11209
11954
Conditional. This operator groups right-to-left.
11210
11955
11211
11956
@cindex @code{+} (plus sign), @code{+=} operator
11222
11967
@cindex percent sign (@code{%}), @code{%=} operator
11223
11968
@cindex @code{^} (caret), @code{^=} operator
11224
11969
@cindex caret (@code{^}), @code{^=} operator
11225
@item = += -= *= /= %= ^= **=
11970
@item @code{= += -= *= /= %= ^= **=}
11226
11971
Assignment. These operators group right-to-left.
11227
11972
@end table
11228
11973
11239
11984
@section Where You Are Makes A Difference
11240
11985
@cindex locale, definition of
11241
11986
11242
Modern systems support the notion of @dfn{locales}: a way to tell
11243
the system about the local character set and language.
11987
Modern systems support the notion of @dfn{locales}: a way to tell the
11988
system about the local character set and language. The ISO C standard
11989
defines a default @code{"C"} locale, which is an environment that is
11990
typical of what many C programmers are used to.
11244
11991
11245
11992
Once upon a time, the locale setting used to affect regexp matching
11246
11993
(@pxref{Ranges and Locales}), but this is no longer true.
11247
11994
11248
Locales can affect record splitting.
11249
For the normal case of @samp{RS = "\n"}, the locale is largely irrelevant.
11250
For other single-character record separators, setting @samp{LC_ALL=C}
11251
in the environment
11252
will give you much better performance when reading records. Otherwise,
11995
Locales can affect record splitting. For the normal case of @samp{RS =
11996
"\n"}, the locale is largely irrelevant. For other single-character
11997
record separators, setting @samp{LC_ALL=C} in the environment will
11998
give you much better performance when reading records. Otherwise,
11253
11999
@command{gawk} has to make several function calls, @emph{per input
11254
12000
character}, to find the record terminator.
11255
12001
11256
According to POSIX, string comparison is also affected by locales
11257
(similar to regular expressions). The details are presented in
11258
@ref{POSIX String Comparison}.
12002
Locales can affect how dates and times are formatted (@pxref{Time
12003
Functions}). For example, a common way to abbreviate the date September
12004
4, 2015 in the United States is ``9/4/15.'' In many countries in
12005
Europe, however, it is abbreviated ``4.9.15.'' Thus, the @samp{%x}
12006
specification in a @code{"US"} locale might produce @samp{9/4/15},
12007
while in a @code{"EUROPE"} locale, it might produce @samp{4.9.15}.
12008
12009
According to POSIX, string comparison is also affected by locales (similar
12010
to regular expressions). The details are presented in @ref{POSIX String
12011
Comparison}.
11259
12012
11260
12013
Finally, the locale affects the value of the decimal point character
11261
used when @command{gawk} parses input data. This is discussed in
11262
detail in @ref{Conversion}.
12014
used when @command{gawk} parses input data. This is discussed in detail
12015
in @ref{Conversion}.
12016
12017
@node Expressions Summary
12018
@section Summary
12019
12020
@itemize @value{BULLET}
12021
@item
12022
Expressions are the basic elements of computation in programs. They are
12023
built from constants, variables, function calls and combinations of the
12024
various kinds of values with operators.
12025
12026
@item
12027
@command{awk} supplies three kinds of constants: numeric, string, and
12028
regexp. @command{gawk} lets you specify numeric constants in octal
12029
and hexadecimal (bases 8 and 16) in addition to decimal (base 10).
12030
In certain contexts, a standalone regexp constant such as @code{/foo/}
12031
has the same meaning as @samp{$0 ~ /foo/}.
12032
12033
@item
12034
Variables hold values between uses in computations. A number of built-in
12035
variables provide information to your @command{awk} program, and a number
12036
of others let you control how @command{awk} behaves.
12037
12038
@item
12039
Numbers are automatically converted to strings, and strings to numbers,
12040
as needed by @command{awk}. Numeric values are converted as if they were
12041
formatted with @code{sprintf()} using the format in @code{CONVFMT}.
12042
Locales can influence the conversions.
12043
12044
@item
12045
@command{awk} provides the usual arithmetic operators (addition,
12046
subtraction, multiplication, division, modulus), and unary plus and minus.
12047
It also provides comparison operators, boolean operators, and regexp
12048
matching operators. String concatenation is accomplished by placing
12049
two expressions next to each other; there is no explicit operator.
12050
The three-operand @samp{?:} operator provides an ``if-else'' test within
12051
expressions.
12052
12053
@item
12054
Assignment operators provide convenient shorthands for common arithmetic
12055
operations.
12056
12057
@item
12058
In @command{awk}, a value is considered to be true if it is non-zero
12059
@emph{or} non-null. Otherwise, the value is false.
12060
12061
@item
12062
A value's type is set upon each assignment and may change over its
12063
lifetime. The type determines how it behaves in comparisons (string
12064
or numeric).
12065
12066
@item
12067
Function calls return a value which may be used as part of a larger
12068
expression. Expressions used to pass parameter values are fully
12069
evaluated before the function is called. @command{awk} provides
12070
built-in and user-defined functions; this is described later on in this
12071
@value{DOCUMENT}.
12072
12073
@item
12074
Operator precedence specifies the order in which operations are performed,
12075
unless explicitly overridden by parentheses. @command{awk}'s operator
12076
precedence is compatible with that of C.
12077
12078
@item
12079
Locales can affect the format of data as output by an @command{awk}
12080
program, and occasionally the format for data read as input.
12081
12082
@end itemize
11263
12083
11264
12084
@c ENDOFRANGE exps
11265
12085
11287
12107
* Statements:: Describes the various control statements in
11288
12108
detail.
11289
12109
* Built-in Variables:: Summarizes the built-in variables.
12110
* Pattern Action Summary:: Patterns and Actions summary.
11290
12111
@end menu
11291
12112
11292
12113
@node Pattern Overview
11317
12138
is nonzero (if a number) or non-null (if a string).
11318
12139
(@xref{Expression Patterns}.)
11319
12140
11320
@item @var{pat1}, @var{pat2}
12141
@item @var{begpat}, @var{endpat}
11321
12142
A pair of patterns separated by a comma, specifying a range of records.
11322
The range includes both the initial record that matches @var{pat1} and
11323
the final record that matches @var{pat2}.
12143
The range includes both the initial record that matches @var{begpat} and
12144
the final record that matches @var{endpat}.
11324
12145
(@xref{Ranges}.)
11325
12146
11326
12147
@item BEGIN
11332
12153
@item BEGINFILE
11333
12154
@itemx ENDFILE
11334
12155
Special patterns for you to supply startup or cleanup actions to be
11335
done on a per file basis.
12156
done on a per-file basis.
11336
12157
(@xref{BEGINFILE/ENDFILE}.)
11337
12158
11338
12159
@item @var{empty}
11382
12203
is used as a dynamic regular expression
11383
12204
(@pxref{Computed Regexps}).
11384
12205
The following example prints the second field of each input record
11385
whose first field is precisely @samp{foo}:
12206
whose first field is precisely @samp{li}:
11386
12207
11387
12208
@cindex @code{/} (forward slash), patterns and
11388
12209
@cindex forward slash (@code{/}), patterns and
11391
12212
@cindex @code{!} (exclamation point), @code{!~} operator
11392
12213
@cindex exclamation point (@code{!}), @code{!~} operator
11393
12214
@example
11394
$ @kbd{awk '$1 == "foo" @{ print $2 @}' BBS-list}
12215
$ @kbd{awk '$1 == "li" @{ print $2 @}' mail-list}
11395
12216
@end example
11396
12217
11397
12218
@noindent
11398
(There is no output, because there is no BBS site with the exact name @samp{foo}.)
12219
(There is no output, because there is no person with the exact name @samp{li}.)
11399
12220
Contrast this with the following regular expression match, which
11400
accepts any record with a first field that contains @samp{foo}:
12221
accepts any record with a first field that contains @samp{li}:
11401
12222
11402
12223
@example
11403
$ @kbd{awk '$1 ~ /foo/ @{ print $2 @}' BBS-list}
11404
@print{} 555-1234
12224
$ @kbd{awk '$1 ~ /foo/ @{ print $2 @}' mail-list}
12225
@print{} 555-5553
11405
12226
@print{} 555-6699
11406
@print{} 555-6480
11407
@print{} 555-2127
11408
12227
@end example
11409
12228
11410
12229
@cindex regexp constants, as patterns
11411
12230
@cindex patterns, regexp constants as
11412
12231
A regexp constant as a pattern is also a special case of an expression
11413
pattern. The expression @code{/foo/} has the value one if @samp{foo}
11414
appears in the current input record. Thus, as a pattern, @code{/foo/}
11415
matches any record containing @samp{foo}.
12232
pattern. The expression @code{/li/} has the value one if @samp{li}
12233
appears in the current input record. Thus, as a pattern, @code{/li/}
12234
matches any record containing @samp{li}.
11416
12235
11417
12236
@cindex Boolean expressions, as patterns
11418
12237
Boolean expressions are also commonly used as patterns.
11419
12238
Whether the pattern
11420
12239
matches an input record depends on whether its subexpressions match.
11421
12240
For example, the following command prints all the records in
11422
@file{BBS-list} that contain both @samp{2400} and @samp{foo}:
12241
@file{mail-list} that contain both @samp{edu} and @samp{li}:
11423
12242
11424
12243
@example
11425
$ @kbd{awk '/2400/ && /foo/' BBS-list}
11426
@print{} fooey 555-1234 2400/1200/300 B
12244
$ @kbd{awk '/edu/ && /li/' mail-list}
12245
@print{} Samuel 555-3430 samuel.lanceolis@@shu.edu A
11427
12246
@end example
11428
12247
11429
12248
The following command prints all records in
11430
@file{BBS-list} that contain @emph{either} @samp{2400} or @samp{foo}
12249
@file{mail-list} that contain @emph{either} @samp{edu} or @samp{li}
11431
12250
(or both, of course):
11432
12251
11433
12252
@example
11434
$ @kbd{awk '/2400/ || /foo/' BBS-list}
11435
@print{} alpo-net 555-3412 2400/1200/300 A
11436
@print{} bites 555-1675 2400/1200/300 A
11437
@print{} fooey 555-1234 2400/1200/300 B
11438
@print{} foot 555-6699 1200/300 B
11439
@print{} macfoo 555-6480 1200/300 A
11440
@print{} sdace 555-3430 2400/1200/300 A
11441
@print{} sabafoo 555-2127 1200/300 C
12253
$ @kbd{awk '/edu/ || /li/' mail-list}
12254
@print{} Amelia 555-5553 amelia.zodiacusque@@gmail.com F
12255
@print{} Broderick 555-0542 broderick.aliquotiens@@yahoo.com R
12256
@print{} Fabius 555-1234 fabius.undevicesimus@@ucb.edu F
12257
@print{} Julie 555-6699 julie.perscrutabor@@skeeve.com F
12258
@print{} Samuel 555-3430 samuel.lanceolis@@shu.edu A
12259
@print{} Jean-Paul 555-2127 jeanpaul.campanorum@@nyu.edu R
11442
12260
@end example
11443
12261
11444
12262
The following command prints all records in
11445
@file{BBS-list} that do @emph{not} contain the string @samp{foo}:
12263
@file{mail-list} that do @emph{not} contain the string @samp{li}:
11446
12264
11447
12265
@example
11448
$ @kbd{awk '! /foo/' BBS-list}
11449
@print{} aardvark 555-5553 1200/300 B
11450
@print{} alpo-net 555-3412 2400/1200/300 A
11451
@print{} barfly 555-7685 1200/300 A
11452
@print{} bites 555-1675 2400/1200/300 A
11453
@print{} camelot 555-0542 300 C
11454
@print{} core 555-2912 1200/300 C
11455
@print{} sdace 555-3430 2400/1200/300 A
12266
$ @kbd{awk '! /li/' mail-list}
12267
@print{} Anthony 555-3412 anthony.asserturo@@hotmail.com A
12268
@print{} Becky 555-7685 becky.algebrarum@@gmail.com A
12269
@print{} Bill 555-1675 bill.drowning@@hotmail.com A
12270
@print{} Camilla 555-2912 camilla.infusarum@@skynet.be R
12271
@print{} Fabius 555-1234 fabius.undevicesimus@@ucb.edu F
12272
@print{} Martin 555-6480 martin.codicibus@@hotmail.com A
12273
@print{} Jean-Paul 555-2127 jeanpaul.campanorum@@nyu.edu R
11456
12274
@end example
11457
12275
11458
12276
@cindex @code{BEGIN} pattern, Boolean patterns and
11496
12314
@dfn{turned on} and the range pattern matches this record as well. As long as
11497
12315
the range pattern stays turned on, it automatically matches every input
11498
12316
record read. The range pattern also matches @var{endpat} against every
11499
input record; when this succeeds, the range pattern is turned off again
12317
input record; when this succeeds, the range pattern is @dfn{turned off} again
11500
12318
for the following record. Then the range pattern goes back to checking
11501
12319
@var{begpat} against each record.
11502
12320
11556
12374
@error{} gawk: cmd. line:1: ^ syntax error
11557
12375
@end example
11558
12376
12377
@cindex range patterns, line continuation and
12378
As a minor point of interest, although it is poor style,
12379
POSIX allows you to put a newline after the comma in
12380
a range pattern. @value{DARKCORNER}
12381
11559
12382
@node BEGIN/END
11560
12383
@subsection The @code{BEGIN} and @code{END} Special Patterns
11561
12384
11580
12403
@node Using BEGIN/END
11581
12404
@subsubsection Startup and Cleanup Actions
11582
12405
12406
@cindex @code{BEGIN} pattern
12407
@cindex @code{END} pattern
11583
12408
A @code{BEGIN} rule is executed once only, before the first input record
11584
12409
is read. Likewise, an @code{END} rule is executed once only, after all the
11585
12410
input is read. For example:
11586
12411
11587
12412
@example
11588
12413
$ @kbd{awk '}
11589
> @kbd{BEGIN @{ print "Analysis of \"foo\"" @}}
11590
> @kbd{/foo/ @{ ++n @}}
11591
> @kbd{END @{ print "\"foo\" appears", n, "times." @}' BBS-list}
11592
@print{} Analysis of "foo"
11593
@print{} "foo" appears 4 times.
12414
> @kbd{BEGIN @{ print "Analysis of \"li\"" @}}
12415
> @kbd{/li/ @{ ++n @}}
12416
> @kbd{END @{ print "\"li\" appears in", n, "records." @}' mail-list}
12417
@print{} Analysis of "li"
12418
@print{} "li" appears in 4 records.
11594
12419
@end example
11595
12420
11596
12421
@cindex @code{BEGIN} pattern, operators and
11597
12422
@cindex @code{END} pattern, operators and
11598
This program finds the number of records in the input file @file{BBS-list}
11599
that contain the string @samp{foo}. The @code{BEGIN} rule prints a title
12423
This program finds the number of records in the input file @file{mail-list}
12424
that contain the string @samp{li}. The @code{BEGIN} rule prints a title
11600
12425
for the report. There is no need to use the @code{BEGIN} rule to
11601
12426
initialize the counter @code{n} to zero, since @command{awk} does this
11602
12427
automatically (@pxref{Variables}).
11603
12428
The second rule increments the variable @code{n} every time a
11604
record containing the pattern @samp{foo} is read. The @code{END} rule
12429
record containing the pattern @samp{li} is read. The @code{END} rule
11605
12430
prints the value of @code{n} at the end of the run.
11606
12431
11607
12432
The special patterns @code{BEGIN} and @code{END} cannot be used in ranges
11643
12468
@subsubsection Input/Output from @code{BEGIN} and @code{END} Rules
11644
12469
11645
12470
@cindex input/output, from @code{BEGIN} and @code{END}
11646
There are several (sometimes subtle) points to remember when doing I/O
12471
There are several (sometimes subtle) points to be aware of when doing I/O
11647
12472
from a @code{BEGIN} or @code{END} rule.
11648
12473
The first has to do with the value of @code{$0} in a @code{BEGIN}
11649
12474
rule. Because @code{BEGIN} rules are executed before any input is read,
11654
12479
without a variable (@pxref{Getline}).
11655
12480
Another way is simply to assign a value to @code{$0}.
11656
12481
12482
@cindex Brian Kernighan's @command{awk}
11657
12483
@cindex differences in @command{awk} and @command{gawk}, @code{BEGIN}/@code{END} patterns
11658
12484
@cindex POSIX @command{awk}, @code{BEGIN}/@code{END} patterns
11659
12485
@cindex @code{print} statement, @code{BEGIN}/@code{END} patterns and
11703
12529
11704
12530
Two special kinds of rule, @code{BEGINFILE} and @code{ENDFILE}, give
11705
12531
you ``hooks'' into @command{gawk}'s command-line file processing loop.
11706
As with the @code{BEGIN} and @code{END} rules (@pxref{BEGIN/END}), all
11707
@code{BEGINFILE} rules in a program are merged, in the order they are
12532
As with the @code{BEGIN} and @code{END} rules
12533
@ifnottex
12534
@ifnotdocbook
12535
(@pxref{BEGIN/END}),
12536
@end ifnotdocbook
12537
@end ifnottex
12538
@iftex
12539
(see the previous section),
12540
@end iftex
12541
@ifdocbook
12542
(see the previous section),
12543
@end ifdocbook
12544
all @code{BEGINFILE} rules in a program are merged, in the order they are
11708
12545
read by @command{gawk}, and all @code{ENDFILE} rules are merged as well.
11709
12546
11710
12547
The body of the @code{BEGINFILE} rules is executed just before
11714
12551
The @code{BEGINFILE} rule provides you the opportunity to accomplish two tasks
11715
12552
that would otherwise be difficult or impossible to perform:
11716
12553
11717
@itemize @bullet
12554
@itemize @value{BULLET}
11718
12555
@item
11719
12556
You can test if the file is readable. Normally, it is a fatal error if a
11720
12557
file named on the command line cannot be opened for reading. However,
11722
12559
command line.
11723
12560
11724
12561
@cindex @command{gawk}, @code{ERRNO} variable in
11725
@cindex @code{ERRNO} variable
12562
@cindex @code{ERRNO} variable, with @code{BEGINFILE} pattern
11726
12563
@cindex @code{nextfile} statement, @code{BEGINFILE}/@code{ENDFILE} patterns and
11727
12564
You do this by checking if the @code{ERRNO} variable is not the empty
11728
12565
string; if so, then @command{gawk} was not able to open the file. In
11732
12569
fatal error.
11733
12570
11734
12571
@item
11735
If you have written extensions that modify the record handling (by inserting
11736
an ``input parser''), you can invoke them at this point, before @command{gawk}
11737
has started processing the file. (This is a @emph{very} advanced feature,
11738
currently used only by the @uref{http://gawkextlib.sourceforge.net, @code{gawkextlib} project}.)
12572
If you have written extensions that modify the record handling (by
12573
inserting an ``input parser,'' @pxref{Input Parsers}), you can invoke
12574
them at this point, before @command{gawk} has started processing the file.
12575
(This is a @emph{very} advanced feature, currently used only by the
12576
@uref{http://gawkextlib.sourceforge.net, @code{gawkextlib} project}.)
11739
12577
@end itemize
11740
12578
11741
12579
The @code{ENDFILE} rule is called when @command{gawk} has finished processing
11757
12595
11758
12596
@cindex @code{getline} statement, @code{BEGINFILE}/@code{ENDFILE} patterns and
11759
12597
The @code{getline} statement (@pxref{Getline}) is restricted inside
11760
both @code{BEGINFILE} and @code{ENDFILE}. Only the @samp{getline
11761
@var{variable} < @var{file}} form is allowed.
12598
both @code{BEGINFILE} and @code{ENDFILE}: only redirected
12599
forms of @code{getline} are allowed.
11762
12600
11763
12601
@code{BEGINFILE} and @code{ENDFILE} are @command{gawk} extensions.
11764
12602
In most other @command{awk} implementations, or if @command{gawk} is in
11765
12603
compatibility mode (@pxref{Options}), they are not special.
11766
12604
11767
@c FIXME: For 4.1 maybe deal with this?
12605
@c FIXME: For 4.2 maybe deal with this?
11768
12606
@ignore
11769
12607
Date: Tue, 17 May 2011 02:06:10 PDT
11770
12608
From: rankin@pactechdata.com (Pat Rankin)
11795
12633
input record. For example, the program:
11796
12634
11797
12635
@example
11798
awk '@{ print $1 @}' BBS-list
12636
awk '@{ print $1 @}' mail-list
11799
12637
@end example
11800
12638
11801
12639
@noindent
11818
12656
@cindex shells, quoting
11819
12657
The most common method is to use shell quoting to substitute
11820
12658
the variable's value into the program inside the script.
11821
For example, in the following program:
12659
For example, consider the following program:
11822
12660
11823
12661
@example
11824
12662
printf "Enter search pattern: "
11828
12666
@end example
11829
12667
11830
12668
@noindent
11831
the @command{awk} program consists of two pieces of quoted text
12669
The @command{awk} program consists of two pieces of quoted text
11832
12670
that are concatenated together to form the program.
11833
12671
The first part is double-quoted, which allows substitution of
11834
12672
the @code{pattern} shell variable inside the quotes.
11842
12680
11843
12681
A better method is to use @command{awk}'s variable assignment feature
11844
12682
(@pxref{Assignment Options})
11845
to assign the shell variable's value to an @command{awk} variable's
11846
value. Then use dynamic regexps to match the pattern
12683
to assign the shell variable's value to an @command{awk} variable.
12684
Then use dynamic regexps to match the pattern
11847
12685
(@pxref{Computed Regexps}).
11848
12686
The following shows how to redo the
11849
12687
previous example using this technique:
11881
12719
@command{awk} what to do once a match for the pattern is found. Thus,
11882
12720
in outline, an @command{awk} program generally looks like this:
11883
12721
11884
@example
11885
@r{[}@var{pattern}@r{]} @{ @var{action} @}
11886
@var{pattern} @r{[}@{ @var{action} @}@r{]}
11887
@dots{}
11888
function @var{name}(@var{args}) @{ @dots{} @}
11889
@dots{}
11890
@end example
12722
@display
12723
[@var{pattern}] @code{@{ @var{action} @}}
12724
@var{pattern} [@code{@{ @var{action} @}}]
12725
@dots{}
12726
@code{function @var{name}(@var{args}) @{ @dots{} @}}
12727
@dots{}
12728
@end display
11891
12729
11892
12730
@cindex @code{@{@}} (braces), actions and
11893
12731
@cindex braces (@code{@{@}}), actions and
11896
12734
@cindex @code{;} (semicolon), separating statements in actions
11897
12735
@cindex semicolon (@code{;}), separating statements in actions
11898
12736
An action consists of one or more @command{awk} @dfn{statements}, enclosed
11899
in curly braces (@samp{@{@dots{}@}}). Each statement specifies one
12737
in braces (@samp{@{@r{@dots{}}@}}). Each statement specifies one
11900
12738
thing to do. The statements are separated by newlines or semicolons.
11901
The curly braces around an action must be used even if the action
12739
The braces around an action must be used even if the action
11902
12740
contains only one statement, or if it contains no statements at
11903
all. However, if you omit the action entirely, omit the curly braces as
12741
all. However, if you omit the action entirely, omit the braces as
11904
12742
well. An omitted action is equivalent to @samp{@{ print $0 @}}:
11905
12743
11906
12744
@example
11926
12764
special ones (@pxref{Statements}).
11927
12765
11928
12766
@item Compound statements
11929
Consist of one or more statements enclosed in
11930
curly braces. A compound statement is used in order to put several
11931
statements together in the body of an @code{if}, @code{while}, @code{do},
11932
or @code{for} statement.
12767
Enclose one or more statements in braces. A compound statement
12768
is used in order to put several statements together in the body of an
12769
@code{if}, @code{while}, @code{do}, or @code{for} statement.
11933
12770
11934
12771
@item Input statements
11935
12772
Use the @code{getline} command
11975
12812
@code{if} statement contains another statement that may or may not be
11976
12813
executed. The contained statement is called the @dfn{body}.
11977
12814
To include more than one statement in the body, group them into a
11978
single @dfn{compound statement} with curly braces, separating them with
12815
single @dfn{compound statement} with braces, separating them with
11979
12816
newlines or semicolons.
11980
12817
11981
12818
@menu
12003
12840
The @code{if}-@code{else} statement is @command{awk}'s decision-making
12004
12841
statement. It looks like this:
12005
12842
12006
@example
12007
if (@var{condition}) @var{then-body} @r{[}else @var{else-body}@r{]}
12008
@end example
12843
@display
12844
@code{if (@var{condition}) @var{then-body}} [@code{else @var{else-body}}]
12845
@end display
12009
12846
12010
12847
@noindent
12011
12848
The @var{condition} is an expression that controls what the rest of the
12029
12866
statement is executed.
12030
12867
If the @code{else} keyword appears on the same line as @var{then-body} and
12031
12868
@var{then-body} is not a compound statement (i.e., not surrounded by
12032
curly braces), then a semicolon must separate @var{then-body} from
12869
braces), then a semicolon must separate @var{then-body} from
12033
12870
the @code{else}.
12034
12871
To illustrate this, the previous example can be rewritten as:
12035
12872
12048
12885
@subsection The @code{while} Statement
12049
12886
@cindex @code{while} statement
12050
12887
@cindex loops
12888
@cindex loops, @code{while}
12051
12889
@cindex loops, See Also @code{while} statement
12052
12890
12053
12891
In programming, a @dfn{loop} is a part of a program that can
12108
12946
@node Do Statement
12109
12947
@subsection The @code{do}-@code{while} Statement
12110
12948
@cindex @code{do}-@code{while} statement
12949
@cindex loops, @code{do}-@code{while}
12111
12950
12112
12951
The @code{do} loop is a variation of the @code{while} looping statement.
12113
12952
The @code{do} loop executes the @var{body} once and then repeats the
12153
12992
@node For Statement
12154
12993
@subsection The @code{for} Statement
12155
12994
@cindex @code{for} statement
12995
@cindex loops, @code{for}, iterative
12156
12996
12157
12997
The @code{for} statement makes it more convenient to count iterations of a
12158
12998
loop. The general form of the @code{for} statement looks like this:
12259
13099
@cindex @code{case} keyword
12260
13100
@cindex @code{default} keyword
12261
13101
13102
This @value{SECTION} describes a @command{gawk}-specific feature.
13103
If @command{gawk} is in compatibility mode (@pxref{Options}),
13104
it is not available.
13105
12262
13106
The @code{switch} statement allows the evaluation of an expression and
12263
13107
the execution of statements based on a @code{case} match. Case statements
12264
13108
are checked for a match in the order they are defined. If no suitable
12314
13158
the @minus{}1 case will also be executed since the @code{default} does
12315
13159
not halt execution.
12316
13160
12317
This @code{switch} statement is a @command{gawk} extension.
12318
If @command{gawk} is in compatibility mode
12319
(@pxref{Options}),
12320
it is not available.
12321
12322
13161
@node Break Statement
12323
13162
@subsection The @code{break} Statement
12324
13163
@cindex @code{break} statement
12325
13164
@cindex loops, exiting
13165
@cindex loops, @code{break} statement and
12326
13166
12327
13167
The @code{break} statement jumps out of the innermost @code{for},
12328
13168
@code{while}, or @code{do} loop that encloses it. The following example
12332
13172
@example
12333
13173
# find smallest divisor of num
12334
13174
@{
12335
num = $1
12336
for (div = 2; div * div <= num; div++) @{
12337
if (num % div == 0)
12338
break
12339
@}
12340
if (num % div == 0)
12341
printf "Smallest divisor of %d is %d\n", num, div
12342
else
12343
printf "%d is prime\n", num
13175
num = $1
13176
for (div = 2; div * div <= num; div++) @{
13177
if (num % div == 0)
13178
break
13179
@}
13180
if (num % div == 0)
13181
printf "Smallest divisor of %d is %d\n", num, div
13182
else
13183
printf "%d is prime\n", num
12344
13184
@}
12345
13185
@end example
12346
13186
12358
13198
@example
12359
13199
# find smallest divisor of num
12360
13200
@{
12361
num = $1
12362
for (div = 2; ; div++) @{
12363
if (num % div == 0) @{
12364
printf "Smallest divisor of %d is %d\n", num, div
12365
break
12366
@}
12367
if (div * div > num) @{
12368
printf "%d is prime\n", num
12369
break
12370
@}
12371
@}
13201
num = $1
13202
for (div = 2; ; div++) @{
13203
if (num % div == 0) @{
13204
printf "Smallest divisor of %d is %d\n", num, div
13205
break
13206
@}
13207
if (div * div > num) @{
13208
printf "%d is prime\n", num
13209
break
13210
@}
13211
@}
12372
13212
@}
12373
13213
@end example
12374
13214
12382
13222
@cindex POSIX @command{awk}, @code{break} statement and
12383
13223
@cindex dark corner, @code{break} statement
12384
13224
@cindex @command{gawk}, @code{break} statement in
13225
@cindex Brian Kernighan's @command{awk}
12385
13226
The @code{break} statement has no meaning when
12386
13227
used outside the body of a loop or @code{switch}.
12387
13228
However, although it was never documented,
12446
13287
@cindex POSIX @command{awk}, @code{continue} statement and
12447
13288
@cindex dark corner, @code{continue} statement
12448
13289
@cindex @command{gawk}, @code{continue} statement in
13290
@cindex Brian Kernighan's @command{awk}
12449
13291
The @code{continue} statement has no special meaning with respect to the
12450
13292
@code{switch} statement, nor does it have any meaning when used outside the
12451
13293
body of a loop. Historical versions of @command{awk} treated a @code{continue}
12515
13357
@cindex POSIX @command{awk}, @code{next}/@code{nextfile} statements and
12516
13358
@cindex @code{next} statement, user-defined functions and
12517
13359
@cindex functions, user-defined, @code{next}/@code{nextfile} statements and
12518
According to the POSIX standard, the behavior is undefined if
12519
the @code{next} statement is used in a @code{BEGIN} or @code{END} rule.
12520
@command{gawk} treats it as a syntax error.
12521
Although POSIX permits it,
12522
some other @command{awk} implementations don't allow the @code{next}
12523
statement inside function bodies
12524
(@pxref{User-defined}).
12525
Just as with any other @code{next} statement, a @code{next} statement inside a
12526
function body reads the next record and starts processing it with the
12527
first rule in the program.
13360
According to the POSIX standard, the behavior is undefined if the
13361
@code{next} statement is used in a @code{BEGIN} or @code{END} rule.
13362
@command{gawk} treats it as a syntax error. Although POSIX permits it,
13363
most other @command{awk} implementations don't allow the @code{next}
13364
statement inside function bodies (@pxref{User-defined}). Just as with any
13365
other @code{next} statement, a @code{next} statement inside a function
13366
body reads the next record and starts processing it with the first rule
13367
in the program.
12528
13368
12529
13369
@node Nextfile Statement
12530
13370
@subsection The @code{nextfile} Statement
12534
13374
is similar to the @code{next} statement.
12535
13375
However, instead of abandoning processing of the current record, the
12536
13376
@code{nextfile} statement instructs @command{awk} to stop processing the
12537
current data file.
13377
current @value{DF}.
12538
13378
12539
13379
Upon execution of the @code{nextfile} statement,
12540
13380
@code{FILENAME} is
12541
updated to the name of the next data file listed on the command line,
13381
updated to the name of the next @value{DF} listed on the command line,
12542
13382
@code{FNR} is reset to one,
12543
13383
and processing
12544
13384
starts over with the first rule in the program.
12547
13387
when @code{nextfile} is invoked during execution of any statement in an
12548
13388
@code{END} rule; In this case, it causes the program to stop immediately. @xref{BEGIN/END}.
12549
13389
12550
The @code{nextfile} statement is useful when there are many data files
13390
The @code{nextfile} statement is useful when there are many @value{DF}s
12551
13391
to process but it isn't necessary to process every record in every file.
12552
13392
Without @code{nextfile},
12553
in order to move on to the next data file, a program
13393
in order to move on to the next @value{DF}, a program
12554
13394
would have to continue scanning the unwanted records. The @code{nextfile}
12555
13395
statement accomplishes this much more efficiently.
12556
13396
12583
13423
12584
13424
@cindex functions, user-defined, @code{next}/@code{nextfile} statements and
12585
13425
@cindex @code{nextfile} statement, user-defined functions and
12586
The current version of the Brian Kernighan's @command{awk} (@pxref{Other
12587
Versions}) also supports @code{nextfile}. However, it doesn't allow the
13426
@cindex Brian Kernighan's @command{awk}
13427
@cindex @command{mawk} utility
13428
The current version of the Brian Kernighan's @command{awk}, and @command{mawk} (@pxref{Other
13429
Versions}) also support @code{nextfile}. However, they don't allow the
12588
13430
@code{nextfile} statement inside function bodies (@pxref{User-defined}).
12589
13431
@command{gawk} does; a @code{nextfile} inside a function body reads the
12590
13432
next record and starts processing it with the first rule in the program,
12598
13440
executing the current rule and to stop processing input; any remaining input
12599
13441
is ignored. The @code{exit} statement is written as follows:
12600
13442
12601
@example
12602
exit @r{[}@var{return code}@r{]}
12603
@end example
13443
@display
13444
@code{exit} [@var{return code}]
13445
@end display
12604
13446
12605
13447
@cindex @code{BEGIN} pattern, @code{exit} statement and
12606
13448
@cindex @code{END} pattern, @code{exit} statement and
12633
13475
In the case where an argument
12634
13476
is supplied to a first @code{exit} statement, and then @code{exit} is
12635
13477
called a second time from an @code{END} rule with no argument,
12636
@command{awk} uses the previously supplied exit value.
12637
@value{DARKCORNER}
13478
@command{awk} uses the previously supplied exit value. @value{DARKCORNER}
12638
13479
@xref{Exit Status}, for more information.
12639
13480
12640
13481
@cindex programming conventions, @code{exit} statement
12646
13487
12647
13488
@example
12648
13489
BEGIN @{
12649
if (("date" | getline date_now) <= 0) @{
12650
print "Can't get system date" > "/dev/stderr"
12651
exit 1
12652
@}
12653
print "current date is", date_now
12654
close("date")
13490
if (("date" | getline date_now) <= 0) @{
13491
print "Can't get system date" > "/dev/stderr"
13492
exit 1
13493
@}
13494
print "current date is", date_now
13495
close("date")
12655
13496
@}
12656
13497
@end example
12657
13498
12682
13523
internal workings of @command{awk} to your program.
12683
13524
12684
13525
@cindex @command{gawk}, built-in variables and
12685
This @value{SECTION} documents all the built-in variables of
12686
@command{gawk}, most of which are also documented in the chapters
12687
describing their areas of activity.
13526
This @value{SECTION} documents all of @command{gawk}'s built-in variables,
13527
most of which are also documented in the @value{CHAPTER}s describing
13528
their areas of activity.
12688
13529
12689
13530
@menu
12690
13531
* User-modified:: Built-in variables that you change to control
12702
13543
@cindex user-modifiable variables
12703
13544
12704
13545
The following is an alphabetical list of variables that you can change to
12705
control how @command{awk} does certain things. The variables that are
12706
specific to @command{gawk} are marked with a pound sign@w{ (@samp{#}).}
13546
control how @command{awk} does certain things.
13547
13548
The variables that are specific to @command{gawk} are marked with a pound
13549
sign (@samp{#}). These variables are @command{gawk} extensions. In other
13550
@command{awk} implementations or if @command{gawk} is in compatibility
13551
mode (@pxref{Options}), they are not special. (Any exceptions are noted
13552
in the description of each variable.)
12707
13553
12708
13554
@table @code
12709
13555
@cindex @code{BINMODE} variable
12710
13556
@cindex binary input/output
12711
13557
@cindex input/output, binary
13558
@cindex differences in @command{awk} and @command{gawk}, @code{BINMODE} variable
12712
13559
@item BINMODE #
12713
On non-POSIX systems, this variable specifies use of binary mode for all I/O.
12714
Numeric values of one, two, or three specify that input files, output files, or
12715
all files, respectively, should use binary I/O.
12716
A numeric value less than zero is treated as zero, and a numeric value greater than
12717
three is treated as three.
12718
Alternatively,
12719
string values of @code{"r"} or @code{"w"} specify that input files and
12720
output files, respectively, should use binary I/O.
12721
A string value of @code{"rw"} or @code{"wr"} indicates that all
12722
files should use binary I/O.
12723
Any other string value is treated the same as @code{"rw"},
12724
but causes @command{gawk}
12725
to generate a warning message.
12726
@code{BINMODE} is described in more detail in
12727
@ref{PC Using}.
12728
12729
@cindex differences in @command{awk} and @command{gawk}, @code{BINMODE} variable
12730
This variable is a @command{gawk} extension.
12731
In other @command{awk} implementations
12732
(except @command{mawk},
12733
@pxref{Other Versions}),
12734
or if @command{gawk} is in compatibility mode
12735
(@pxref{Options}),
12736
it is not special.
13560
On non-POSIX systems, this variable specifies use of binary mode
13561
for all I/O. Numeric values of one, two, or three specify that input
13562
files, output files, or all files, respectively, should use binary I/O.
13563
A numeric value less than zero is treated as zero, and a numeric value
13564
greater than three is treated as three. Alternatively, string values
13565
of @code{"r"} or @code{"w"} specify that input files and output files,
13566
respectively, should use binary I/O. A string value of @code{"rw"} or
13567
@code{"wr"} indicates that all files should use binary I/O. Any other
13568
string value is treated the same as @code{"rw"}, but causes @command{gawk}
13569
to generate a warning message. @code{BINMODE} is described in more
13570
detail in @ref{PC Using}. @command{mawk} @pxref{Other Versions}),
13571
also supports this variable, but only using numeric values.
12737
13572
12738
13573
@cindex @code{CONVFMT} variable
12739
13574
@cindex POSIX @command{awk}, @code{CONVFMT} variable and
12740
13575
@cindex numbers, converting, to strings
12741
13576
@cindex strings, converting, numbers to
12742
@item CONVFMT
13577
@item @code{CONVFMT}
12743
13578
This string controls conversion of numbers to
12744
13579
strings (@pxref{Conversion}).
12745
13580
It works by being passed, in effect, as the first argument to the
12754
13589
@cindex field separators, @code{FIELDWIDTHS} variable and
12755
13590
@cindex separators, field, @code{FIELDWIDTHS} variable and
12756
13591
@item FIELDWIDTHS #
12757
This is a space-separated list of columns that tells @command{gawk}
13592
A space-separated list of columns that tells @command{gawk}
12758
13593
how to split input with fixed columnar boundaries.
12759
13594
Assigning a value to @code{FIELDWIDTHS}
12760
13595
overrides the use of @code{FS} and @code{FPAT} for field splitting.
12761
13596
@xref{Constant Size}, for more information.
12762
13597
12763
If @command{gawk} is in compatibility mode
12764
(@pxref{Options}), then @code{FIELDWIDTHS}
12765
has no special meaning, and field-splitting operations occur based
12766
exclusively on the value of @code{FS}.
12767
12768
13598
@cindex @command{gawk}, @code{FPAT} variable in
12769
13599
@cindex @code{FPAT} variable
12770
13600
@cindex differences in @command{awk} and @command{gawk}, @code{FPAT} variable
12771
13601
@cindex field separators, @code{FPAT} variable and
12772
13602
@cindex separators, field, @code{FPAT} variable and
12773
13603
@item FPAT #
12774
This is a regular expression (as a string) that tells @command{gawk}
13604
A regular expression (as a string) that tells @command{gawk}
12775
13605
to create the fields based on text that matches the regular expression.
12776
13606
Assigning a value to @code{FPAT}
12777
13607
overrides the use of @code{FS} and @code{FIELDWIDTHS} for field splitting.
12778
13608
@xref{Splitting By Content}, for more information.
12779
13609
12780
If @command{gawk} is in compatibility mode
12781
(@pxref{Options}), then @code{FPAT}
12782
has no special meaning, and field-splitting operations occur based
12783
exclusively on the value of @code{FS}.
12784
12785
13610
@cindex @code{FS} variable
12786
13611
@cindex separators, field
12787
13612
@cindex field separators
12788
13613
@item FS
12789
This is the input field separator
12790
(@pxref{Field Separators}).
13614
The input field separator (@pxref{Field Separators}).
12791
13615
The value is a single-character string or a multicharacter regular
12792
13616
expression that matches the separations between fields in an input
12793
13617
record. If the value is the null string (@code{""}), then each
12821
13645
@cindex @command{gawk}, @code{IGNORECASE} variable in
12822
13646
@cindex @code{IGNORECASE} variable
12823
13647
@cindex differences in @command{awk} and @command{gawk}, @code{IGNORECASE} variable
12824
@cindex case sensitivity, string comparisons and
12825
@cindex case sensitivity, regexps and
13648
@cindex case sensitivity, and string comparisons
13649
@cindex case sensitivity, and regexps
12826
13650
@cindex regular expressions, case sensitivity
12827
13651
@item IGNORECASE #
12828
13652
If @code{IGNORECASE} is nonzero or non-null, then all string comparisons
12837
13661
field separator.
12838
13662
@xref{Case-sensitivity}.
12839
13663
12840
If @command{gawk} is in compatibility mode
12841
(@pxref{Options}),
12842
then @code{IGNORECASE} has no special meaning. Thus, string
12843
and regexp operations are always case-sensitive.
12844
12845
13664
@cindex @command{gawk}, @code{LINT} variable in
12846
13665
@cindex @code{LINT} variable
12847
13666
@cindex differences in @command{awk} and @command{gawk}, @code{LINT} variable
12848
13667
@cindex lint checking
12849
13668
@item LINT #
12850
13669
When this variable is true (nonzero or non-null), @command{gawk}
12851
behaves as if the @option{--lint} command-line option is in effect.
13670
behaves as if the @option{--lint} command-line option is in effect
12852
13671
(@pxref{Options}).
12853
13672
With a value of @code{"fatal"}, lint warnings become fatal errors.
12854
13673
With a value of @code{"invalid"}, only warnings about things that are
12869
13688
@cindex numbers, converting, to strings
12870
13689
@cindex strings, converting, numbers to
12871
13690
@item OFMT
12872
This string controls conversion of numbers to
13691
Controls conversion of numbers to
12873
13692
strings (@pxref{Conversion}) for
12874
13693
printing with the @code{print} statement. It works by being passed
12875
13694
as the first argument to the @code{sprintf()} function
12890
13709
12891
13710
@cindex @code{ORS} variable
12892
13711
@item ORS
12893
This is the output record separator. It is output at the end of every
13712
The output record separator. It is output at the end of every
12894
13713
@code{print} statement. Its default value is @code{"\n"}, the newline
12895
13714
character. (@xref{Output Separators}.)
12896
13715
12897
13716
@cindex @code{PREC} variable
12898
13717
@item PREC #
12899
13718
The working precision of arbitrary precision floating-point numbers,
12900
53 bits by default (@pxref{Setting Precision}).
13719
53 bits by default (@pxref{Setting precision}).
12901
13720
12902
13721
@cindex @code{ROUNDMODE} variable
12903
13722
@item ROUNDMODE #
12904
13723
The rounding mode to use for arbitrary precision arithmetic on
12905
13724
numbers, by default @code{"N"} (@samp{roundTiesToEven} in
12906
the IEEE-754 standard)
12907
(@pxref{Setting Rounding Mode}).
13725
the IEEE 754 standard; @pxref{Setting the rounding mode}).
12908
13726
12909
13727
@cindex @code{RS} variable
12910
13728
@cindex separators, for records
12911
13729
@cindex record separators
12912
@item RS
12913
This is @command{awk}'s input record separator. Its default value is a string
13730
@item @code{RS}
13731
The input record separator. Its default value is a string
12914
13732
containing a single newline character, which means that an input record
12915
13733
consists of a single line of text.
12916
13734
It can also be the null string, in which case records are separated by
12929
13747
@cindex @code{SUBSEP} variable
12930
13748
@cindex separators, subscript
12931
13749
@cindex subscript separators
12932
@item SUBSEP
12933
This is the subscript separator. It has the default value of
13750
@item @code{SUBSEP}
13751
The subscript separator. It has the default value of
12934
13752
@code{"\034"} and is used to separate the parts of the indices of a
12935
13753
multidimensional array. Thus, the expression @code{@w{foo["A", "B"]}}
12936
13754
really accesses @code{foo["A\034B"]}
12941
13759
@cindex differences in @command{awk} and @command{gawk}, @code{TEXTDOMAIN} variable
12942
13760
@cindex internationalization, localization
12943
13761
@item TEXTDOMAIN #
12944
This variable is used for internationalization of programs at the
13762
Used for internationalization of programs at the
12945
13763
@command{awk} level. It sets the default text domain for specially
12946
13764
marked string constants in the source text, as well as for the
12947
13765
@code{dcgettext()}, @code{dcngettext()} and @code{bindtextdomain()} functions
12948
13766
(@pxref{Internationalization}).
12949
13767
The default value of @code{TEXTDOMAIN} is @code{"messages"}.
12950
12951
This variable is a @command{gawk} extension.
12952
In other @command{awk} implementations,
12953
or if @command{gawk} is in compatibility mode
12954
(@pxref{Options}),
12955
it is not special.
12956
13768
@end table
12957
13769
@c ENDOFRANGE bvar
12958
13770
@c ENDOFRANGE varb
12968
13780
@cindex variables, built-in, conveying information
12969
13781
The following is an alphabetical list of variables that @command{awk}
12970
13782
sets automatically on certain occasions in order to provide
12971
information to your program. The variables that are specific to
12972
@command{gawk} are marked with a pound sign@w{ (@samp{#}).}
12973
12974
@table @code
13783
information to your program.
13784
13785
The variables that are specific to @command{gawk} are marked with a pound
13786
sign (@samp{#}). These variables are @command{gawk} extensions. In other
13787
@command{awk} implementations or if @command{gawk} is in compatibility
13788
mode (@pxref{Options}), they are not special.
13789
13790
@c @asis for docbook
13791
@table @asis
12975
13792
@cindex @code{ARGC}/@code{ARGV} variables
12976
13793
@cindex arguments, command-line
12977
13794
@cindex command line, arguments
12978
@item ARGC@r{,} ARGV
13795
@item @code{ARGC}, @code{ARGV}
12979
13796
The command-line arguments available to @command{awk} programs are stored in
12980
13797
an array called @code{ARGV}. @code{ARGC} is the number of command-line
12981
13798
arguments present. @xref{Other Arguments}.
12987
13804
$ @kbd{awk 'BEGIN @{}
12988
13805
> @kbd{for (i = 0; i < ARGC; i++)}
12989
13806
> @kbd{print ARGV[i]}
12990
> @kbd{@}' inventory-shipped BBS-list}
13807
> @kbd{@}' inventory-shipped mail-list}
12991
13808
@print{} awk
12992
13809
@print{} inventory-shipped
12993
@print{} BBS-list
13810
@print{} mail-list
12994
13811
@end example
12995
13812
12996
13813
@noindent
12997
13814
@code{ARGV[0]} contains @samp{awk}, @code{ARGV[1]}
12998
13815
contains @samp{inventory-shipped}, and @code{ARGV[2]} contains
12999
@samp{BBS-list}. The value of @code{ARGC} is three, one more than the
13816
@samp{mail-list}. The value of @code{ARGC} is three, one more than the
13000
13817
index of the last element in @code{ARGV}, because the elements are numbered
13001
13818
from zero.
13002
13819
13015
13832
13016
13833
@cindex @code{ARGIND} variable
13017
13834
@cindex differences in @command{awk} and @command{gawk}, @code{ARGIND} variable
13018
@item ARGIND #
13835
@item @code{ARGIND #}
13019
13836
The index in @code{ARGV} of the current file being processed.
13020
Every time @command{gawk} opens a new data file for processing, it sets
13021
@code{ARGIND} to the index in @code{ARGV} of the file name.
13837
Every time @command{gawk} opens a new @value{DF} for processing, it sets
13838
@code{ARGIND} to the index in @code{ARGV} of the @value{FN}.
13022
13839
When @command{gawk} is processing the input files,
13023
13840
@samp{FILENAME == ARGV[ARGIND]} is always true.
13024
13841
13025
13842
@cindex files, processing@comma{} @code{ARGIND} variable and
13026
13843
This variable is useful in file processing; it allows you to tell how far
13027
along you are in the list of data files as well as to distinguish between
13028
successive instances of the same file name on the command line.
13844
along you are in the list of @value{DF}s as well as to distinguish between
13845
successive instances of the same @value{FN} on the command line.
13029
13846
13030
13847
@cindex file names, distinguishing
13031
13848
While you can change the value of @code{ARGIND} within your @command{awk}
13032
13849
program, @command{gawk} automatically sets it to a new value when the
13033
13850
next file is opened.
13034
13851
13035
This variable is a @command{gawk} extension.
13036
In other @command{awk} implementations,
13037
or if @command{gawk} is in compatibility mode
13038
(@pxref{Options}),
13039
it is not special.
13040
13041
13852
@cindex @code{ENVIRON} array
13042
@cindex environment variables
13043
@item ENVIRON
13853
@cindex environment variables, in @code{ENVIRON} array
13854
@item @code{ENVIRON}
13044
13855
An associative array containing the values of the environment. The array
13045
13856
indices are the environment variable names; the elements are the values of
13046
13857
the particular environment variables. For example,
13047
@code{ENVIRON["HOME"]} might be @file{/home/arnold}.
13858
@code{ENVIRON["HOME"]} might be @code{/home/arnold}.
13048
13859
13049
13860
For POSIX @command{awk}, changing this array does not affect the
13050
13861
environment passed on to any programs that @command{awk} may spawn via
13059
13870
13060
13871
Some operating systems may not have environment variables.
13061
13872
On such systems, the @code{ENVIRON} array is empty (except for
13062
@w{@code{ENVIRON["AWKPATH"]}},
13063
@pxref{AWKPATH Variable} and
13064
@w{@code{ENVIRON["AWKLIBPATH"]}},
13873
@w{@code{ENVIRON["AWKPATH"]}} and
13874
@w{@code{ENVIRON["AWKLIBPATH"]}};
13875
@pxref{AWKPATH Variable}, and
13065
13876
@pxref{AWKLIBPATH Variable}).
13066
13877
13067
13878
@cindex @command{gawk}, @code{ERRNO} variable in
13068
13879
@cindex @code{ERRNO} variable
13069
13880
@cindex differences in @command{awk} and @command{gawk}, @code{ERRNO} variable
13070
13881
@cindex error handling, @code{ERRNO} variable and
13071
@item ERRNO #
13072
If a system error occurs during a redirection for @code{getline},
13073
during a read for @code{getline}, or during a @code{close()} operation,
13074
then @code{ERRNO} contains a string describing the error.
13075
13076
In addition, @command{gawk} clears @code{ERRNO}
13077
before opening each command-line input file. This enables checking if
13078
the file is readable inside a @code{BEGINFILE} pattern (@pxref{BEGINFILE/ENDFILE}).
13079
13080
Otherwise,
13081
@code{ERRNO} works similarly to the C variable @code{errno}.
13082
Except for the case just mentioned,
13083
@command{gawk} @emph{never} clears it (sets it
13084
to zero or @code{""}). Thus, you should only expect its value
13085
to be meaningful when an I/O operation returns a failure
13086
value, such as @code{getline} returning @minus{}1.
13087
You are, of course, free to clear it yourself before doing an
13088
I/O operation.
13089
13090
This variable is a @command{gawk} extension.
13091
In other @command{awk} implementations,
13092
or if @command{gawk} is in compatibility mode
13093
(@pxref{Options}),
13094
it is not special.
13882
@item @code{ERRNO #}
13883
If a system error occurs during a redirection for @code{getline}, during
13884
a read for @code{getline}, or during a @code{close()} operation, then
13885
@code{ERRNO} contains a string describing the error.
13886
13887
In addition, @command{gawk} clears @code{ERRNO} before opening each
13888
command-line input file. This enables checking if the file is readable
13889
inside a @code{BEGINFILE} pattern (@pxref{BEGINFILE/ENDFILE}).
13890
13891
Otherwise, @code{ERRNO} works similarly to the C variable @code{errno}.
13892
Except for the case just mentioned, @command{gawk} @emph{never} clears
13893
it (sets it to zero or @code{""}). Thus, you should only expect its
13894
value to be meaningful when an I/O operation returns a failure value,
13895
such as @code{getline} returning @minus{}1. You are, of course, free
13896
to clear it yourself before doing an I/O operation.
13095
13897
13096
13898
@cindex @code{FILENAME} variable
13097
13899
@cindex dark corner, @code{FILENAME} variable
13098
@item FILENAME
13099
The name of the file that @command{awk} is currently reading.
13100
When no data files are listed on the command line, @command{awk} reads
13101
from the standard input and @code{FILENAME} is set to @code{"-"}.
13102
@code{FILENAME} is changed each time a new file is read
13103
(@pxref{Reading Files}).
13104
Inside a @code{BEGIN} rule, the value of @code{FILENAME} is
13105
@code{""}, since there are no input files being processed
13106
yet.@footnote{Some early implementations of Unix @command{awk} initialized
13107
@code{FILENAME} to @code{"-"}, even if there were data files to be
13108
processed. This behavior was incorrect and should not be relied
13109
upon in your programs.}
13110
@value{DARKCORNER}
13111
Note, though, that using @code{getline}
13112
(@pxref{Getline})
13113
inside a @code{BEGIN} rule can give
13114
@code{FILENAME} a value.
13900
@item @code{FILENAME}
13901
The name of the current input file. When no @value{DF}s are listed
13902
on the command line, @command{awk} reads from the standard input and
13903
@code{FILENAME} is set to @code{"-"}. @code{FILENAME} changes each
13904
time a new file is read (@pxref{Reading Files}). Inside a @code{BEGIN}
13905
rule, the value of @code{FILENAME} is @code{""}, since there are no input
13906
files being processed yet.@footnote{Some early implementations of Unix
13907
@command{awk} initialized @code{FILENAME} to @code{"-"}, even if there
13908
were @value{DF}s to be processed. This behavior was incorrect and should
13909
not be relied upon in your programs.} @value{DARKCORNER} Note, though,
13910
that using @code{getline} (@pxref{Getline}) inside a @code{BEGIN} rule
13911
can give @code{FILENAME} a value.
13115
13912
13116
13913
@cindex @code{FNR} variable
13117
@item FNR
13914
@item @code{FNR}
13118
13915
The current record number in the current file. @code{FNR} is
13119
13916
incremented each time a new record is read
13120
13917
(@pxref{Records}). It is reinitialized
13121
13918
to zero each time a new input file is started.
13122
13919
13123
13920
@cindex @code{NF} variable
13124
@item NF
13921
@item @code{NF}
13125
13922
The number of fields in the current input record.
13126
13923
@code{NF} is set each time a new record is read, when a new field is
13127
13924
created or when @code{$0} changes (@pxref{Fields}).
13128
13925
13129
Unlike most of the variables described in this
13130
@ifnotinfo
13131
section,
13132
@end ifnotinfo
13133
@ifinfo
13134
node,
13135
@end ifinfo
13926
Unlike most of the variables described in this @value{SUBSECTION},
13136
13927
assigning a value to @code{NF} has the potential to affect
13137
13928
@command{awk}'s internal workings. In particular, assignments
13138
13929
to @code{NF} can be used to create or remove fields from the
13141
13932
@cindex @code{FUNCTAB} array
13142
13933
@cindex @command{gawk}, @code{FUNCTAB} array in
13143
13934
@cindex differences in @command{awk} and @command{gawk}, @code{FUNCTAB} variable
13144
@item FUNCTAB #
13935
@item @code{FUNCTAB #}
13145
13936
An array whose indices and corresponding values are the names of all
13146
13937
the user-defined or extension functions in the program.
13147
13938
13148
13939
@quotation NOTE
13149
13940
Attempting to use the @code{delete} statement with the @code{FUNCTAB}
13150
array will cause a fatal error. Any attempt to assign to an element of
13151
the @code{FUNCTAB} array will also cause a fatal error.
13941
array causes a fatal error. Any attempt to assign to an element of
13942
@code{FUNCTAB} also causes a fatal error.
13152
13943
@end quotation
13153
13944
13154
13945
@cindex @code{NR} variable
13155
@item NR
13946
@item @code{NR}
13156
13947
The number of input records @command{awk} has processed since
13157
13948
the beginning of the program's execution
13158
13949
(@pxref{Records}).
13161
13952
@cindex @command{gawk}, @code{PROCINFO} array in
13162
13953
@cindex @code{PROCINFO} array
13163
13954
@cindex differences in @command{awk} and @command{gawk}, @code{PROCINFO} array
13164
@item PROCINFO #
13955
@item @code{PROCINFO #}
13165
13956
The elements of this array provide access to information about the
13166
13957
running @command{awk} program.
13167
13958
The following elements (listed alphabetically)
13168
13959
are guaranteed to be available:
13169
13960
13170
13961
@table @code
13962
@cindex effective group ID of @command{gawk} user
13171
13963
@item PROCINFO["egid"]
13172
13964
The value of the @code{getegid()} system call.
13173
13965
13174
13966
@item PROCINFO["euid"]
13967
@cindex effective user ID of @command{gawk} user
13175
13968
The value of the @code{geteuid()} system call.
13176
13969
13177
13970
@item PROCINFO["FS"]
13181
13974
or @code{"FPAT"} if field matching with @code{FPAT} is in effect.
13182
13975
13183
13976
@item PROCINFO["identifiers"]
13977
@cindex program identifiers
13184
13978
A subarray, indexed by the names of all identifiers used in the
13185
13979
text of the AWK program. For each identifier, the value of the element is one of the following:
13186
13980
13209
14003
while the program runs.
13210
14004
13211
14005
@item PROCINFO["gid"]
14006
@cindex group ID of @command{gawk} user
13212
14007
The value of the @code{getgid()} system call.
13213
14008
13214
14009
@item PROCINFO["pgrpid"]
14010
@cindex process group idIDof @command{gawk} process
13215
14011
The process group ID of the current process.
13216
14012
13217
14013
@item PROCINFO["pid"]
14014
@cindex process ID of @command{gawk} process
13218
14015
The process ID of the current process.
13219
14016
13220
14017
@item PROCINFO["ppid"]
14018
@cindex parent process ID of @command{gawk} process
13221
14019
The parent process ID of the current process.
13222
14020
13223
14021
@item PROCINFO["sorted_in"]
13224
14022
If this element exists in @code{PROCINFO}, its value controls the
13225
14023
order in which array indices will be processed by
13226
@samp{for (index in array) @dots{}} loops.
14024
@samp{for (@var{index} in @var{array})} loops.
13227
14025
Since this is an advanced feature, we defer the
13228
14026
full description until later; see
13229
14027
@ref{Scanning an Array}.
13237
14035
The value of the @code{getuid()} system call.
13238
14036
13239
14037
@item PROCINFO["version"]
14038
@cindex version of @command{gawk}
14039
@cindex @command{gawk} version
13240
14040
The version of @command{gawk}.
13241
14041
@end table
13242
14042
13246
14046
(@pxref{Arbitrary Precision Arithmetic}):
13247
14047
13248
14048
@table @code
14049
@cindex version of GNU MPFR library
13249
14050
@item PROCINFO["mpfr_version"]
13250
14051
The version of the GNU MPFR library.
13251
14052
13252
14053
@item PROCINFO["gmp_version"]
14054
@cindex version of GNU MP library
13253
14055
The version of the GNU MP library.
13254
14056
13255
14057
@item PROCINFO["prec_max"]
14058
@cindex maximum precision supported by MPFR library
13256
14059
The maximum precision supported by MPFR.
13257
14060
13258
14061
@item PROCINFO["prec_min"]
14062
@cindex minimum precision supported by MPFR library
13259
14063
The minimum precision required by MPFR.
13260
14064
@end table
13261
14065
13266
14070
13267
14071
@table @code
13268
14072
@item PROCINFO["api_major"]
14073
@cindex version of @command{gawk} extension API
14074
@cindex extension API, version number
13269
14075
The major version of the extension API.
13270
14076
13271
14077
@item PROCINFO["api_minor"]
13272
14078
The minor version of the extension API.
13273
14079
@end table
13274
14080
14081
@cindex supplementary groups of @command{gawk} process
13275
14082
On some systems, there may be elements in the array, @code{"group1"}
13276
14083
through @code{"group@var{N}"} for some @var{N}. @var{N} is the number of
13277
14084
supplementary groups that the process has. Use the @code{in} operator
13279
14086
(@pxref{Reference to Elements}).
13280
14087
13281
14088
@cindex @command{gawk}, @code{PROCINFO} array in
13282
@cindex @code{PROCINFO} array
14089
@cindex @code{PROCINFO} array, uses
13283
14090
The @code{PROCINFO} array has the following additional uses:
13284
14091
13285
@itemize @bullet
14092
@itemize @value{BULLET}
13286
14093
@item
13287
It may be
13288
used to cause coprocesses
13289
to communicate over pseudo-ttys instead of through two-way pipes;
13290
this is discussed further in @ref{Two-way I/O}.
14094
It may be used to cause coprocesses to communicate over pseudo-ttys
14095
instead of through two-way pipes; this is discussed further in
14096
@ref{Two-way I/O}.
13291
14097
13292
14098
@item
13293
14099
It may be used to provide a timeout when reading from any
13295
14101
@xref{Read Timeout}, for more information.
13296
14102
@end itemize
13297
14103
13298
This array is a @command{gawk} extension.
13299
In other @command{awk} implementations,
13300
or if @command{gawk} is in compatibility mode
13301
(@pxref{Options}),
13302
it is not special.
13303
13304
14104
@cindex @code{RLENGTH} variable
13305
@item RLENGTH
14105
@item @code{RLENGTH}
13306
14106
The length of the substring matched by the
13307
14107
@code{match()} function
13308
14108
(@pxref{String Functions}).
13310
14110
is the length of the matched string, or @minus{}1 if no match is found.
13311
14111
13312
14112
@cindex @code{RSTART} variable
13313
@item RSTART
14113
@item @code{RSTART}
13314
14114
The start-index in characters of the substring that is matched by the
13315
14115
@code{match()} function
13316
14116
(@pxref{String Functions}).
13321
14121
@cindex @command{gawk}, @code{RT} variable in
13322
14122
@cindex @code{RT} variable
13323
14123
@cindex differences in @command{awk} and @command{gawk}, @code{RT} variable
13324
@item RT #
13325
This is set each time a record is read. It contains the input text
13326
that matched the text denoted by @code{RS}, the record separator.
13327
13328
This variable is a @command{gawk} extension.
13329
In other @command{awk} implementations,
13330
or if @command{gawk} is in compatibility mode
13331
(@pxref{Options}),
13332
it is not special.
14124
@item @code{RT #}
14125
The input text that matched the text denoted by @code{RS},
14126
the record separator. It is set every time a record is read.
13333
14127
13334
14128
@cindex @command{gawk}, @code{SYMTAB} array in
13335
14129
@cindex @code{SYMTAB} array
13336
14130
@cindex differences in @command{awk} and @command{gawk}, @code{SYMTAB} variable
13337
@item SYMTAB #
14131
@item @code{SYMTAB #}
13338
14132
An array whose indices are the names of all currently defined
13339
14133
global variables and arrays in the program. The array may be used
13340
14134
for indirect access to read or write the value of a variable:
13351
14145
Also, you may not use the @code{delete} statement with the
13352
14146
@code{SYMTAB} array.
13353
14147
13354
You may use an index for @code{SYMTAB} that is not a predefined identifer:
14148
You may use an index for @code{SYMTAB} that is not a predefined identifier:
13355
14149
13356
14150
@example
13357
14151
SYMTAB["xxx"] = 5
13363
14157
a regular array. The only difference is that you can't then delete
13364
14158
@code{SYMTAB["xxx"]}.
13365
14159
14160
@cindex Schorr, Andrew
13366
14161
The @code{SYMTAB} array is more interesting than it looks. Andrew Schorr
13367
14162
points out that it effectively gives @command{awk} data pointers. Consider his
13368
14163
example:
13377
14172
@end example
13378
14173
13379
14174
@quotation NOTE
13380
In order to avoid severe time-travel paradoxes@footnote{Not to mention difficult
13381
implementation issues.}, neither @code{FUNCTAB} nor @code{SYMTAB}
14175
In order to avoid severe time-travel paradoxes,@footnote{Not to mention difficult
14176
implementation issues.} neither @code{FUNCTAB} nor @code{SYMTAB}
13382
14177
are available as elements within the @code{SYMTAB} array.
13383
14178
@end quotation
13384
14179
@end table
13419
14214
13420
14215
@node ARGC and ARGV
13421
14216
@subsection Using @code{ARGC} and @code{ARGV}
13422
@cindex @code{ARGC}/@code{ARGV} variables
14217
@cindex @code{ARGC}/@code{ARGV} variables, how to use
13423
14218
@cindex arguments, command-line
13424
14219
@cindex command line, arguments
13425
14220
13431
14226
$ @kbd{awk 'BEGIN @{}
13432
14227
> @kbd{for (i = 0; i < ARGC; i++)}
13433
14228
> @kbd{print ARGV[i]}
13434
> @kbd{@}' inventory-shipped BBS-list}
14229
> @kbd{@}' inventory-shipped mail-list}
13435
14230
@print{} awk
13436
14231
@print{} inventory-shipped
13437
@print{} BBS-list
14232
@print{} mail-list
13438
14233
@end example
13439
14234
13440
14235
@noindent
13441
14236
In this example, @code{ARGV[0]} contains @samp{awk}, @code{ARGV[1]}
13442
14237
contains @samp{inventory-shipped}, and @code{ARGV[2]} contains
13443
@samp{BBS-list}.
14238
@samp{mail-list}.
13444
14239
Notice that the @command{awk} program is not entered in @code{ARGV}. The
13445
14240
other command-line options, with their arguments, are also not
13446
14241
entered. This includes variable assignments done with the @option{-v}
13481
14276
If the value of @code{ARGC} is decreased, that eliminates input files
13482
14277
from the end of the list. By recording the old value of @code{ARGC}
13483
14278
elsewhere, a program can treat the eliminated arguments as
13484
something other than file names.
14279
something other than @value{FN}s.
13485
14280
13486
14281
To eliminate a file from the middle of the list, store the null string
13487
14282
(@code{""}) into @code{ARGV} in place of the file's name. As a
13488
special feature, @command{awk} ignores file names that have been
14283
special feature, @command{awk} ignores @value{FN}s that have been
13489
14284
replaced with the null string.
13490
14285
Another option is to
13491
14286
use the @code{delete} statement to remove elements from
13544
14339
(@xref{Getopt Function}, for an @command{awk} library function
13545
14340
that parses command-line options.)
13546
14341
14342
@node Pattern Action Summary
14343
@section Summary
14344
14345
@itemize @value{BULLET}
14346
@item
14347
Pattern-action pairs make up the basic elements of an @command{awk}
14348
program. Patterns are either normal expressions, range expressions,
14349
regexp constants, one of the special keywords @code{BEGIN}, @code{END},
14350
@code{BEGINFILE}, @code{ENDFILE}, or empty. The action executes if
14351
the current record matches the pattern. Empty (missing) patterns match
14352
all records.
14353
14354
@item
14355
I/O from @code{BEGIN} and @code{END} rules have certain constraints.
14356
This is also true, only more so, for @code{BEGINFILE} and @code{ENDFILE}
14357
rules. The latter two give you ``hooks'' into @command{gawk}'s file
14358
processing, allowing you to recover from a file that otherwise would
14359
cause a fatal error (such as a file that cannot be opened).
14360
14361
@item
14362
Shell variables can be used in @command{awk} programs by careful
14363
use of shell quoting. It is easier to pass a shell variable into
14364
@command{awk} by using the @option{-v} option and an @command{awk}
14365
variable.
14366
14367
@item
14368
Actions consist of statements enclosed in curly braces. Statements
14369
are built up from expressions, control statements, compound statements,
14370
input and output statements, and deletion statements.
14371
14372
@item
14373
The control statements in @command{awk} are @code{if}-@code{else},
14374
@code{while}, @code{for}, and @code{do}-@code{while}. @command{gawk}
14375
adds the @code{switch} statement. There are two flavors of @code{for}
14376
statement: one for for performing general looping, and the other iterating
14377
through an array.
14378
14379
@item
14380
@code{break} and @code{continue} let you exit early or start the next
14381
iteration of a loop (or get out of a @code{switch}).
14382
14383
@item
14384
@code{next} and @code{nextfile} let you read the next record and start
14385
over at the top of your program, or skip to the next input file and
14386
start over, respectively.
14387
14388
@item
14389
The @code{exit} statement terminates your program. When executed
14390
from an action (or function body) it transfers control to the
14391
@code{END} statements. From an @code{END} statement body, it exits
14392
immediately. You may pass an optional numeric value to be used
14393
at @command{awk}'s exit status.
14394
14395
@item
14396
Some built-in variables provide control over @command{awk}, mainly for I/O.
14397
Other variables convey information from @command{awk} to your program.
14398
14399
@end itemize
14400
13547
14401
@node Arrays
13548
14402
@chapter Arrays in @command{awk}
13549
14403
@c STARTOFRANGE arrs
13560
14414
arrays, as well as some of the less obvious points about array usage.
13561
14415
The @value{CHAPTER} moves on to discuss @command{gawk}'s facility
13562
14416
for sorting arrays, and ends with a brief description of @command{gawk}'s
13563
ability to support true multidimensional arrays.
14417
ability to support true arrays of arrays.
13564
14418
13565
14419
@cindex variables, names of
13566
14420
@cindex functions, names of
13567
@cindex arrays, names of
14421
@cindex arrays, names of, and names of functions/variables
13568
14422
@cindex names, arrays/variables
13569
14423
@cindex namespace issues
13570
14424
@command{awk} maintains a single set
13583
14437
* Multidimensional:: Emulating multidimensional arrays in
13584
14438
@command{awk}.
13585
14439
* Arrays of Arrays:: True multidimensional arrays.
14440
* Arrays Summary:: Summary of arrays.
13586
14441
@end menu
13587
14442
13588
14443
@node Array Basics
13644
14499
13645
14500
A contiguous array of four elements might look like the following example,
13646
14501
conceptually, if the element values are 8, @code{"foo"},
13647
@code{""}, and 30:
13648
13649
@c @strong{FIXME: NEXT ED:} Use real images here, and an @float
13650
@iftex
13651
@c from Karl Berry, much thanks for the help.
13652
@tex
13653
\bigskip % space above the table (about 1 linespace)
13654
\offinterlineskip
13655
\newdimen\width \width = 1.5cm
13656
\newdimen\hwidth \hwidth = 4\width \advance\hwidth by 2pt % 5 * 0.4pt
13657
\centerline{\vbox{
13658
\halign{\strut\hfil\ignorespaces#&&\vrule#&\hbox to\width{\hfil#\unskip\hfil}\cr
13659
\noalign{\hrule width\hwidth}
13660
&&{\tt 8} &&{\tt "foo"} &&{\tt ""} &&{\tt 30} &&\quad Value\cr
13661
\noalign{\hrule width\hwidth}
13662
\noalign{\smallskip}
13663
&\omit&0&\omit &1 &\omit&2 &\omit&3 &\omit&\quad Index\cr
13664
}
13665
}}
13666
@end tex
13667
@end iftex
13668
@ifnottex
13669
@example
13670
+---------+---------+--------+---------+
13671
| 8 | "foo" | "" | 30 | @r{Value}
13672
+---------+---------+--------+---------+
13673
0 1 2 3 @r{Index}
13674
@end example
13675
@end ifnottex
14502
@code{""}, and 30
14503
@ifnotdocbook
14504
as shown in @ref{figure-array-elements}:
14505
@end ifnotdocbook
14506
@ifdocbook
14507
as shown in @inlineraw{docbook, <xref linkend="figure-array-elements"/>}:
14508
@end ifdocbook
14509
14510
@ifnotdocbook
14511
@float Figure,figure-array-elements
14512
@caption{A Contiguous Array}
14513
@ifinfo
14514
@center @image{array-elements, , , Basic Program Stages, txt}
14515
@end ifinfo
14516
@ifnotinfo
14517
@center @image{array-elements, , , Basic Program Stages}
14518
@end ifnotinfo
14519
@end float
14520
@end ifnotdocbook
14521
14522
@docbook
14523
<figure id="figure-array-elements" float="0">
14524
<title>A Contiguous Array</title>
14525
<mediaobject>
14526
<imageobject role="web"><imagedata fileref="array-elements.png" format="PNG"/></imageobject>
14527
</mediaobject>
14528
</figure>
14529
@end docbook
13676
14530
13677
14531
@noindent
13678
14532
Only the values are stored; the indices are implicit from the order of
13689
14543
that each array is a collection of pairs: an index and its corresponding
13690
14544
array element value:
13691
14545
14546
@ifnotdocbook
13692
14547
@example
13693
14548
@r{Index} 3 @r{Value} 30
13694
14549
@r{Index} 1 @r{Value} "foo"
13695
14550
@r{Index} 0 @r{Value} 8
13696
14551
@r{Index} 2 @r{Value} ""
13697
14552
@end example
14553
@end ifnotdocbook
14554
14555
@docbook
14556
<informaltable>
14557
<tgroup cols="2">
14558
<colspec colname="1" align="center"/>
14559
<colspec colname="2" align="center"/>
14560
<thead>
14561
<row>
14562
<entry>Index</entry>
14563
<entry>Value</entry>
14564
</row>
14565
</thead>
14566
14567
<tbody>
14568
<row>
14569
<entry><literal>3</literal></entry>
14570
<entry><literal>30</literal></entry>
14571
</row>
14572
14573
<row>
14574
<entry><literal>1</literal></entry>
14575
<entry><literal>"foo"</literal></entry>
14576
</row>
14577
14578
<row>
14579
<entry><literal>0</literal></entry>
14580
<entry><literal>8</literal></entry>
14581
</row>
14582
14583
<row>
14584
<entry><literal>2</literal></entry>
14585
<entry><literal>""</literal></entry>
14586
</row>
14587
14588
</tbody>
14589
</tgroup>
14590
</informaltable>
14591
14592
@end docbook
13698
14593
13699
14594
@noindent
13700
14595
The pairs are shown in jumbled order because their order is irrelevant.
13703
14598
at any time. For example, suppose a tenth element is added to the array
13704
14599
whose value is @w{@code{"number ten"}}. The result is:
13705
14600
14601
@ifnotdocbook
13706
14602
@example
13707
14603
@r{Index} 10 @r{Value} "number ten"
13708
14604
@r{Index} 3 @r{Value} 30
13710
14606
@r{Index} 0 @r{Value} 8
13711
14607
@r{Index} 2 @r{Value} ""
13712
14608
@end example
14609
@end ifnotdocbook
14610
14611
@docbook
14612
<informaltable>
14613
<tgroup cols="2">
14614
<colspec colname="1" align="center"/>
14615
<colspec colname="2" align="center"/>
14616
<thead>
14617
<row>
14618
<entry>Index</entry>
14619
<entry>Value</entry>
14620
</row>
14621
</thead>
14622
<tbody>
14623
14624
<row>
14625
<entry><literal>10</literal></entry>
14626
<entry><literal>"number ten"</literal></entry>
14627
</row>
14628
14629
<row>
14630
<entry><literal>3</literal></entry>
14631
<entry><literal>30</literal></entry>
14632
</row>
14633
14634
<row>
14635
<entry><literal>1</literal></entry>
14636
<entry><literal>"foo"</literal></entry>
14637
</row>
14638
14639
<row>
14640
<entry><literal>0</literal></entry>
14641
<entry><literal>8</literal></entry>
14642
</row>
14643
14644
<row>
14645
<entry><literal>2</literal></entry>
14646
<entry><literal>""</literal></entry>
14647
</row>
14648
14649
</tbody>
14650
</tgroup>
14651
</informaltable>
14652
14653
@end docbook
13713
14654
13714
14655
@noindent
13715
14656
@cindex sparse arrays
13722
14663
an index. For example, the following is an array that translates words from
13723
14664
English to French:
13724
14665
14666
@ifnotdocbook
13725
14667
@example
13726
14668
@r{Index} "dog" @r{Value} "chien"
13727
14669
@r{Index} "cat" @r{Value} "chat"
13728
14670
@r{Index} "one" @r{Value} "un"
13729
14671
@r{Index} 1 @r{Value} "un"
13730
14672
@end example
14673
@end ifnotdocbook
14674
14675
@docbook
14676
<informaltable>
14677
<tgroup cols="2">
14678
<colspec colname="1" align="center"/>
14679
<colspec colname="2" align="center"/>
14680
<thead>
14681
<row>
14682
<entry>Index</entry>
14683
<entry>Value</entry>
14684
</row>
14685
</thead>
14686
<tbody>
14687
<row>
14688
<entry><literal>"dog"</literal></entry>
14689
<entry><literal>"chien"</literal></entry>
14690
</row>
14691
14692
<row>
14693
<entry><literal>"cat"</literal></entry>
14694
<entry><literal>"chat"</literal></entry>
14695
</row>
14696
14697
<row>
14698
<entry><literal>"one"</literal></entry>
14699
<entry><literal>"un"</literal></entry>
14700
</row>
14701
14702
<row>
14703
<entry><literal>1</literal></entry>
14704
<entry><literal>"un"</literal></entry>
14705
</row>
14706
14707
</tbody>
14708
</tgroup>
14709
</informaltable>
14710
14711
@end docbook
13731
14712
13732
14713
@noindent
13733
14714
Here we decided to translate the number one in both spelled-out and
13734
14715
numeric form---thus illustrating that a single array can have both
13735
14716
numbers and strings as indices.
13736
In fact, array subscripts are always strings; this is discussed
14717
(In fact, array subscripts are always strings; this is discussed
13737
14718
in more detail in
13738
@ref{Numeric Array Subscripts}.
14719
@ref{Numeric Array Subscripts}.)
13739
14720
Here, the number @code{1} isn't double-quoted, since @command{awk}
13740
14721
automatically converts it to a string.
13741
14722
13742
14723
@cindex @command{gawk}, @code{IGNORECASE} variable in
13743
@cindex @code{IGNORECASE} variable
13744
14724
@cindex case sensitivity, array indices and
13745
@cindex arrays, @code{IGNORECASE} variable and
13746
@cindex @code{IGNORECASE} variable, array subscripts and
14725
@cindex arrays, and @code{IGNORECASE} variable
14726
@cindex @code{IGNORECASE} variable, and array indices
13747
14727
The value of @code{IGNORECASE} has no effect upon array subscripting.
13748
14728
The identical string value used to store an array element must be used
13749
14729
to retrieve it.
13759
14739
13760
14740
@node Reference to Elements
13761
14741
@subsection Referring to an Array Element
13762
@cindex arrays, elements, referencing
13763
@cindex elements in arrays
14742
@cindex arrays, referencing elements
14743
@cindex array members
14744
@cindex elements of arrays
13764
14745
13765
14746
The principal way to use an array is to refer to one of its elements.
13766
14747
An array reference is an expression as follows:
13777
14758
element. For example, @code{foo[4.3]} is an expression for the element
13778
14759
of array @code{foo} at index @samp{4.3}.
13779
14760
14761
@cindex arrays, unassigned elements
14762
@cindex unassigned array elements
14763
@cindex empty array elements
13780
14764
A reference to an array element that has no recorded value yields a value of
13781
14765
@code{""}, the null string. This includes elements
13782
14766
that have not been assigned any value as well as elements that have been
13783
14767
deleted (@pxref{Delete}).
13784
14768
14769
@cindex non-existent array elements
14770
@cindex arrays, elements that don't exist
13785
14771
@quotation NOTE
13786
14772
A reference to an element that does not exist @emph{automatically} creates
13787
14773
that array element, with the null string as its value. (In some cases,
13801
14787
@end quotation
13802
14788
13803
14789
@c @cindex arrays, @code{in} operator and
13804
@cindex @code{in} operator
14790
@cindex @code{in} operator, testing if array element exists
13805
14791
To determine whether an element exists in an array at a certain index, use
13806
14792
the following expression:
13807
14793
13808
14794
@example
13809
@var{ind} in @var{array}
14795
@var{indx} in @var{array}
13810
14796
@end example
13811
14797
13812
14798
@cindex side effects, array indexing
13813
14799
@noindent
13814
This expression tests whether the particular index @var{ind} exists,
14800
This expression tests whether the particular index @var{indx} exists,
13815
14801
without the side effect of creating that element if it is not present.
13816
The expression has the value one (true) if @code{@var{array}[@var{ind}]}
14802
The expression has the value one (true) if @code{@var{array}[@var{indx}]}
13817
14803
exists and zero (false) if it does not exist.
13818
14804
For example, this statement tests whether the array @code{frequencies}
13819
14805
contains the index @samp{2}:
13836
14822
13837
14823
@node Assigning Elements
13838
14824
@subsection Assigning Array Elements
13839
@cindex arrays, elements, assigning
13840
@cindex elements in arrays, assigning
14825
@cindex arrays, elements, assigning values
14826
@cindex elements in arrays, assigning values
13841
14827
13842
14828
Array elements can be assigned values just like
13843
14829
@command{awk} variables:
13854
14840
13855
14841
@node Array Example
13856
14842
@subsection Basic Array Example
14843
@cindex arrays, an example of using
13857
14844
13858
14845
The following program takes a list of lines, each beginning with a line
13859
14846
number, and prints them out in order of line number. The line numbers
13923
14910
@node Scanning an Array
13924
14911
@subsection Scanning All Elements of an Array
13925
14912
@cindex elements in arrays, scanning
14913
@cindex scanning arrays
13926
14914
@cindex arrays, scanning
14915
@cindex loops, @code{for}, array scanning
13927
14916
13928
14917
In programs that use arrays, it is often necessary to use a loop that
13929
14918
executes once for each element of an array. In other languages, where
13940
14929
@end example
13941
14930
13942
14931
@noindent
13943
@cindex @code{in} operator
14932
@cindex @code{in} operator, use in loops
13944
14933
This loop executes @var{body} once for each index in @var{array} that the
13945
14934
program has previously used, with the variable @var{var} set to that index.
13946
14935
13979
14968
@xref{Word Sorting},
13980
14969
for a more detailed example of this type.
13981
14970
13982
@cindex arrays, elements, order of
13983
@cindex elements in arrays, order of
14971
@cindex arrays, elements, order of access by @code{in} operator
14972
@cindex elements in arrays, order of access by @code{in} operator
14973
@cindex @code{in} operator, order of array access
13984
14974
The order in which elements of the array are accessed by this statement
13985
14975
is determined by the internal arrangement of the array elements within
13986
@command{awk} and normally cannot be controlled or changed. This can lead to
13987
problems if new elements are added to @var{array} by statements in
13988
the loop body; it is not predictable whether the @code{for} loop will
13989
reach them. Similarly, changing @var{var} inside the loop may produce
13990
strange results. It is best to avoid such things.
14976
@command{awk} and in standard @command{awk} cannot be controlled
14977
or changed. This can lead to problems if new elements are added to
14978
@var{array} by statements in the loop body; it is not predictable whether
14979
the @code{for} loop will reach them. Similarly, changing @var{var} inside
14980
the loop may produce strange results. It is best to avoid such things.
14981
14982
As a point of information, @command{gawk} sets up the list of elements
14983
to be iterated over before the loop starts, and does not change it.
14984
But not all @command{awk} versions do so. Consider this program, named
14985
@file{loopcheck.awk}:
14986
14987
@example
14988
BEGIN @{
14989
a["here"] = "here"
14990
a["is"] = "is"
14991
a["a"] = "a"
14992
a["loop"] = "loop"
14993
for (i in a) @{
14994
j++
14995
a[j] = j
14996
print i
14997
@}
14998
@}
14999
@end example
15000
15001
Here is what happens when run with @command{gawk}:
15002
15003
@example
15004
$ @kbd{gawk -f loopcheck.awk}
15005
@print{} here
15006
@print{} loop
15007
@print{} a
15008
@print{} is
15009
@end example
15010
15011
Contrast this to Brian Kernighan's @command{awk}:
15012
15013
@example
15014
$ @kbd{nawk -f loopcheck.awk}
15015
@print{} loop
15016
@print{} here
15017
@print{} is
15018
@print{} a
15019
@print{} 1
15020
@end example
13991
15021
13992
15022
@node Controlling Scanning
13993
@subsection Using Predefined Array Scanning Orders
15023
@subsection Using Predefined Array Scanning Orders With @command{gawk}
15024
15025
This @value{SUBSECTION} describes a feature that is specific to @command{gawk}.
13994
15026
13995
15027
By default, when a @code{for} loop traverses an array, the order
13996
15028
is undefined, meaning that the @command{awk} implementation
13998
15030
This order is usually based on the internal implementation of arrays
13999
15031
and will vary from one version of @command{awk} to the next.
14000
15032
15033
@cindex array scanning order, controlling
15034
@cindex controlling array scanning order
14001
15035
Often, though, you may wish to do something simple, such as
14002
15036
``traverse the array by comparing the indices in ascending order,''
14003
15037
or ``traverse the array by comparing the values in descending order.''
14004
15038
@command{gawk} provides two mechanisms which give you this control.
14005
15039
14006
@itemize @bullet
15040
@itemize @value{BULLET}
14007
15041
@item
14008
15042
Set @code{PROCINFO["sorted_in"]} to one of a set of predefined values.
14009
15043
We describe this now.
14014
15048
is described later, in @ref{Array Sorting}.
14015
15049
@end itemize
14016
15050
15051
@cindex @code{PROCINFO}, values of @code{sorted_in}
14017
15052
The following special values for @code{PROCINFO["sorted_in"]} are available:
14018
15053
14019
15054
@table @code
14109
15144
Here are some additional things to bear in mind about sorted
14110
15145
array traversal.
14111
15146
14112
@itemize @bullet
15147
@itemize @value{BULLET}
14113
15148
@item
14114
15149
The value of @code{PROCINFO["sorted_in"]} is global. That is, it affects
14115
15150
all array traversal @code{for} loops. If you need to change it within your
14174
15209
print "This will never be printed"
14175
15210
@end example
14176
15211
14177
@cindex null strings, array elements and
15212
@cindex null strings, and deleting array elements
14178
15213
It is important to note that deleting an element is @emph{not} the
14179
15214
same as assigning it a null value (the empty string, @code{""}).
14180
15215
For example:
14196
15231
@cindex extensions, common@comma{} @code{delete} to delete entire arrays
14197
15232
@cindex arrays, deleting entire contents
14198
15233
@cindex deleting entire arrays
15234
@cindex @code{delete} @var{array}
14199
15235
@cindex differences in @command{awk} and @command{gawk}, array elements, deleting
14200
15236
All the elements of an array may be deleted with a single statement
14201
15237
by leaving off the subscript in the @code{delete} statement,
14210
15246
more efficient than the equivalent loop that deletes each element one
14211
15247
at a time.
14212
15248
15249
@cindex Brian Kernighan's @command{awk}
14213
15250
@quotation NOTE
14214
15251
For many years,
14215
15252
using @code{delete} without a subscript was a @command{gawk} extension.
14252
15289
@section Using Numbers to Subscript Arrays
14253
15290
14254
15291
@cindex numbers, as array subscripts
14255
@cindex arrays, subscripts
15292
@cindex arrays, numeric subscripts
14256
15293
@cindex subscripts in arrays, numbers as
14257
@cindex @code{CONVFMT} variable, array subscripts and
15294
@cindex @code{CONVFMT} variable, and array subscripts
14258
15295
An important aspect to remember about arrays is that @emph{array subscripts
14259
15296
are always strings}. When a numeric value is used as a subscript,
14260
15297
it is converted to a string value before being used for subscripting
14284
15321
@code{CONVFMT} only allows two significant digits. This test fails,
14285
15322
since @code{"12.15"} is different from @code{"12.153"}.
14286
15323
14287
@cindex converting, during subscripting
15324
@cindex converting integer array subscripts
15325
@cindex integer array indices
14288
15326
According to the rules for conversions
14289
15327
(@pxref{Conversion}), integer
14290
15328
values are always converted to strings as integers, no matter what the
14338
15376
@print{} line 2
14339
15377
@end example
14340
15378
14341
Unfortunately, the very first line of input data did not come out in the
15379
Unfortunately, the very first line of input data did not appear in the
14342
15380
output!
14343
15381
14344
15382
Upon first glance, we would think that this program should have worked.
14378
15416
@section Multidimensional Arrays
14379
15417
14380
15418
@menu
14381
* Multiscanning:: Scanning multidimensional arrays.
15419
* Multiscanning:: Scanning multidimensional arrays.
14382
15420
@end menu
14383
15421
14384
15422
@cindex subscripts in arrays, multidimensional
14390
15428
two-dimensional array named @code{grid} is with
14391
15429
@code{grid[@var{x},@var{y}]}.
14392
15430
14393
@cindex @code{SUBSEP} variable, multidimensional arrays
15431
@cindex @code{SUBSEP} variable, and multidimensional arrays
14394
15432
Multidimensional arrays are supported in @command{awk} through
14395
15433
concatenation of indices into one string.
14396
15434
@command{awk} converts the indices into strings
14422
15460
"b@@c"]}} are indistinguishable because both are actually
14423
15461
stored as @samp{foo["a@@b@@c"]}.
14424
15462
15463
@cindex @code{in} operator, index existence in multidimensional arrays
14425
15464
To test whether a particular index sequence exists in a
14426
15465
multidimensional array, use the same operator (@code{in}) that is
14427
15466
used for single dimensional arrays. Write the whole sequence of indices
14487
15526
14488
15527
@cindex subscripts in arrays, multidimensional, scanning
14489
15528
@cindex arrays, multidimensional, scanning
15529
@cindex scanning multidimensional arrays
14490
15530
However, if your program has an array that is always accessed as
14491
15531
multidimensional, you can get the effect of scanning it by combining
14492
15532
the scanning @code{for} statement
14528
15568
14529
15569
@node Arrays of Arrays
14530
15570
@section Arrays of Arrays
15571
@cindex arrays of arrays
14531
15572
14532
15573
@command{gawk} goes beyond standard @command{awk}'s multidimensional
14533
15574
array access and provides true arrays of
14534
15575
arrays. Elements of a subarray are referred to by their own indices
14535
15576
enclosed in square brackets, just like the elements of the main array.
14536
For example, the following creates a two-element subarray at index @samp{1}
15577
For example, the following creates a two-element subarray at index @code{1}
14537
15578
of the main array @code{a}:
14538
15579
14539
15580
@example
14557
15598
elements of an array or its subarray do not all have to have the same
14558
15599
type. This means that the main array and any of its subarrays can be
14559
15600
non-rectangular, or jagged in structure. One can assign a scalar value to
14560
the index @samp{4} of the main array @code{a}:
15601
the index @code{4} of the main array @code{a}:
14561
15602
14562
15603
@example
14563
15604
a[4] = "An element in a jagged array"
14578
15619
@end example
14579
15620
14580
15621
@noindent
14581
This removes the scalar value from index @samp{4} and then inserts a
15622
This removes the scalar value from index @code{4} and then inserts a
14582
15623
subarray of subarray of subarray containing a scalar. You can also
14583
15624
delete an entire subarray or subarray of subarrays:
14584
15625
14679
15720
$ @kbd{gawk 'BEGIN @{ b[1][1] = ""; split("a b c d", b[1]); print b[1][1] @}'}
14680
15721
@print{} a
14681
15722
@end example
15723
15724
@node Arrays Summary
15725
@section Summary
15726
15727
@itemize @value{BULLET}
15728
@item
15729
Standard @command{awk} provides one-dimensional associative arrays
15730
(arrays indexed by string values). All arrays are associative; numeric
15731
indices are converted automatically to strings.
15732
15733
@item
15734
Array elements are referenced as @code{@var{array}[@var{indx}]}.
15735
Referencing an element creates it if it did not exist previously.
15736
15737
@item
15738
The proper way to see if an array has an element with a given index
15739
is to use the @code{in} operator: @samp{@var{indx} in @var{array}}.
15740
15741
@item
15742
Use @samp{for (@var{indx} in @var{array}) @dots{}} to scan through all the
15743
individual elements of an array. In the body of the loop, @var{indx} takes
15744
on the value of each element's index in turn.
15745
15746
@item
15747
The order in which a @samp{for (@var{indx} in @var{array})} loop
15748
traverses an array is undefined in POSIX @command{awk} and varies among
15749
implementations. @command{gawk} lets you control the order by assigning
15750
special predefined values to @code{PROCINFO["sorted_in"]}.
15751
15752
@item
15753
Use @samp{delete @var{array}[@var{indx}]} to delete an individual element.
15754
You may also use @samp{delete @var{array}} to delete all of the elements
15755
in the array. This latter feature has been a common extension for many
15756
years and is now standard, but may not be supported by all commercial
15757
versions of @command{awk}.
15758
15759
@item
15760
Standard @command{awk} simulates multidimensional arrays by separating
15761
subscript values with a comma. The values are concatenated into a
15762
single string, separated by the value of @code{SUBSEP}. The fact
15763
that such a subscript was created in this way is not retained; thus
15764
changing @code{SUBSEP} may have unexpected consequences. You can use
15765
@samp{(@var{sub1}, @var{sub2}, @dots{}) in @var{array}} to see if such
15766
a multidimensional subscript exists in @var{array}.
15767
15768
@item
15769
@command{gawk} provides true arrays of arrays. You use a separate
15770
set of square brackets for each dimension in such an array:
15771
@code{data[row][col]}, for example. Array elements may thus be either
15772
scalar values (number or string) or another array.
15773
15774
@item
15775
Use the @code{isarray()} built-in function to determine if an array
15776
element is itself a subarray.
15777
15778
@end itemize
15779
14682
15780
@c ENDOFRANGE arrs
14683
15781
14684
15782
@node Functions
14703
15801
* Built-in:: Summarizes the built-in functions.
14704
15802
* User-defined:: Describes User-defined functions in detail.
14705
15803
* Indirect Calls:: Choosing the function to call at runtime.
15804
* Functions Summary:: Summary of functions.
14706
15805
@end menu
14707
15806
14708
15807
@node Built-in
14787
15886
14788
15887
@node Numeric Functions
14789
15888
@subsection Numeric Functions
15889
@cindex numeric functions
14790
15890
14791
15891
The following list describes all of
14792
15892
the built-in functions that work with numbers.
14793
15893
Optional parameters are enclosed in square brackets@w{ ([ ]):}
14794
15894
14795
@table @code
14796
@item atan2(@var{y}, @var{x})
14797
@cindex @code{atan2()} function
15895
@c @asis for docbook
15896
@table @asis
15897
@item @code{atan2(@var{y}, @var{x})}
15898
@cindexawkfunc{atan2}
15899
@cindex arctangent
14798
15900
Return the arctangent of @code{@var{y} / @var{x}} in radians.
14799
You can use @samp{pi = atan2(0, -1)} to retrieve the value of @value{PI}.
15901
You can use @samp{pi = atan2(0, -1)} to retrieve the value of
15902
@value{PI}.
14800
15903
14801
@item cos(@var{x})
14802
@cindex @code{cos()} function
15904
@item @code{cos(@var{x})}
15905
@cindexawkfunc{cos}
15906
@cindex cosine
14803
15907
Return the cosine of @var{x}, with @var{x} in radians.
14804
15908
14805
@item exp(@var{x})
14806
@cindex @code{exp()} function
15909
@item @code{exp(@var{x})}
15910
@cindexawkfunc{exp}
15911
@cindex exponent
14807
15912
Return the exponential of @var{x} (@code{e ^ @var{x}}) or report
14808
15913
an error if @var{x} is out of range. The range of values @var{x} can have
14809
15914
depends on your machine's floating-point representation.
14810
15915
14811
@item int(@var{x})
14812
@cindex @code{int()} function
15916
@item @code{int(@var{x})}
15917
@cindexawkfunc{int}
15918
@cindex round to nearest integer
14813
15919
Return the nearest integer to @var{x}, located between @var{x} and zero and
14814
15920
truncated toward zero.
14815
15921
14816
15922
For example, @code{int(3)} is 3, @code{int(3.9)} is 3, @code{int(-3.9)}
14817
15923
is @minus{}3, and @code{int(-3)} is @minus{}3 as well.
14818
15924
14819
@item log(@var{x})
14820
@cindex @code{log()} function
15925
@item @code{log(@var{x})}
15926
@cindexawkfunc{log}
15927
@cindex logarithm
14821
15928
Return the natural logarithm of @var{x}, if @var{x} is positive;
14822
15929
otherwise, report an error.
14823
15930
14824
@item rand()
14825
@cindex @code{rand()} function
15931
@item @code{rand()}
15932
@cindexawkfunc{rand}
14826
15933
@cindex random numbers, @code{rand()}/@code{srand()} functions
14827
15934
Return a random number. The values of @code{rand()} are
14828
15935
uniformly distributed between zero and one.
14864
15971
@}
14865
15972
@end example
14866
15973
14867
@cindex numbers, random
15974
@cindex seeding random number generator
14868
15975
@cindex random numbers, seed of
14869
15976
@quotation CAUTION
14870
15977
In most @command{awk} implementations, including @command{gawk},
14879
15986
use @code{srand()}.
14880
15987
@end quotation
14881
15988
14882
@item sin(@var{x})
14883
@cindex @code{sin()} function
15989
@item @code{sin(@var{x})}
15990
@cindexawkfunc{sin}
15991
@cindex sine
14884
15992
Return the sine of @var{x}, with @var{x} in radians.
14885
15993
14886
@item sqrt(@var{x})
14887
@cindex @code{sqrt()} function
15994
@item @code{sqrt(@var{x})}
15995
@cindexawkfunc{sqrt}
15996
@cindex square root
14888
15997
Return the positive square root of @var{x}.
14889
15998
@command{gawk} prints a warning message
14890
15999
if @var{x} is negative. Thus, @code{sqrt(4)} is 2.
14891
16000
14892
@item srand(@r{[}@var{x}@r{]})
14893
@cindex @code{srand()} function
16001
@item @code{srand(}[@var{x}]@code{)}
16002
@cindexawkfunc{srand}
14894
16003
Set the starting point, or seed,
14895
16004
for generating random numbers to the value @var{x}.
14896
16005
14920
16029
14921
16030
@node String Functions
14922
16031
@subsection String-Manipulation Functions
16032
@cindex string-manipulation functions
14923
16033
14924
16034
The functions in this @value{SECTION} look at or change the text of one
14925
16035
or more strings.
14932
16042
and not the number of bytes used to represent those characters. Similarly,
14933
16043
@code{index()} works with character indices, and not byte indices.
14934
16044
16045
@quotation CAUTION
16046
A number of functions deal with indices into strings. For these
16047
functions, the first character of a string is at position (index) one.
16048
This is different from C and the languages descended from it, where the
16049
first character is at position zero. You need to remember this when
16050
doing index calculations, particularly if you are used to C.
16051
@end quotation
16052
14935
16053
In the following list, optional parameters are enclosed in square brackets@w{ ([ ]).}
14936
16054
Several functions perform string substitution; the full discussion is
14937
16055
provided in the description of the @code{sub()} function, which comes
14938
16056
towards the end since the list is presented in alphabetic order.
16057
14939
16058
Those functions that are specific to @command{gawk} are marked with a
14940
pound sign@w{ (@samp{#}):}
16059
pound sign (@samp{#}). They are not available in compatibility mode
16060
(@pxref{Options}):
16061
14941
16062
14942
16063
@menu
14943
16064
* Gory Details:: More than you want to know about @samp{\} and
14945
16066
@code{gensub()}.
14946
16067
@end menu
14947
16068
14948
@table @code
14949
@item asort(@var{source} @r{[}, @var{dest} @r{[}, @var{how} @r{]} @r{]}) #
14950
@itemx asorti(@var{source} @r{[}, @var{dest} @r{[}, @var{how} @r{]} @r{]}) #
14951
@cindex @code{asorti()} function (@command{gawk})
16069
@c @asis for docbook
16070
@table @asis
16071
@item @code{asort(}@var{source} [@code{,} @var{dest} [@code{,} @var{how} ] ]@code{) #}
16072
@itemx @code{asorti(}@var{source} [@code{,} @var{dest} [@code{,} @var{how} ] ]@code{) #}
16073
@cindexgawkfunc{asorti}
16074
@cindex sort array
14952
16075
@cindex arrays, elements, retrieving number of
14953
@cindex @code{asort()} function (@command{gawk})
14954
@cindex @command{gawk}, @code{IGNORECASE} variable in
14955
@cindex @code{IGNORECASE} variable
16076
@cindexgawkfunc{asort}
16077
@cindex sort array indices
14956
16078
These two functions are similar in behavior, so they are described
14957
16079
together.
14958
16080
14970
16092
is specified, then @var{source} is duplicated into @var{dest}. @var{dest}
14971
16093
is then sorted, leaving the indices of @var{source} unchanged.
14972
16094
14973
When comparing strings, @code{IGNORECASE} affects the sorting. If the
16095
@cindex @command{gawk}, @code{IGNORECASE} variable in
16096
When comparing strings, @code{IGNORECASE} affects the sorting
16097
(@pxref{Array Sorting Functions}). If the
14974
16098
@var{source} array contains subarrays as values (@pxref{Arrays of
14975
16099
Arrays}), they will come last, after all scalar values.
14976
16100
15009
16133
a[3] = "middle"
15010
16134
@end example
15011
16135
15012
@code{asort()} and @code{asorti()} are @command{gawk} extensions; they
15013
are not available in compatibility mode (@pxref{Options}).
15014
15015
@item gensub(@var{regexp}, @var{replacement}, @var{how} @r{[}, @var{target}@r{]}) #
15016
@cindex @code{gensub()} function (@command{gawk})
16136
@item @code{gensub(@var{regexp}, @var{replacement}, @var{how}} [@code{, @var{target}}]@code{) #}
16137
@cindexgawkfunc{gensub}
16138
@cindex search and replace in strings
16139
@cindex substitute in string
15017
16140
Search the target string @var{target} for matches of the regular
15018
16141
expression @var{regexp}. If @var{how} is a string beginning with
15019
16142
@samp{g} or @samp{G} (short for ``global''), then replace all matches of @var{regexp} with
15022
16145
use @code{$0}. It returns the modified string as the result
15023
16146
of the function and the original target string is @emph{not} changed.
15024
16147
15025
@code{gensub()} is a general substitution function. It's purpose is
16148
@code{gensub()} is a general substitution function. Its purpose is
15026
16149
to provide more features than the standard @code{sub()} and @code{gsub()}
15027
16150
functions.
15028
16151
15072
16195
If @var{regexp} does not match @var{target}, @code{gensub()}'s return value
15073
16196
is the original unchanged value of @var{target}.
15074
16197
15075
@code{gensub()} is a @command{gawk} extension; it is not available
15076
in compatibility mode (@pxref{Options}).
15077
15078
@item gsub(@var{regexp}, @var{replacement} @r{[}, @var{target}@r{]})
15079
@cindex @code{gsub()} function
16198
@item @code{gsub(@var{regexp}, @var{replacement}} [@code{, @var{target}}]@code{)}
16199
@cindexawkfunc{gsub}
15080
16200
Search @var{target} for
15081
16201
@emph{all} of the longest, leftmost, @emph{nonoverlapping} matching
15082
16202
substrings it can find and replace them with @var{replacement}.
15097
16217
As in @code{sub()}, the characters @samp{&} and @samp{\} are special,
15098
16218
and the third argument must be assignable.
15099
16219
15100
@item index(@var{in}, @var{find})
15101
@cindex @code{index()} function
15102
@cindex searching
16220
@item @code{index(@var{in}, @var{find})}
16221
@cindexawkfunc{index}
16222
@cindex search in string
16223
@cindex find substring in string
15103
16224
Search the string @var{in} for the first occurrence of the string
15104
16225
@var{find}, and return the position in characters where that occurrence
15105
16226
begins in the string @var{in}. Consider the following example:
15111
16232
15112
16233
@noindent
15113
16234
If @var{find} is not found, @code{index()} returns zero.
15114
(Remember that string indices in @command{awk} start at one.)
15115
16235
15116
16236
It is a fatal error to use a regexp constant for @var{find}.
15117
16237
15118
@item length(@r{[}@var{string}@r{]})
15119
@cindex @code{length()} function
16238
@item @code{length(}[@var{string}]@code{)}
16239
@cindexawkfunc{length}
16240
@cindex string length
16241
@cindex length of string
15120
16242
Return the number of characters in @var{string}. If
15121
16243
@var{string} is a number, the length of the digit string representing
15122
16244
that number is returned. For example, @code{length("abcde")} is five. By
15123
contrast, @code{length(15 * 35)} works out to three. In this example, 15 * 35 =
15124
525, and 525 is then converted to the string @code{"525"}, which has
16245
contrast, @code{length(15 * 35)} works out to three. In this example,
16246
@iftex
16247
@math{15 @cdot 35 = 525},
16248
@end iftex
16249
@ifnottex
16250
@ifnotdocbook
16251
15 * 35 = 525,
16252
@end ifnotdocbook
16253
@end ifnottex
16254
@docbook
16255
15 ⋅ 35 = 525, @c
16256
@end docbook
16257
and 525 is then converted to the string @code{"525"}, which has
15125
16258
three characters.
15126
16259
16260
@cindex length of input record
16261
@cindex input record, length of
15127
16262
If no argument is supplied, @code{length()} returns the length of @code{$0}.
15128
16263
15129
16264
@c @cindex historical features
15162
16297
@cindex common extensions, @code{length()} applied to an array
15163
16298
@cindex extensions, common@comma{} @code{length()} applied to an array
15164
16299
@cindex differences between @command{gawk} and @command{awk}
16300
@cindex number of array elements
16301
@cindex array, number of elements
15165
16302
With @command{gawk} and several other @command{awk} implementations, when given an
15166
16303
array argument, the @code{length()} function returns the number of elements
15167
16304
in the array. @value{COMMONEXT}
15174
16311
If @option{--posix} is supplied, using an array argument is a fatal error
15175
16312
(@pxref{Arrays}).
15176
16313
15177
@item match(@var{string}, @var{regexp} @r{[}, @var{array}@r{]})
15178
@cindex @code{match()} function
16314
@item @code{match(@var{string}, @var{regexp}} [@code{, @var{array}}]@code{)}
16315
@cindexawkfunc{match}
16316
@cindex string, regular expression match
16317
@cindex match regexp in string
15179
16318
Search @var{string} for the
15180
16319
longest, leftmost substring matched by the regular expression,
15181
@var{regexp} and return the character position, or @dfn{index},
16320
@var{regexp} and return the character position (index)
15182
16321
at which that substring begins (one, if it starts at the beginning of
15183
16322
@var{string}). If no match is found, return zero.
15184
16323
15185
16324
The @var{regexp} argument may be either a regexp constant
15186
(@code{/@dots{}/}) or a string constant (@code{"@dots{}"}).
16325
(@code{/}@dots{}@code{/}) or a string constant (@code{"}@dots{}@code{"}).
15187
16326
In the latter case, the string is treated as a regexp to be matched.
15188
16327
@xref{Computed Regexps}, for a
15189
16328
discussion of the difference between the two forms, and the
15289
16428
(@pxref{Options}),
15290
16429
using a third argument is a fatal error.
15291
16430
15292
@item patsplit(@var{string}, @var{array} @r{[}, @var{fieldpat} @r{[}, @var{seps} @r{]} @r{]}) #
15293
@cindex @code{patsplit()} function (@command{gawk})
16431
@item @code{patsplit(@var{string}, @var{array}} [@code{, @var{fieldpat}} [@code{, @var{seps}} ] ]@code{) #}
16432
@cindexgawkfunc{patsplit}
16433
@cindex split string into array
15294
16434
Divide
15295
16435
@var{string} into pieces defined by @var{fieldpat}
15296
16436
and store the pieces in @var{array} and the separator strings in the
15314
16454
Before splitting the string, @code{patsplit()} deletes any previously existing
15315
16455
elements in the arrays @var{array} and @var{seps}.
15316
16456
15317
@cindex troubleshooting, @code{patsplit()} function
15318
The @code{patsplit()} function is a
15319
@command{gawk} extension. In compatibility mode
15320
(@pxref{Options}),
15321
it is not available.
15322
15323
@item split(@var{string}, @var{array} @r{[}, @var{fieldsep} @r{[}, @var{seps} @r{]} @r{]})
15324
@cindex @code{split()} function
16457
@item @code{split(@var{string}, @var{array}} [@code{, @var{fieldsep}} [@code{, @var{seps}} ] ]@code{)}
16458
@cindexawkfunc{split}
15325
16459
Divide @var{string} into pieces separated by @var{fieldsep}
15326
16460
and store the pieces in @var{array} and the separator strings in the
15327
16461
@var{seps} array. The first piece is stored in
15350
16484
@end example
15351
16485
15352
16486
@noindent
15353
@cindex strings, splitting
16487
@cindex strings splitting, example
15354
16488
splits the string @samp{cul-de-sac} into three fields using @samp{-} as the
15355
16489
separator. It sets the contents of the array @code{a} as follows:
15356
16490
15405
16539
@var{array} has one element only. The value of that element is the original
15406
16540
@var{string}.
15407
16541
15408
@item sprintf(@var{format}, @var{expression1}, @dots{})
15409
@cindex @code{sprintf()} function
16542
In POSIX mode (@pxref{Options}), the fourth argument is not allowed.
16543
16544
@item @code{sprintf(@var{format}, @var{expression1}, @dots{})}
16545
@cindexawkfunc{sprintf}
16546
@cindex formatting strings
15410
16547
Return (without printing) the string that @code{printf} would
15411
16548
have printed out with the same arguments
15412
16549
(@pxref{Printf}).
15419
16556
@noindent
15420
16557
assigns the string @w{@samp{pi = 3.14 (approx.)}} to the variable @code{pival}.
15421
16558
15422
@cindex @code{strtonum()} function (@command{gawk})
15423
@item strtonum(@var{str}) #
16559
@cindexgawkfunc{strtonum}
16560
@cindex convert string to number
16561
@item @code{strtonum(@var{str}) #}
15424
16562
Examine @var{str} and return its numeric value. If @var{str}
15425
16563
begins with a leading @samp{0}, @code{strtonum()} assumes that @var{str}
15426
16564
is an octal number. If @var{str} begins with a leading @samp{0x} or
15442
16580
Note also that @code{strtonum()} uses the current locale's decimal point
15443
16581
for recognizing numbers (@pxref{Locales}).
15444
16582
15445
@cindex differences in @command{awk} and @command{gawk}, @code{strtonum()} function (@command{gawk})
15446
@code{strtonum()} is a @command{gawk} extension; it is not available
15447
in compatibility mode (@pxref{Options}).
15448
15449
@item sub(@var{regexp}, @var{replacement} @r{[}, @var{target}@r{]})
15450
@cindex @code{sub()} function
16583
@item @code{sub(@var{regexp}, @var{replacement}} [@code{, @var{target}}]@code{)}
16584
@cindexawkfunc{sub}
16585
@cindex replace in string
15451
16586
Search @var{target}, which is treated as a string, for the
15452
16587
leftmost, longest substring matched by the regular expression @var{regexp}.
15453
16588
Modify the entire string
15456
16591
Return the number of substitutions made (zero or one).
15457
16592
15458
16593
The @var{regexp} argument may be either a regexp constant
15459
(@code{/@dots{}/}) or a string constant (@code{"@dots{}"}).
16594
(@code{/}@dots{}@code{/}) or a string constant (@code{"}@dots{}@code{"}).
15460
16595
In the latter case, the string is treated as a regexp to be matched.
15461
16596
@xref{Computed Regexps}, for a
15462
16597
discussion of the difference between the two forms, and the
15546
16681
Finally, if the @var{regexp} is not a regexp constant, it is converted into a
15547
16682
string, and then the value of that string is treated as the regexp to match.
15548
16683
15549
@item substr(@var{string}, @var{start} @r{[}, @var{length}@r{]})
15550
@cindex @code{substr()} function
16684
@item @code{substr(@var{string}, @var{start}} [@code{, @var{length}} ]@code{)}
16685
@cindexawkfunc{substr}
16686
@cindex substring
15551
16687
Return a @var{length}-character-long substring of @var{string},
15552
16688
starting at character number @var{start}. The first character of a
15553
16689
string is character number one.@footnote{This is different from
15561
16697
if @var{length} is greater than the number of characters remaining
15562
16698
in the string, counting from character @var{start}.
15563
16699
16700
@cindex Brian Kernighan's @command{awk}
15564
16701
If @var{start} is less than one, @code{substr()} treats it as
15565
16702
if it was one. (POSIX doesn't specify what to do in this case:
15566
16703
Brian Kernighan's @command{awk} acts this way, and therefore @command{gawk}
15603
16740
@end example
15604
16741
15605
16742
@cindex case sensitivity, converting case
15606
@cindex converting, case
15607
@item tolower(@var{string})
15608
@cindex @code{tolower()} function
16743
@cindex strings, converting letter case
16744
@item @code{tolower(@var{string})}
16745
@cindexawkfunc{tolower}
16746
@cindex convert string to lower case
15609
16747
Return a copy of @var{string}, with each uppercase character
15610
16748
in the string replaced with its corresponding lowercase character.
15611
16749
Nonalphabetic characters are left unchanged. For example,
15612
16750
@code{tolower("MiXeD cAsE 123")} returns @code{"mixed case 123"}.
15613
16751
15614
@item toupper(@var{string})
15615
@cindex @code{toupper()} function
16752
@item @code{toupper(@var{string})}
16753
@cindexawkfunc{toupper}
16754
@cindex convert string to upper case
15616
16755
Return a copy of @var{string}, with each lowercase character
15617
16756
in the string replaced with its corresponding uppercase character.
15618
16757
Nonalphabetic characters are left unchanged. For example,
15636
16775
15637
16776
First, there is the @dfn{lexical} level, which is when @command{awk} reads
15638
16777
your program
15639
and builds an internal copy of it that can be executed.
16778
and builds an internal copy of it to execute.
15640
16779
Then there is the runtime level, which is when @command{awk} actually scans the
15641
16780
replacement string to determine what to generate.
15642
16781
16782
@cindex Brian Kernighan's @command{awk}
15643
16783
At both levels, @command{awk} looks for a defined set of characters that
15644
16784
can come after a backslash. At the lexical level, it looks for the
15645
16785
escape sequences listed in @ref{Escape Sequences}.
15778
16918
following character, but for anything other than @samp{\} and @samp{&},
15779
16919
such special meaning is undefined. This wording leads to two problems:
15780
16920
15781
@itemize @bullet
16921
@itemize @value{BULLET}
15782
16922
@item
15783
16923
Backslashes must now be doubled in the @var{replacement} string, breaking
15784
16924
historical @command{awk} programs.
15909
17049
The only case where the difference is noticeable is the last one: @samp{\\\\}
15910
17050
is seen as @samp{\\} and produces @samp{\} instead of @samp{\\}.
15911
17051
15912
Starting with version 3.1.4, @command{gawk} followed the POSIX rules
17052
Starting with @value{PVERSION} 3.1.4, @command{gawk} followed the POSIX rules
15913
17053
when @option{--posix} is specified (@pxref{Options}). Otherwise,
15914
17054
it continued to follow the 1996 proposed rules, since
15915
17055
that had been its behavior for many years.
15916
17056
15917
When version 4.0.0 was released, the @command{gawk} maintainer
17057
When @value{PVERSION} 4.0.0 was released, the @command{gawk} maintainer
15918
17058
made the POSIX rules the default, breaking well over a decade's worth
15919
17059
of backwards compatibility.@footnote{This was rather naive of him, despite
15920
17060
there being a note in this section indicating that the next major version
15921
17061
would move to the POSIX rules.} Needless to say, this was a bad idea,
15922
and as of version 4.0.1, @command{gawk} resumed its historical
17062
and as of @value{PVERSION} 4.0.1, @command{gawk} resumed its historical
15923
17063
behavior, and only follows the POSIX rules when @option{--posix} is given.
15924
17064
15925
17065
The rules for @code{gensub()} are considerably simpler. At the runtime
16004
17144
16005
17145
@node I/O Functions
16006
17146
@subsection Input/Output Functions
17147
@cindex input/output functions
16007
17148
16008
17149
The following functions relate to input/output (I/O).
16009
17150
Optional parameters are enclosed in square brackets ([ ]):
16010
17151
16011
@table @code
16012
@item close(@var{filename} @r{[}, @var{how}@r{]})
16013
@cindex @code{close()} function
17152
@table @asis
17153
@item @code{close(}@var{filename} [@code{,} @var{how}]@code{)}
17154
@cindexawkfunc{close}
16014
17155
@cindex files, closing
17156
@cindex close file or coprocess
16015
17157
Close the file @var{filename} for input or output. Alternatively, the
16016
17158
argument may be a shell command that was used for creating a coprocess, or
16017
17159
for redirecting to or from a pipe; then the coprocess or pipe is closed.
16027
17169
@xref{Two-way I/O},
16028
17170
which discusses this feature in more detail and gives an example.
16029
17171
16030
@item fflush(@r{[}@var{filename}@r{]})
16031
@cindex @code{fflush()} function
17172
Note that the second argument to @code{close()} is a @command{gawk}
17173
extension; it is not available in compatibility mode (@pxref{Options}).
17174
17175
@item @code{fflush(}[@var{filename}]@code{)}
17176
@cindexawkfunc{fflush}
17177
@cindex flush buffered output
16032
17178
Flush any buffered output associated with @var{filename}, which is either a
16033
17179
file opened for writing or a shell command for redirecting output to
16034
17180
a pipe or coprocess.
16046
17192
buffers its output and the @code{fflush()} function forces
16047
17193
@command{gawk} to flush its buffers.
16048
17194
16049
@code{fflush()} was added to Brian Kernighan's
16050
version of @command{awk} in 1994.
16051
For over two decades, it was not part of the POSIX standard.
16052
As of December, 2012, it was accepted for
16053
inclusion into the POSIX standard.
17195
@cindex extensions, common@comma{} @code{fflush()} function
17196
@cindex Brian Kernighan's @command{awk}
17197
@code{fflush()} was added to Brian Kernighan's @command{awk} in
17198
April of 1992. For two decades, it was not part of the POSIX standard.
17199
As of December, 2012, it was accepted for inclusion into the POSIX
17200
standard.
16054
17201
See @uref{http://austingroupbugs.net/view.php?id=634, the Austin Group website}.
16055
17202
16056
17203
POSIX standardizes @code{fflush()} as follows: If there
16059
17206
and pipes.
16060
17207
16061
17208
@quotation NOTE
16062
Prior to version 4.0.2, @command{gawk}
17209
Prior to @value{PVERSION} 4.0.2, @command{gawk}
16063
17210
would flush only the standard output if there was no argument,
16064
17211
and flush all output files and pipes if the argument was the null
16065
17212
string. This was changed in order to be compatible with Brian
16075
17222
@c @cindex warnings, automatic
16076
17223
@cindex troubleshooting, @code{fflush()} function
16077
17224
@code{fflush()} returns zero if the buffer is successfully flushed;
16078
otherwise, it returns non-zero (@command{gawk} returns @minus{}1).
17225
otherwise, it returns non-zero. (@command{gawk} returns @minus{}1.)
16079
17226
In the case where all buffers are flushed, the return value is zero
16080
17227
only if all buffers were flushed successfully. Otherwise, it is
16081
17228
@minus{}1, and @command{gawk} warns about the problem @var{filename}.
16085
17232
or if @var{filename} is not an open file, pipe, or coprocess.
16086
17233
In such a case, @code{fflush()} returns @minus{}1, as well.
16087
17234
16088
@item system(@var{command})
16089
@cindex @code{system()} function
17235
@item @code{system(@var{command})}
17236
@cindexawkfunc{system}
17237
@cindex invoke shell command
16090
17238
@cindex interacting with other programs
16091
17239
Execute the operating-system
16092
17240
command @var{command} and then return to the @command{awk} program.
16117
17265
16118
17266
@noindent
16119
17267
@cindex troubleshooting, @code{system()} function
16120
@cindex @code{--sandbox} option, disabling @code{system()} function
17268
@cindex @option{--sandbox} option, disabling @code{system()} function
16121
17269
However, if your @command{awk}
16122
17270
program is interactive, @code{system()} is useful for running large
16123
17271
self-contained programs, such as a shell or an editor.
16233
17381
16234
17382
@node Time Functions
16235
17383
@subsection Time Functions
17384
@cindex time functions
16236
17385
16237
17386
@c STARTOFRANGE tst
16238
17387
@cindex timestamps
16249
17398
in the form returned by the @code{time()} system call, which is the
16250
17399
number of seconds since a particular epoch. On POSIX-compliant systems,
16251
17400
it is the number of seconds since
16252
1970-01-01 00:00:00 UTC, not counting leap seconds.@footnote{@xref{Glossary},
16253
especially the entries ``Epoch'' and ``UTC.''}
17401
1970-01-01 00:00:00 UTC, not counting leap
17402
@ifclear FOR_PRINT
17403
seconds.@footnote{@xref{Glossary}, especially the entries ``Epoch'' and ``UTC.''}
17404
@end ifclear
17405
@ifset FOR_PRINT
17406
seconds.
17407
@end ifset
16254
17408
All known POSIX-compliant systems support timestamps from 0 through
16255
@math{2^{31} - 1}, which is sufficient to represent times through
17409
@iftex
17410
@math{2^{31} - 1},
17411
@end iftex
17412
@ifnottex
17413
@ifnotdocbook
17414
2^31 - 1,
17415
@end ifnotdocbook
17416
@end ifnottex
17417
@docbook
17418
2<superscript>31</superscript> − 1, @c
17419
@end docbook
17420
which is sufficient to represent times through
16256
17421
2038-01-19 03:14:07 UTC. Many systems support a wider range of timestamps,
16257
17422
including negative timestamps that represent times before the
16258
17423
epoch.
16269
17434
of @command{mawk} (@pxref{Other Versions}) also support these functions.
16270
17435
Optional parameters are enclosed in square brackets ([ ]):
16271
17436
16272
@table @code
16273
@item mktime(@var{datespec})
16274
@cindex @code{mktime()} function (@command{gawk})
17437
@c @asis for docbook
17438
@table @asis
17439
@item @code{mktime(@var{datespec})}
17440
@cindexgawkfunc{mktime}
17441
@cindex generate time values
16275
17442
Turn @var{datespec} into a timestamp in the same form
16276
17443
as is returned by @code{systime()}. It is similar to the function of the
16277
17444
same name in ISO C. The argument, @var{datespec}, is a string of the form
16299
17466
16300
17467
@cindex @command{gawk}, @code{PROCINFO} array in
16301
17468
@cindex @code{PROCINFO} array
16302
@item strftime(@r{[}@var{format} @r{[}, @var{timestamp} @r{[}, @var{utc-flag}@r{]]]})
17469
@item @code{strftime(} [@var{format} [@code{,} @var{timestamp} [@code{,} @var{utc-flag}] ] ]@code{)}
16303
17470
@c STARTOFRANGE strf
16304
@cindex @code{strftime()} function (@command{gawk})
17471
@cindexgawkfunc{strftime}
17472
@cindex format time string
16305
17473
Format the time specified by @var{timestamp}
16306
17474
based on the contents of the @var{format} string and return the result.
16307
17475
It is similar to the function of the same name in ISO C.
16318
17486
@code{@w{"%a %b %e %H:%M:%S %Z %Y"}}. This format string produces
16319
17487
output that is equivalent to that of the @command{date} utility.
16320
17488
You can assign a new value to @code{PROCINFO["strftime"]} to
16321
change the default format.
17489
change the default format; see below for the various format directives.
16322
17490
16323
@item systime()
16324
@cindex @code{systime()} function (@command{gawk})
17491
@item @code{systime()}
17492
@cindexgawkfunc{systime}
16325
17493
@cindex timestamps
17494
@cindex current system time
16326
17495
Return the current time as the number of seconds since
16327
17496
the system epoch. On POSIX systems, this is the number of seconds
16328
17497
since 1970-01-01 00:00:00 UTC, not counting leap seconds.
16394
17563
16395
17564
@item %g
16396
17565
The year modulo 100 of the ISO 8601 week number, as a decimal number (00--99).
16397
For example, January 1, 1993 is in week 53 of 1992. Thus, the year
16398
of its ISO 8601 week number is 1992, even though its year is 1993.
16399
Similarly, December 31, 1973 is in week 1 of 1974. Thus, the year
16400
of its ISO week number is 1974, even though its year is 1973.
17566
For example, January 1, 2012 is in week 53 of 2011. Thus, the year
17567
of its ISO 8601 week number is 2011, even though its year is 2012.
17568
Similarly, December 31, 2012 is in week 1 of 2013. Thus, the year
17569
of its ISO week number is 2013, even though its year is 2012.
16401
17570
16402
17571
@item %G
16403
17572
The full year of the ISO week number, as a decimal number.
16478
17647
The year modulo 100 as a decimal number (00--99).
16479
17648
16480
17649
@item %Y
16481
The full year as a decimal number (e.g., 2011).
17650
The full year as a decimal number (e.g., 2015).
16482
17651
16483
17652
@c @cindex RFC 822
16484
17653
@c @cindex RFC 1036
16512
17681
Typically, the conversion specifier either does not appear in the
16513
17682
returned string or appears literally.}
16514
17683
16515
@c @cindex locale, definition of
16516
Informally, a @dfn{locale} is the geographic place in which a program
16517
is meant to run. For example, a common way to abbreviate the date
16518
September 4, 2012 in the United States is ``9/4/12.''
16519
In many countries in Europe, however, it is abbreviated ``4.9.12.''
16520
Thus, the @samp{%x} specification in a @code{"US"} locale might produce
16521
@samp{9/4/12}, while in a @code{"EUROPE"} locale, it might produce
16522
@samp{4.9.12}. The ISO C standard defines a default @code{"C"}
16523
locale, which is an environment that is typical of what many C programmers
16524
are used to.
16525
16526
17684
For systems that are not yet fully standards-compliant,
16527
17685
@command{gawk} supplies a copy of
16528
17686
@code{strftime()} from the GNU C Library.
16575
17733
16576
17734
@example
16577
17735
$ date '+Today is %A, %B %d, %Y.'
16578
@print{} Today is Wednesday, March 30, 2011.
17736
@print{} Today is Monday, May 05, 2014.
16579
17737
@end example
16580
17738
16581
17739
Here is the @command{gawk} version of the @command{date} utility.
16595
17753
esac
16596
17754
16597
17755
gawk 'BEGIN @{
16598
format = "%a %b %e %H:%M:%S %Z %Y"
17756
format = PROCINFO["strftime"]
16599
17757
exitval = 0
16600
17758
16601
17759
if (ARGC > 2)
16616
17774
16617
17775
@node Bitwise Functions
16618
17776
@subsection Bit-Manipulation Functions
17777
@cindex bit-manipulation functions
16619
17778
@c STARTOFRANGE bit
16620
17779
@cindex bitwise, operations
16621
17780
@c STARTOFRANGE and
16682
17841
@end tex
16683
17842
16684
17843
@docbook
16685
<!-- FIXME: Fix ID and add xref in text. -->
16686
<table id="table-bitwise-ops">
16687
<title>Bitwise Operations</title>
17844
<informaltable>
16688
17845
16689
17846
<tgroup cols="7" colsep="1">
16690
17847
<colspec colname="c1"/>
16744
17901
16745
17902
</tbody>
16746
17903
</tgroup>
16747
</table>
17904
</informaltable>
16748
17905
@end docbook
16749
17906
@end float
16750
17907
16778
17935
16779
17936
@cindex @command{gawk}, bitwise operations in
16780
17937
@table @code
16781
@cindex @code{and()} function (@command{gawk})
16782
@item and(@var{v1}, @var{v2} @r{[}, @r{@dots{}]})
17938
@cindexgawkfunc{and}
17939
@cindex bitwise AND
17940
@item @code{and(@var{v1}, @var{v2}} [@code{,} @dots{}]@code{)}
16783
17941
Return the bitwise AND of the arguments. There must be at least two.
16784
17942
16785
@cindex @code{compl()} function (@command{gawk})
16786
@item compl(@var{val})
17943
@cindexgawkfunc{compl}
17944
@cindex bitwise complement
17945
@item @code{compl(@var{val})}
16787
17946
Return the bitwise complement of @var{val}.
16788
17947
16789
@cindex @code{lshift()} function (@command{gawk})
16790
@item lshift(@var{val}, @var{count})
17948
@cindexgawkfunc{lshift}
17949
@cindex left shift
17950
@item @code{lshift(@var{val}, @var{count})}
16791
17951
Return the value of @var{val}, shifted left by @var{count} bits.
16792
17952
16793
@cindex @code{or()} function (@command{gawk})
16794
@item or(@var{v1}, @var{v2} @r{[}, @r{@dots{}]})
17953
@cindexgawkfunc{or}
17954
@cindex bitwise OR
17955
@item @code{or(@var{v1}, @var{v2}} [@code{,} @dots{}]@code{)}
16795
17956
Return the bitwise OR of the arguments. There must be at least two.
16796
17957
16797
@cindex @code{rshift()} function (@command{gawk})
16798
@item rshift(@var{val}, @var{count})
17958
@cindexgawkfunc{rshift}
17959
@cindex right shift
17960
@item @code{rshift(@var{val}, @var{count})}
16799
17961
Return the value of @var{val}, shifted right by @var{count} bits.
16800
17962
16801
@cindex @code{xor()} function (@command{gawk})
16802
@item xor(@var{v1}, @var{v2} @r{[}, @r{@dots{}]})
17963
@cindexgawkfunc{xor}
17964
@cindex bitwise XOR
17965
@item @code{xor(@var{v1}, @var{v2}} [@code{,} @dots{}]@code{)}
16803
17966
Return the bitwise XOR of the arguments. There must be at least two.
16804
17967
@end table
16805
17968
16890
18053
@cindex strings, converting
16891
18054
@cindex numbers, converting
16892
18055
@cindex converting, numbers to strings
18056
@cindex number as string of bits
16893
18057
The @code{bits2str()} function turns a binary number into a string.
16894
18058
The number @code{1} represents a binary value where the rightmost bit
16895
18059
is set to 1. Using this mask,
16921
18085
16922
18086
@command{gawk} provides a single function that lets you distinguish
16923
18087
an array from a scalar variable. This is necessary for writing code
16924
that traverses every element of a true multidimensional array
18088
that traverses every element of an array of arrays.
16925
18089
(@pxref{Arrays of Arrays}).
16926
18090
16927
18091
@table @code
16928
@cindex @code{isarray()} function (@command{gawk})
18092
@cindexgawkfunc{isarray}
18093
@cindex scalar or array
16929
18094
@item isarray(@var{x})
16930
18095
Return a true value if @var{x} is an array. Otherwise return false.
16931
18096
@end table
16933
18098
@code{isarray()} is meant for use in two circumstances. The first is when
16934
18099
traversing a multidimensional array: you can test if an element is itself
16935
18100
an array or not. The second is inside the body of a user-defined function
16936
(not discussed yet; @pxref{User-defined}), to test if a paramater is an
18101
(not discussed yet; @pxref{User-defined}), to test if a parameter is an
16937
18102
array or not.
16938
18103
16939
18104
Note, however, that using @code{isarray()} at the global level to test
16947
18112
@subsection String-Translation Functions
16948
18113
@cindex @command{gawk}, string-translation functions
16949
18114
@cindex functions, string-translation
18115
@cindex string-translation functions
16950
18116
@cindex internationalization
16951
18117
@cindex @command{awk} programs, internationalizing
16952
18118
16957
18123
for the full story.
16958
18124
Optional parameters are enclosed in square brackets ([ ]):
16959
18125
16960
@table @code
16961
@cindex @code{bindtextdomain()} function (@command{gawk})
16962
@item bindtextdomain(@var{directory} @r{[}, @var{domain}@r{]})
18126
@table @asis
18127
@cindexgawkfunc{bindtextdomain}
18128
@cindex set directory of message catalogs
18129
@item @code{bindtextdomain(@var{directory}} [@code{,} @var{domain}]@code{)}
16963
18130
Set the directory in which
16964
18131
@command{gawk} will look for message translation files, in case they
16965
18132
will not or cannot be placed in the ``standard'' locations
16971
18138
@code{bindtextdomain()} returns the current binding for the
16972
18139
given @var{domain}.
16973
18140
16974
@cindex @code{dcgettext()} function (@command{gawk})
16975
@item dcgettext(@var{string} @r{[}, @var{domain} @r{[}, @var{category}@r{]]})
18141
@cindexgawkfunc{dcgettext}
18142
@cindex translate string
18143
@item @code{dcgettext(@var{string}} [@code{,} @var{domain} [@code{,} @var{category}] ]@code{)}
16976
18144
Return the translation of @var{string} in
16977
18145
text domain @var{domain} for locale category @var{category}.
16978
18146
The default value for @var{domain} is the current value of @code{TEXTDOMAIN}.
16979
18147
The default value for @var{category} is @code{"LC_MESSAGES"}.
16980
18148
16981
@cindex @code{dcngettext()} function (@command{gawk})
16982
@item dcngettext(@var{string1}, @var{string2}, @var{number} @r{[}, @var{domain} @r{[}, @var{category}@r{]]})
18149
@cindexgawkfunc{dcngettext}
18150
@item @code{dcngettext(@var{string1}, @var{string2}, @var{number}} [@code{,} @var{domain} [@code{,} @var{category}] ]@code{)}
16983
18151
Return the plural form used for @var{number} of the
16984
18152
translation of @var{string1} and @var{string2} in text domain
16985
18153
@var{domain} for locale category @var{category}. @var{string1} is the
16995
18163
@section User-Defined Functions
16996
18164
16997
18165
@c STARTOFRANGE udfunc
16998
@cindex user-defined, functions
18166
@cindex user-defined functions
16999
18167
@c STARTOFRANGE funcud
17000
18168
@cindex functions, user-defined
17001
18169
Complicated @command{awk} programs can often be simplified by defining
17027
18195
17028
18196
The definition of a function named @var{name} looks like this:
17029
18197
17030
@example
17031
function @var{name}(@r{[}@var{parameter-list}@r{]})
17032
@{
18198
@display
18199
@code{function} @var{name}@code{(}[@var{parameter-list}]@code{)}
18200
@code{@{}
17033
18201
@var{body-of-function}
17034
@}
17035
@end example
18202
@code{@}}
18203
@end display
17036
18204
17037
18205
@cindex names, functions
17038
18206
@cindex functions, names of
17047
18215
@var{parameter-list} is an optional list of the function's arguments and local
17048
18216
variable names, separated by commas. When the function is called,
17049
18217
the argument names are used to hold the argument values given in
17050
the call. The local variables are initialized to the empty string.
18218
the call.
18219
17051
18220
A function cannot have two parameters with the same name, nor may it
17052
18221
have a parameter with the same name as the function itself.
18222
In addition, according to the POSIX standard, function parameters
18223
cannot have the same name as one of the special built-in variables
18224
(@pxref{Built-in Variables}). Not all versions of @command{awk} enforce
18225
this restriction.)
17053
18226
17054
In addition, according to the POSIX standard, function parameters cannot have the same
17055
name as one of the special built-in variables
17056
(@pxref{Built-in Variables}. Not all versions of @command{awk}
17057
enforce this restriction.
18227
Local variables act like the empty string if referenced where a string
18228
value is required, and like zero if referenced where a numeric value
18229
is required. This is the same as regular variables that have never been
18230
assigned a value. (There is more to understand about local variables;
18231
@pxref{Dynamic Typing}.)
17058
18232
17059
18233
The @var{body-of-function} consists of @command{awk} statements. It is the
17060
18234
most important part of the definition, because it says what the function
17081
18255
the local variables, in order to document how your function is supposed to be used.
17082
18256
17083
18257
@cindex variables, shadowing
18258
@cindex shadowing of variable values
17084
18259
During execution of the function body, the arguments and local variable
17085
18260
values hide, or @dfn{shadow}, any variables of the same names used in the
17086
18261
rest of the program. The shadowed variables are not accessible in the
17101
18276
The act of a function calling itself is called @dfn{recursion}.
17102
18277
17103
18278
All the built-in functions return a value to their caller.
17104
User-defined functions can do also, using the @code{return} statement,
18279
User-defined functions can do so also, using the @code{return} statement,
17105
18280
which is described in detail in @ref{Return Statement}.
17106
18281
Many of the subsequent examples in this @value{SECTION} use
17107
18282
the @code{return} statement.
17139
18314
17140
18315
@node Function Example
17141
18316
@subsection Function Definition Examples
18317
@cindex function definition example
17142
18318
17143
18319
Here is an example of a user-defined function, called @code{myprint()}, that
17144
18320
takes a number and prints it in a specific format:
17193
18369
to repeat this loop everywhere that you need to clear out
17194
18370
an array, your program can just call @code{delarray}.
17195
18371
(This guarantees portability. The use of @samp{delete @var{array}} to delete
17196
the contents of an entire array is a nonstandard extension.)
18372
the contents of an entire array is a recent@footnote{Late in 2012.}
18373
addition to the POSIX standard.)
17197
18374
17198
18375
The following is an example of a recursive function. It takes a string
17199
18376
as an input parameter and returns the string in backwards order.
17236
18413
17237
18414
function ctime(ts, format)
17238
18415
@{
17239
format = "%a %b %e %H:%M:%S %Z %Y"
18416
format = PROCINFO["strftime"]
17240
18417
if (ts == 0)
17241
18418
ts = systime() # use current time as default
17242
18419
return strftime(format, ts)
17249
18426
@subsection Calling User-Defined Functions
17250
18427
17251
18428
@c STARTOFRANGE fudc
17252
This section describes how to call a user-defined function.
18429
@cindex functions, user-defined, calling
18430
@dfn{Calling a function} means causing the function to run and do its job.
18431
A function call is an expression and its value is the value returned by
18432
the function.
17253
18433
17254
18434
@menu
17255
18435
* Calling A Function:: Don't use spaces.
17260
18440
@node Calling A Function
17261
18441
@subsubsection Writing A Function Call
17262
18442
17263
@cindex functions, user-defined, calling
17264
@dfn{Calling a function} means causing the function to run and do its job.
17265
A function call is an expression and its value is the value returned by
17266
the function.
17267
17268
18443
A function call consists of the function name followed by the arguments
17269
18444
in parentheses. @command{awk} expressions are what you write in the
17270
18445
call for the arguments. Each time the call is executed, these
17288
18463
@node Variable Scope
17289
18464
@subsubsection Controlling Variable Scope
17290
18465
17291
@cindex local variables
17292
@cindex variables, local
17293
There is no way to make a variable local to a @code{@{ @dots{} @}} block in
18466
@cindex local variables, in a function
18467
@cindex variables, local to a function
18468
Unlike many languages,
18469
there is no way to make a variable local to a @code{@{} @dots{} @code{@}} block in
17294
18470
@command{awk}, but you can make a variable local to a function. It is
17295
18471
good practice to do so whenever a variable is needed only in that
17296
18472
function.
17552
18728
can also be used to return a value for use in the rest of the @command{awk}
17553
18729
program. It looks like this:
17554
18730
17555
@example
17556
return @r{[}@var{expression}@r{]}
17557
@end example
18731
@display
18732
@code{return} [@var{expression}]
18733
@end display
17558
18734
17559
18735
The @var{expression} part is optional.
17560
18736
Due most likely to an oversight, POSIX does not define what the return
17561
18737
value is if you omit the @var{expression}. Technically speaking, this
17562
make the returned value undefined, and therefore, unpredictable.
18738
makes the returned value undefined, and therefore, unpredictable.
17563
18739
In practice, though, all versions of @command{awk} simply return the
17564
18740
null string, which acts like zero if used in a numeric context.
17565
18741
17662
18838
@end example
17663
18839
17664
18840
In this example, the first call to @code{foo()} generates
17665
a fatal error, so @command{gawk} will not report the second
17666
error. If you comment out that call, though, then @command{gawk}
17667
will report the second error.
18841
a fatal error, so @command{awk} will not report the second
18842
error. If you comment out that call, though, then @command{awk}
18843
does report the second error.
17668
18844
17669
18845
Usually, such things aren't a big issue, but it's worth
17670
18846
being aware of them.
17734
18910
17735
18911
@example
17736
18912
the_func = "sum"
17737
result = @@the_func() # calls the `sum' function
18913
result = @@the_func() # calls the sum() function
17738
18914
@end example
17739
18915
17740
18916
Here is a full program that processes the previously shown data,
17855
19031
@ignore
17856
19032
@c file eg/lib/quicksort.awk
17857
19033
#
17858
# Arnold Robbins, arnold@skeeve.com, Public Domain
19034
# Arnold Robbins, arnold@@skeeve.com, Public Domain
17859
19035
# January 2009
19036
17860
19037
@c endfile
17861
19038
17862
19039
@end ignore
17929
19106
17930
19107
Next comes a sorting function. It is parameterized with the starting and
17931
19108
ending field numbers and the comparison function. It builds an array with
17932
the data and calls @code{quicksort} appropriately, and then formats the
19109
the data and calls @code{quicksort()} appropriately, and then formats the
17933
19110
results as a single string:
17934
19111
17935
19112
@example
17977
19154
@c endfile
17978
19155
@end example
17979
19156
17980
Here is an extended version of the data file:
19157
Here is an extended version of the @value{DF}:
17981
19158
17982
19159
@example
17983
19160
@c file eg/data/class_data2
18028
19205
@noindent
18029
19206
@code{gawk} will look up the actual function to call only once.
18030
19207
19208
@node Functions Summary
19209
@section Summary
19210
19211
@itemize @value{BULLET}
19212
@item
19213
@command{awk} provides built-in functions and lets you define your own
19214
functions.
19215
19216
@item
19217
POSIX @command{awk} provides three kinds of built-in functions: numeric,
19218
string, and I/O. @command{gawk} provides functions that work with values
19219
representing time, do bit manipulation, sort arrays, and internationalize
19220
and localize programs. @command{gawk} also provides several extensions to
19221
some of standard functions, typically in the form of additional arguments.
19222
19223
@item
19224
Functions accept zero or more arguments and return a value. The
19225
expressions that provide the argument values are completely evaluated
19226
before the function is called. Order of evaluation is not defined.
19227
The return value can be ignored.
19228
19229
@item
19230
The handling of backslash in @code{sub()} and @code{gsub()} is not simple.
19231
It is more straightforward in @command{gawk}'s @code{gensub()} function,
19232
but that function still requires care in its use.
19233
19234
@item
19235
User-defined functions provide important capabilities but come with
19236
some syntactic inelegancies. In a function call, there cannot be any
19237
space between the function name and the opening left parenthesis of the
19238
argument list. Also, there is no provision for local variables, so the
19239
convention is to add extra parameters, and to separate them visually
19240
from the real parameters by extra whitespace.
19241
19242
@item
19243
User-defined functions may call other user-defined (and built-in)
19244
functions and may call themselves recursively. Function parameters
19245
``hide'' any global variables of the same names.
19246
19247
@item
19248
Scalar values are passed to user-defined functions by value. Array
19249
parameters are passed by reference; any changes made by the function to
19250
array parameters are thus visible after the function has returned.
19251
19252
@item
19253
Use the @code{return} statement to return from a user-defined function.
19254
An optional expression becomes the function's return value. Only scalar
19255
values may be returned by a function.
19256
19257
@item
19258
If a variable that has never been used is passed to a user-defined
19259
function, how that function treats the variable can set its nature:
19260
either scalar or array.
19261
19262
@item
19263
@command{gawk} provides indirect function calls using a special syntax.
19264
By setting a variable to the name of a user-defined function, you can
19265
determine at runtime what function will be called at that point in the
19266
program. This is equivalent to function pointers in C and C++.
19267
19268
@end itemize
19269
18031
19270
@c ENDOFRANGE funcud
18032
19271
18033
@iftex
18034
@part Part II:@* Problem Solving With @command{awk}
18035
@end iftex
19272
@ifnotinfo
19273
@part @value{PART2}Problem Solving With @command{awk}
19274
@end ifnotinfo
18036
19275
18037
@ignore
18038
19276
@ifdocbook
18039
@part Part II:@* Problem Solving With @command{awk}
18040
18041
19277
Part II shows how to use @command{awk} and @command{gawk} for problem solving.
18042
19278
There is lots of code here for you to read and learn from.
18043
19279
It contains the following chapters:
18044
19280
18045
@itemize @bullet
19281
@itemize @value{BULLET}
18046
19282
@item
18047
19283
@ref{Library Functions}.
18048
19284
18050
19286
@ref{Sample Programs}.
18051
19287
@end itemize
18052
19288
@end ifdocbook
18053
@end ignore
18054
19289
18055
19290
@node Library Functions
18056
19291
@chapter A Library of @command{awk} Functions
18067
19302
place. It simplifies programming, making program development more
18068
19303
manageable, and making programs more readable.
18069
19304
19305
@cindex Kernighan, Brian
19306
@cindex Plauger, P.J.@:
18070
19307
In their seminal 1976 book, @cite{Software Tools},@footnote{Sadly, over 35
18071
19308
years later, many of the lessons taught by this book have yet to be
18072
19309
learned by a vast number of practicing programmers.} Brian Kernighan
18086
19323
Programs}, provide a good-sized body of code for you to read, and we hope,
18087
19324
to learn from.
18088
19325
18089
@c 2e: USE TEXINFO-2 FUNCTION DEFINITION STUFF!!!!!!!!!!!!!
18090
19326
This @value{CHAPTER} presents a library of useful @command{awk} functions.
18091
19327
Many of the sample programs presented later in this @value{DOCUMENT}
18092
19328
use these functions.
18099
19335
for this @value{DOCUMENT}.
18100
19336
(This has already been done as part of the @command{gawk} distribution.)
18101
19337
19338
@ifclear FOR_PRINT
18102
19339
If you have written one or more useful, general-purpose @command{awk} functions
18103
19340
and would like to contribute them to the @command{awk} user community, see
18104
19341
@ref{How To Contribute}, for more information.
19342
@end ifclear
18105
19343
18106
19344
@cindex portability, example programs
18107
19345
The programs in this @value{CHAPTER} and in
18110
19348
Rewriting these programs for different implementations of @command{awk}
18111
19349
is pretty straightforward.
18112
19350
18113
@itemize @bullet
19351
@itemize @value{BULLET}
18114
19352
@item
18115
19353
Diagnostic error messages are sent to @file{/dev/stderr}.
18116
19354
Use @samp{| "cat 1>&2"} instead of @samp{> "/dev/stderr"} if your system
18154
19392
* Passwd Functions:: Functions for getting user information.
18155
19393
* Group Functions:: Functions for getting group information.
18156
19394
* Walking Arrays:: A function to walk arrays of arrays.
19395
* Library Functions Summary:: Summary of library functions.
19396
* Library exercises:: Exercises.
18157
19397
@end menu
18158
19398
18159
19399
@node Library Names
18196
19436
@cindex underscore (@code{_}), in names of private variables
18197
19437
In addition, several of the library functions use a prefix that helps
18198
19438
indicate what function or set of functions use the variables---for example,
18199
@code{_pw_byname} in the user database routines
19439
@code{_pw_byname()} in the user database routines
18200
19440
(@pxref{Passwd Functions}).
18201
19441
This convention is recommended, since it even further decreases the
18202
19442
chance of inadvertent conflict among variable names. Note that this
18215
19455
the variable name is not all capital letters indicates that the variable is
18216
19456
not one of @command{awk}'s built-in variables, such as @code{FS}.
18217
19457
18218
@cindex @code{--dump-variables} option
19458
@cindex @option{--dump-variables} option, using for library functions
18219
19459
It is also important that @emph{all} variables in library
18220
19460
functions that do not need to save state are, in fact, declared
18221
19461
local.@footnote{@command{gawk}'s @option{--dump-variables} command-line
18288
19528
#
18289
19529
# Arnold Robbins, arnold@@skeeve.com, Public Domain
18290
19530
# February, 2004
19531
# Revised June, 2014
18291
19532
18292
19533
@c endfile
18293
19534
@end ignore
18294
19535
@c file eg/lib/strtonum.awk
18295
function mystrtonum(str, ret, chars, n, i, k, c)
19536
function mystrtonum(str, ret, n, i, k, c)
18296
19537
@{
18297
19538
if (str ~ /^0[0-7]*$/) @{
18298
19539
# octal
18305
19546
18306
19547
ret = ret * 8 + k
18307
19548
@}
18308
@} else if (str ~ /^0[xX][[:xdigit:]]+/) @{
19549
@} else if (str ~ /^0[xX][[:xdigit:]]+$/) @{
18309
19550
# hexadecimal
18310
19551
str = substr(str, 3) # lop off leading 0x
18311
19552
n = length(str)
18313
19554
for (i = 1; i <= n; i++) @{
18314
19555
c = substr(str, i, 1)
18315
19556
c = tolower(c)
18316
if ((k = index("0123456789", c)) > 0)
18317
k-- # adjust for 1-basing in awk
18318
else if ((k = index("abcdef", c)) > 0)
18319
k += 9
19557
k = index("123456789abcdef", c)
18320
19558
18321
19559
ret = ret * 16 + k
18322
19560
@}
18484
19722
to the program calling @code{assert()}. Normally, if a program consists
18485
19723
of just a @code{BEGIN} rule, the input files and/or standard input are
18486
19724
not read. However, now that the program has an @code{END} rule, @command{awk}
18487
attempts to read the input data files or standard input
19725
attempts to read the input @value{DF}s or standard input
18488
19726
(@pxref{Using BEGIN/END}),
18489
19727
most likely causing the program to hang as it waits for input.
18490
19728
18510
19748
The way @code{printf} and @code{sprintf()}
18511
19749
(@pxref{Printf})
18512
19750
perform rounding often depends upon the system's C @code{sprintf()}
18513
subroutine. On many machines, @code{sprintf()} rounding is ``unbiased,''
18514
which means it doesn't always round a trailing @samp{.5} up, contrary
18515
to naive expectations. In unbiased rounding, @samp{.5} rounds to even,
19751
subroutine. On many machines, @code{sprintf()} rounding is @dfn{unbiased},
19752
which means it doesn't always round a trailing .5 up, contrary
19753
to naive expectations. In unbiased rounding, .5 rounds to even,
18516
19754
rather than always up, so 1.5 rounds to 2 but 4.5 rounds to 4. This means
18517
19755
that if you are using a format that does rounding (e.g., @code{"%.0f"}),
18518
19756
you should check what your system does. The following function does
18561
19799
@c don't include test harness in the file that gets installed
18562
19800
18563
19801
# test harness
18564
@{ print $0, round($0) @}
19802
# @{ print $0, round($0) @}
18565
19803
@end example
18566
19804
18567
19805
@node Cliff Random Function
18628
19866
18629
19867
@cindex @code{ord()} user-defined function
18630
19868
@cindex @code{chr()} user-defined function
19869
@cindex @code{_ord_init()} user-defined function
18631
19870
@example
18632
19871
@c file eg/lib/ord.awk
18633
19872
# ord.awk --- do ord and chr
18674
19913
@cindex character sets (machine character encodings)
18675
19914
@cindex ASCII
18676
19915
@cindex EBCDIC
19916
@cindex Unicode
18677
19917
@cindex mark parity
18678
Some explanation of the numbers used by @code{chr} is worthwhile.
19918
Some explanation of the numbers used by @code{_ord_init()} is worthwhile.
18679
19919
The most prominent character set in use today is ASCII.@footnote{This
18680
19920
is changing; many systems use Unicode, a very large character set
18681
19921
that includes ASCII as a subset. On systems with full Unicode support,
18686
19926
defines characters that use the values from 0 to 127.@footnote{ASCII
18687
19927
has been extended in many countries to use the values from 128 to 255
18688
19928
for country-specific characters. If your system uses these extensions,
18689
you can simplify @code{_ord_init} to loop from 0 to 255.}
19929
you can simplify @code{_ord_init()} to loop from 0 to 255.}
18690
19930
In the now distant past,
18691
19931
at least one minicomputer manufacturer
18692
19932
@c Pr1me, blech
18853
20093
now = systime()
18854
20094
18855
20095
# return date(1)-style output
18856
ret = strftime("%a %b %e %H:%M:%S %Z %Y", now)
20096
ret = strftime(PROCINFO["strftime"], now)
18857
20097
18858
20098
# clear out target array
18859
20099
delete time
18969
20209
test would be @samp{contents == ""}.
18970
20210
18971
20211
@node Data File Management
18972
@section Data File Management
20212
@section @value{DDF} Management
18973
20213
18974
20214
@c STARTOFRANGE dataf
18975
20215
@cindex files, managing
18978
20218
@c STARTOFRANGE flibdataf
18979
20219
@cindex functions, library, managing data files
18980
20220
This @value{SECTION} presents functions that are useful for managing
18981
command-line data files.
20221
command-line @value{DF}s.
18982
20222
18983
20223
@menu
18984
20224
* Filetrans Function:: A function for handling data file transitions.
18989
20229
@end menu
18990
20230
18991
20231
@node Filetrans Function
18992
@subsection Noting Data File Boundaries
20232
@subsection Noting @value{DDF} Boundaries
18993
20233
18994
20234
@cindex files, managing, data file boundaries
18995
20235
@cindex files, initialization and cleanup
18997
20237
the beginning and end of your @command{awk} program, respectively
18998
20238
(@pxref{BEGIN/END}).
18999
20239
We (the @command{gawk} authors) once had a user who mistakenly thought that the
19000
@code{BEGIN} rule is executed at the beginning of each data file and the
19001
@code{END} rule is executed at the end of each data file.
20240
@code{BEGIN} rule is executed at the beginning of each @value{DF} and the
20241
@code{END} rule is executed at the end of each @value{DF}.
19002
20242
19003
20243
When informed
19004
20244
that this was not the case, the user requested that we add new special
19009
20249
the job can be done cleanly in @command{awk} itself, as illustrated
19010
20250
by the following library program.
19011
20251
It arranges to call two user-supplied functions, @code{beginfile()} and
19012
@code{endfile()}, at the beginning and end of each data file.
20252
@code{endfile()}, at the beginning and end of each @value{DF}.
19013
20253
Besides solving the problem in only nine(!) lines of code, it does so
19014
20254
@emph{portably}; this works with any implementation of @command{awk}:
19015
20255
19040
20280
rule it supplies is executed first.
19041
20281
19042
20282
This rule relies on @command{awk}'s @code{FILENAME} variable that
19043
automatically changes for each new data file. The current file name is
20283
automatically changes for each new @value{DF}. The current @value{FN} is
19044
20284
saved in a private variable, @code{_oldfilename}. If @code{FILENAME} does
19045
not equal @code{_oldfilename}, then a new data file is being processed and
20285
not equal @code{_oldfilename}, then a new @value{DF} is being processed and
19046
20286
it is necessary to call @code{endfile()} for the old file. Because
19047
20287
@code{endfile()} should only be called if a file has been processed, the
19048
20288
program first checks to make sure that @code{_oldfilename} is not the null
19049
string. The program then assigns the current file name to
20289
string. The program then assigns the current @value{FN} to
19050
20290
@code{_oldfilename} and calls @code{beginfile()} for the file.
19051
20291
Because, like all @command{awk} variables, @code{_oldfilename} is
19052
20292
initialized to the null string, this rule executes correctly even for the
19053
first data file.
20293
first @value{DF}.
19054
20294
19055
20295
The program also supplies an @code{END} rule to do the final processing for
19056
20296
the last file. Because this @code{END} rule comes before any @code{END} rules
19059
20299
19060
20300
@cindex @code{beginfile()} user-defined function
19061
20301
@cindex @code{endfile()} user-defined function
19062
If the same data file occurs twice in a row on the command line, then
20302
If the same @value{DF} occurs twice in a row on the command line, then
19063
20303
@code{endfile()} and @code{beginfile()} are not executed at the end of the
19064
20304
first pass and at the beginning of the second pass.
19065
20305
The following version solves the problem:
19174
20414
(@pxref{Nextfile Statement}).
19175
20415
19176
20416
@node File Checking
19177
@subsection Checking for Readable Data Files
20417
@subsection Checking for Readable @value{DDF}s
19178
20418
19179
20419
@cindex troubleshooting, readable data files
19180
20420
@cindex readable data files@comma{} checking
19181
20421
@cindex files, skipping
19182
Normally, if you give @command{awk} a data file that isn't readable,
19183
it stops with a fatal error. There are times when you
19184
might want to just ignore such files and keep going. You can
19185
do this by prepending the following program to your @command{awk}
19186
program:
20422
Normally, if you give @command{awk} a @value{DF} that isn't readable,
20423
it stops with a fatal error. There are times when you might want to
20424
just ignore such files and keep going.@footnote{The @code{BEGINFILE}
20425
special pattern (@pxref{BEGINFILE/ENDFILE}) provides an alternative
20426
mechanism for dealing with files that can't be opened. However, the
20427
code here provides a portable solution.} You can do this by prepending
20428
the following program to your @command{awk} program:
19187
20429
19188
20430
@cindex @code{readable.awk} program
19189
20431
@example
19221
20463
See also @ref{ARGC and ARGV}.
19222
20464
19223
20465
@node Empty Files
19224
@subsection Checking For Zero-length Files
20466
@subsection Checking for Zero-length Files
19225
20467
19226
20468
All known @command{awk} implementations silently skip over zero-length files.
19227
20469
This is a by-product of @command{awk}'s implicit
19228
20470
read-a-record-and-match-against-the-rules loop: when @command{awk}
19229
20471
tries to read a record from an empty file, it immediately receives an
19230
20472
end of file indication, closes the file, and proceeds on to the next
19231
command-line data file, @emph{without} executing any user-level
20473
command-line @value{DF}, @emph{without} executing any user-level
19232
20474
@command{awk} program code.
19233
20475
19234
20476
Using @command{gawk}'s @code{ARGIND} variable
19235
20477
(@pxref{Built-in Variables}), it is possible to detect when an empty
19236
data file has been skipped. Similar to the library file presented
20478
@value{DF} has been skipped. Similar to the library file presented
19237
20479
in @ref{Filetrans Function}, the following library file calls a function named
19238
20480
@code{zerofile()} that the user must provide. The arguments passed are
19239
the file name and the position in @code{ARGV} where it was found:
20481
the @value{FN} and the position in @code{ARGV} where it was found:
19240
20482
19241
20483
@cindex @code{zerofile.awk} program
19242
20484
@example
19283
20525
condition of the @code{for} loop uses the @samp{<=} operator,
19284
20526
not @samp{<}.
19285
20527
19286
As an exercise, you might consider whether this same problem can
19287
be solved without relying on @command{gawk}'s @code{ARGIND} variable.
19288
19289
As a second exercise, revise this code to handle the case where
19290
an intervening value in @code{ARGV} is a variable assignment.
19291
19292
@ignore
19293
# zerofile2.awk --- same thing, portably
19294
19295
BEGIN @{
19296
ARGIND = Argind = 0
19297
for (i = 1; i < ARGC; i++)
19298
Fnames[ARGV[i]]++
19299
19300
@}
19301
FNR == 1 @{
19302
while (ARGV[ARGIND] != FILENAME)
19303
ARGIND++
19304
Seen[FILENAME]++
19305
if (Seen[FILENAME] == Fnames[FILENAME])
19306
do
19307
ARGIND++
19308
while (ARGV[ARGIND] != FILENAME)
19309
@}
19310
ARGIND > Argind + 1 @{
19311
for (Argind++; Argind < ARGIND; Argind++)
19312
zerofile(ARGV[Argind], Argind)
19313
@}
19314
ARGIND != Argind @{
19315
Argind = ARGIND
19316
@}
19317
END @{
19318
if (ARGIND < ARGC - 1)
19319
ARGIND = ARGC - 1
19320
if (ARGIND > Argind)
19321
for (Argind++; Argind <= ARGIND; Argind++)
19322
zerofile(ARGV[Argind], Argind)
19323
@}
19324
@end ignore
19325
19326
20528
@node Ignoring Assigns
19327
@subsection Treating Assignments as File Names
20529
@subsection Treating Assignments as @value{FFN}s
19328
20530
19329
20531
@cindex assignments as filenames
19330
20532
@cindex filenames, assignments as
19331
20533
Occasionally, you might not want @command{awk} to process command-line
19332
20534
variable assignments
19333
20535
(@pxref{Assignment Options}).
19334
In particular, if you have a file name that contain an @samp{=} character,
19335
@command{awk} treats the file name as an assignment, and does not process it.
20536
In particular, if you have a @value{FN} that contains an @samp{=} character,
20537
@command{awk} treats the @value{FN} as an assignment, and does not process it.
19336
20538
19337
20539
Some users have suggested an additional command-line option for @command{gawk}
19338
20540
to disable command-line assignments. However, some simple programming with
19376
20578
The function works by looping through the arguments.
19377
20579
It prepends @samp{./} to
19378
20580
any argument that matches the form
19379
of a variable assignment, turning that argument into a file name.
20581
of a variable assignment, turning that argument into a @value{FN}.
19380
20582
19381
20583
The use of @code{No_command_assign} allows you to disable command-line
19382
20584
assignments at invocation time, by giving the variable a true value.
19460
20662
19461
20663
@item optopt
19462
20664
The letter representing the command-line option.
19463
@c While not usually documented, most versions supply this variable.
19464
20665
@end table
19465
20666
19466
20667
The following C fragment shows how @code{getopt()} might process command-line
19511
20712
function was written before @command{gawk} acquired the ability to
19512
20713
split strings into single characters using @code{""} as the separator.
19513
20714
We have left it alone, since using @code{substr()} is more portable.}
19514
@c FIXME: could use split(str, a, "") to do it more easily.
19515
20715
19516
20716
The discussion that follows walks through the code a bit at a time:
19517
20717
19694
20894
# test program
19695
20895
if (_getopt_test) @{
19696
20896
while ((_go_c = getopt(ARGC, ARGV, "ab:cd")) != -1)
19697
printf("c = <%c>, optarg = <%s>\n",
20897
printf("c = <%c>, Optarg = <%s>\n",
19698
20898
_go_c, Optarg)
19699
20899
printf("non-option arguments:\n")
19700
20900
for (; Optind < ARGC; Optind++)
19710
20910
19711
20911
@example
19712
20912
$ @kbd{awk -f getopt.awk -v _getopt_test=1 -- -a -cbARG bax -x}
19713
@print{} c = <a>, optarg = <>
19714
@print{} c = <c>, optarg = <>
19715
@print{} c = <b>, optarg = <ARG>
20913
@print{} c = <a>, Optarg = <>
20914
@print{} c = <c>, Optarg = <>
20915
@print{} c = <b>, Optarg = <ARG>
19716
20916
@print{} non-option arguments:
19717
20917
@print{} ARGV[3] = <bax>
19718
20918
@print{} ARGV[4] = <-x>
19719
20919
19720
20920
$ @kbd{awk -f getopt.awk -v _getopt_test=1 -- -a -x -- xyz abc}
19721
@print{} c = <a>, optarg = <>
20921
@print{} c = <a>, Optarg = <>
19722
20922
@error{} x -- invalid option
19723
@print{} c = <?>, optarg = <>
20923
@print{} c = <?>, Optarg = <>
19724
20924
@print{} non-option arguments:
19725
20925
@print{} ARGV[4] = <xyz>
19726
20926
@print{} ARGV[5] = <abc>
19727
20927
@end example
19728
20928
19729
In both runs,
19730
the first @option{--} terminates the arguments to @command{awk}, so that it does
19731
not try to interpret the @option{-a}, etc., as its own options.
20929
In both runs, the first @option{--} terminates the arguments to
20930
@command{awk}, so that it does not try to interpret the @option{-a},
20931
etc., as its own options.
19732
20932
19733
20933
@quotation NOTE
19734
After @code{getopt()} is through, it is the responsibility of the user level
19735
code to
19736
clear out all the elements of @code{ARGV} from 1 to @code{Optind},
19737
so that @command{awk} does not try to process the command-line options
19738
as file names.
20934
After @code{getopt()} is through, it is the responsibility of the
20935
user level code to clear out all the elements of @code{ARGV} from 1
20936
to @code{Optind}, so that @command{awk} does not try to process the
20937
command-line options as @value{FN}s.
19739
20938
@end quotation
19740
20939
19741
20940
Several of the sample programs presented in
19752
20951
@c STARTOFRANGE libfudata
19753
20952
@cindex libraries of @command{awk} functions, user database, reading
19754
20953
@c STARTOFRANGE flibudata
19755
@cindex functions, library, user database, reading
20954
@cindex functions, library, user database@comma{} reading
19756
20955
@c STARTOFRANGE udatar
19757
20956
@cindex user database@comma{} reading
19758
20957
@c STARTOFRANGE dataur
19797
20996
happens, the C program should call @code{endpwent()} to close the database.
19798
20997
Following is @command{pwcat}, a C program that ``cats'' the password database:
19799
20998
19800
@c Use old style function header for portability to old systems (SunOS, HP/UX).
19801
19802
20999
@example
19803
21000
@c file eg/lib/pwcat.c
19804
21001
/*
19805
21002
* pwcat.c
19806
21003
*
19807
* Generate a printable version of the password database
21004
* Generate a printable version of the password database.
19808
21005
*/
19809
21006
@c endfile
19810
21007
@ignore
20001
21198
or her
20002
21199
own way of splitting records and fields.
20003
21200
20004
@cindex @code{PROCINFO} array
21201
@cindex @code{PROCINFO} array, testing the field splitting
20005
21202
The @code{using_fw} variable checks @code{PROCINFO["FS"]}, which
20006
21203
is @code{"FIELDWIDTHS"} if field splitting is being done with
20007
21204
@code{FIELDWIDTHS}. This makes it possible to restore the correct
20010
21207
or on some other @command{awk} implementation.
20011
21208
20012
21209
The code that checks for using @code{FPAT}, using @code{using_fpat}
20013
and @code{PROCINFO["FS"]} is similar.
21210
and @code{PROCINFO["FS"]}, is similar.
20014
21211
20015
21212
The main part of the function uses a loop to read database lines, split
20016
21213
the line into fields, and then store the line into each array as necessary.
20040
21237
@end example
20041
21238
20042
21239
@cindex @code{getpwuid()} function (C library)
20043
Similarly,
20044
the @code{getpwuid} function takes a user ID number argument. If that
20045
user number is in the database, it returns the appropriate line. Otherwise, it
20046
returns the null string:
21240
Similarly, the @code{getpwuid()} function takes a user ID number
21241
argument. If that user number is in the database, it returns the
21242
appropriate line. Otherwise, it returns the null string:
20047
21243
20048
21244
@cindex @code{getpwuid()} user-defined function
20049
21245
@example
20120
21316
@c STARTOFRANGE libfgdata
20121
21317
@cindex libraries of @command{awk} functions, group database, reading
20122
21318
@c STARTOFRANGE flibgdata
20123
@cindex functions, library, group database, reading
21319
@cindex functions, library, group database@comma{} reading
20124
21320
@c STARTOFRANGE gdatar
20125
21321
@cindex group database, reading
20126
21322
@c STARTOFRANGE datagr
20127
21323
@cindex database, group, reading
20128
@cindex @code{PROCINFO} array
21324
@cindex @code{PROCINFO} array, and group membership
20129
21325
@cindex @code{getgrent()} function (C library)
20130
21326
@cindex @code{getgrent()} user-defined function
20131
21327
@cindex groups@comma{} information about
20151
21347
/*
20152
21348
* grcat.c
20153
21349
*
20154
* Generate a printable version of the group database
21350
* Generate a printable version of the group database.
20155
21351
*/
20156
21352
@c endfile
20157
21353
@ignore
20238
21434
20239
21435
@item Group ID Number
20240
21436
The group's numeric group ID number;
20241
this number must be unique within the file.
21437
the association of name to number must be unique within the file.
20242
21438
(On some systems it's a C @code{long}, and not an @code{int}. Thus
20243
21439
we cast it to @code{long} for all cases.)
20244
21440
20368
21564
of members. A pair of such entries might look like the following:
20369
21565
20370
21566
@example
20371
tvpeople:*:101:johnny,jay,arsenio
21567
tvpeople:*:101:johny,jay,arsenio
20372
21568
tvpeople:*:101:david,conan,tom,joan
20373
21569
@end example
20374
21570
20375
21571
For this reason, @code{_gr_init()} looks to see if a group name or
20376
21572
group ID number is already seen. If it is, then the user names are
20377
simply concatenated onto the previous list of users. (There is actually a
21573
simply concatenated onto the previous list of users.@footnote{There is actually a
20378
21574
subtle problem with the code just presented. Suppose that
20379
21575
the first time there were no names. This code adds the names with
20380
a leading comma. It also doesn't check that there is a @code{$4}.)
21576
a leading comma. It also doesn't check that there is a @code{$4}.}
20381
21577
20382
21578
Finally, @code{_gr_init()} closes the pipeline to @command{grcat}, restores
20383
21579
@code{FS} (and @code{FIELDWIDTHS} or @code{FPAT} if necessary), @code{RS}, and @code{$0},
20537
21733
@print{} a[3] = 3
20538
21734
@end example
20539
21735
20540
Walking an array and processing each element is a general-purpose
20541
operation. You might want to consider generalizing the @code{walk_array()}
20542
function by adding an additional parameter named @code{process}.
20543
20544
Then, inside the loop, instead of simply printing the array element's
20545
index and value, use the indirect function call syntax
20546
(@pxref{Indirect Calls}) on @code{process}, passing it the index
20547
and the value.
20548
20549
When calling @code{walk_array()}, you would pass the name of a user-defined
20550
function that expects to receive and index and a value, and then processes
20551
the element.
20552
20553
20554
21736
@c ENDOFRANGE libfgdata
20555
21737
@c ENDOFRANGE flibgdata
20556
21738
@c ENDOFRANGE gdatar
20557
21739
@c ENDOFRANGE libf
21740
21741
@node Library Functions Summary
21742
@section Summary
21743
21744
@itemize @value{BULLET}
21745
@item
21746
Reading programs is an excellent way to learn Good Programming.
21747
The functions provided in this @value{CHAPTER} and the next are intended
21748
to serve that purpose.
21749
21750
@item
21751
When writing general-purpose library functions, put some thought into how
21752
to name any global variables so that they won't conflict with variables
21753
from a user's program.
21754
21755
@item
21756
The functions presented here fit into the following categories:
21757
21758
@c nested list
21759
@table @asis
21760
@item General problems
21761
Number to string conversion, assertions, rounding, random number
21762
generation, converting characters to numbers, joining strings, getting
21763
easily usable time-of-day information, and reading a whole file in
21764
one shot.
21765
21766
@item Managing @value{DF}s
21767
Noting @value{DF} boundaries, rereading the current file, checking for
21768
readable files, checking for zero-length files, and treating assignments
21769
as @value{FN}s.
21770
21771
@item Processing command-line options
21772
An @command{awk} version of the standard C @code{getopt()} function.
21773
21774
@item Reading the user and group databases
21775
Two sets of routines that parallel the C library versions.
21776
21777
@item Traversing arrays of arrays
21778
A simple function to traverse an array of arrays to any depth.
21779
@end table
21780
@c end nested list
21781
21782
@end itemize
21783
21784
@node Library exercises
21785
@section Exercises
21786
21787
@enumerate
21788
@item
21789
In @ref{Empty Files}, we presented the @file{zerofile.awk} program,
21790
which made use of @command{gawk}'s @code{ARGIND} variable. Can this
21791
problem be solved without relying on @code{ARGIND}? If so, how?
21792
21793
@ignore
21794
# zerofile2.awk --- same thing, portably
21795
21796
BEGIN @{
21797
ARGIND = Argind = 0
21798
for (i = 1; i < ARGC; i++)
21799
Fnames[ARGV[i]]++
21800
21801
@}
21802
FNR == 1 @{
21803
while (ARGV[ARGIND] != FILENAME)
21804
ARGIND++
21805
Seen[FILENAME]++
21806
if (Seen[FILENAME] == Fnames[FILENAME])
21807
do
21808
ARGIND++
21809
while (ARGV[ARGIND] != FILENAME)
21810
@}
21811
ARGIND > Argind + 1 @{
21812
for (Argind++; Argind < ARGIND; Argind++)
21813
zerofile(ARGV[Argind], Argind)
21814
@}
21815
ARGIND != Argind @{
21816
Argind = ARGIND
21817
@}
21818
END @{
21819
if (ARGIND < ARGC - 1)
21820
ARGIND = ARGC - 1
21821
if (ARGIND > Argind)
21822
for (Argind++; Argind <= ARGIND; Argind++)
21823
zerofile(ARGV[Argind], Argind)
21824
@}
21825
@end ignore
21826
21827
@item
21828
As a related challenge, revise that code to handle the case where
21829
an intervening value in @code{ARGV} is a variable assignment.
21830
21831
@item
21832
@ref{Walking Arrays}, presented a function that walked a multidimensional
21833
array to print it out. However, walking an array and processing
21834
each element is a general-purpose operation. Generalize the
21835
@code{walk_array()} function by adding an additional parameter named
21836
@code{process}.
21837
21838
Then, inside the loop, instead of printing the array element's index and
21839
value, use the indirect function call syntax (@pxref{Indirect Calls})
21840
on @code{process}, passing it the index and the value.
21841
21842
When calling @code{walk_array()}, you would pass the name of a
21843
user-defined function that expects to receive an index and a value,
21844
and then processes the element.
21845
21846
Test your new version by printing the array; you should end up with
21847
output identical to that of the original version.
21848
21849
@end enumerate
21850
20558
21851
@c ENDOFRANGE flib
20559
21852
@c ENDOFRANGE fudlib
20560
21853
@c ENDOFRANGE datagr
20564
21857
@c STARTOFRANGE awkpex
20565
21858
@cindex @command{awk} programs, examples of
20566
21859
21860
@c FULLXREF ON
20567
21861
@ref{Library Functions},
20568
21862
presents the idea that reading programs in a language contributes to
20569
21863
learning that language. This @value{CHAPTER} continues that theme,
20570
21864
presenting a potpourri of @command{awk} programs for your reading
20571
21865
enjoyment.
21866
@c FULLXREF OFF
20572
21867
@ifnotinfo
20573
21868
There are three sections.
20574
21869
The first describes how to run the programs presented
20595
21890
* Running Examples:: How to run these examples.
20596
21891
* Clones:: Clones of common utilities.
20597
21892
* Miscellaneous Programs:: Some interesting @command{awk} programs.
21893
* Programs Summary:: Summary of programs.
21894
* Programs Exercises:: Exercises.
20598
21895
@end menu
20599
21896
20600
21897
@node Running Examples
20609
21906
@noindent
20610
21907
Here, @var{program} is the name of the @command{awk} program (such as
20611
21908
@file{cut.awk}), @var{options} are any command-line options for the
20612
program that start with a @samp{-}, and @var{files} are the actual data files.
21909
program that start with a @samp{-}, and @var{files} are the actual @value{DF}s.
20613
21910
20614
21911
If your system supports the @samp{#!} executable interpreter mechanism
20615
21912
(@pxref{Executable Scripts}),
20747
22044
20748
22045
@noindent
20749
22046
The variables @code{e1} and @code{e2} are used so that the function
20750
fits nicely on the
20751
@ifnotinfo
20752
page.
20753
@end ifnotinfo
20754
@ifnottex
20755
screen.
20756
@end ifnottex
22047
fits nicely on the @value{PAGE}.
20757
22048
20758
22049
@cindex @code{BEGIN} pattern, running @command{awk} programs and
20759
22050
@cindex @code{FS} variable, running @command{awk} programs and
20783
22074
OFS = ""
20784
22075
@} else if (c == "d") @{
20785
22076
if (length(Optarg) > 1) @{
20786
printf("Using first character of %s" \
22077
printf("cut: using first character of %s" \
20787
22078
" for delimiter\n", Optarg) > "/dev/stderr"
20788
22079
Optarg = substr(Optarg, 1, 1)
20789
22080
@}
20792
22083
if (FS == " ") # defeat awk semantics
20793
22084
FS = "[ ]"
20794
22085
@} else if (c == "s")
20795
suppress++
22086
suppress = 1
20796
22087
else
20797
22088
usage()
20798
22089
@}
20814
22105
we have to
20815
22106
clear out all the elements of @code{ARGV} from 1 to @code{Optind},
20816
22107
so that @command{awk} does not try to process the command-line options
20817
as file names.
22108
as @value{FN}s.
20818
22109
20819
22110
After dealing with the command-line options, the program verifies that the
20820
22111
options make sense. Only one or the other of @option{-c} and @option{-f}
20864
22155
m = split(f[i], g, "-")
20865
22156
@group
20866
22157
if (m != 2 || g[1] >= g[2]) @{
20867
printf("bad field list: %s\n",
22158
printf("cut: bad field list: %s\n",
20868
22159
f[i]) > "/dev/stderr"
20869
22160
exit 1
20870
22161
@}
20901
22192
20902
22193
@example
20903
22194
@c file eg/prog/cut.awk
20904
function set_charlist( field, i, j, f, g, t,
22195
function set_charlist( field, i, j, f, g, n, m, t,
20905
22196
filler, last, len)
20906
22197
@{
20907
22198
field = 1 # count total fields
20911
22202
if (index(f[i], "-") != 0) @{ # range
20912
22203
m = split(f[i], g, "-")
20913
22204
if (m != 2 || g[1] >= g[2]) @{
20914
printf("bad character list: %s\n",
22205
printf("cut: bad character list: %s\n",
20915
22206
f[i]) > "/dev/stderr"
20916
22207
exit 1
20917
22208
@}
20987
22278
@c ENDOFRANGE ficut
20988
22279
@c ENDOFRANGE colcut
20989
22280
20990
@c Exercise: Rewrite using split with "".
20991
22281
20992
22282
@node Egrep Program
20993
22283
@subsection Searching for Regular Expressions in Files
20998
22288
@cindex searching, files for regular expressions
20999
22289
@c STARTOFRANGE fsregexp
21000
22290
@cindex files, searching for regular expressions
22291
@c STARTOFRANGE egrep
21001
22292
@cindex @command{egrep} utility
21002
22293
The @command{egrep} utility searches files for patterns. It uses regular
21003
22294
expressions that are almost identical to those available in @command{awk}
21004
22295
(@pxref{Regexp}).
21005
22296
You invoke it as follows:
21006
22297
21007
@example
21008
egrep @r{[} @var{options} @r{]} '@var{pattern}' @var{files} @dots{}
21009
@end example
22298
@display
22299
@command{egrep} [@var{options}] @code{'@var{pattern}'} @var{files} @dots{}
22300
@end display
21010
22301
21011
22302
The @var{pattern} is a regular expression. In typical usage, the regular
21012
22303
expression is quoted to prevent the shell from expanding any of the
21013
special characters as file name wildcards. Normally, @command{egrep}
21014
prints the lines that matched. If multiple file names are provided on
22304
special characters as @value{FN} wildcards. Normally, @command{egrep}
22305
prints the lines that matched. If multiple @value{FN}s are provided on
21015
22306
the command line, each output line is preceded by the name of the file
21016
22307
and a colon.
21017
22308
21102
22393
command line is used. The @command{awk} command-line arguments up to @code{ARGV[Optind]}
21103
22394
are cleared, so that @command{awk} won't try to process them as files. If no
21104
22395
files are specified, the standard input is used, and if multiple files are
21105
specified, we make sure to note this so that the file names can precede the
22396
specified, we make sure to note this so that the @value{FN}s can precede the
21106
22397
matched lines in the output:
21107
22398
21108
22399
@example
21136
22427
The rule is
21137
22428
commented out since it is not necessary with @command{gawk}:
21138
22429
21139
@c Exercise: Fix this, w/array and new line as key to original line
21140
21141
22430
@example
21142
22431
@c file eg/prog/egrep.awk
21143
22432
#@{
21188
22477
@c endfile
21189
22478
@end example
21190
22479
22480
The @code{BEGINFILE} and @code{ENDFILE} special patterns
22481
(@pxref{BEGINFILE/ENDFILE}) could be used, but then the program would be
22482
@command{gawk}-specific. Additionally, this example was written before
22483
@command{gawk} acquired @code{BEGINFILE} and @code{ENDFILE}.
22484
21191
22485
The following rule does most of the work of matching lines. The variable
21192
22486
@code{matches} is true if the line matched the pattern. If the user
21193
22487
wants lines that did not match, the sense of @code{matches} is inverted
21200
22494
are not counting lines. First, if the user only wants exit status
21201
22495
(@code{no_print} is true), then it is enough to know that @emph{one}
21202
22496
line in this file matched, and we can skip on to the next file with
21203
@code{nextfile}. Similarly, if we are only printing file names, we can
21204
print the file name, and then skip to the next file with @code{nextfile}.
21205
Finally, each line is printed, with a leading file name and colon
22497
@code{nextfile}. Similarly, if we are only printing @value{FN}s, we can
22498
print the @value{FN}, and then skip to the next file with @code{nextfile}.
22499
Finally, each line is printed, with a leading @value{FN} and colon
21206
22500
if necessary:
21207
22501
21208
22502
@cindex @code{!} (exclamation point), @code{!} operator
21244
22538
@c file eg/prog/egrep.awk
21245
22539
END \
21246
22540
@{
21247
if (total == 0)
21248
exit 1
21249
exit 0
22541
exit (total == 0)
21250
22542
@}
21251
22543
@c endfile
21252
22544
@end example
21283
22575
@c ENDOFRANGE regexps
21284
22576
@c ENDOFRANGE sfregexp
21285
22577
@c ENDOFRANGE fsregexp
22578
@c ENDOFRANGE egrep
21286
22579
21287
22580
@node Id Program
21288
22581
@subsection Printing out User Information
21289
22582
21290
22583
@cindex printing, user information
21291
22584
@cindex users, information about, printing
22585
@c STARTOFRANGE id
21292
22586
@cindex @command{id} utility
21293
22587
The @command{id} utility lists a user's real and effective user ID numbers,
21294
22588
real and effective group ID numbers, and the user's group set, if any.
21298
22592
21299
22593
@example
21300
22594
$ @kbd{id}
21301
@print{} uid=500(arnold) gid=500(arnold) groups=6(disk),7(lp),19(floppy)
22595
@print{} uid=1000(arnold) gid=1000(arnold) groups=1000(arnold),4(adm),7(lp),27(sudo)
21302
22596
@end example
21303
22597
21304
@cindex @code{PROCINFO} array
22598
@cindex @code{PROCINFO} array, and user and group ID numbers
21305
22599
This information is part of what is provided by @command{gawk}'s
21306
22600
@code{PROCINFO} array (@pxref{Built-in Variables}).
21307
22601
However, the @command{id} utility provides a more palatable output than just
21334
22628
# Arnold Robbins, arnold@@skeeve.com, Public Domain
21335
22629
# May 1993
21336
22630
# Revised February 1996
22631
# Revised May 2014
21337
22632
21338
22633
@c endfile
21339
22634
@end ignore
21353
22648
21354
22649
printf("uid=%d", uid)
21355
22650
pw = getpwuid(uid)
21356
if (pw != "") @{
21357
split(pw, a, ":")
21358
printf("(%s)", a[1])
21359
@}
22651
if (pw != "")
22652
pr_first_field(pw)
21360
22653
21361
22654
if (euid != uid) @{
21362
22655
printf(" euid=%d", euid)
21363
22656
pw = getpwuid(euid)
21364
if (pw != "") @{
21365
split(pw, a, ":")
21366
printf("(%s)", a[1])
21367
@}
22657
if (pw != "")
22658
pr_first_field(pw)
21368
22659
@}
21369
22660
21370
22661
printf(" gid=%d", gid)
21371
22662
pw = getgrgid(gid)
21372
if (pw != "") @{
21373
split(pw, a, ":")
21374
printf("(%s)", a[1])
21375
@}
22663
if (pw != "")
22664
pr_first_field(pw)
21376
22665
21377
22666
if (egid != gid) @{
21378
22667
printf(" egid=%d", egid)
21379
22668
pw = getgrgid(egid)
21380
if (pw != "") @{
21381
split(pw, a, ":")
21382
printf("(%s)", a[1])
21383
@}
22669
if (pw != "")
22670
pr_first_field(pw)
21384
22671
@}
21385
22672
21386
22673
for (i = 1; ("group" i) in PROCINFO; i++) @{
21389
22676
group = PROCINFO["group" i]
21390
22677
printf("%d", group)
21391
22678
pw = getgrgid(group)
21392
if (pw != "") @{
21393
split(pw, a, ":")
21394
printf("(%s)", a[1])
21395
@}
22679
if (pw != "")
22680
pr_first_field(pw)
21396
22681
if (("group" (i+1)) in PROCINFO)
21397
22682
printf(",")
21398
22683
@}
21399
22684
21400
22685
print ""
21401
22686
@}
22687
22688
function pr_first_field(str, a)
22689
@{
22690
split(str, a, ":")
22691
printf("(%s)", a[1])
22692
@}
21402
22693
@c endfile
21403
22694
@end example
21404
22695
21405
@cindex @code{in} operator
21406
22696
The test in the @code{for} loop is worth noting.
21407
22697
Any supplementary groups in the @code{PROCINFO} array have the
21408
22698
indices @code{"group1"} through @code{"group@var{N}"} for some
21412
22702
21413
22703
This loop works by starting at one, concatenating the value with
21414
22704
@code{"group"}, and then using @code{in} to see if that value is
21415
in the array. Eventually, @code{i} is incremented past
22705
in the array (@pxref{Reference to Elements}). Eventually, @code{i} is incremented past
21416
22706
the last group in the array and the loop exits.
21417
22707
21418
22708
The loop is also correct if there are @emph{no} supplementary
21419
22709
groups; then the condition is false the first time it's
21420
22710
tested, and the loop body never executes.
21421
22711
21422
@c exercise!!!
21423
@ignore
21424
The POSIX version of @command{id} takes arguments that control which
21425
information is printed. Modify this version to accept the same
21426
arguments and perform in the same way.
21427
@end ignore
22712
The @code{pr_first_field()} function simply isolates out some
22713
code that is used repeatedly, making the whole program
22714
slightly shorter and cleaner.
22715
22716
@c ENDOFRANGE id
21428
22717
21429
22718
@node Split Program
21430
22719
@subsection Splitting a Large File into Pieces
21433
22722
21434
22723
@c STARTOFRANGE filspl
21435
22724
@cindex files, splitting
22725
@c STARTOFRANGE split
21436
22726
@cindex @code{split} utility
21437
22727
The @command{split} program splits large text files into smaller pieces.
21438
22728
Usage is as follows:@footnote{This is the traditional usage. The
21439
22729
POSIX usage is different, but not relevant for what the program
21440
22730
aims to demonstrate.}
21441
22731
21442
@example
21443
split @r{[}-@var{count}@r{]} file @r{[} @var{prefix} @r{]}
21444
@end example
22732
@display
22733
@command{split} [@code{-@var{count}}] [@var{file}] [@var{prefix}]
22734
@end display
21445
22735
21446
22736
By default,
21447
22737
the output files are named @file{xaa}, @file{xab}, and so on. Each file has
21450
22740
preceded with a minus; e.g., @samp{-500} for files with 500 lines in them
21451
22741
instead of 1000. To change the name of the output files to something like
21452
22742
@file{myfileaa}, @file{myfileab}, and so on, supply an additional
21453
argument that specifies the file name prefix.
22743
argument that specifies the @value{FN} prefix.
21454
22744
21455
22745
Here is a version of @command{split} in @command{awk}. It uses the
21456
22746
@code{ord()} and @code{chr()} functions presented in
21460
22750
not too many arguments. It then looks at each argument in turn. The
21461
22751
first argument could be a minus sign followed by a number. If it is, this happens
21462
22752
to look like a negative number, so it is made positive, and that is the
21463
count of lines. The data file name is skipped over and the final argument
21464
is used as the prefix for the output file names:
22753
count of lines. The @value{DF} name is skipped over and the final argument
22754
is used as the prefix for the output @value{FN}s:
21465
22755
21466
22756
@cindex @code{split.awk} program
21467
22757
@example
21475
22765
#
21476
22766
# Arnold Robbins, arnold@@skeeve.com, Public Domain
21477
22767
# May 1993
22768
# Revised slightly, May 2014
21478
22769
21479
22770
@c endfile
21480
22771
@end ignore
21481
22772
@c file eg/prog/split.awk
21482
# usage: split [-num] [file] [outname]
22773
# usage: split [-count] [file] [outname]
21483
22774
21484
22775
BEGIN @{
21485
22776
outfile = "x" # default
21488
22779
usage()
21489
22780
21490
22781
i = 1
21491
if (ARGV[i] ~ /^-[[:digit:]]+$/) @{
22782
if (i in ARGV && ARGV[i] ~ /^-[[:digit:]]+$/) @{
21492
22783
count = -ARGV[i]
21493
22784
ARGV[i] = ""
21494
22785
i++
21510
22801
The next rule does most of the work. @code{tcount} (temporary count) tracks
21511
22802
how many lines have been printed to the output file so far. If it is greater
21512
22803
than @code{count}, it is time to close the current file and start a new one.
21513
@code{s1} and @code{s2} track the current suffixes for the file name. If
22804
@code{s1} and @code{s2} track the current suffixes for the @value{FN}. If
21514
22805
they are both @samp{z}, the file is just too big. Otherwise, @code{s1}
21515
22806
moves to the next letter in the alphabet and @code{s2} starts over again at
21516
22807
@samp{a}:
21542
22833
@c endfile
21543
22834
@end example
21544
22835
21545
@c Exercise: do this with just awk builtin functions, index("abc..."), substr, etc.
21546
21547
22836
@noindent
21548
22837
The @code{usage()} function simply prints an error message and exits:
21549
22838
21560
22849
21561
22850
@noindent
21562
22851
The variable @code{e} is used so that the function
21563
fits nicely on the
21564
@ifinfo
21565
screen.
21566
@end ifinfo
21567
@ifnotinfo
21568
page.
21569
@end ifnotinfo
22852
fits nicely on the @value{PAGE}.
21570
22853
21571
22854
This program is a bit sloppy; it relies on @command{awk} to automatically close the last file
21572
22855
instead of doing it in an @code{END} rule.
21573
22856
It also assumes that letters are contiguous in the character set,
21574
22857
which isn't true for EBCDIC systems.
21575
22858
21576
@c Exercise: Fix these problems.
21577
@c BFD...
21578
22859
@c ENDOFRANGE filspl
22860
@c ENDOFRANGE split
21579
22861
21580
22862
@node Tee Program
21581
22863
@subsection Duplicating Output into Multiple Files
21582
22864
21583
22865
@cindex files, multiple@comma{} duplicating output into
21584
22866
@cindex output, duplicating into files
22867
@c STARTOFRANGE tee
21585
22868
@cindex @code{tee} utility
21586
22869
The @code{tee} program is known as a ``pipe fitting.'' @code{tee} copies
21587
22870
its standard input to its standard output and also duplicates it to the
21588
22871
files named on the command line. Its usage is as follows:
21589
22872
21590
@example
21591
tee @r{[}-a@r{]} file @dots{}
21592
@end example
22873
@display
22874
@command{tee} [@option{-a}] @var{file} @dots{}
22875
@end display
21593
22876
21594
22877
The @option{-a} option tells @code{tee} to append to the named files, instead of
21595
22878
truncating them and starting over.
21598
22881
into an array named @code{copy}.
21599
22882
@code{ARGV[0]} is not copied, since it is not needed.
21600
22883
@code{tee} cannot use @code{ARGV} directly, since @command{awk} attempts to
21601
process each file name in @code{ARGV} as input data.
22884
process each @value{FN} in @code{ARGV} as input data.
21602
22885
21603
22886
@cindex flag variables
21604
22887
If the first argument is @option{-a}, then the flag variable
21605
22888
@code{append} is set to true, and both @code{ARGV[1]} and
21606
22889
@code{copy[1]} are deleted. If @code{ARGC} is less than two, then no
21607
file names were supplied and @code{tee} prints a usage message and exits.
22890
@value{FN}s were supplied and @code{tee} prints a usage message and exits.
21608
22891
Finally, @command{awk} is forced to read the standard input by setting
21609
22892
@code{ARGV[1]} to @code{"-"} and @code{ARGC} to two:
21610
22893
21696
22979
@}
21697
22980
@c endfile
21698
22981
@end example
22982
@c ENDOFRANGE tee
21699
22983
21700
22984
@node Uniq Program
21701
22985
@subsection Printing Nonduplicated Lines of Text
21706
22990
@cindex printing, unduplicated lines of text
21707
22991
@c STARTOFRANGE tpul
21708
22992
@cindex text@comma{} printing, unduplicated lines of
22993
@c STARTOFRANGE uniq
21709
22994
@cindex @command{uniq} utility
21710
22995
The @command{uniq} utility reads sorted lines of data on its standard
21711
22996
input, and by default removes duplicate lines. In other words, it only
21712
22997
prints unique lines---hence the name. @command{uniq} has a number of
21713
22998
options. The usage is as follows:
21714
22999
21715
@example
21716
uniq @r{[}-udc @r{[}-@var{n}@r{]]} @r{[}+@var{n}@r{]} @r{[} @var{input file} @r{[} @var{output file} @r{]]}
21717
@end example
23000
@display
23001
@command{uniq} [@option{-udc} [@code{-@var{n}}]] [@code{+@var{n}}] [@var{inputfile} [@var{outputfile}]]
23002
@end display
21718
23003
21719
23004
The options for @command{uniq} are:
21720
23005
21738
23023
Skip @var{n} characters before comparing lines. Any fields specified with
21739
23024
@samp{-@var{n}} are skipped first.
21740
23025
21741
@item @var{input file}
23026
@item @var{inputfile}
21742
23027
Data is read from the input file named on the command line, instead of from
21743
23028
the standard input.
21744
23029
21745
@item @var{output file}
23030
@item @var{outputfile}
21746
23031
The generated output is sent to the named output file, instead of to the
21747
23032
standard output.
21748
23033
@end table
21957
23242
@end example
21958
23243
@c ENDOFRANGE prunt
21959
23244
@c ENDOFRANGE tpul
23245
@c ENDOFRANGE uniq
21960
23246
21961
23247
@node Wc Program
21962
23248
@subsection Counting Things
21973
23259
@cindex characters, counting
21974
23260
@c STARTOFRANGE lico
21975
23261
@cindex lines, counting
23262
@c STARTOFRANGE wc
21976
23263
@cindex @command{wc} utility
21977
23264
The @command{wc} (word count) utility counts lines, words, and characters in
21978
23265
one or more input files. Its usage is as follows:
21979
23266
21980
@example
21981
wc @r{[}-lwc@r{]} @r{[} @var{files} @dots{} @r{]}
21982
@end example
23267
@display
23268
@command{wc} [@option{-lwc}] [@var{files} @dots{}]
23269
@end display
21983
23270
21984
23271
If no files are specified on the command line, @command{wc} reads its standard
21985
23272
input. If there are multiple files, it also prints total counts for all
22066
23353
@end example
22067
23354
22068
23355
The @code{beginfile()} function is simple; it just resets the counts of lines,
22069
words, and characters to zero, and saves the current file name in
23356
words, and characters to zero, and saves the current @value{FN} in
22070
23357
@code{fname}:
22071
23358
22072
23359
@example
22079
23366
@c endfile
22080
23367
@end example
22081
23368
22082
The @code{endfile()} function adds the current file's numbers to the running
22083
totals of lines, words, and characters.@footnote{@command{wc} can't just use the value of
22084
@code{FNR} in @code{endfile()}. If you examine
22085
the code in
22086
@ref{Filetrans Function},
22087
you will see that
22088
@code{FNR} has already been reset by the time
22089
@code{endfile()} is called.} It then prints out those numbers
22090
for the file that was just read. It relies on @code{beginfile()} to reset the
22091
numbers for the following data file:
22092
@c FIXME: ONE DAY: make the above footnote an exercise,
22093
@c instead of giving away the answer.
23369
The @code{endfile()} function adds the current file's numbers to the
23370
running totals of lines, words, and characters. It then prints out those
23371
numbers for the file that was just read. It relies on @code{beginfile()}
23372
to reset the numbers for the following @value{DF}:
22094
23373
22095
23374
@example
22096
23375
@c file eg/prog/wc.awk
22155
23434
@c ENDOFRANGE lico
22156
23435
@c ENDOFRANGE woco
22157
23436
@c ENDOFRANGE chco
23437
@c ENDOFRANGE wc
22158
23438
@c ENDOFRANGE posimawk
22159
23439
22160
23440
@node Miscellaneous Programs
22259
23539
@i{Nothing cures insomnia like a ringing alarm clock.}
22260
23540
@author Arnold Robbins
22261
23541
@end quotation
23542
@cindex Quanstrom, Erik
23543
@ignore
23544
Date: Sat, 15 Feb 2014 16:47:09 -0500
23545
Subject: Re: 9atom install question
23546
Message-ID: <l2jcvx6j6mey60xnrkb0hhob.1392500829294@email.android.com>
23547
From: Erik Quanstrom <quanstro@quanstro.net>
23548
To: Aharon Robbins <arnold@skeeve.com>
23549
23550
yes.
23551
23552
- erik
23553
23554
Aharon Robbins <arnold@skeeve.com> wrote:
23555
23556
>> sleep is for web developers.
23557
>
23558
>Can I quote you, in the gawk manual?
23559
>
23560
>Thanks,
23561
>
23562
>Arnold
23563
@end ignore
23564
@quotation
23565
@i{Sleep is for web developers.}
23566
@author Erik Quanstrom
23567
@end quotation
22262
23568
22263
23569
@c STARTOFRANGE tialarm
22264
23570
@cindex time, alarm clock example program
22380
23686
# how long to sleep for
22381
23687
naptime = target - current
22382
23688
if (naptime <= 0) @{
22383
print "time is in the past!" > "/dev/stderr"
23689
print "alarm: time is in the past!" > "/dev/stderr"
22384
23690
exit 1
22385
23691
@}
22386
23692
@c endfile
22423
23729
22424
23730
@c STARTOFRANGE chtra
22425
23731
@cindex characters, transliterating
23732
@c STARTOFRANGE tr
22426
23733
@cindex @command{tr} utility
22427
23734
The system @command{tr} utility transliterates characters. For example, it is
22428
23735
often used to map uppercase letters into lowercase for further processing:
22432
23739
@end example
22433
23740
22434
23741
@command{tr} requires two lists of characters.@footnote{On some older
22435
systems,
22436
including Solaris,
22437
@command{tr} may require that the lists be written as
22438
range expressions enclosed in square brackets (@samp{[a-z]}) and quoted,
22439
to prevent the shell from attempting a file name expansion. This is
22440
not a feature.} When processing the input, the first character in the
22441
first list is replaced with the first character in the second list,
22442
the second character in the first list is replaced with the second
22443
character in the second list, and so on. If there are more characters
22444
in the ``from'' list than in the ``to'' list, the last character of the
22445
``to'' list is used for the remaining characters in the ``from'' list.
23742
systems, including Solaris, the system version of @command{tr} may require
23743
that the lists be written as range expressions enclosed in square brackets
23744
(@samp{[a-z]}) and quoted, to prevent the shell from attempting a file
23745
name expansion. This is not a feature.} When processing the input, the
23746
first character in the first list is replaced with the first character
23747
in the second list, the second character in the first list is replaced
23748
with the second character in the second list, and so on. If there are
23749
more characters in the ``from'' list than in the ``to'' list, the last
23750
character of the ``to'' list is used for the remaining characters in the
23751
``from'' list.
22446
23752
22447
Some time ago,
23753
Once upon a time,
22448
23754
@c early or mid-1989!
22449
23755
a user proposed that a transliteration function should
22450
23756
be added to @command{gawk}.
22462
23768
(@pxref{String Functions}).@footnote{This
22463
23769
program was written before @command{gawk} acquired the ability to
22464
23770
split each character in a string into separate array elements.}
22465
@c Exercise: How might you use this new feature to simplify the program?
22466
23771
There are two functions. The first, @code{stranslate()}, takes three
22467
23772
arguments:
22468
23773
22558
23863
While it is possible to do character transliteration in a user-level
22559
23864
function, it is not necessarily efficient, and we (the @command{gawk}
22560
23865
authors) started to consider adding a built-in function. However,
22561
shortly after writing this program, we learned that the System V Release 4
22562
@command{awk} had added the @code{toupper()} and @code{tolower()} functions
22563
(@pxref{String Functions}).
22564
These functions handle the vast majority of the
22565
cases where character transliteration is necessary, and so we chose to
22566
simply add those functions to @command{gawk} as well and then leave well
22567
enough alone.
23866
shortly after writing this program, we learned that Brian Kernighan
23867
had added the @code{toupper()} and @code{tolower()} functions to his
23868
@command{awk} (@pxref{String Functions}). These functions handle the
23869
vast majority of the cases where character transliteration is necessary,
23870
and so we chose to simply add those functions to @command{gawk} as well
23871
and then leave well enough alone.
22568
23872
22569
23873
An obvious improvement to this program would be to set up the
22570
23874
@code{t_ar} array only once, in a @code{BEGIN} rule. However, this
22571
23875
assumes that the ``from'' and ``to'' lists
22572
23876
will never change throughout the lifetime of the program.
22573
23877
@c ENDOFRANGE chtra
23878
@c ENDOFRANGE tr
22574
23879
22575
23880
@node Labels Program
22576
23881
@subsection Printing Mailing Labels
22596
23901
@command{awk} splits records at blank lines
22597
23902
(@pxref{Records}).
22598
23903
It sets @code{MAXLINES} to 100, since 100 is the maximum number
22599
of lines on the page (20 * 5 = 100).
23904
of lines on the page
23905
@iftex
23906
(@math{20 @cdot 5 = 100}).
23907
@end iftex
23908
@ifnottex
23909
@ifnotdocbook
23910
(20 * 5 = 100).
23911
@end ifnotdocbook
23912
@end ifnottex
23913
@docbook
23914
(20 ⋅ 5 = 100). @c
23915
@end docbook
22600
23916
22601
23917
Most of the work is done in the @code{printpage()} function.
22602
23918
The label lines are stored sequentially in the @code{line} array. But they
22630
23946
The @code{END} rule arranges to flush the final page of labels; there may
22631
23947
not have been an even multiple of 20 labels in the data:
22632
23948
23949
@c STARTOFRANGE labels
22633
23950
@cindex @code{labels.awk} program
22634
23951
@example
22635
23952
@c file eg/prog/labels.awk
22697
24014
@end example
22698
24015
@c ENDOFRANGE prml
22699
24016
@c ENDOFRANGE mlprint
24017
@c ENDOFRANGE labels
22700
24018
22701
24019
@node Word Sorting
22702
24020
@subsection Generating Word-Usage Counts
22706
24024
22707
24025
When working with large amounts of text, it can be interesting to know
22708
24026
how often different words appear. For example, an author may overuse
22709
certain words, in which case she might wish to find synonyms to substitute
24027
certain words, in which case he or she might wish to find synonyms to substitute
22710
24028
for words that appear too often. This @value{SUBSECTION} develops a
22711
24029
program for counting words and presenting the frequency information
22712
24030
in a useful format.
22736
24054
This program has several problems that prevent it from being
22737
24055
useful on real text files:
22738
24056
22739
@itemize @bullet
24057
@itemize @value{BULLET}
22740
24058
@item
22741
24059
The @command{awk} language considers upper- and lowercase characters to be
22742
24060
distinct. Therefore, ``bartender'' and ``Bartender'' are not treated
22763
24081
by using the system @command{sort} utility to process the output of the
22764
24082
@command{awk} script. Here is the new version of the program:
22765
24083
24084
@c STARTOFRANGE wordfreq
22766
24085
@cindex @code{wordfreq.awk} program
22767
24086
@example
22768
24087
@c file eg/prog/wordfreq.awk
22783
24102
@}
22784
24103
@end example
22785
24104
24105
The regexp @samp{/[^[:alnum:]_[:blank:]]/} might have been written
24106
@samp{/[[:punct:]]/}, but then underscores would also be removed,
24107
and we want to keep them.
24108
22786
24109
Assuming we have saved this program in a file named @file{wordfreq.awk},
22787
24110
and that the data is in @file{file1}, the following pipeline:
22788
24111
22824
24147
See the general operating system documentation for more information on how
22825
24148
to use the @command{sort} program.
22826
24149
@c ENDOFRANGE worus
24150
@c ENDOFRANGE wordfreq
22827
24151
22828
24152
@node History Sorting
22829
24153
@subsection Removing Duplicates from Unsorted Text
22834
24158
(@pxref{Uniq Program}),
22835
24159
removes duplicate lines from @emph{sorted} data.
22836
24160
22837
Suppose, however, you need to remove duplicate lines from a data file but
24161
Suppose, however, you need to remove duplicate lines from a @value{DF} but
22838
24162
that you want to preserve the order the lines are in. A good example of
22839
24163
this might be a shell history file. The history file keeps a copy of all
22840
24164
the commands you have entered, and it is not unusual to repeat a command
22853
24177
The @code{END} rule simply prints out the lines, in order:
22854
24178
22855
24179
@cindex Rakitzis, Byron
24180
@c STARTOFRANGE histsort
22856
24181
@cindex @code{histsort.awk} program
22857
24182
@example
22858
24183
@c file eg/prog/histsort.awk
22892
24217
print data[lines[i]], lines[i]
22893
24218
@end example
22894
24219
24220
@noindent
22895
24221
This works because @code{data[$0]} is incremented each time a line is
22896
24222
seen.
22897
24223
@c ENDOFRANGE lidu
24224
@c ENDOFRANGE histsort
22898
24225
22899
24226
@node Extract Program
22900
24227
@subsection Extracting Programs from Texinfo Source Files
22926
24253
@ifnotinfo
22927
24254
Texinfo is fully documented in the book
22928
24255
@cite{Texinfo---The GNU Documentation Format},
22929
available from the Free Software Foundation.
24256
available from the Free Software Foundation,
24257
and also available @uref{http://www.gnu.org/software/texinfo/manual/texinfo/, online}.
22930
24258
@end ifnotinfo
22931
24259
@ifinfo
22932
24260
The Texinfo language is described fully, starting with
22936
24264
For our purposes, it is enough to know three things about Texinfo input
22937
24265
files:
22938
24266
22939
@itemize @bullet
24267
@itemize @value{BULLET}
22940
24268
@item
22941
24269
The ``at'' symbol (@samp{@@}) is special in Texinfo, much as
22942
24270
the backslash (@samp{\}) is in C
23004
24332
given (@code{NF} is at least three) and also checking that the command
23005
24333
exits with a zero exit status, signifying OK:
23006
24334
24335
@c STARTOFRANGE extract
23007
24336
@cindex @code{extract.awk} program
23008
24337
@example
23009
24338
@c file eg/prog/extract.awk
23025
24354
/^@@c(omment)?[ \t]+system/ \
23026
24355
@{
23027
24356
if (NF < 3) @{
23028
e = (FILENAME ":" FNR)
24357
e = ("extract: " FILENAME ":" FNR)
23029
24358
e = (e ": badly formed `system' line")
23030
24359
print e > "/dev/stderr"
23031
24360
next
23034
24363
$2 = ""
23035
24364
stat = system($0)
23036
24365
if (stat != 0) @{
23037
e = (FILENAME ":" FNR)
24366
e = ("extract: " FILENAME ":" FNR)
23038
24367
e = (e ": warning: system returned " stat)
23039
24368
print e > "/dev/stderr"
23040
24369
@}
23044
24373
23045
24374
@noindent
23046
24375
The variable @code{e} is used so that the rule
23047
fits nicely on the
23048
@ifnotinfo
23049
page.
23050
@end ifnotinfo
23051
@ifnottex
23052
screen.
23053
@end ifnottex
24376
fits nicely on the @value{PAGE}.
23054
24377
23055
24378
The second rule handles moving data into files. It verifies that a
23056
file name is given in the directive. If the file named is not the
24379
@value{FN} is given in the directive. If the file named is not the
23057
24380
current file, then the current file is closed. Keeping the current file
23058
24381
open until a new file is encountered allows the use of the @samp{>}
23059
24382
redirection for printing the contents, keeping open file management
23077
24400
The @samp{@@} symbol is used as the separator character.
23078
24401
Each element of @code{a} that is empty indicates two successive @samp{@@}
23079
24402
symbols in the original line. For each two empty elements (@samp{@@@@} in
23080
the original file), we have to add a single @samp{@@} symbol back
23081
in.@footnote{This program was written before @command{gawk} had the
23082
@code{gensub()} function. Consider how you might use it to simplify the code.}
24403
the original file), we have to add a single @samp{@@} symbol back in.
23083
24404
23084
24405
When the processing of the array is finished, @code{join()} is called with the
23085
value of @code{SUBSEP}, to rejoin the pieces back into a single
24406
value of @code{SUBSEP} (@pxref{Multidimensional}),
24407
to rejoin the pieces back into a single
23086
24408
line. That line is then printed to the output file:
23087
24409
23088
24410
@example
23090
24412
/^@@c(omment)?[ \t]+file/ \
23091
24413
@{
23092
24414
if (NF != 3) @{
23093
e = (FILENAME ":" FNR ": badly formed `file' line")
24415
e = ("extract: " FILENAME ":" FNR ": badly formed `file' line")
23094
24416
print e > "/dev/stderr"
23095
24417
next
23096
24418
@}
23135
24457
(@pxref{Redirection}).
23136
24458
This makes it easy to mix program text and explanatory prose for the same
23137
24459
sample source file (as has been done here!) without any hassle. The file is
23138
only closed when a new data file name is encountered or at the end of the
24460
only closed when a new @value{DF} name is encountered or at the end of the
23139
24461
input file.
23140
24462
23141
24463
Finally, the function @code{@w{unexpected_eof()}} prints an appropriate
23142
24464
error message and then exits.
23143
24465
The @code{END} rule handles the final cleanup, closing the open file:
23144
24466
23145
@c function lb put on same line for page breaking. sigh
23146
24467
@example
23147
24468
@c file eg/prog/extract.awk
23148
24469
@group
23149
24470
function unexpected_eof()
23150
24471
@{
23151
printf("%s:%d: unexpected EOF or error\n",
24472
printf("extract: %s:%d: unexpected EOF or error\n",
23152
24473
FILENAME, FNR) > "/dev/stderr"
23153
24474
exit 1
23154
24475
@}
23162
24483
@end example
23163
24484
@c ENDOFRANGE texse
23164
24485
@c ENDOFRANGE fitex
24486
@c ENDOFRANGE extract
23165
24487
23166
24488
@node Simple Sed
23167
24489
@subsection A Simple Stream Editor
23187
24509
23188
24510
The following program, @file{awksed.awk}, accepts at least two command-line
23189
24511
arguments: the pattern to look for and the text to replace it with. Any
23190
additional arguments are treated as data file names to process. If none
24512
additional arguments are treated as @value{DF} names to process. If none
23191
24513
are provided, the standard input is used:
23192
24514
23193
24515
@cindex Brennan, Michael
24516
@c STARTOFRANGE awksed
23194
24517
@cindex @command{awksed.awk} program
23195
24518
@c @cindex simple stream editor
23196
24519
@c @cindex stream editor, simple
23260
24583
of arguments and calling @code{usage()} if there is a problem. Then it sets
23261
24584
@code{RS} and @code{ORS} from the command-line arguments and sets
23262
24585
@code{ARGV[1]} and @code{ARGV[2]} to the null string, so that they are
23263
not treated as file names
24586
not treated as @value{FN}s
23264
24587
(@pxref{ARGC and ARGV}).
23265
24588
23266
24589
The @code{usage()} function prints an error message and exits.
23267
24590
Finally, the single rule handles the printing scheme outlined above,
23268
24591
using @code{print} or @code{printf} as appropriate, depending upon the
23269
24592
value of @code{RT}.
23270
23271
@ignore
23272
Exercise, compare the performance of this version with the more
23273
straightforward:
23274
23275
BEGIN {
23276
pat = ARGV[1]
23277
repl = ARGV[2]
23278
ARGV[1] = ARGV[2] = ""
23279
}
23280
23281
{ gsub(pat, repl); print }
23282
23283
Exercise: what are the advantages and disadvantages of this version versus sed?
23284
Advantage: egrep regexps
23285
speed (?)
23286
Disadvantage: no & in replacement text
23287
23288
Others?
23289
@end ignore
24593
@c ENDOFRANGE awksed
23290
24594
23291
24595
@node Igawk Program
23292
24596
@subsection An Easy Way to Use Library Functions
23328
24632
The following program, @file{igawk.sh}, provides this service.
23329
24633
It simulates @command{gawk}'s searching of the @env{AWKPATH} variable
23330
24634
and also allows @dfn{nested} includes; i.e., a file that is included
23331
with @samp{@@include} can contain further @samp{@@include} statements.
24635
with @code{@@include} can contain further @code{@@include} statements.
23332
24636
@command{igawk} makes an effort to only include files once, so that nested
23333
24637
includes don't accidentally include a library function twice.
23334
24638
23358
24662
text is just appended directly.
23359
24663
23360
24664
@item
23361
Source file names, provided with @option{-f}. We use a neat trick and append
24665
Source @value{FN}s, provided with @option{-f}. We use a neat trick and append
23362
24666
@samp{@@include @var{filename}} to the shell variable's contents. Since the file-inclusion
23363
24667
program works the way @command{gawk} does, this gets the text
23364
24668
of the file included into the program at the correct point.
23366
24670
23367
24671
@item
23368
24672
Run an @command{awk} program (naturally) over the shell variable's contents to expand
23369
@samp{@@include} statements. The expanded program is placed in a second
24673
@code{@@include} statements. The expanded program is placed in a second
23370
24674
shell variable.
23371
24675
23372
24676
@item
23373
24677
Run the expanded program with @command{gawk} and any other original command-line
23374
arguments that the user supplied (such as the data file names).
24678
arguments that the user supplied (such as the @value{DF} names).
23375
24679
@end enumerate
23376
24680
23377
24681
This program uses shell variables extensively: for storing command-line arguments,
23386
24690
The next part loops through all the command-line arguments.
23387
24691
There are several cases of interest:
23388
24692
23389
@table @code
23390
@item --
24693
@c @asis for docbook
24694
@table @asis
24695
@item @option{--}
23391
24696
This ends the arguments to @command{igawk}. Anything else should be passed on
23392
24697
to the user's @command{awk} program without being evaluated.
23393
24698
23394
@item -W
24699
@item @option{-W}
23395
24700
This indicates that the next option is specific to @command{gawk}. To make
23396
24701
argument processing easier, the @option{-W} is appended to the front of the
23397
24702
remaining arguments and the loop continues. (This is an @command{sh}
23398
24703
programming trick. Don't worry about it if you are not familiar with
23399
24704
@command{sh}.)
23400
24705
23401
@item -v@r{,} -F
24706
@item @option{-v}, @option{-F}
23402
24707
These are saved and passed on to @command{gawk}.
23403
24708
23404
@item -f@r{,} --file@r{,} --file=@r{,} -Wfile=
23405
The file name is appended to the shell variable @code{program} with an
23406
@samp{@@include} statement.
24709
@item @option{-f}, @option{--file}, @option{--file=}, @option{-Wfile=}
24710
The @value{FN} is appended to the shell variable @code{program} with an
24711
@code{@@include} statement.
23407
24712
The @command{expr} utility is used to remove the leading option part of the
23408
24713
argument (e.g., @samp{--file=}).
23409
24714
(Typical @command{sh} usage would be to use the @command{echo} and @command{sed}
23411
24716
escape sequences in their arguments, possibly mangling the program text.
23412
24717
Using @command{expr} avoids this problem.)
23413
24718
23414
@item --source@r{,} --source=@r{,} -Wsource=
24719
@item @option{--source}, @option{--source=}, @option{-Wsource=}
23415
24720
The source text is appended to @code{program}.
23416
24721
23417
@item --version@r{,} -Wversion
24722
@item @option{--version}, @option{-Wversion}
23418
24723
@command{igawk} prints its version number, runs @samp{gawk --version}
23419
24724
to get the @command{gawk} version information, and then exits.
23420
24725
@end table
23430
24735
23431
24736
The program is as follows:
23432
24737
24738
@c STARTOFRANGE igawk
23433
24739
@cindex @code{igawk.sh} program
23434
24740
@example
23435
24741
@c file eg/prog/igawk.sh
23521
24827
@c endfile
23522
24828
@end example
23523
24829
23524
The @command{awk} program to process @samp{@@include} directives
24830
The @command{awk} program to process @code{@@include} directives
23525
24831
is stored in the shell variable @code{expand_prog}. Doing this keeps
23526
24832
the shell script readable. The @command{awk} program
23527
24833
reads through the user's program, one line at a time, using @code{getline}
23528
24834
(@pxref{Getline}). The input
23529
file names and @samp{@@include} statements are managed using a stack.
23530
As each @samp{@@include} is encountered, the current file name is
23531
``pushed'' onto the stack and the file named in the @samp{@@include}
23532
directive becomes the current file name. As each file is finished,
24835
@value{FN}s and @code{@@include} statements are managed using a stack.
24836
As each @code{@@include} is encountered, the current @value{FN} is
24837
``pushed'' onto the stack and the file named in the @code{@@include}
24838
directive becomes the current @value{FN}. As each file is finished,
23533
24839
the stack is ``popped,'' and the previous input file becomes the current
23534
24840
input file again. The process is started by making the original file
23535
24841
the first one on the stack.
23538
24844
a file. It simulates @command{gawk}'s behavior when searching the
23539
24845
@env{AWKPATH} environment variable
23540
24846
(@pxref{AWKPATH Variable}).
23541
If a file name has a @samp{/} in it, no path search is done.
23542
Similarly, if the file name is @code{"-"}, then that string is
24847
If a @value{FN} has a @samp{/} in it, no path search is done.
24848
Similarly, if the @value{FN} is @code{"-"}, then that string is
23543
24849
used as-is. Otherwise,
23544
the file name is concatenated with the name of each directory in
23545
the path, and an attempt is made to open the generated file name.
24850
the @value{FN} is concatenated with the name of each directory in
24851
the path, and an attempt is made to open the generated @value{FN}.
23546
24852
The only way to test if a file can be read in @command{awk} is to go
23547
24853
ahead and try to read it with @code{getline}; this is what @code{pathto()}
23548
24854
does.@footnote{On some very old versions of @command{awk}, the test
23549
24855
@samp{getline junk < t} can loop forever if the file exists but is empty.
23550
Caveat emptor.} If the file can be read, it is closed and the file name
24856
Caveat emptor.} If the file can be read, it is closed and the @value{FN}
23551
24857
is returned:
23552
24858
23553
24859
@ignore
23602
24908
@c endfile
23603
24909
@end example
23604
24910
23605
The stack is initialized with @code{ARGV[1]}, which will be @file{/dev/stdin}.
24911
The stack is initialized with @code{ARGV[1]}, which will be @code{"/dev/stdin"}.
23606
24912
The main loop comes next. Input lines are read in succession. Lines that
23607
do not start with @samp{@@include} are printed verbatim.
23608
If the line does start with @samp{@@include}, the file name is in @code{$2}.
24913
do not start with @code{@@include} are printed verbatim.
24914
If the line does start with @code{@@include}, the @value{FN} is in @code{$2}.
23609
24915
@code{pathto()} is called to generate the full path. If it cannot, then the program
23610
24916
prints an error message and continues.
23611
24917
23612
24918
The next thing to check is if the file is included already. The
23613
@code{processed} array is indexed by the full file name of each included
24919
@code{processed} array is indexed by the full @value{FN} of each included
23614
24920
file and it tracks this information for us. If the file is
23615
seen again, a warning message is printed. Otherwise, the new file name is
24921
seen again, a warning message is printed. Otherwise, the new @value{FN} is
23616
24922
pushed onto the stack and processing continues.
23617
24923
23618
24924
Finally, when @code{getline} encounters the end of the input file, the file
23633
24939
fpath = pathto($2)
23634
24940
@group
23635
24941
if (fpath == "") @{
23636
printf("igawk:%s:%d: cannot find %s\n",
24942
printf("igawk: %s:%d: cannot find %s\n",
23637
24943
input[stackptr], FNR, $2) > "/dev/stderr"
23638
24944
continue
23639
24945
@}
23673
24979
23674
24980
@enumerate
23675
24981
@item
23676
Run @command{gawk} with the @samp{@@include}-processing program (the
24982
Run @command{gawk} with the @code{@@include}-processing program (the
23677
24983
value of the @code{expand_prog} shell variable) on standard input.
23678
24984
23679
24985
@item
23690
24996
23691
24997
@c this causes more problems than it solves, so leave it out.
23692
24998
@ignore
23693
The special file @file{/dev/null} is passed as a data file to @command{gawk}
24999
The special file @file{/dev/null} is passed as a @value{DF} to @command{gawk}
23694
25000
to handle an interesting case. Suppose that the user's program only has
23695
a @code{BEGIN} rule and there are no data files to read.
23696
The program should exit without reading any data files.
25001
a @code{BEGIN} rule and there are no @value{DF}s to read.
25002
The program should exit without reading any @value{DF}s.
23697
25003
However, suppose that an included library file defines an @code{END}
23698
25004
rule of its own. In this case, @command{gawk} will hang, reading standard
23699
25005
input. In order to avoid this, @file{/dev/null} is explicitly added to the
23712
25018
The @command{eval} command is a shell construct that reruns the shell's parsing
23713
25019
process. This keeps things properly quoted.
23714
25020
23715
This version of @command{igawk} represents my fifth version of this program.
25021
This version of @command{igawk} represents the fifth version of this program.
23716
25022
There are four key simplifications that make the program work better:
23717
25023
23718
@itemize @bullet
25024
@itemize @value{BULLET}
23719
25025
@item
23720
Using @samp{@@include} even for the files named with @option{-f} makes building
25026
Using @code{@@include} even for the files named with @option{-f} makes building
23721
25027
the initial collected @command{awk} program much simpler; all the
23722
@samp{@@include} processing can be done once.
25028
@code{@@include} processing can be done once.
23723
25029
23724
25030
@item
23725
25031
Not trying to save the line read with @code{getline}
23726
25032
in the @code{pathto()} function when testing for the
23727
25033
file's accessibility for use with the main program simplifies things
23728
25034
considerably.
23729
@c what problem does this engender though - exercise
23730
@c answer, reading from "-" or /dev/stdin
23731
25035
23732
25036
@item
23733
25037
Using a @code{getline} loop in the @code{BEGIN} rule does it all in one
23734
25038
place. It is not necessary to call out to a separate loop for processing
23735
nested @samp{@@include} statements.
25039
nested @code{@@include} statements.
23736
25040
23737
25041
@item
23738
25042
Instead of saving the expanded program in a temporary file, putting it in a shell variable
23752
25056
features to a program; they can often be layered on top.
23753
25057
@ignore
23754
25058
With @command{igawk},
23755
there is no real reason to build @samp{@@include} processing into
25059
there is no real reason to build @code{@@include} processing into
23756
25060
@command{gawk} itself.
23757
25061
@end ignore
23758
23759
@cindex search paths
23760
@cindex search paths, for source files
23761
@cindex source files@comma{} search path for
23762
@cindex files, source@comma{} search path for
23763
@cindex directories, searching
23764
As an additional example of this, consider the idea of having two
23765
files in a directory in the search path:
23766
23767
@table @file
23768
@item default.awk
23769
This file contains a set of default library functions, such
23770
as @code{getopt()} and @code{assert()}.
23771
23772
@item site.awk
23773
This file contains library functions that are specific to a site or
23774
installation; i.e., locally developed functions.
23775
Having a separate file allows @file{default.awk} to change with
23776
new @command{gawk} releases, without requiring the system administrator to
23777
update it each time by adding the local functions.
23778
@end table
23779
23780
One user
23781
@c Karl Berry, karl@ileaf.com, 10/95
23782
suggested that @command{gawk} be modified to automatically read these files
23783
upon startup. Instead, it would be very simple to modify @command{igawk}
23784
to do this. Since @command{igawk} can process nested @samp{@@include}
23785
directives, @file{default.awk} could simply contain @samp{@@include}
23786
statements for the desired library functions.
23787
23788
@c Exercise: make this change
23789
25062
@c ENDOFRANGE libfex
23790
25063
@c ENDOFRANGE flibex
23791
25064
@c ENDOFRANGE awkpex
25065
@c ENDOFRANGE igawk
23792
25066
23793
25067
@node Anagram Program
23794
25068
@subsection Finding Anagrams From A Dictionary
23795
25069
25070
@cindex anagrams, finding
23796
25071
An interesting programming challenge is to
23797
25072
search for @dfn{anagrams} in a
23798
25073
word list (such as
23812
25087
words with the same signature and array sorting to print the words
23813
25088
in sorted order.
23814
25089
25090
@c STARTOFRANGE anagram
23815
25091
@cindex @code{anagram.awk} program
23816
25092
@example
23817
25093
@c file eg/prog/anagram.awk
23920
25196
@dots{}
23921
25197
@end example
23922
25198
25199
@c ENDOFRANGE anagram
25200
23923
25201
@node Signature Program
23924
25202
@subsection And Now For Something Completely Different
23925
25203
25204
@cindex signature program
25205
@cindex Brini, Davide
23926
25206
The following program was written by Davide Brini
23927
25207
@c (@email{dave_br@@gmx.com})
23928
25208
and is published on @uref{http://backreference.org/2011/02/03/obfuscated-awk/,
23947
25227
O+X*(o*(o+O)+O),+x+O+X*o,x*(x-o),(o+X+x)*o*o-(x-O-O),O+(X-x)*(X+O),x-O@}'
23948
25228
@end example
23949
25229
23950
We leave it to you to determine what the program does.
25230
@cindex Johansen, Chris
25231
We leave it to you to determine what the program does. (If you are
25232
truly desperate to understand it, see Chris Johansen's explanation,
25233
which is embedded in the Texinfo source file for this @value{DOCUMENT}.)
23951
25234
23952
25235
@ignore
23953
25236
To: "Arnold Robbins" <arnold@skeeve.com>
24027
25310
}
24028
25311
@end ignore
24029
25312
24030
@iftex
24031
@part Part III:@* Moving Beyond Standard @command{awk} With @command{gawk}
24032
@end iftex
24033
24034
@ignore
25313
@node Programs Summary
25314
@section Summary
25315
25316
@itemize @value{BULLET}
25317
@item
25318
The functions provided in this @value{CHAPTER} and the previous one
25319
continue on the theme that reading programs is an excellent way to learn
25320
Good Programming.
25321
25322
@item
25323
Using @samp{#!} to make @command{awk} programs directly runnable makes
25324
them easier to use. Otherwise, invoke the program using @samp{awk
25325
-f @dots{}}.
25326
25327
@item
25328
Reimplementing standard POSIX programs in @command{awk} is a pleasant
25329
exercise; @command{awk}'s expressive power lets you write such programs
25330
in relatively few lines of code, yet they are functionally complete
25331
and usable.
25332
25333
@item
25334
One of standard @command{awk}'s weaknesses is working with individual
25335
characters. The ability to use @code{split()} with the empty string as
25336
the separator can considerably simplify such tasks.
25337
25338
@item
25339
The library functions from @ref{Library Functions}, proved their
25340
usefulness for a number of real (if small) programs.
25341
25342
@item
25343
Besides reinventing POSIX wheels, other programs solved a selection of
25344
interesting problems, such as finding duplicates words in text, printing
25345
mailing labels, and finding anagrams.
25346
25347
@end itemize
25348
25349
@node Programs Exercises
25350
@section Exercises
25351
25352
@enumerate
25353
@item
25354
Rewrite @file{cut.awk} (@pxref{Cut Program})
25355
using @code{split()} with @code{""} as the seperator.
25356
25357
@item
25358
In @ref{Egrep Program}, we mentioned that @samp{egrep -i} could be
25359
simulated in versions of @command{awk} without @code{IGNORECASE} by
25360
using @code{tolower()} on the line and the pattern. In a footnote there,
25361
we also mentioned that this solution has a bug: the translated line is
25362
output, and not the original one. Fix this problem.
25363
@c Exercise: Fix this, w/array and new line as key to original line
25364
25365
@item
25366
The POSIX version of @command{id} takes options that control which
25367
information is printed. Modify the @command{awk} version
25368
(@pxref{Id Program}) to accept the same arguments and perform in the
25369
same way.
25370
25371
@item
25372
The @code{split.awk} program (@pxref{Split Program}) uses
25373
the @code{chr()} and @code{ord()} functions to move through the
25374
letters of the alphabet.
25375
Modify the program to instead use only the @command{awk}
25376
built-in functions, such as @code{index()} and @code{substr()}.
25377
25378
@item
25379
The @code{split.awk} program (@pxref{Split Program}) assumes
25380
that letters are contiguous in the character set,
25381
which isn't true for EBCDIC systems.
25382
Fix this problem.
25383
25384
@item
25385
Why can't the @file{wc.awk} program (@pxref{Wc Program}) just
25386
use the value of @code{FNR} in @code{endfile()}?
25387
Hint: Examine the code in @ref{Filetrans Function}.
25388
25389
@ignore
25390
@command{wc} can't just use the value of @code{FNR} in
25391
@code{endfile()}. If you examine the code in @ref{Filetrans Function},
25392
you will see that @code{FNR} has already been reset by the time
25393
@code{endfile()} is called.
25394
@end ignore
25395
25396
@item
25397
Manipulation of individual characters in the @command{translate} program
25398
(@pxref{Translate Program}) is painful using standard @command{awk}
25399
functions. Given that @command{gawk} can split strings into individual
25400
characters using @code{""} as the separator, how might you use this
25401
feature to simplify the program?
25402
25403
@item
25404
The @file{extract.awk} program (@pxref{Extract Program}) was written
25405
before @command{gawk} had the @code{gensub()} function. Use it
25406
to simplify the code.
25407
25408
@item
25409
Compare the performance of the @file{awksed.awk} program
25410
(@pxref{Simple Sed}) with the more straightforward:
25411
25412
@example
25413
BEGIN @{
25414
pat = ARGV[1]
25415
repl = ARGV[2]
25416
ARGV[1] = ARGV[2] = ""
25417
@}
25418
25419
@{ gsub(pat, repl); print @}
25420
@end example
25421
25422
@item
25423
What are the advantages and disadvantages of @file{awksed.awk} versus
25424
the real @command{sed} utility?
25425
25426
@ignore
25427
Advantage: egrep regexps
25428
speed (?)
25429
Disadvantage: no & in replacement text
25430
25431
Others?
25432
@end ignore
25433
25434
@item
25435
In @ref{Igawk Program}, we mentioned that not trying to save the line
25436
read with @code{getline} in the @code{pathto()} function when testing
25437
for the file's accessibility for use with the main program simplifies
25438
things considerably. What problem does this engender though?
25439
@c answer, reading from "-" or /dev/stdin
25440
25441
@cindex search paths
25442
@cindex search paths, for source files
25443
@cindex source files@comma{} search path for
25444
@cindex files, source@comma{} search path for
25445
@cindex directories, searching
25446
@item
25447
As an additional example of the idea that it is not always necessary to
25448
add new features to a program, consider the idea of having two files in
25449
a directory in the search path:
25450
25451
@table @file
25452
@item default.awk
25453
This file contains a set of default library functions, such
25454
as @code{getopt()} and @code{assert()}.
25455
25456
@item site.awk
25457
This file contains library functions that are specific to a site or
25458
installation; i.e., locally developed functions.
25459
Having a separate file allows @file{default.awk} to change with
25460
new @command{gawk} releases, without requiring the system administrator to
25461
update it each time by adding the local functions.
25462
@end table
25463
25464
One user
25465
@c Karl Berry, karl@ileaf.com, 10/95
25466
suggested that @command{gawk} be modified to automatically read these files
25467
upon startup. Instead, it would be very simple to modify @command{igawk}
25468
to do this. Since @command{igawk} can process nested @code{@@include}
25469
directives, @file{default.awk} could simply contain @code{@@include}
25470
statements for the desired library functions.
25471
Make this change.
25472
25473
@item
25474
Modify @file{anagram.awk} (@pxref{Anagram Program}), to avoid
25475
the use of the external @command{sort} utility.
25476
25477
@end enumerate
25478
25479
@ifnotinfo
25480
@part @value{PART3}Moving Beyond Standard @command{awk} With @command{gawk}
25481
@end ifnotinfo
25482
24035
25483
@ifdocbook
24036
24037
@part Part III:@* Moving Beyond Standard @command{awk} With @command{gawk}
24038
24039
25484
Part III focuses on features specific to @command{gawk}.
24040
25485
It contains the following chapters:
24041
25486
24042
@itemize @bullet
25487
@itemize @value{BULLET}
24043
25488
@item
24044
25489
@ref{Advanced Features}.
24045
25490
24054
25499
24055
25500
@item
24056
25501
@ref{Dynamic Extensions}.
25502
@end itemize
24057
25503
@end ifdocbook
24058
@end ignore
24059
25504
24060
25505
@node Advanced Features
24061
25506
@chapter Advanced Features of @command{gawk}
24062
@cindex advanced features, network connections, See Also networks, connections
24063
25507
@c STARTOFRANGE gawadv
24064
25508
@cindex @command{gawk}, features, advanced
24065
25509
@c STARTOFRANGE advgaw
24072
25516
"Write documentation as if whoever reads it is a violent psychopath
24073
25517
who knows where you live."
24074
25518
@end ignore
25519
@cindex Langston, Peter
25520
@cindex English, Steve
24075
25521
@quotation
24076
25522
@i{Write documentation as if whoever reads it is
24077
25523
a violent psychopath who knows where you live.}
24091
25537
can @dfn{profile} an @command{awk} program, making it possible to tune
24092
25538
it for performance.
24093
25539
25540
@c FULLXREF ON
24094
25541
A number of advanced features require separate @value{CHAPTER}s of their
24095
25542
own:
24096
25543
24097
@itemize @bullet
25544
@itemize @value{BULLET}
24098
25545
@item
24099
25546
@ref{Internationalization}, discusses how to internationalize
24100
25547
your @command{awk} programs, so that they can speak multiple
24113
25560
discusses the ability to dynamically add new built-in functions to
24114
25561
@command{gawk}.
24115
25562
@end itemize
25563
@c FULLXREF OFF
24116
25564
24117
25565
@menu
24118
25566
* Nondecimal Data:: Allowing nondecimal input data.
24121
25569
* Two-way I/O:: Two-way communications with another process.
24122
25570
* TCP/IP Networking:: Using @command{gawk} for network programming.
24123
25571
* Profiling:: Profiling your @command{awk} programs.
25572
* Advanced Features Summary:: Summary of advanced features.
24124
25573
@end menu
24125
25574
24126
25575
@node Nondecimal Data
24127
25576
@section Allowing Nondecimal Input Data
24128
@cindex @code{--non-decimal-data} option
25577
@cindex @option{--non-decimal-data} option
24129
25578
@cindex advanced features, nondecimal input data
24130
25579
@cindex input, data@comma{} nondecimal
24131
25580
@cindex constants, nondecimal
24153
25602
The @code{print} statement treats its expressions as strings.
24154
25603
Although the fields can act as numbers when necessary,
24155
25604
they are still strings, so @code{print} does not try to treat them
24156
numerically. You may need to add zero to a field to force it to
25605
numerically. You need to add zero to a field to force it to
24157
25606
be treated as a number. For example:
24158
25607
24159
25608
@example
24169
25618
disabled. If you want it, you must explicitly request it.
24170
25619
24171
25620
@cindex programming conventions, @code{--non-decimal-data} option
24172
@cindex @code{--non-decimal-data} option, @code{strtonum()} function and
25621
@cindex @option{--non-decimal-data} option, @code{strtonum()} function and
24173
25622
@cindex @code{strtonum()} function (@command{gawk}), @code{--non-decimal-data} option and
24174
25623
@quotation CAUTION
24175
25624
@emph{Use of this option is not recommended.}
24176
25625
It can break old programs very badly.
24177
25626
Instead, use the @code{strtonum()} function to convert your data
24178
(@pxref{Nondecimal-numbers}).
25627
(@pxref{String Functions}).
24179
25628
This makes your programs easier to write and easier to read, and
24180
25629
leads to less surprising results.
24181
25630
@end quotation
24209
25658
24210
25659
@ref{Controlling Scanning}, describes how you can assign special,
24211
25660
pre-defined values to @code{PROCINFO["sorted_in"]} in order to
24212
control the order in which @command{gawk} will traverse an array
25661
control the order in which @command{gawk} traverses an array
24213
25662
during a @code{for} loop.
24214
25663
24215
25664
In addition, the value of @code{PROCINFO["sorted_in"]} can be a function name.
24466
25915
@subsection Sorting Array Values and Indices with @command{gawk}
24467
25916
24468
25917
@cindex arrays, sorting
24469
@cindex @code{asort()} function (@command{gawk})
25918
@cindexgawkfunc{asort}
24470
25919
@cindex @code{asort()} function (@command{gawk}), arrays@comma{} sorting
24471
@cindex @code{asorti()} function (@command{gawk})
25920
@cindexgawkfunc{asorti}
24472
25921
@cindex @code{asorti()} function (@command{gawk}), arrays@comma{} sorting
24473
25922
@cindex sort function, arrays, sorting
24474
25923
In most @command{awk} implementations, sorting an array requires writing
24533
25982
24534
25983
So far, so good. Now it starts to get interesting. Both @code{asort()}
24535
25984
and @code{asorti()} accept a third string argument to control comparison
24536
of array elements. In @ref{String Functions}, we ignored this third
24537
argument; however, the time has now come to describe how this argument
24538
affects these two functions.
25985
of array elements. When we introduced @code{asort()} and @code{asorti()}
25986
in @ref{String Functions}, we ignored this third argument; however,
25987
now is the time to describe how this argument affects these two functions.
24539
25988
24540
25989
Basically, the third argument specifies how the array is to be sorted.
24541
25990
There are two possibilities. As with @code{PROCINFO["sorted_in"]},
24563
26012
24564
26013
@c Document It And Call It A Feature. Sigh.
24565
26014
@cindex @command{gawk}, @code{IGNORECASE} variable in
24566
@cindex @code{IGNORECASE} variable
24567
@cindex arrays, sorting, @code{IGNORECASE} variable and
24568
@cindex @code{IGNORECASE} variable, array sorting and
26015
@cindex arrays, sorting, and @code{IGNORECASE} variable
26016
@cindex @code{IGNORECASE} variable, and array sorting functions
24569
26017
Because @code{IGNORECASE} affects string comparisons, the value
24570
26018
of @code{IGNORECASE} also affects sorting for both @code{asort()} and @code{asorti()}.
24571
26019
Note also that the locale's sorting order does @emph{not}
24644
26092
termed a @dfn{coprocess}, since it runs in parallel with @command{gawk}.
24645
26093
The two-way connection is created using the @samp{|&} operator
24646
26094
(borrowed from the Korn shell, @command{ksh}):@footnote{This is very
24647
different from the same operator in the C shell.}
26095
different from the same operator in the C shell and in Bash.}
24648
26096
24649
26097
@example
24650
26098
do @{
24666
26114
24667
26115
There are some cautionary items to be aware of:
24668
26116
24669
@itemize @bullet
26117
@itemize @value{BULLET}
24670
26118
@item
24671
26119
As the code inside @command{gawk} currently stands, the coprocess's
24672
26120
standard error goes to the same place that the parent @command{gawk}'s
24732
26180
24733
26181
As a side note, the assignment @samp{LC_ALL=C} in the @command{sort}
24734
26182
command ensures traditional Unix (ASCII) sorting from @command{sort}.
26183
This is not strictly necessary here, but it's good to know how to do this.
24735
26184
24736
26185
@cindex @command{gawk}, @code{PROCINFO} array in
24737
@cindex @code{PROCINFO} array
26186
@cindex @code{PROCINFO} array, and communications via ptys
24738
26187
You may also use pseudo-ttys (ptys) for
24739
26188
two-way communication instead of pipes, if your system supports them.
24740
26189
This is done on a per-command basis, by setting a special element
24750
26199
@end example
24751
26200
24752
26201
@noindent
24753
Using ptys avoids the buffer deadlock issues described earlier, at some
26202
Using ptys usually avoids the buffer deadlock issues described earlier, at some
24754
26203
loss in performance. If your system does not have ptys, or if all the
24755
26204
system's ptys are in use, @command{gawk} automatically falls back to
24756
26205
using regular pipes.
24785
26234
You can think of this as just a @emph{very long} two-way pipeline to
24786
26235
a coprocess.
24787
26236
The way @command{gawk} decides that you want to use TCP/IP networking is
24788
by recognizing special file names that begin with one of @samp{/inet/},
26237
by recognizing special @value{FN}s that begin with one of @samp{/inet/},
24789
26238
@samp{/inet4/} or @samp{/inet6}.
24790
26239
24791
The full syntax of the special file name is
26240
The full syntax of the special @value{FN} is
24792
26241
@file{/@var{net-type}/@var{protocol}/@var{local-port}/@var{remote-host}/@var{remote-port}}.
24793
26242
The components are:
24794
26243
24854
26303
@inforef{Top, , General Introduction, gawkinet, TCP/IP Internetworking with @command{gawk}},
24855
26304
@end ifinfo
24856
26305
@ifnotinfo
24857
See @cite{TCP/IP Internetworking with @command{gawk}},
26306
See
26307
@uref{http://www.gnu.org/software/gawk/manual/gawkinet/,
26308
@cite{TCP/IP Internetworking with @command{gawk}}},
24858
26309
which comes as part of the @command{gawk} distribution,
24859
26310
@end ifnotinfo
24860
26311
for a much more complete introduction and discussion, as well as
24877
26328
named @file{awkprof.out}. Because it is profiling, it also executes up to 45% slower than
24878
26329
@command{gawk} normally does.
24879
26330
24880
@cindex @code{--profile} option
26331
@cindex @option{--profile} option
24881
26332
As shown in the following example,
24882
26333
the @option{--profile} option can be used to change the name of the file
24883
26334
where @command{gawk} will write the profile:
24932
26383
junk
24933
26384
@end example
24934
26385
24935
Here is the @file{awkprof.out} that results from running the @command{gawk}
24936
profiler on this program and data (this example also illustrates that @command{awk}
24937
programmers sometimes have to work late):
26386
Here is the @file{awkprof.out} that results from running the
26387
@command{gawk} profiler on this program and data. (This example also
26388
illustrates that @command{awk} programmers sometimes get up very early
26389
in the morning to work.)
24938
26390
24939
@cindex @code{BEGIN} pattern
24940
@cindex @code{END} pattern
26391
@cindex @code{BEGIN} pattern, and profiling
26392
@cindex @code{END} pattern, and profiling
24941
26393
@example
24942
# gawk profile, created Sun Aug 13 00:00:15 2000
24943
24944
# BEGIN block(s)
24945
24946
BEGIN @{
24947
1 print "First BEGIN rule"
24948
1 print "Second BEGIN rule"
24949
@}
24950
24951
# Rule(s)
24952
24953
5 /foo/ @{ # 2
24954
2 print "matched /foo/, gosh"
24955
6 for (i = 1; i <= 3; i++) @{
24956
6 sing()
24957
@}
24958
@}
24959
24960
5 @{
24961
5 if (/foo/) @{ # 2
24962
2 print "if is true"
24963
3 @} else @{
24964
3 print "else is true"
24965
@}
24966
@}
24967
24968
# END block(s)
24969
24970
END @{
24971
1 print "First END rule"
24972
1 print "Second END rule"
24973
@}
24974
24975
# Functions, listed alphabetically
24976
24977
6 function sing(dummy)
24978
@{
24979
6 print "I gotta be me!"
24980
@}
26394
# gawk profile, created Thu Feb 27 05:16:21 2014
26395
26396
# BEGIN block(s)
26397
26398
BEGIN @{
26399
1 print "First BEGIN rule"
26400
@}
26401
26402
BEGIN @{
26403
1 print "Second BEGIN rule"
26404
@}
26405
26406
# Rule(s)
26407
26408
5 /foo/ @{ # 2
26409
2 print "matched /foo/, gosh"
26410
6 for (i = 1; i <= 3; i++) @{
26411
6 sing()
26412
@}
26413
@}
26414
26415
5 @{
26416
5 if (/foo/) @{ # 2
26417
2 print "if is true"
26418
3 @} else @{
26419
3 print "else is true"
26420
@}
26421
@}
26422
26423
# END block(s)
26424
26425
END @{
26426
1 print "First END rule"
26427
@}
26428
26429
END @{
26430
1 print "Second END rule"
26431
@}
26432
26433
26434
# Functions, listed alphabetically
26435
26436
6 function sing(dummy)
26437
@{
26438
6 print "I gotta be me!"
26439
@}
24981
26440
@end example
24982
26441
24983
26442
This example illustrates many of the basic features of profiling output.
24984
26443
They are as follows:
24985
26444
24986
@itemize @bullet
26445
@itemize @value{BULLET}
24987
26446
@item
24988
The program is printed in the order @code{BEGIN} rule,
24989
@code{BEGINFILE} rule,
26447
The program is printed in the order @code{BEGIN} rules,
26448
@code{BEGINFILE} rules,
24990
26449
pattern/action rules,
24991
@code{ENDFILE} rule, @code{END} rule and functions, listed
26450
@code{ENDFILE} rules, @code{END} rules and functions, listed
24992
26451
alphabetically.
24993
Multiple @code{BEGIN} and @code{END} rules are merged together,
24994
as are multiple @code{BEGINFILE} and @code{ENDFILE} rules.
26452
Multiple @code{BEGIN} and @code{END} rules retain their
26453
separate identities, as do
26454
multiple @code{BEGINFILE} and @code{ENDFILE} rules.
24995
26455
24996
@cindex patterns, counts
26456
@cindex patterns, counts, in a profile
24997
26457
@item
24998
26458
Pattern-action rules have two counts.
24999
26459
The first count, to the left of the rule, shows how many times
25013
26473
The count for the @code{else}
25014
26474
indicates how many times the test failed.
25015
26475
25016
@cindex loops, count for header
26476
@cindex loops, count for header, in a profile
25017
26477
@item
25018
26478
The count for a loop header (such as @code{for}
25019
26479
or @code{while}) shows how many times the loop test was executed.
25021
26481
statement in a rule to determine how many times the rule was executed.
25022
26482
If the first statement is a loop, the count is misleading.)
25023
26483
25024
@cindex functions, user-defined, counts
25025
@cindex user-defined, functions, counts
26484
@cindex functions, user-defined, counts, in a profile
26485
@cindex user-defined, functions, counts, in a profile
25026
26486
@item
25027
26487
For user-defined functions, the count next to the @code{function}
25028
26488
keyword indicates how many times the function was called.
25036
26496
Braces are used everywhere, even when
25037
26497
the body of an @code{if}, @code{else}, or loop is only a single statement.
25038
26498
25039
@cindex @code{()} (parentheses)
25040
@cindex parentheses @code{()}
26499
@cindex @code{()} (parentheses), in a profile
26500
@cindex parentheses @code{()}, in a profile
25041
26501
@item
25042
26502
Parentheses are used only where needed, as indicated by the structure
25043
26503
of the program and the precedence rules.
25044
@c extra verbiage here satisfies the copyeditor. ugh.
25045
26504
For example, @samp{(3 + 5) * 4} means add three plus five, then multiply
25046
26505
the total by four. However, @samp{3 + 5 * 4} has no parentheses, and
25047
26506
means @samp{3 + (5 * 4)}.
25072
26531
profiled version by ``pretty printing'' its internal representation of
25073
26532
the program. The advantage to this is that @command{gawk} can produce
25074
26533
a standard representation. The disadvantage is that all source-code
25075
comments are lost, as are the distinctions among multiple @code{BEGIN},
25076
@code{END}, @code{BEGINFILE}, and @code{ENDFILE} rules. Also, things such as:
26534
comments are lost.
26535
Also, things such as:
25077
26536
25078
26537
@example
25079
26538
/foo/
25093
26552
25094
26553
@cindex profiling @command{awk} programs, dynamically
25095
26554
@cindex @command{gawk} program, dynamic profiling
26555
@cindex dynamic profiling
25096
26556
Besides creating profiles when a program has completed,
25097
26557
@command{gawk} can produce a profile while it is running.
25098
26558
This is useful if your @command{awk} program goes into an
25106
26566
@end example
25107
26567
25108
26568
@cindex @command{kill} command@comma{} dynamic profiling
25109
@cindex @code{USR1} signal
25110
@cindex @code{SIGUSR1} signal
25111
@cindex signals, @code{USR1}/@code{SIGUSR1}
26569
@cindex @code{USR1} signal, for dynamic profiling
26570
@cindex @code{SIGUSR1} signal, for dynamic profiling
26571
@cindex signals, @code{USR1}/@code{SIGUSR1}, for profiling
25112
26572
@noindent
25113
26573
The shell prints a job number and process ID number; in this case, 13992.
25114
26574
Use the @command{kill} command to send the @code{USR1} signal
25123
26583
@file{awkprof.out}, or to a different file if one specified with
25124
26584
the @option{--profile} option.
25125
26585
25126
Along with the regular profile, as shown earlier, the profile
26586
Along with the regular profile, as shown earlier, the profile file
25127
26587
includes a trace of any active functions:
25128
26588
25129
26589
@example
25139
26599
Each time, the profile and function call trace are appended to the output
25140
26600
profile file.
25141
26601
25142
@cindex @code{HUP} signal
25143
@cindex @code{SIGHUP} signal
25144
@cindex signals, @code{HUP}/@code{SIGHUP}
26602
@cindex @code{HUP} signal, for dynamic profiling
26603
@cindex @code{SIGHUP} signal, for dynamic profiling
26604
@cindex signals, @code{HUP}/@code{SIGHUP}, for profiling
25145
26605
If you use the @code{HUP} signal instead of the @code{USR1} signal,
25146
26606
@command{gawk} produces the profile and the function call trace and then exits.
25147
26607
25163
26623
Finally, @command{gawk} also accepts another option, @option{--pretty-print}.
25164
26624
When called this way, @command{gawk} ``pretty prints'' the program into
25165
26625
@file{awkprof.out}, without any execution counts.
26626
26627
@quotation NOTE
26628
The @option{--pretty-print} option still runs your program.
26629
This will change in the next major release.
26630
@end quotation
26631
@c ENDOFRANGE awkp
26632
@c ENDOFRANGE proawk
26633
26634
@node Advanced Features Summary
26635
@section Summary
26636
26637
@itemize @value{BULLET}
26638
@item
26639
The @option{--non-decimal-data} option causes @command{gawk} to treat
26640
octal- and hexadecimal-looking input data as octal and hexadecimal.
26641
This option should be used with caution or not at all; use of @code{strtonum()}
26642
is preferable.
26643
26644
@item
26645
You can take over complete control of sorting in @samp{for (@var{indx} in @var{array})}
26646
array traversal by setting @code{PROCINFO["sorted_in"]} to the name of a user-defined
26647
function that does the comparison of array elements based on index and value.
26648
26649
@item
26650
Similarly, you can supply the name of a user-defined comparison function as the
26651
third argument to either @code{asort()} or @command{asorti()} to control how
26652
those functions sort arrays. Or you may provide one of the predefined control
26653
strings that work for @code{PROCINFO["sorted_in"]}.
26654
26655
@item
26656
You can use the @samp{|&} operator to create a two-way pipe to a co-process.
26657
You read from the co-process with @code{getline} and write to it with @code{print}
26658
or @code{printf}. Use @code{close()} to close off the co-process completely, or
26659
optionally, close off one side of the two-way communications.
26660
26661
@item
26662
By using special ``@value{FN}s'' with the @samp{|&} operator, you can open a
26663
TCP/IP (or UDP/IP) connection to remote hosts in the Internet. @command{gawk}
26664
supports both IPv4 an IPv6.
26665
26666
@item
26667
You can generate statement count profiles of your program. This can help you
26668
determine which parts of your program may be taking the most time and let
26669
you tune them more easily. Sending the @code{USR1} signal while profiling causes
26670
@command{gawk} to dump the profile and keep going, including a function call stack.
26671
26672
@item
26673
You can also just ``pretty print'' the program. This currently also runs
26674
the program, but that will change in the next major release.
26675
26676
@end itemize
26677
25166
26678
@c ENDOFRANGE advgaw
25167
26679
@c ENDOFRANGE gawadv
25168
@c ENDOFRANGE awkp
25169
@c ENDOFRANGE proawk
25170
26680
25171
26681
@node Internationalization
25172
26682
@chapter Internationalization with @command{gawk}
25196
26706
25197
26707
@menu
25198
26708
* I18N and L10N:: Internationalization and Localization.
25199
* Explaining gettext:: How GNU @code{gettext} works.
26709
* Explaining gettext:: How GNU @command{gettext} works.
25200
26710
* Programmer i18n:: Features for the programmer.
25201
26711
* Translator i18n:: Features for the translator.
25202
26712
* I18N Example:: A simple i18n example.
25203
26713
* Gawk I18N:: @command{gawk} is also internationalized.
26714
* I18N Summary:: Summary of I18N stuff.
25204
26715
@end menu
25205
26716
25206
26717
@node I18N and L10N
25220
26731
monetary values are printed and read.
25221
26732
25222
26733
@node Explaining gettext
25223
@section GNU @code{gettext}
26734
@section GNU @command{gettext}
25224
26735
25225
26736
@cindex internationalizing a program
25226
26737
@c STARTOFRANGE gettex
25227
@cindex @code{gettext} library
25228
The facilities in GNU @code{gettext} focus on messages; strings printed
26738
@cindex @command{gettext} library
26739
@command{gawk} uses GNU @command{gettext} to provide its internationalization
26740
features.
26741
The facilities in GNU @command{gettext} focus on messages; strings printed
25229
26742
by a program, either directly or via formatting with @code{printf} or
25230
26743
@code{sprintf()}.@footnote{For some operating systems, the @command{gawk}
25231
port doesn't support GNU @code{gettext}.
26744
port doesn't support GNU @command{gettext}.
25232
26745
Therefore, these features are not available
25233
26746
if you are using one of those operating systems. Sorry.}
25234
26747
25235
@cindex portability, @code{gettext} library and
25236
When using GNU @code{gettext}, each application has its own
26748
@cindex portability, @command{gettext} library and
26749
When using GNU @command{gettext}, each application has its own
25237
26750
@dfn{text domain}. This is a unique name, such as @samp{kpilot} or @samp{gawk},
25238
26751
that identifies the application.
25239
26752
A complete application may have multiple components---programs written
25257
26770
@cindex @code{textdomain()} function (C library)
25258
26771
@item
25259
26772
The programmer indicates the application's text domain
25260
(@code{"guide"}) to the @code{gettext} library,
26773
(@command{"guide"}) to the @command{gettext} library,
25261
26774
by calling the @code{textdomain()} function.
25262
26775
25263
26776
@cindex @code{.pot} files
25274
26787
25275
26788
@cindex @code{.po} files
25276
26789
@cindex files, @code{.po}
26790
@c STARTOFRANGE portobfi
25277
26791
@cindex portable object files
25278
26792
@cindex files, portable object
25279
26793
@item
25285
26799
@cindex @code{.gmo} files
25286
26800
@cindex files, @code{.gmo}
25287
26801
@cindex message object files
26802
@c STARTOFRANGE portmsgfi
25288
26803
@cindex files, message object
25289
26804
@item
25290
26805
Each language's @file{.po} file is converted into a binary
25299
26814
25300
26815
@cindex @code{bindtextdomain()} function (C library)
25301
26816
@item
25302
For testing and development, it is possible to tell @code{gettext}
26817
For testing and development, it is possible to tell @command{gettext}
25303
26818
to use @file{.gmo} files in a different directory than the standard
25304
26819
one by using the @code{bindtextdomain()} function.
25305
26820
25332
26847
25333
26848
@cindex @code{_} (underscore), C macro
25334
26849
@cindex underscore (@code{_}), C macro
25335
The GNU @code{gettext} developers, recognizing that typing
26850
The GNU @command{gettext} developers, recognizing that typing
25336
26851
@samp{gettext(@dots{})} over and over again is both painful and ugly to look
25337
26852
at, use the macro @samp{_} (an underscore) to make things easier:
25338
26853
25345
26860
@end example
25346
26861
25347
26862
@cindex internationalization, localization, locale categories
25348
@cindex @code{gettext} library, locale categories
26863
@cindex @command{gettext} library, locale categories
25349
26864
@cindex locale categories
25350
26865
@noindent
25351
26866
This reduces the typing overhead to just three extra characters per string
25353
26868
25354
26869
There are locale @dfn{categories}
25355
26870
for different types of locale-related information.
25356
The defined locale categories that @code{gettext} knows about are:
26871
The defined locale categories that @command{gettext} knows about are:
25357
26872
25358
26873
@table @code
25359
26874
@cindex @code{LC_MESSAGES} locale category
25360
26875
@item LC_MESSAGES
25361
Text messages. This is the default category for @code{gettext}
26876
Text messages. This is the default category for @command{gettext}
25362
26877
operations, but it is possible to supply a different one explicitly,
25363
26878
if necessary. (It is almost never necessary to supply a different category.)
25364
26879
25406
26921
25407
26922
@cindex @code{LC_ALL} locale category
25408
26923
@item LC_ALL
25409
All of the above. (Not too useful in the context of @code{gettext}.)
26924
All of the above. (Not too useful in the context of @command{gettext}.)
25410
26925
@end table
25411
26926
@c ENDOFRANGE gettex
25412
26927
25422
26937
@cindex @code{TEXTDOMAIN} variable
25423
26938
@item TEXTDOMAIN
25424
26939
This variable indicates the application's text domain.
25425
For compatibility with GNU @code{gettext}, the default
26940
For compatibility with GNU @command{gettext}, the default
25426
26941
value is @code{"messages"}.
25427
26942
25428
26943
@cindex internationalization, localization, marked strings
25432
26947
are candidates for translation at runtime.
25433
26948
String constants without a leading underscore are not translated.
25434
26949
25435
@cindex @code{dcgettext()} function (@command{gawk})
25436
@item dcgettext(@var{string} @r{[}, @var{domain} @r{[}, @var{category}@r{]]})
26950
@cindexgawkfunc{dcgettext}
26951
@item @code{dcgettext(@var{string}} [@code{,} @var{domain} [@code{,} @var{category}]]@code{)}
25437
26952
Return the translation of @var{string} in
25438
26953
text domain @var{domain} for locale category @var{category}.
25439
26954
The default value for @var{domain} is the current value of @code{TEXTDOMAIN}.
25458
26973
default arguments.
25459
26974
@end quotation
25460
26975
25461
@cindex @code{dcngettext()} function (@command{gawk})
25462
@item dcngettext(@var{string1}, @var{string2}, @var{number} @r{[}, @var{domain} @r{[}, @var{category}@r{]]})
26976
@cindexgawkfunc{dcngettext}
26977
@item @code{dcngettext(@var{string1}, @var{string2}, @var{number}} [@code{,} @var{domain} [@code{,} @var{category}]]@code{)}
25463
26978
Return the plural form used for @var{number} of the
25464
26979
translation of @var{string1} and @var{string2} in text domain
25465
26980
@var{domain} for locale category @var{category}. @var{string1} is the
25474
26989
@cindex files, @code{.gmo}, specifying directory of
25475
26990
@cindex message object files, specifying directory of
25476
26991
@cindex files, message object, specifying directory of
25477
@cindex @code{bindtextdomain()} function (@command{gawk})
25478
@item bindtextdomain(@var{directory} @r{[}, @var{domain}@r{]})
26992
@cindexgawkfunc{bindtextdomain}
26993
@item @code{bindtextdomain(@var{directory}} [@code{,} @var{domain} ]@code{)}
25479
26994
Change the directory in which
25480
@code{gettext} looks for @file{.gmo} files, in case they
26995
@command{gettext} looks for @file{.gmo} files, in case they
25481
26996
will not or cannot be placed in the standard locations
25482
26997
(e.g., during testing).
25483
26998
Return the directory in which @var{domain} is ``bound.''
25576
27091
@cindex portable object files
25577
27092
@cindex files, portable object
25578
27093
Once a program's translatable strings have been marked, they must
25579
be extracted to create the initial @file{.po} file.
27094
be extracted to create the initial @file{.pot} file.
25580
27095
As part of translation, it is often helpful to rearrange the order
25581
27096
in which arguments to @code{printf} are output.
25582
27097
25596
27111
@subsection Extracting Marked Strings
25597
27112
@cindex strings, extracting
25598
27113
@cindex marked strings@comma{} extracting
25599
@cindex @code{--gen-pot} option
27114
@cindex @option{--gen-pot} option
25600
27115
@cindex command-line options, string extraction
25601
27116
@cindex string extraction (internationalization)
25602
27117
@cindex marked string extraction (internationalization)
25603
27118
@cindex extraction, of marked strings (internationalization)
25604
27119
25605
@cindex @code{--gen-pot} option
27120
@cindex @option{--gen-pot} option
25606
27121
Once your @command{awk} program is working, and all the strings have
25607
27122
been marked and you've set (and perhaps bound) the text domain,
25608
27123
it is time to produce translations.
25616
27131
@cindex @code{xgettext} utility
25617
27132
When run with @option{--gen-pot}, @command{gawk} does not execute your
25618
27133
program. Instead, it parses it as usual and prints all marked strings
25619
to standard output in the format of a GNU @code{gettext} Portable Object
27134
to standard output in the format of a GNU @command{gettext} Portable Object
25620
27135
file. Also included in the output are any constant strings that
25621
27136
appear as the first argument to @code{dcgettext()} or as the first and
25622
27137
second argument to @code{dcngettext()}.@footnote{The
25623
27138
@command{xgettext} utility that comes with GNU
25624
@code{gettext} can handle @file{.awk} files.}
27139
@command{gettext} can handle @file{.awk} files.}
25625
27140
@xref{I18N Example},
25626
27141
for the full list of steps to go through to create and test
25627
27142
translations for @command{guide}.
27143
@c ENDOFRANGE portobfi
27144
@c ENDOFRANGE portmsgfi
25628
27145
25629
27146
@node Printf Ordering
25630
27147
@subsection Rearranging @code{printf} Arguments
25635
27152
(@pxref{Printf})
25636
27153
present a special problem for translation.
25637
27154
Consider the following:@footnote{This example is borrowed
25638
from the GNU @code{gettext} manual.}
27155
from the GNU @command{gettext} manual.}
25639
27156
25640
@c line broken here only for smallbook format
25641
27157
@example
25642
27158
printf(_"String `%s' has %d characters\n",
25643
27159
string, length(string)))
25671
27187
@example
25672
27188
$ @kbd{gawk 'BEGIN @{}
25673
27189
> @kbd{string = "Dont Panic"}
25674
> @kbd{printf _"%2$d characters live in \"%1$s\"\n",}
27190
> @kbd{printf "%2$d characters live in \"%1$s\"\n",}
25675
27191
> @kbd{string, length(string)}
25676
27192
> @kbd{@}'}
25677
27193
@print{} 10 characters live in "Dont Panic"
25705
27221
and those with positional specifiers in the same string:
25706
27222
25707
27223
@example
25708
$ @kbd{gawk 'BEGIN @{ printf _"%d %3$s\n", 1, 2, "hi" @}'}
27224
$ @kbd{gawk 'BEGIN @{ printf "%d %3$s\n", 1, 2, "hi" @}'}
25709
27225
@error{} gawk: cmd. line:1: fatal: must use `count$' on all formats or none
25710
27226
@end example
25711
27227
25745
27261
However, it is actually almost portable, requiring very little
25746
27262
change:
25747
27263
25748
@itemize @bullet
27264
@itemize @value{BULLET}
25749
27265
@cindex @code{TEXTDOMAIN} variable, portability and
25750
27266
@item
25751
27267
Assignments to @code{TEXTDOMAIN} won't have any effect,
25885
27401
@cindex Linux
25886
27402
@cindex GNU/Linux
25887
27403
The next step is to make the directory to hold the binary message object
25888
file and then to create the @file{guide.gmo} file.
25889
The directory layout shown here is standard for GNU @code{gettext} on
25890
GNU/Linux systems. Other versions of @code{gettext} may use a different
27404
file and then to create the @file{guide.mo} file.
27405
We pretend that our file is to be used in the @code{en_US.UTF-8} locale.
27406
The directory layout shown here is standard for GNU @command{gettext} on
27407
GNU/Linux systems. Other versions of @command{gettext} may use a different
25891
27408
layout:
25892
27409
25893
27410
@example
25894
$ @kbd{mkdir en_US en_US/LC_MESSAGES}
27411
$ @kbd{mkdir en_US.UTF-8 en_US.UTF-8/LC_MESSAGES}
25895
27412
@end example
25896
27413
25897
@cindex @code{.po} files, converting to @code{.gmo}
25898
@cindex files, @code{.po}, converting to @code{.gmo}
25899
@cindex @code{.gmo} files, converting from @code{.po}
25900
@cindex files, @code{.gmo}, converting from @code{.po}
27414
@cindex @code{.po} files, converting to @code{.mo}
27415
@cindex files, @code{.po}, converting to @code{.mo}
27416
@cindex @code{.mo} files, converting from @code{.po}
27417
@cindex files, @code{.mo}, converting from @code{.po}
25901
27418
@cindex portable object files, converting to message object files
25902
27419
@cindex files, portable object, converting to message object files
25903
27420
@cindex message object files, converting from portable object files
25904
27421
@cindex files, message object, converting from portable object files
25905
27422
@cindex @command{msgfmt} utility
25906
27423
The @command{msgfmt} utility does the conversion from human-readable
25907
@file{.po} file to machine-readable @file{.gmo} file.
27424
@file{.po} file to machine-readable @file{.mo} file.
25908
27425
By default, @command{msgfmt} creates a file named @file{messages}.
25909
27426
This file must be renamed and placed in the proper directory so that
25910
27427
@command{gawk} can find it:
25911
27428
25912
27429
@example
25913
27430
$ @kbd{msgfmt guide-mellow.po}
25914
$ @kbd{mv messages en_US/LC_MESSAGES/guide.gmo}
27431
$ @kbd{mv messages en_US.UTF-8/LC_MESSAGES/guide.mo}
25915
27432
@end example
25916
27433
25917
27434
Finally, we run the program to test it:
25940
27457
@section @command{gawk} Can Speak Your Language
25941
27458
25942
27459
@command{gawk} itself has been internationalized
25943
using the GNU @code{gettext} package.
25944
(GNU @code{gettext} is described in
27460
using the GNU @command{gettext} package.
27461
(GNU @command{gettext} is described in
25945
27462
complete detail in
25946
27463
@ifinfo
25947
@inforef{Top, , GNU @code{gettext} utilities, gettext, GNU gettext tools}.)
27464
@inforef{Top, , GNU @command{gettext} utilities, gettext, GNU gettext tools}.)
25948
27465
@end ifinfo
25949
27466
@ifnotinfo
25950
@cite{GNU gettext tools}.)
27467
@uref{http://www.gnu.org/software/gettext/manual/,
27468
@cite{GNU gettext tools}}.)
25951
27469
@end ifnotinfo
25952
As of this writing, the latest version of GNU @code{gettext} is
25953
@uref{ftp://ftp.gnu.org/gnu/gettext/gettext-0.18.2.1.tar.gz, version 0.18.2.1}.
27470
As of this writing, the latest version of GNU @command{gettext} is
27471
@uref{ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.1.tar.gz,
27472
@value{PVERSION} 0.19.1}.
25954
27473
25955
27474
If a translation of @command{gawk}'s messages exists,
25956
27475
then @command{gawk} produces usage messages, warnings,
25957
27476
and fatal errors in the local language.
27477
27478
@node I18N Summary
27479
@section Summary
27480
27481
@itemize @value{BULLET}
27482
@item
27483
Internationalization means writing a program such that it can use multiple
27484
languages without requiring source-code changes. Localization means
27485
providing the data necessary for an internationalized program to work
27486
in a particular language.
27487
27488
@item
27489
@command{gawk} uses GNU @command{gettext} to let you internationalize
27490
and localize @command{awk} programs. A program's text domain identifies
27491
the program for grouping all messages and other data together.
27492
27493
@item
27494
You mark a program's strings for translation by preceding them with
27495
an underscore. Once that is done, the strings are extracted into a
27496
@file{.pot} file. This file is copied for each language into a @file{.po}
27497
file, and the @file{.po} files are compiled into @file{.gmo} files for
27498
use at runtime.
27499
27500
@item
27501
You can use position specifications with @code{sprintf()} and
27502
@code{printf} to rearrange the placement of argument values in formatted
27503
strings and output. This is useful for the translations of format
27504
control strings.
27505
27506
@item
27507
The internationalization features have been designed so that they
27508
can be easily worked around in a standard @command{awk}.
27509
27510
@item
27511
@command{gawk} itself has been internationalized and ships with
27512
a number of translations for its messages.
27513
27514
@end itemize
27515
25958
27516
@c ENDOFRANGE inloc
25959
27517
25960
@c The original text for this chapter was contributed by Efraim Yawitz.
25961
@c FIXME: Add more indexing.
25962
25963
27518
@node Debugger
25964
27519
@chapter Debugging @command{awk} Programs
25965
27520
@cindex debugging @command{awk} programs
25966
27521
27522
@c The original text for this chapter was contributed by Efraim Yawitz.
27523
@c FIXME: Add more indexing.
27524
25967
27525
It would be nice if computer programs worked perfectly the first time they
25968
27526
were run, but in real life, this rarely happens for programs of
25969
27527
any complexity. Thus, most programming languages have facilities available
25980
27538
* List of Debugger Commands:: Main debugger commands.
25981
27539
* Readline Support:: Readline support.
25982
27540
* Limitations:: Limitations and future plans.
27541
* Debugging Summary:: Debugging summary.
25983
27542
@end menu
25984
27543
25985
27544
@node Debugging
25986
@section Introduction to @command{gawk} Debugger
27545
@section Introduction to The @command{gawk} Debugger
25987
27546
25988
27547
This @value{SECTION} introduces debugging in general and begins
25989
27548
the discussion of debugging in @command{gawk}.
26008
27567
depends on the language being debugged, but in general, you can expect at
26009
27568
least the following:
26010
27569
26011
@itemize @bullet
27570
@itemize @value{BULLET}
26012
27571
@item
26013
27572
The ability to watch a program execute its instructions one by one,
26014
27573
giving you, the programmer, the opportunity to think about what is happening
26046
27605
this @value{CHAPTER}.
26047
27606
26048
27607
@table @dfn
27608
@cindex stack frame
26049
27609
@item Stack Frame
26050
27610
Programs generally call functions during the course of their execution.
26051
27611
One function can call another, or a function can call itself (recursion).
26067
27627
each stack frame (as detailed later on).
26068
27628
26069
27629
@item Breakpoint
27630
@cindex breakpoint
26070
27631
During debugging, you often wish to let the program run until it
26071
27632
reaches a certain point, and then continue execution from there one
26072
27633
statement (or instruction) at a time. The way to do this is to set
26076
27637
as many breakpoints as you like.
26077
27638
26078
27639
@item Watchpoint
27640
@cindex watchpoint
26079
27641
A watchpoint is similar to a breakpoint. The difference is that
26080
27642
breakpoints are oriented around the code: stop when a certain point in the
26081
27643
code is reached. A watchpoint, however, specifies that program execution
26107
27669
26108
27670
@node Sample Debugging Session
26109
27671
@section Sample Debugging Session
27672
@cindex sample debugging session
26110
27673
26111
27674
In order to illustrate the use of @command{gawk} as a debugger, let's look at a sample
26112
27675
debugging session. We will use the @command{awk} implementation of the
26120
27683
26121
27684
@node Debugger Invocation
26122
27685
@subsection How to Start the Debugger
27686
@cindex starting the debugger
27687
@cindex debugger, how to start
26123
27688
26124
Starting the debugger is almost exactly like running @command{awk}, except you have to
26125
pass an additional option @option{--debug} or the corresponding short option @option{-D}.
26126
The file(s) containing the program and any supporting code are given on the command
26127
line as arguments to one or more @option{-f} options. (@command{gawk} is not designed
26128
to debug command-line programs, only programs contained in files.) In our case,
26129
we invoke the debugger like this:
27689
Starting the debugger is almost exactly like running @command{gawk},
27690
except you have to pass an additional option @option{--debug} or the
27691
corresponding short option @option{-D}. The file(s) containing the
27692
program and any supporting code are given on the command line as arguments
27693
to one or more @option{-f} options. (@command{gawk} is not designed
27694
to debug command-line programs, only programs contained in files.)
27695
In our case, we invoke the debugger like this:
26130
27696
26131
27697
@example
26132
27698
$ @kbd{gawk -D -f getopt.awk -f join.awk -f uniq.awk inputfile}
26259
27825
26260
27826
@noindent
26261
27827
So we can see that @code{are_equal()} was only called for the second record
26262
of the file. Of course, this is because our program contained a rule for
27828
of the file. Of course, this is because our program contains a rule for
26263
27829
@samp{NR == 1}:
26264
27830
26265
27831
@example
26291
27857
decides whether to give the lines the special ``field skipping'' treatment
26292
27858
indicated by the @option{-f} command-line option. (Notice that we skipped
26293
27859
from where we were before at line 64 to here, since the condition in line 64
26294
26295
@example
26296
if (fcount == 0 && charcount == 0)
26297
@end example
26298
26299
@noindent
26300
was false.)
27860
@samp{if (fcount == 0 && charcount == 0)} was false.)
26301
27861
26302
27862
Continuing to step, we now get to the splitting of the current and
26303
27863
last records:
26406
27966
The @command{gawk} debugger command set can be divided into the
26407
27967
following categories:
26408
27968
26409
@itemize @bullet{}
27969
@itemize @value{BULLET}
26410
27970
26411
27971
@item
26412
27972
Breakpoint control
26432
27992
show the abbreviation on a second description line.
26433
27993
A debugger command name may also be truncated if that partial
26434
27994
name is unambiguous. The debugger has the built-in capability to
26435
automatically repeat the previous command when just hitting @key{Enter}.
27995
automatically repeat the previous command just by hitting @key{Enter}.
26436
27996
This works for the commands @code{list}, @code{next}, @code{nexti}, @code{step}, @code{stepi}
26437
27997
and @code{continue} executed without any argument.
26438
27998
26459
28019
@cindex debugger commands, @code{break}
26460
28020
@cindex @code{break} debugger command
26461
28021
@cindex @code{b} debugger command (alias for @code{break})
28022
@cindex set breakpoint
28023
@cindex breakpoint, setting
26462
28024
@item @code{break} [[@var{filename}@code{:}]@var{n} | @var{function}] [@code{"@var{expression}"}]
26463
28025
@itemx @code{b} [[@var{filename}@code{:}]@var{n} | @var{function}] [@code{"@var{expression}"}]
26464
28026
Without any argument, set a breakpoint at the next instruction
26465
28027
to be executed in the selected stack frame.
26466
28028
Arguments can be one of the following:
26467
28029
28030
@c @asis for docbook
26468
28031
@c nested table
26469
@table @var
26470
@item n
28032
@table @asis
28033
@item @var{n}
26471
28034
Set a breakpoint at line number @var{n} in the current source file.
26472
28035
26473
@item filename@code{:}n
28036
@item @var{filename}@code{:}@var{n}
26474
28037
Set a breakpoint at line number @var{n} in source file @var{filename}.
26475
28038
26476
@item function
28039
@item @var{function}
26477
28040
Set a breakpoint at entry to (the first instruction of)
26478
28041
function @var{function}.
26479
28042
@end table
26489
28052
26490
28053
@cindex debugger commands, @code{clear}
26491
28054
@cindex @code{clear} debugger command
28055
@cindex delete breakpoint at location
28056
@cindex breakpoint at location, how to delete
26492
28057
@item @code{clear} [[@var{filename}@code{:}]@var{n} | @var{function}]
26493
28058
Without any argument, delete any breakpoint at the next instruction
26494
28059
to be executed in the selected stack frame. If the program stops at
26496
28061
does not stop at that location again. Arguments can be one of the following:
26497
28062
26498
28063
@c nested table
26499
@table @var
26500
@item n
28064
@table @asis
28065
@item @var{n}
26501
28066
Delete breakpoint(s) set at line number @var{n} in the current source file.
26502
28067
26503
@item filename@code{:}n
28068
@item @var{filename}@code{:}@var{n}
26504
28069
Delete breakpoint(s) set at line number @var{n} in source file @var{filename}.
26505
28070
26506
@item function
28071
@item @var{function}
26507
28072
Delete breakpoint(s) set at entry to function @var{function}.
26508
28073
@end table
26509
28074
26510
28075
@cindex debugger commands, @code{condition}
26511
28076
@cindex @code{condition} debugger command
28077
@cindex breakpoint condition
26512
28078
@item @code{condition} @var{n} @code{"@var{expression}"}
26513
28079
Add a condition to existing breakpoint or watchpoint @var{n}. The
26514
28080
condition is an @command{awk} expression that the debugger evaluates
26522
28088
@cindex debugger commands, @code{delete}
26523
28089
@cindex @code{delete} debugger command
26524
28090
@cindex @code{d} debugger command (alias for @code{delete})
28091
@cindex delete breakpoint by number
28092
@cindex breakpoint, delete by number
26525
28093
@item @code{delete} [@var{n1 n2} @dots{}] [@var{n}--@var{m}]
26526
28094
@itemx @code{d} [@var{n1 n2} @dots{}] [@var{n}--@var{m}]
26527
28095
Delete specified breakpoints or a range of breakpoints. Deletes
26529
28097
26530
28098
@cindex debugger commands, @code{disable}
26531
28099
@cindex @code{disable} debugger command
28100
@cindex disable breakpoint
28101
@cindex breakpoint, how to disable or enable
26532
28102
@item @code{disable} [@var{n1 n2} @dots{} | @var{n}--@var{m}]
26533
28103
Disable specified breakpoints or a range of breakpoints. Without
26534
28104
any argument, disables all breakpoints.
26537
28107
@cindex debugger commands, @code{enable}
26538
28108
@cindex @code{enable} debugger command
26539
28109
@cindex @code{e} debugger command (alias for @code{enable})
28110
@cindex enable breakpoint
26540
28111
@item @code{enable} [@code{del} | @code{once}] [@var{n1 n2} @dots{}] [@var{n}--@var{m}]
26541
28112
@itemx @code{e} [@code{del} | @code{once}] [@var{n1 n2} @dots{}] [@var{n}--@var{m}]
26542
28113
Enable specified breakpoints or a range of breakpoints. Without
26556
28127
26557
28128
@cindex debugger commands, @code{ignore}
26558
28129
@cindex @code{ignore} debugger command
28130
@cindex ignore breakpoint
26559
28131
@item @code{ignore} @var{n} @var{count}
26560
28132
Ignore breakpoint number @var{n} the next @var{count} times it is
26561
28133
hit.
26564
28136
@cindex debugger commands, @code{tbreak}
26565
28137
@cindex @code{tbreak} debugger command
26566
28138
@cindex @code{t} debugger command (alias for @code{tbreak})
28139
@cindex temporary breakpoint
26567
28140
@item @code{tbreak} [[@var{filename}@code{:}]@var{n} | @var{function}]
26568
28141
@itemx @code{t} [[@var{filename}@code{:}]@var{n} | @var{function}]
26569
28142
Set a temporary breakpoint (enabled for only one stop).
26584
28157
@cindex @code{silent} debugger command
26585
28158
@cindex debugger commands, @code{end}
26586
28159
@cindex @code{end} debugger command
28160
@cindex breakpoint commands
28161
@cindex commands to execute at breakpoint
26587
28162
@item @code{commands} [@var{n}]
26588
28163
@itemx @code{silent}
26589
28164
@itemx @dots{}
26611
28186
26612
28187
@cindex debugger commands, @code{c} (@code{continue})
26613
28188
@cindex debugger commands, @code{continue}
28189
@cindex continue program, in debugger
26614
28190
@item @code{continue} [@var{count}]
26615
28191
@itemx @code{c} [@var{count}]
26616
28192
Resume program execution. If continued from a breakpoint and @var{count} is
26627
28203
@cindex debugger commands, @code{next}
26628
28204
@cindex @code{next} debugger command
26629
28205
@cindex @code{n} debugger command (alias for @code{next})
28206
@cindex single-step execution, in the debugger
26630
28207
@item @code{next} [@var{count}]
26631
28208
@itemx @code{n} [@var{count}]
26632
28209
Continue execution to the next source line, stepping over function calls.
26721
28298
26722
28299
@cindex debugger commands, @code{eval}
26723
28300
@cindex @code{eval} debugger command
28301
@cindex evaluate expressions, in debugger
26724
28302
@item @code{eval "@var{awk statements}"}
26725
28303
Evaluate @var{awk statements} in the context of the running program.
26726
28304
You can do anything that an @command{awk} program would do: assign
26738
28316
@cindex debugger commands, @code{print}
26739
28317
@cindex @code{print} debugger command
26740
28318
@cindex @code{p} debugger command (alias for @code{print})
28319
@cindex print variables, in debugger
26741
28320
@item @code{print} @var{var1}[@code{,} @var{var2} @dots{}]
26742
28321
@itemx @code{p} @var{var1}[@code{,} @var{var2} @dots{}]
26743
28322
Print the value of a @command{gawk} variable or field.
26771
28350
26772
28351
@cindex debugger commands, @code{set}
26773
28352
@cindex @code{set} debugger command
28353
@cindex assign values to variables, in debugger
26774
28354
@item @code{set} @var{var}@code{=}@var{value}
26775
28355
Assign a constant (number or string) value to an @command{awk} variable
26776
28356
or field.
26777
String values must be enclosed between double quotes (@code{"@dots{}"}).
28357
String values must be enclosed between double quotes (@code{"}@dots{}@code{"}).
26778
28358
26779
28359
You can also set special @command{awk} variables, such as @code{FS},
26780
28360
@code{NF}, @code{NR}, etc.
26783
28363
@cindex debugger commands, @code{watch}
26784
28364
@cindex @code{watch} debugger command
26785
28365
@cindex @code{w} debugger command (alias for @code{watch})
28366
@cindex set watchpoint
26786
28367
@item @code{watch} @var{var} | @code{$}@var{n} [@code{"@var{expression}"}]
26787
28368
@itemx @code{w} @var{var} | @code{$}@var{n} [@code{"@var{expression}"}]
26788
28369
Add variable @var{var} (or field @code{$@var{n}}) to the watch list.
26799
28380
26800
28381
@cindex debugger commands, @code{undisplay}
26801
28382
@cindex @code{undisplay} debugger command
28383
@cindex stop automatic display, in debugger
26802
28384
@item @code{undisplay} [@var{n}]
26803
28385
Remove item number @var{n} (or all items, if no argument) from the
26804
28386
automatic display list.
26805
28387
26806
28388
@cindex debugger commands, @code{unwatch}
26807
28389
@cindex @code{unwatch} debugger command
28390
@cindex delete watchpoint
26808
28391
@item @code{unwatch} [@var{n}]
26809
28392
Remove item number @var{n} (or all items, if no argument) from the
26810
28393
watch list.
26825
28408
@cindex debugger commands, @code{backtrace}
26826
28409
@cindex @code{backtrace} debugger command
26827
28410
@cindex @code{bt} debugger command (alias for @code{backtrace})
28411
@cindex call stack, display in debugger
28412
@cindex traceback, display in debugger
26828
28413
@item @code{backtrace} [@var{count}]
26829
28414
@itemx @code{bt} [@var{count}]
26830
28415
Print a backtrace of all function calls (stack frames), or innermost @var{count}
26831
28416
frames if @var{count} > 0. Print the outermost @var{count} frames if
26832
28417
@var{count} < 0. The backtrace displays the name and arguments to each
26833
function, the source file name, and the line number.
28418
function, the source @value{FN}, and the line number.
26834
28419
26835
28420
@cindex debugger commands, @code{down}
26836
28421
@cindex @code{down} debugger command
26844
28429
@cindex @code{f} debugger command (alias for @code{frame})
26845
28430
@item @code{frame} [@var{n}]
26846
28431
@itemx @code{f} [@var{n}]
26847
Select and print (frame number, function and argument names, source file,
26848
and the source line) stack frame @var{n}. Frame 0 is the currently executing,
26849
or @dfn{innermost}, frame (function call), frame 1 is the frame that called the
26850
innermost one. The highest numbered frame is the one for the main program.
28432
Select and print stack frame @var{n}. Frame 0 is the currently executing,
28433
or @dfn{innermost}, frame (function call), frame 1 is the frame that
28434
called the innermost one. The highest numbered frame is the one for the
28435
main program. The printed information consists of the frame number,
28436
function and argument names, source file, and the source line.
26851
28437
26852
28438
@cindex debugger commands, @code{up}
26853
28439
@cindex @code{up} debugger command
26878
28464
@c nested table
26879
28465
@table @code
26880
28466
@item args
28467
@cindex show function arguments, in debugger
26881
28468
Arguments of the selected frame.
26882
28469
26883
28470
@item break
28471
@cindex show breakpoints
26884
28472
List all currently set breakpoints.
26885
28473
26886
28474
@item display
28475
@cindex automatic displays, in debugger
26887
28476
List all items in the automatic display list.
26888
28477
26889
28478
@item frame
28479
@cindex describe call stack frame, in debugger
26890
28480
Description of the selected stack frame.
26891
28481
26892
28482
@item functions
26893
List all function definitions including source file names and
28483
@cindex list function definitions, in debugger
28484
List all function definitions including source @value{FN}s and
26894
28485
line numbers.
26895
28486
26896
28487
@item locals
28488
@cindex show local variables, in debugger
26897
28489
Local variables of the selected frame.
26898
28490
26899
28491
@item source
28492
@cindex show name of current source file, in debugger
26900
28493
The name of the current source file. Each time the program stops, the
26901
28494
current source file is the file containing the current instruction.
26902
28495
When the debugger first starts, the current source file is the first file
26905
28498
be used at any time to change the current source.
26906
28499
26907
28500
@item sources
28501
@cindex show all source files, in debugger
26908
28502
List all program sources.
26909
28503
26910
28504
@item variables
28505
@cindex list all global variables, in debugger
26911
28506
List all global variables.
26912
28507
26913
28508
@item watch
28509
@cindex show watchpoints
26914
28510
List all items in the watch list.
26915
28511
@end table
26916
28512
@end table
26924
28520
@cindex debugger commands, @code{option}
26925
28521
@cindex @code{option} debugger command
26926
28522
@cindex @code{o} debugger command (alias for @code{option})
28523
@cindex display debugger options
28524
@cindex debugger options
26927
28525
@item @code{option} [@var{name}[@code{=}@var{value}]]
26928
28526
@itemx @code{o} [@var{name}[@code{=}@var{value}]]
26929
28527
Without an argument, display the available debugger options
26933
28531
The available options are:
26934
28532
26935
28533
@c nested table
26936
@table @code
26937
@item history_size
28534
@c asis for docbook
28535
@table @asis
28536
@item @code{history_size}
28537
@cindex debugger history size
26938
28538
The maximum number of lines to keep in the history file @file{./.gawk_history}.
26939
28539
The default is 100.
26940
28540
26941
@item listsize
28541
@item @code{listsize}
28542
@cindex debugger default list amount
26942
28543
The number of lines that @code{list} prints. The default is 15.
26943
28544
26944
@item outfile
28545
@item @code{outfile}
28546
@cindex redirect @command{gawk} output, in debugger
26945
28547
Send @command{gawk} output to a file; debugger output still goes
26946
28548
to standard output. An empty string (@code{""}) resets output to
26947
28549
standard output.
26948
28550
26949
@item prompt
28551
@item @code{prompt}
28552
@cindex debugger prompt
26950
28553
The debugger prompt. The default is @samp{@w{gawk> }}.
26951
28554
26952
@item save_history @r{[}on @r{|} off@r{]}
28555
@item @code{save_history} [@code{on} | @code{off}]
28556
@cindex debugger history file
26953
28557
Save command history to file @file{./.gawk_history}.
26954
28558
The default is @code{on}.
26955
28559
26956
@item save_options @r{[}on @r{|} off@r{]}
28560
@item @code{save_options} [@code{on} | @code{off}]
28561
@cindex save debugger options
26957
28562
Save current options to file @file{./.gawkrc} upon exit.
26958
28563
The default is @code{on}.
26959
28564
Options are read back in to the next session upon startup.
26960
28565
26961
@item trace @r{[}on @r{|} off@r{]}
28566
@item @code{trace} [@code{on} | @code{off}]
28567
@cindex instruction tracing, in debugger
26962
28568
Turn instruction tracing on or off. The default is @code{off}.
26963
28569
@end table
26964
28570
26965
28571
@item @code{save} @var{filename}
26966
Save the commands from the current session to the given file name,
28572
Save the commands from the current session to the given @value{FN},
26967
28573
so that they can be replayed using the @command{source} command.
26968
28574
26969
28575
@item @code{source} @var{filename}
28576
@cindex debugger, read commands from a file
26970
28577
Run command(s) from a file; an error in any command does not
26971
28578
terminate execution of subsequent commands. Comments (lines starting
26972
28579
with @samp{#}) are allowed in a command file.
27002
28609
27003
28610
@smallexample
27004
28611
gawk> @kbd{dump}
27005
@print{} # BEGIN
28612
@print{} # BEGIN
27006
28613
@print{}
27007
28614
@print{} [ 1:0xfcd340] Op_rule : [in_rule = BEGIN] [source_file = brini.awk]
27008
28615
@print{} [ 1:0xfcc240] Op_push_i : "~" [MALLOC|STRING|STRCUR]
27065
28672
@cindex debugger commands, @code{list}
27066
28673
@cindex @code{list} debugger command
27067
28674
@cindex @code{l} debugger command (alias for @code{list})
27068
@item @code{list} [@code{-} | @code{+} | @var{n} | @var{filename@code{:}n} | @var{n}--@var{m} | @var{function}]
27069
@itemx @code{l} [@code{-} | @code{+} | @var{n} | @var{filename@code{:}n} | @var{n}--@var{m} | @var{function}]
28675
@item @code{list} [@code{-} | @code{+} | @var{n} | @var{filename}@code{:}@var{n} | @var{n}--@var{m} | @var{function}]
28676
@itemx @code{l} [@code{-} | @code{+} | @var{n} | @var{filename}@code{:}@var{n} | @var{n}--@var{m} | @var{function}]
27070
28677
Print the specified lines (default 15) from the current source file
27071
28678
or the file named @var{filename}. The possible arguments to @code{list}
27072
28679
are as follows:
27086
28693
@item @var{n}--@var{m}
27087
28694
Print lines from @var{n} to @var{m}.
27088
28695
27089
@item @var{filename@code{:}n}
28696
@item @var{filename}@code{:}@var{n}
27090
28697
Print lines centered around line number @var{n} in
27091
28698
source file @var{filename}. This command may change the current source file.
27092
28699
27099
28706
@cindex debugger commands, @code{quit}
27100
28707
@cindex @code{quit} debugger command
27101
28708
@cindex @code{q} debugger command (alias for @code{quit})
28709
@cindex exit the debugger
27102
28710
@item @code{quit}
27103
28711
@itemx @code{q}
27104
28712
Exit the debugger. Debugging is great fun, but sometimes we all have
27109
28717
27110
28718
@cindex debugger commands, @code{trace}
27111
28719
@cindex @code{trace} debugger command
27112
@item @code{trace} @code{on} @r{|} @code{off}
28720
@item @code{trace} [@code{on} | @code{off}]
27113
28721
Turn on or off a continuous printing of instructions which are about to
27114
28722
be executed, along with printing the @command{awk} line which they
27115
28723
implement. The default is @code{off}.
27122
28730
27123
28731
@node Readline Support
27124
28732
@section Readline Support
28733
@cindex command completion, in debugger
28734
@cindex history expansion, in debugger
27125
28735
27126
If @command{gawk} is compiled with the @code{readline} library, you
27127
can take advantage of that library's command completion and history expansion
27128
features. The following types of completion are available:
28736
If @command{gawk} is compiled with
28737
@uref{http://cnswww.cns.cwru.edu/php/chet/readline/readline.html,
28738
the @code{readline} library}, you can take advantage of that library's
28739
command completion and history expansion features. The following types
28740
of completion are available:
27129
28741
27130
28742
@table @asis
27131
28743
@item Command completion
27132
28744
Command names.
27133
28745
27134
@item Source file name completion
27135
Source file names. Relevant commands are
28746
@item Source @value{FN} completion
28747
Source @value{FN}s. Relevant commands are
27136
28748
@code{break},
27137
28749
@code{clear},
27138
28750
@code{list},
27162
28774
but as with any program, especially in its early releases, it still has
27163
28775
some limitations. A few which are worth being aware of are:
27164
28776
27165
@itemize @bullet{}
28777
@itemize @value{BULLET}
27166
28778
@item
27167
28779
At this point, the debugger does not give a detailed explanation of
27168
28780
what you did wrong when you type in something it doesn't like. Rather, it just
27175
28787
you will realize that much of the internal manipulation of data
27176
28788
in @command{gawk}, as in many interpreters, is done on a stack.
27177
28789
@code{Op_push}, @code{Op_pop}, etc., are the ``bread and butter'' of
27178
most @command{gawk} code. Unfortunately, as of now, the @command{gawk}
28790
most @command{gawk} code.
28791
28792
Unfortunately, as of now, the @command{gawk}
27179
28793
debugger does not allow you to examine the stack's contents.
27180
27181
28794
That is, the intermediate results of expression evaluation are on the
27182
28795
stack, but cannot be printed. Rather, only variables which are defined
27183
28796
in the program can be printed. Of course, a workaround for
27204
28817
Look forward to a future release when these and other missing features may
27205
28818
be added, and of course feel free to try to add them yourself!
27206
28819
28820
@node Debugging Summary
28821
@section Summary
28822
28823
@itemize @value{BULLET}
28824
@item
28825
Programs rarely work correctly the first time. Finding bugs
28826
is @dfn{debugging} and a program that helps you find bugs is a
28827
@dfn{debugger}. @command{gawk} has a built-in debugger that works very
28828
similarly to the GNU Debugger, GDB.
28829
28830
@item
28831
Debuggers let you step through your program one statement at a time,
28832
examine and change variable and array values, and do a number of other
28833
things that let understand what your program is actually doing (as
28834
opposed to what it is supposed to do).
28835
28836
@item
28837
Like most debuggers, the @command{gawk} debugger works in terms of stack
28838
frames, and lets you set both breakpoints (stop at a point in the code)
28839
and watchpoints (stop when a data value changes).
28840
28841
@item
28842
The debugger command set is fairly complete, providing control over
28843
breakpoints, execution, viewing and changing data, working with the stack,
28844
getting information, and other tasks.
28845
28846
@item
28847
If the @code{readline} library is available when @command{gawk} is
28848
compiled, it is used by the debugger to provide command-line history
28849
and editing.
28850
28851
@end itemize
28852
27207
28853
@node Arbitrary Precision Arithmetic
27208
28854
@chapter Arithmetic and Arbitrary Precision Arithmetic with @command{gawk}
27209
28855
@cindex arbitrary precision
27210
28856
@cindex multiple precision
27211
28857
@cindex infinite precision
27212
@cindex floating-point numbers, arbitrary precision
27213
@cindex MPFR
27214
@cindex GMP
27215
27216
@cindex Knuth, Donald
27217
@quotation
27218
@i{There's a credibility gap: We don't know how much of the computer's answers
27219
to believe. Novice computer users solve this problem by implicitly trusting
27220
in the computer as an infallible authority; they tend to believe that all
27221
digits of a printed answer are significant. Disillusioned computer users have
27222
just the opposite approach; they are constantly afraid that their answers
27223
are almost meaningless.}@footnote{Donald E.@: Knuth.
27224
@cite{The Art of Computer Programming}. Volume 2,
27225
@cite{Seminumerical Algorithms}, third edition,
27226
1998, ISBN 0-201-89683-4, p.@: 229.}
27227
@author Donald Knuth
27228
@end quotation
27229
27230
This @value{CHAPTER} discusses issues that you may encounter
27231
when performing arithmetic. It begins by discussing some of
27232
the general attributes of computer arithmetic, along with how
27233
this can influence what you see when running @command{awk} programs.
27234
This discussion applies to all versions of @command{awk}.
27235
27236
The @value{CHAPTER} then moves on to describe @dfn{arbitrary precision
27237
arithmetic}, a feature which is specific to @command{gawk}.
28858
@cindex floating-point, numbers@comma{} arbitrary precision
28859
28860
This @value{CHAPTER} introduces some basic concepts relating to
28861
how computers do arithmetic and briefly lists the features in
28862
@command{gawk} for performing arbitrary precision floating point
28863
computations. It then proceeds to describe floating-point arithmetic,
28864
which is what @command{awk} uses for all its computations, including a
28865
discussion of arbitrary precision floating point arithmetic, which is
28866
a feature available only in @command{gawk}. It continues on to present
28867
arbitrary precision integers, and concludes with a description of some
28868
points where @command{gawk} and the POSIX standard are not quite in
28869
agreement.
27238
28870
27239
28871
@menu
27240
* General Arithmetic:: An introduction to computer arithmetic.
27241
* Floating-point Programming:: Effective Floating-point Programming.
27242
* Gawk and MPFR:: How @command{gawk} provides
27243
arbitrary-precision arithmetic.
27244
* Arbitrary Precision Floats:: Arbitrary Precision Floating-point Arithmetic
27245
with @command{gawk}.
27246
* Arbitrary Precision Integers:: Arbitrary Precision Integer Arithmetic with
27247
@command{gawk}.
28872
* Computer Arithmetic:: A quick intro to computer math.
28873
* Math Definitions:: Defining terms used.
28874
* MPFR features:: The MPFR features in @command{gawk}.
28875
* FP Math Caution:: Things to know.
28876
* Arbitrary Precision Integers:: Arbitrary Precision Integer Arithmetic with
28877
@command{gawk}.
28878
* POSIX Floating Point Problems:: Standards Versus Existing Practice.
28879
* Floating point summary:: Summary of floating point discussion.
27248
28880
@end menu
27249
28881
27250
@node General Arithmetic
28882
@node Computer Arithmetic
27251
28883
@section A General Description of Computer Arithmetic
27252
28884
27253
@cindex integers
27254
@cindex floating-point, numbers
27255
@cindex numbers, floating-point
27256
Within computers, there are two kinds of numeric values: @dfn{integers}
27257
and @dfn{floating-point}.
27258
In school, integer values were referred to as ``whole'' numbers---that is,
27259
numbers without any fractional part, such as 1, 42, or @minus{}17.
28885
Until now, we have worked with data as either numbers or
28886
strings. Ultimately, however, computers represent everything in terms
28887
of @dfn{binary digits}, or @dfn{bits}. A decimal digit can take on any
28888
of 10 values: zero through nine. A binary digit can take on any of two
28889
values, zero or one. Using binary, computers (and computer software)
28890
can represent and manipulate numerical and character data. In general,
28891
the more bits you can use to represent a particular thing, the greater
28892
the range of possible values it can take on.
28893
28894
Modern computers support at least two, and often more, ways to do
28895
arithmetic. Each kind of arithmetic uses a different representation
28896
(organization of the bits) for the numbers. The kinds of arithmetic
28897
that interest us are:
28898
28899
@table @asis
28900
@item Decimal arithmetic
28901
This is the kind of arithmetic you learned in elementary school, using
28902
paper and pencil (and/or a calculator). In theory, numbers can have an
28903
arbitrary number of digits on either side (or both sides) of the decimal
28904
point, and the results of a computation are always exact.
28905
28906
Some modern system can do decimal arithmetic in hardware, but usually you
28907
need a special software library to provide access to these instructions.
28908
There are also libraries that do decimal arithmetic entirely in software.
28909
28910
Despite the fact that some users expect @command{gawk} to be performing
28911
decimal arithmetic,@footnote{We don't know why they expect this, but
28912
they do.} it does not do so.
28913
28914
@item Integer arithmetic
28915
In school, integer values were referred to as ``whole'' numbers---that
28916
is, numbers without any fractional part, such as 1, 42, or @minus{}17.
27260
28917
The advantage to integer numbers is that they represent values exactly.
27261
The disadvantage is that their range is limited. On most systems,
27262
this range is @minus{}2,147,483,648 to 2,147,483,647.
27263
However, many systems now support a range from
27264
@minus{}9,223,372,036,854,775,808 to 9,223,372,036,854,775,807.
28918
The disadvantage is that their range is limited.
27265
28919
27266
28920
@cindex unsigned integers
27267
28921
@cindex integers, unsigned
27268
Integer values come in two flavors: @dfn{signed} and @dfn{unsigned}.
27269
Signed values may be negative or positive, with the range of values just
27270
described.
27271
Unsigned values are always positive. On most systems,
27272
the range is from 0 to 4,294,967,295.
27273
However, many systems now support a range from
27274
0 to 18,446,744,073,709,551,615.
27275
27276
@cindex double precision floating-point
27277
@cindex single precision floating-point
27278
Floating-point numbers represent what are called ``real'' numbers; i.e.,
27279
those that do have a fractional part, such as 3.1415927.
27280
The advantage to floating-point numbers is that they
27281
can represent a much larger range of values.
27282
The disadvantage is that there are numbers that they cannot represent
27283
exactly.
27284
@command{awk} uses @dfn{double precision} floating-point numbers, which
27285
can hold more digits than @dfn{single precision}
27286
floating-point numbers.
27287
@c Floating-point issues are discussed more fully in
27288
@c @ref{Floating Point Issues}.
27289
27290
There a several important issues to be aware of, described next.
27291
27292
@menu
27293
* Floating Point Issues:: Stuff to know about floating-point numbers.
27294
* Integer Programming:: Effective integer programming.
27295
@end menu
27296
27297
@node Floating Point Issues
27298
@subsection Floating-Point Number Caveats
27299
27300
This @value{SECTION} describes some of the issues
27301
involved in using floating-point numbers.
27302
27303
There is a very nice
27304
@uref{http://www.validlab.com/goldberg/paper.pdf, paper on floating-point arithmetic}
27305
by David Goldberg,
27306
``What Every Computer Scientist Should Know About Floating-point Arithmetic,''
27307
@cite{ACM Computing Surveys} @strong{23}, 1 (1991-03), 5-48.
27308
This is worth reading if you are interested in the details,
27309
but it does require a background in computer science.
27310
27311
@menu
27312
* String Conversion Precision:: The String Value Can Lie.
27313
* Unexpected Results:: Floating Point Numbers Are Not Abstract
27314
Numbers.
27315
* POSIX Floating Point Problems:: Standards Versus Existing Practice.
27316
@end menu
27317
27318
@node String Conversion Precision
27319
@subsubsection The String Value Can Lie
27320
27321
Internally, @command{awk} keeps both the numeric value
27322
(double precision floating-point) and the string value for a variable.
27323
Separately, @command{awk} keeps
27324
track of what type the variable has
27325
(@pxref{Typing and Comparison}),
27326
which plays a role in how variables are used in comparisons.
27327
27328
It is important to note that the string value for a number may not
27329
reflect the full value (all the digits) that the numeric value
27330
actually contains.
27331
The following program, @file{values.awk}, illustrates this:
27332
27333
@example
27334
@{
27335
sum = $1 + $2
27336
# see it for what it is
27337
printf("sum = %.12g\n", sum)
27338
# use CONVFMT
27339
a = "<" sum ">"
27340
print "a =", a
27341
# use OFMT
27342
print "sum =", sum
27343
@}
27344
@end example
27345
27346
@noindent
27347
This program shows the full value of the sum of @code{$1} and @code{$2}
27348
using @code{printf}, and then prints the string values obtained
27349
from both automatic conversion (via @code{CONVFMT}) and
27350
from printing (via @code{OFMT}).
27351
27352
Here is what happens when the program is run:
27353
27354
@example
27355
$ @kbd{echo 3.654321 1.2345678 | awk -f values.awk}
27356
@print{} sum = 4.8888888
27357
@print{} a = <4.88889>
27358
@print{} sum = 4.88889
27359
@end example
27360
27361
This makes it clear that the full numeric value is different from
27362
what the default string representations show.
27363
27364
@code{CONVFMT}'s default value is @code{"%.6g"}, which yields a value with
27365
at most six significant digits. For some applications, you might want to
27366
change it to specify more precision.
27367
On most modern machines, most of the time,
27368
17 digits is enough to capture a floating-point number's
27369
value exactly.@footnote{Pathological cases can require up to
27370
752 digits (!), but we doubt that you need to worry about this.}
27371
27372
@node Unexpected Results
27373
@subsubsection Floating Point Numbers Are Not Abstract Numbers
27374
27375
@cindex floating-point, numbers
27376
Unlike numbers in the abstract sense (such as what you studied in high school
27377
or college arithmetic), numbers stored in computers are limited in certain ways.
27378
They cannot represent an infinite number of digits, nor can they always
27379
represent things exactly.
27380
In particular,
27381
floating-point numbers cannot
27382
always represent values exactly. Here is an example:
27383
27384
@example
27385
$ @kbd{awk '@{ printf("%010d\n", $1 * 100) @}'}
27386
515.79
27387
@print{} 0000051579
27388
515.80
27389
@print{} 0000051579
27390
515.81
27391
@print{} 0000051580
27392
515.82
27393
@print{} 0000051582
27394
@kbd{Ctrl-d}
27395
@end example
27396
27397
@noindent
27398
This shows that some values can be represented exactly,
27399
whereas others are only approximated. This is not a ``bug''
27400
in @command{awk}, but simply an artifact of how computers
27401
represent numbers.
27402
27403
@quotation NOTE
27404
It cannot be emphasized enough that the behavior just
27405
described is fundamental to modern computers. You will
27406
see this kind of thing happen in @emph{any} programming
27407
language using hardware floating-point numbers. It is @emph{not}
27408
a bug in @command{gawk}, nor is it something that can be ``just
27409
fixed.''
27410
@end quotation
27411
27412
@cindex negative zero
27413
@cindex positive zero
27414
@cindex zero@comma{} negative vs.@: positive
27415
Another peculiarity of floating-point numbers on modern systems
27416
is that they often have more than one representation for the number zero!
27417
In particular, it is possible to represent ``minus zero'' as well as
27418
regular, or ``positive'' zero.
27419
27420
This example shows that negative and positive zero are distinct values
27421
when stored internally, but that they are in fact equal to each other,
27422
as well as to ``regular'' zero:
27423
27424
@example
27425
$ @kbd{gawk 'BEGIN @{ mz = -0 ; pz = 0}
27426
> @kbd{printf "-0 = %g, +0 = %g, (-0 == +0) -> %d\n", mz, pz, mz == pz}
27427
> @kbd{printf "mz == 0 -> %d, pz == 0 -> %d\n", mz == 0, pz == 0}
27428
> @kbd{@}'}
27429
@print{} -0 = -0, +0 = 0, (-0 == +0) -> 1
27430
@print{} mz == 0 -> 1, pz == 0 -> 1
27431
@end example
27432
27433
It helps to keep this in mind should you process numeric data
27434
that contains negative zero values; the fact that the zero is negative
27435
is noted and can affect comparisons.
27436
27437
@node POSIX Floating Point Problems
27438
@subsubsection Standards Versus Existing Practice
27439
27440
Historically, @command{awk} has converted any non-numeric looking string
27441
to the numeric value zero, when required. Furthermore, the original
27442
definition of the language and the original POSIX standards specified that
27443
@command{awk} only understands decimal numbers (base 10), and not octal
27444
(base 8) or hexadecimal numbers (base 16).
27445
27446
Changes in the language of the
27447
2001 and 2004 POSIX standards can be interpreted to imply that @command{awk}
27448
should support additional features. These features are:
27449
27450
@itemize @bullet
27451
@item
27452
Interpretation of floating point data values specified in hexadecimal
27453
notation (@samp{0xDEADBEEF}). (Note: data values, @emph{not}
27454
source code constants.)
27455
27456
@item
27457
Support for the special IEEE 754 floating point values ``Not A Number''
27458
(NaN), positive Infinity (``inf'') and negative Infinity (``@minus{}inf'').
27459
In particular, the format for these values is as specified by the ISO 1999
27460
C standard, which ignores case and can allow machine-dependent additional
27461
characters after the @samp{nan} and allow either @samp{inf} or @samp{infinity}.
27462
@end itemize
27463
27464
The first problem is that both of these are clear changes to historical
27465
practice:
27466
27467
@itemize @bullet
27468
@item
27469
The @command{gawk} maintainer feels that supporting hexadecimal floating
27470
point values, in particular, is ugly, and was never intended by the
27471
original designers to be part of the language.
27472
27473
@item
27474
Allowing completely alphabetic strings to have valid numeric
27475
values is also a very severe departure from historical practice.
27476
@end itemize
27477
27478
The second problem is that the @code{gawk} maintainer feels that this
27479
interpretation of the standard, which requires a certain amount of
27480
``language lawyering'' to arrive at in the first place, was not even
27481
intended by the standard developers. In other words, ``we see how you
27482
got where you are, but we don't think that that's where you want to be.''
27483
27484
Recognizing the above issues, but attempting to provide compatibility
27485
with the earlier versions of the standard,
27486
the 2008 POSIX standard added explicit wording to allow, but not require,
27487
that @command{awk} support hexadecimal floating point values and
27488
special values for ``Not A Number'' and infinity.
27489
27490
Although the @command{gawk} maintainer continues to feel that
27491
providing those features is inadvisable,
27492
nevertheless, on systems that support IEEE floating point, it seems
27493
reasonable to provide @emph{some} way to support NaN and Infinity values.
27494
The solution implemented in @command{gawk} is as follows:
27495
27496
@itemize @bullet
27497
@item
27498
With the @option{--posix} command-line option, @command{gawk} becomes
27499
``hands off.'' String values are passed directly to the system library's
27500
@code{strtod()} function, and if it successfully returns a numeric value,
27501
that is what's used.@footnote{You asked for it, you got it.}
27502
By definition, the results are not portable across
27503
different systems. They are also a little surprising:
27504
27505
@example
27506
$ @kbd{echo nanny | gawk --posix '@{ print $1 + 0 @}'}
27507
@print{} nan
27508
$ @kbd{echo 0xDeadBeef | gawk --posix '@{ print $1 + 0 @}'}
27509
@print{} 3735928559
27510
@end example
27511
27512
@item
27513
Without @option{--posix}, @command{gawk} interprets the four strings
27514
@samp{+inf},
27515
@samp{-inf},
27516
@samp{+nan},
27517
and
27518
@samp{-nan}
27519
specially, producing the corresponding special numeric values.
27520
The leading sign acts a signal to @command{gawk} (and the user)
27521
that the value is really numeric. Hexadecimal floating point is
27522
not supported (unless you also use @option{--non-decimal-data},
27523
which is @emph{not} recommended). For example:
27524
27525
@example
27526
$ @kbd{echo nanny | gawk '@{ print $1 + 0 @}'}
27527
@print{} 0
27528
$ @kbd{echo +nan | gawk '@{ print $1 + 0 @}'}
27529
@print{} nan
27530
$ @kbd{echo 0xDeadBeef | gawk '@{ print $1 + 0 @}'}
27531
@print{} 0
27532
@end example
27533
27534
@command{gawk} does ignore case in the four special values.
27535
Thus @samp{+nan} and @samp{+NaN} are the same.
27536
@end itemize
27537
27538
@node Integer Programming
27539
@subsection Mixing Integers And Floating-point
27540
27541
As has been mentioned already, @command{awk} uses hardware double
27542
precision with 64-bit IEEE binary floating-point representation
27543
for numbers on most systems. A large integer like 9,007,199,254,740,997
27544
has a binary representation that, although finite, is more than 53 bits long;
27545
it must also be rounded to 53 bits.
27546
The biggest integer that can be stored in a C @code{double} is usually the same
27547
as the largest possible value of a @code{double}. If your system @code{double}
27548
is an IEEE 64-bit @code{double}, this largest possible value is an integer and
27549
can be represented precisely. What more should one know about integers?
27550
27551
If you want to know what is the largest integer, such that it and
27552
all smaller integers can be stored in 64-bit doubles without losing precision,
27553
then the answer is
27554
@iftex
27555
@math{2^{53}}.
27556
@end iftex
27557
@ifnottex
27558
2^53.
27559
@end ifnottex
27560
The next representable number is the even number
27561
@iftex
27562
@math{2^{53} + 2},
27563
@end iftex
27564
@ifnottex
27565
2^53 + 2,
27566
@end ifnottex
27567
meaning it is unlikely that you will be able to make
27568
@command{gawk} print
27569
@iftex
27570
@math{2^{53} + 1}
27571
@end iftex
27572
@ifnottex
27573
2^53 + 1
27574
@end ifnottex
27575
in integer format.
27576
The range of integers exactly representable by a 64-bit double
27577
is
27578
@iftex
27579
@math{[-2^{53}, 2^{53}]}.
27580
@end iftex
27581
@ifnottex
27582
[@minus{}2^53, 2^53].
27583
@end ifnottex
27584
If you ever see an integer outside this range in @command{awk}
27585
using 64-bit doubles, you have reason to be very suspicious about
27586
the accuracy of the output. Here is a simple program with erroneous output:
27587
27588
@example
27589
$ @kbd{gawk 'BEGIN @{ i = 2^53 - 1; for (j = 0; j < 4; j++) print i + j @}'}
27590
@print{} 9007199254740991
27591
@print{} 9007199254740992
27592
@print{} 9007199254740992
27593
@print{} 9007199254740994
27594
@end example
27595
27596
The lesson is to not assume that any large integer printed by @command{awk}
27597
represents an exact result from your computation, especially if it wraps
27598
around on your screen.
27599
27600
@node Floating-point Programming
27601
@section Understanding Floating-point Programming
27602
27603
Numerical programming is an extensive area; if you need to develop
27604
sophisticated numerical algorithms then @command{gawk} may not be
27605
the ideal tool, and this documentation may not be sufficient.
27606
It might require digesting a book or two@footnote{One recommended title is
27607
@cite{Numerical Computing with IEEE Floating Point Arithmetic}, Michael L.@:
27608
Overton, Society for Industrial and Applied Mathematics, 2004.
27609
ISBN: 0-89871-482-6, ISBN-13: 978-0-89871-482-1. See
27610
@uref{http://www.cs.nyu.edu/cs/faculty/overton/book}.}
27611
to really internalize how to compute
27612
with ideal accuracy and precision,
27613
and the result often depends on the particular application.
27614
27615
@quotation NOTE
27616
A floating-point calculation's @dfn{accuracy} is how close it comes
27617
to the real value. This is as opposed to the @dfn{precision}, which
27618
usually refers to the number of bits used to represent the number
27619
(see @uref{http://en.wikipedia.org/wiki/Accuracy_and_precision,
27620
the Wikipedia article} for more information).
27621
@end quotation
27622
27623
There are two options for doing floating-point calculations:
27624
hardware floating-point (as used by standard @command{awk} and
27625
the default for @command{gawk}), and @dfn{arbitrary-precision}
27626
floating-point, which is software based.
27627
From this point forward, this @value{CHAPTER}
27628
aims to provide enough information to understand both, and then
27629
will focus on @command{gawk}'s facilities for the latter.@footnote{If you
27630
are interested in other tools that perform arbitrary precision arithmetic,
27631
you may want to investigate the POSIX @command{bc} tool. See
27632
@uref{http://pubs.opengroup.org/onlinepubs/009695399/utilities/bc.html,
27633
the POSIX specification for it}, for more information.}
28922
In computers, integer values come in two flavors: @dfn{signed} and
28923
@dfn{unsigned}. Signed values may be negative or positive, whereas
28924
unsigned values are always positive (that is, greater than or equal
28925
to zero).
28926
28927
In computer systems, integer arithmetic is exact, but the possible
28928
range of values is limited. Integer arithmetic is generally faster than
28929
floating point arithmetic.
28930
28931
@item Floating point arithmetic
28932
Floating-point numbers represent what were called in school ``real''
28933
numbers; i.e., those that have a fractional part, such as 3.1415927.
28934
The advantage to floating-point numbers is that they can represent a
28935
much larger range of values than can integers. The disadvantage is that
28936
there are numbers that they cannot represent exactly.
28937
28938
Modern systems support floating point arithmetic in hardware, with a
28939
limited range of values. There are software libraries that allow
28940
the use of arbitrary precision floating point calculations.
28941
28942
POSIX @command{awk} uses @dfn{double precision} floating-point numbers, which
28943
can hold more digits than @dfn{single precision} floating-point numbers.
28944
@command{gawk} has facilities for performing arbitrary precision floating
28945
point arithmetic, which we describe in more detail shortly.
28946
@end table
28947
28948
Computers work with integer and floating point values of different
28949
ranges. Integer values are usually either 32 or 64 bits in size. Single
28950
precision floating point values occupy 32 bits, whereas double precision
28951
floating point values occupy 64 bits. Floating point values are always
28952
signed. The possible ranges of values are shown in the following table.
28953
28954
@multitable @columnfractions .34 .33 .33
28955
@headitem Numeric representation @tab Miniumum value @tab Maximum value
28956
@item 32-bit signed integer @tab @minus{}2,147,483,648 @tab 2,147,483,647
28957
@item 32-bit unsigned integer @tab 0 @tab 4,294,967,295
28958
@item 64-bit signed integer @tab @minus{}9,223,372,036,854,775,808 @tab 9,223,372,036,854,775,807
28959
@item 64-bit unsigned integer @tab 0 @tab 18,446,744,073,709,551,615
28960
@item Single precision floating point (approximate) @tab @code{1.175494e-38} @tab @code{3.402823e+38}
28961
@item Double precision floating point (approximate) @tab @code{2.225074e-308} @tab @code{1.797693e+308}
28962
@end multitable
28963
28964
@node Math Definitions
28965
@section Other Stuff To Know
28966
28967
The rest of this @value{CHAPTER} uses a number of terms. Here are some
28968
informal definitions that should help you work your way through the material
28969
here.
28970
28971
@table @dfn
28972
@item Accuracy
28973
A floating-point calculation's accuracy is how close it comes
28974
to the real (paper and pencil) value.
28975
28976
@item Error
28977
The difference between what the result of a computation ``should be''
28978
and what it actually is. It is best to minimize error as much
28979
as possible.
28980
28981
@item Exponent
28982
The order of magnitude of a value;
28983
some number of bits in a floating-point value store the exponent.
28984
28985
@item Inf
28986
A special value representing infinity. Operations involving another
28987
number and infinity produce infinity.
28988
28989
@item NaN
28990
``Not A Number.'' A special value indicating a result that can't
28991
happen in real math, but that can happen in floating-point computations.
28992
28993
@item Normalized
28994
How the significand (see later in this list) is usually stored. The
28995
value is adjusted so that the first bit is one, and then that leading
28996
one is assumed instead of physically stored. This provides one
28997
extra bit of precision.
28998
28999
@item Precision
29000
The number of bits used to represent a floating-point number.
29001
The more bits, the more digits you can represent.
29002
Binary and decimal precisions are related approximately, according to the
29003
formula:
29004
29005
@display
29006
@iftex
29007
@math{prec = 3.322 @cdot dps}
29008
@end iftex
29009
@ifnottex
29010
@ifnotdocbook
29011
@var{prec} = 3.322 * @var{dps}
29012
@end ifnotdocbook
29013
@end ifnottex
29014
@docbook
29015
<emphasis>prec</emphasis> = 3.322 ⋅ <emphasis>dps</emphasis> @c
29016
@end docbook
29017
@end display
29018
29019
@noindent
29020
Here, @var{prec} denotes the binary precision
29021
(measured in bits) and @var{dps} (short for decimal places)
29022
is the decimal digits.
29023
29024
@item Rounding mode
29025
How numbers are rounded up or down when necessary.
29026
More details are provided later.
29027
29028
@item Significand
29029
A floating point value consists the significand multiplied by 10
29030
to the power of the exponent. For example, in @code{1.2345e67},
29031
the significand is @code{1.2345}.
29032
29033
@item Stability
29034
From @uref{http://en.wikipedia.org/wiki/Numerical_stability,
29035
the Wikipedia article on numerical stability}:
29036
``Calculations that can be proven not to magnify approximation errors
29037
are called @dfn{numerically stable}.''
29038
@end table
29039
29040
See @uref{http://en.wikipedia.org/wiki/Accuracy_and_precision,
29041
the Wikipedia article on accuracy and precision} for more information
29042
on some of those terms.
29043
29044
On modern systems, floating-point hardware uses the representation and
29045
operations defined by the IEEE 754 standard.
29046
Three of the standard IEEE 754 types are 32-bit single precision,
29047
64-bit double precision and 128-bit quadruple precision.
29048
The standard also specifies extended precision formats
29049
to allow greater precisions and larger exponent ranges.
29050
(@command{awk} uses only the 64-bit double precision format.)
29051
29052
@ref{table-ieee-formats} lists the precision and exponent
29053
field values for the basic IEEE 754 binary formats:
29054
29055
@float Table,table-ieee-formats
29056
@caption{Basic IEEE Format Context Values}
29057
@multitable @columnfractions .20 .20 .20 .20 .20
29058
@headitem Name @tab Total bits @tab Precision @tab emin @tab emax
29059
@item Single @tab 32 @tab 24 @tab @minus{}126 @tab +127
29060
@item Double @tab 64 @tab 53 @tab @minus{}1022 @tab +1023
29061
@item Quadruple @tab 128 @tab 113 @tab @minus{}16382 @tab +16383
29062
@end multitable
29063
@end float
29064
29065
@quotation NOTE
29066
The precision numbers include the implied leading one that gives them
29067
one extra bit of significand.
29068
@end quotation
29069
29070
@node MPFR features
29071
@section Arbitrary Precison Arithmetic Features In @command{gawk}
29072
29073
By default, @command{gawk} uses the double precision floating point values
29074
supplied by the hardware of the system it runs on. However, if it was
29075
compiled to do, @command{gawk} uses the @uref{http://www.mpfr.org, GNU
29076
MPFR} and @uref{http://gmplib.org, GNU MP} (GMP) libraries for arbitrary
29077
precision arithmetic on numbers. You can see if MPFR support is available
29078
like so:
29079
29080
@example
29081
$ @kbd{gawk --version}
29082
@print{} GNU Awk 4.1.1, API: 1.1 (GNU MPFR 3.1.0-p3, GNU MP 5.0.2)
29083
@print{} Copyright (C) 1989, 1991-2014 Free Software Foundation.
29084
@dots{}
29085
@end example
29086
29087
@noindent
29088
(You may see different version numbers than what's shown here. That's OK;
29089
what's important is to see that GNU MPFR and GNU MP are listed in
29090
the output.)
29091
29092
Additionally, there are a few elements available in the @code{PROCINFO}
29093
array to provide information about the MPFR and GMP libraries
29094
(@pxref{Auto-set}).
29095
29096
The MPFR library provides precise control over precisions and rounding
29097
modes, and gives correctly rounded, reproducible, platform-independent
29098
results. With either of the command-line options @option{--bignum} or
29099
@option{-M}, all floating-point arithmetic operators and numeric functions
29100
can yield results to any desired precision level supported by MPFR.
29101
29102
Two built-in variables, @code{PREC} and @code{ROUNDMODE},
29103
provide control over the working precision and the rounding mode.
29104
The precision and the rounding mode are set globally for every operation
29105
to follow.
29106
@xref{Auto-set}, for more information.
29107
29108
@node FP Math Caution
29109
@section Floating Point Arithmetic: Caveat Emptor!
29110
29111
@quotation
29112
Math class is tough!
29113
@author Late 1980's Barbie
29114
@end quotation
29115
29116
This @value{SECTION} provides a high level overview of the issues
29117
involved when doing lots of floating-point arithmetic.@footnote{There
29118
is a very nice @uref{http://www.validlab.com/goldberg/paper.pdf,
29119
paper on floating-point arithmetic} by David Goldberg, ``What Every
29120
Computer Scientist Should Know About Floating-point Arithmetic,''
29121
@cite{ACM Computing Surveys} @strong{23}, 1 (1991-03), 5-48. This is
29122
worth reading if you are interested in the details, but it does require
29123
a background in computer science.}
29124
The discussion applies to both hardware and arbitrary-precision
29125
floating-point arithmetic.
29126
29127
@quotation CAUTION
29128
The material here is purposely general. If you need to do serious
29129
computer arithmetic, you should do some research first, and not
29130
rely just on what we tell you.
29131
@end quotation
29132
29133
@menu
29134
* Inexactness of computations:: Floating point math is not exact.
29135
* Getting Accuracy:: Getting more accuracy takes some work.
29136
* Try To Round:: Add digits and round.
29137
* Setting precision:: How to set the precision.
29138
* Setting the rounding mode:: How to set the rounding mode.
29139
@end menu
29140
29141
@node Inexactness of computations
29142
@subsection Floating Point Arithmetic Is Not Exact
27634
29143
27635
29144
Binary floating-point representations and arithmetic are inexact.
27636
29145
Simple values like 0.1 cannot be precisely represented using
27642
29151
but then you cannot be sure of the number of significant decimal places
27643
29152
in the final result.
27644
29153
27645
Sometimes, before you start to write any code, you should think more
29154
@menu
29155
* Inexact representation:: Numbers are not exactly represented.
29156
* Comparing FP Values:: How to compare floating point values.
29157
* Errors accumulate:: Errors get bigger as they go.
29158
@end menu
29159
29160
@node Inexact representation
29161
@subsubsection Many Numbers Cannot Be Represented Exactly
29162
29163
So, before you start to write any code, you should think
27646
29164
about what you really want and what's really happening. Consider the
27647
29165
two numbers in the following example:
27648
29166
27672
29190
Usually this is a format string like @code{"%.15g"}, which when
27673
29191
used in the previous example, produces an output identical to the input.
27674
29192
29193
@node Comparing FP Values
29194
@subsubsection Be Careful Comparing Values
29195
27675
29196
Because the underlying representation can be a little bit off from the exact value,
27676
comparing floating-point values to see if they are equal is generally not a good idea.
27677
Here is an example where it does not work like you expect:
29197
comparing floating-point values to see if they are exactly equal is generally a bad idea.
29198
Here is an example where it does not work like you would expect:
27678
29199
27679
29200
@example
27680
29201
$ @kbd{gawk 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'}
27681
29202
@print{} 0
27682
29203
@end example
27683
29204
27684
The loss of accuracy during a single computation with floating-point numbers
27685
usually isn't enough to worry about. However, if you compute a value
27686
which is the result of a sequence of floating point operations,
29205
The general wisdom when comparing floating-point values is to see if
29206
they are within some small range of each other (called a @dfn{delta},
29207
or @dfn{tolerance}).
29208
You have to decide how small a delta is important to you. Code to do
29209
this looks something like this:
29210
29211
@example
29212
delta = 0.00001 # for example
29213
difference = abs(a) - abs(b) # subtract the two values
29214
if (difference < delta)
29215
# all ok
29216
else
29217
# not ok
29218
@end example
29219
29220
@node Errors accumulate
29221
@subsubsection Errors Accumulate
29222
29223
The loss of accuracy during a single computation with floating-point
29224
numbers usually isn't enough to worry about. However, if you compute a
29225
value which is the result of a sequence of floating point operations,
27687
29226
the error can accumulate and greatly affect the computation itself.
27688
Here is an attempt to compute the value of the constant
27689
@value{PI} using one of its many series representations:
29227
Here is an attempt to compute the value of @value{PI} using one of its
29228
many series representations:
27690
29229
27691
29230
@example
27692
29231
BEGIN @{
27700
29239
@}
27701
29240
@end example
27702
29241
27703
When run, the early errors propagating through later computations
27704
cause the loop to terminate prematurely after an attempt to divide by zero.
29242
When run, the early errors propagate through later computations,
29243
causing the loop to terminate prematurely after attempting to divide by zero:
27705
29244
27706
29245
@example
27707
29246
$ @kbd{gawk -f pi.awk}
27728
29267
@print{} 4
27729
29268
@end example
27730
29269
27731
Can computation using arbitrary precision help with the previous examples?
27732
If you are impatient to know, see
27733
@ref{Exact Arithmetic}.
29270
@node Getting Accuracy
29271
@subsection Getting The Accuracy You Need
29272
29273
Can arbitrary precision arithmetic give exact results? There are
29274
no easy answers. The standard rules of algebra often do not apply
29275
when using floating-point arithmetic.
29276
Among other things, the distributive and associative laws
29277
do not hold completely, and order of operation may be important
29278
for your computation. Rounding error, cumulative precision loss
29279
and underflow are often troublesome.
29280
29281
When @command{gawk} tests the expressions @samp{0.1 + 12.2} and
29282
@samp{12.3} for equality using the machine double precision arithmetic,
29283
it decides that they are not equal! (@xref{Comparing FP Values}.)
29284
You can get the result you want by increasing the precision; 56 bits in
29285
this case does the job:
29286
29287
@example
29288
$ @kbd{gawk -M -v PREC=56 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'}
29289
@print{} 1
29290
@end example
29291
29292
If adding more bits is good, perhaps adding even more bits of
29293
precision is better?
29294
Here is what happens if we use an even larger value of @code{PREC}:
29295
29296
@example
29297
$ @kbd{gawk -M -v PREC=201 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'}
29298
@print{} 0
29299
@end example
29300
29301
This is not a bug in @command{gawk} or in the MPFR library.
29302
It is easy to forget that the finite number of bits used to store the value
29303
is often just an approximation after proper rounding.
29304
The test for equality succeeds if and only if @emph{all} bits in the two operands
29305
are exactly the same. Since this is not necessarily true after floating-point
29306
computations with a particular precision and effective rounding rule,
29307
a straight test for equality may not work. Instead, compare the
29308
two numbers to see if they are within the desirable delta of each other.
29309
29310
In applications where 15 or fewer decimal places suffice,
29311
hardware double precision arithmetic can be adequate, and is usually much faster.
29312
But you need to keep in mind that every floating-point operation
29313
can suffer a new rounding error with catastrophic consequences as illustrated
29314
by our earlier attempt to compute the value of @value{PI}.
29315
Extra precision can greatly enhance the stability and the accuracy
29316
of your computation in such cases.
29317
29318
Repeated addition is not necessarily equivalent to multiplication
29319
in floating-point arithmetic. In the example in
29320
@ref{Errors accumulate}:
29321
29322
@example
29323
$ @kbd{gawk 'BEGIN @{}
29324
> @kbd{for (d = 1.1; d <= 1.5; d += 0.1) # loop five times (?)}
29325
> @kbd{i++}
29326
> @kbd{print i}
29327
> @kbd{@}'}
29328
@print{} 4
29329
@end example
29330
29331
@noindent
29332
you may or may not succeed in getting the correct result by choosing
29333
an arbitrarily large value for @code{PREC}. Reformulation of
29334
the problem at hand is often the correct approach in such situations.
29335
29336
@node Try To Round
29337
@subsection Try A Few Extra Bits of Precision and Rounding
27734
29338
27735
29339
Instead of arbitrary precision floating-point arithmetic,
27736
29340
often all you need is an adjustment of your logic
27737
29341
or a different order for the operations in your calculation.
27738
The stability and the accuracy of the computation of the constant @value{PI}
29342
The stability and the accuracy of the computation of @value{PI}
27739
29343
in the earlier example can be enhanced by using the following
27740
29344
simple algebraic transformation:
27741
29345
27742
29346
@example
27743
(sqrt(x * x + 1) - 1) / x = x / (sqrt(x * x + 1) + 1)
29347
(sqrt(x * x + 1) - 1) / x @equiv{} x / (sqrt(x * x + 1) + 1)
27744
29348
@end example
27745
29349
27746
29350
@noindent
27747
After making this, change the program does converge to
29351
After making this, change the program converges to
27748
29352
@value{PI} in under 30 iterations:
27749
29353
27750
29354
@example
27759
29363
@print{} 3.141592653589797
27760
29364
@end example
27761
29365
27762
There is no need to be unduly suspicious about the results from
27763
floating-point arithmetic. The lesson to remember is that
27764
floating-point arithmetic is always more complex than arithmetic using
27765
pencil and paper. In order to take advantage of the power
27766
of computer floating-point, you need to know its limitations
27767
and work within them. For most casual use of floating-point arithmetic,
27768
you will often get the expected result in the end if you simply round
27769
the display of your final results to the correct number of significant
27770
decimal digits.
27771
27772
As general advice, avoid presenting numerical data in a manner that
27773
implies better precision than is actually the case.
27774
27775
@menu
27776
* Floating-point Representation:: Binary floating-point representation.
27777
* Floating-point Context:: Floating-point context.
27778
* Rounding Mode:: Floating-point rounding mode.
27779
@end menu
27780
27781
@node Floating-point Representation
27782
@subsection Binary Floating-point Representation
27783
@cindex IEEE-754 format
27784
27785
Although floating-point representations vary from machine to machine,
27786
the most commonly encountered representation is that defined by the
27787
IEEE 754 Standard. An IEEE-754 format value has three components:
27788
27789
@itemize @bullet
27790
@item
27791
A sign bit telling whether the number is positive or negative.
27792
27793
@item
27794
An @dfn{exponent}, @var{e}, giving its order of magnitude.
27795
27796
@item
27797
A @dfn{significand}, @var{s},
27798
specifying the actual digits of the number.
27799
@end itemize
27800
27801
The value of the
27802
number is then
27803
@iftex
27804
@math{s @cdot 2^e}.
27805
@end iftex
27806
@ifnottex
27807
@var{s * 2^e}.
27808
@end ifnottex
27809
The first bit of a non-zero binary significand
27810
is always one, so the significand in an IEEE-754 format only includes the
27811
fractional part, leaving the leading one implicit.
27812
The significand is stored in @dfn{normalized} format,
27813
which means that the first bit is always a one.
27814
27815
Three of the standard IEEE-754 types are 32-bit single precision,
27816
64-bit double precision and 128-bit quadruple precision.
27817
The standard also specifies extended precision formats
27818
to allow greater precisions and larger exponent ranges.
27819
27820
@node Floating-point Context
27821
@subsection Floating-point Context
27822
@cindex context, floating-point
27823
27824
A floating-point @dfn{context} defines the environment for arithmetic operations.
27825
It governs precision, sets rules for rounding, and limits the range for exponents.
27826
The context has the following primary components:
27827
27828
@table @dfn
27829
@item Precision
27830
Precision of the floating-point format in bits.
27831
27832
@item emax
27833
Maximum exponent allowed for the format.
27834
27835
@item emin
27836
Minimum exponent allowed for the format.
27837
27838
@item Underflow behavior
27839
The format may or may not support gradual underflow.
27840
27841
@item Rounding
27842
The rounding mode of the context.
27843
@end table
27844
27845
@ref{table-ieee-formats} lists the precision and exponent
27846
field values for the basic IEEE-754 binary formats:
27847
27848
@float Table,table-ieee-formats
27849
@caption{Basic IEEE Format Context Values}
27850
@multitable @columnfractions .20 .20 .20 .20 .20
27851
@headitem Name @tab Total bits @tab Precision @tab emin @tab emax
27852
@item Single @tab 32 @tab 24 @tab @minus{}126 @tab +127
27853
@item Double @tab 64 @tab 53 @tab @minus{}1022 @tab +1023
27854
@item Quadruple @tab 128 @tab 113 @tab @minus{}16382 @tab +16383
27855
@end multitable
27856
@end float
27857
27858
@quotation NOTE
27859
The precision numbers include the implied leading one that gives them
27860
one extra bit of significand.
27861
@end quotation
27862
27863
A floating-point context can also determine which signals are treated
27864
as exceptions, and can set rules for arithmetic with special values.
27865
Please consult the IEEE-754 standard or other resources for details.
27866
27867
@command{gawk} ordinarily uses the hardware double precision
27868
representation for numbers. On most systems, this is IEEE-754
27869
floating-point format, corresponding to 64-bit binary with 53 bits
27870
of precision.
27871
27872
@quotation NOTE
27873
In case an underflow occurs, the standard allows, but does not require,
27874
the result from an arithmetic operation to be a number smaller than
27875
the smallest nonzero normalized number. Such numbers do
27876
not have as many significant digits as normal numbers, and are called
27877
@dfn{denormals} or @dfn{subnormals}. The alternative, simply returning a zero,
27878
is called @dfn{flush to zero}. The basic IEEE-754 binary formats
27879
support subnormal numbers.
27880
@end quotation
27881
27882
@node Rounding Mode
27883
@subsection Floating-point Rounding Mode
27884
@cindex rounding mode, floating-point
27885
27886
The @dfn{rounding mode} specifies the behavior for the results of numerical
27887
operations when discarding extra precision. Each rounding mode indicates
27888
how the least significant returned digit of a rounded result is to
27889
be calculated.
27890
@ref{table-rounding-modes} lists the IEEE-754 defined
27891
rounding modes:
27892
27893
@float Table,table-rounding-modes
27894
@caption{IEEE 754 Rounding Modes}
27895
@multitable @columnfractions .45 .55
27896
@headitem Rounding Mode @tab IEEE Name
27897
@item Round to nearest, ties to even @tab @code{roundTiesToEven}
27898
@item Round toward plus Infinity @tab @code{roundTowardPositive}
27899
@item Round toward negative Infinity @tab @code{roundTowardNegative}
27900
@item Round toward zero @tab @code{roundTowardZero}
27901
@item Round to nearest, ties away from zero @tab @code{roundTiesToAway}
27902
@end multitable
27903
@end float
27904
27905
The default mode @code{roundTiesToEven} is the most preferred,
27906
but the least intuitive. This method does the obvious thing for most values,
27907
by rounding them up or down to the nearest digit.
27908
For example, rounding 1.132 to two digits yields 1.13,
27909
and rounding 1.157 yields 1.16.
27910
27911
However, when it comes to rounding a value that is exactly halfway between,
27912
things do not work the way you probably learned in school.
27913
In this case, the number is rounded to the nearest even digit.
27914
So rounding 0.125 to two digits rounds down to 0.12,
27915
but rounding 0.6875 to three digits rounds up to 0.688.
27916
You probably have already encountered this rounding mode when
27917
using @code{printf} to format floating-point numbers.
27918
For example:
27919
27920
@example
27921
BEGIN @{
27922
x = -4.5
27923
for (i = 1; i < 10; i++) @{
27924
x += 1.0
27925
printf("%4.1f => %2.0f\n", x, x)
27926
@}
27927
@}
27928
@end example
27929
27930
@noindent
27931
produces the following output when run on the author's system:@footnote{It
27932
is possible for the output to be completely different if the
27933
C library in your system does not use the IEEE-754 even-rounding
27934
rule to round halfway cases for @code{printf}.}
27935
27936
@example
27937
-3.5 => -4
27938
-2.5 => -2
27939
-1.5 => -2
27940
-0.5 => 0
27941
0.5 => 0
27942
1.5 => 2
27943
2.5 => 2
27944
3.5 => 4
27945
4.5 => 4
27946
@end example
27947
27948
The theory behind the rounding mode @code{roundTiesToEven} is that
27949
it more or less evenly distributes upward and downward rounds
27950
of exact halves, which might cause any round-off error
27951
to cancel itself out. This is the default rounding mode used
27952
in IEEE-754 computing functions and operators.
27953
27954
The other rounding modes are rarely used.
27955
Round toward positive infinity (@code{roundTowardPositive})
27956
and round toward negative infinity (@code{roundTowardNegative})
27957
are often used to implement interval arithmetic,
27958
where you adjust the rounding mode to calculate upper and lower bounds
27959
for the range of output. The @code{roundTowardZero}
27960
mode can be used for converting floating-point numbers to integers.
27961
The rounding mode @code{roundTiesToAway} rounds the result to the
27962
nearest number and selects the number with the larger magnitude
27963
if a tie occurs.
27964
27965
Some numerical analysts will tell you that your choice of rounding style
27966
has tremendous impact on the final outcome, and advise you to wait until
27967
final output for any rounding. Instead, you can often avoid round-off error problems by
27968
setting the precision initially to some value sufficiently larger than
27969
the final desired precision, so that the accumulation of round-off error
27970
does not influence the outcome.
27971
If you suspect that results from your computation are
27972
sensitive to accumulation of round-off error,
27973
one way to be sure is to look for a significant difference in output
27974
when you change the rounding mode.
27975
27976
@node Gawk and MPFR
27977
@section @command{gawk} + MPFR = Powerful Arithmetic
27978
27979
The rest of this @value{CHAPTER} describes how to use the arbitrary precision
27980
(also known as @dfn{multiple precision} or @dfn{infinite precision}) numeric
27981
capabilities in @command{gawk} to produce maximally accurate results
27982
when you need it.
27983
27984
But first you should check if your version of
27985
@command{gawk} supports arbitrary precision arithmetic.
27986
The easiest way to find out is to look at the output of
27987
the following command:
27988
27989
@example
27990
$ @kbd{gawk --version}
27991
@print{} GNU Awk 4.1.0, API: 1.0 (GNU MPFR 3.1.0-p3, GNU MP 5.0.2)
27992
@print{} Copyright (C) 1989, 1991-2013 Free Software Foundation.
27993
@dots{}
27994
@end example
27995
27996
@command{gawk} uses the
27997
@uref{http://www.mpfr.org, GNU MPFR}
27998
and
27999
@uref{http://gmplib.org, GNU MP} (GMP)
28000
libraries for arbitrary precision
28001
arithmetic on numbers. So if you do not see the names of these libraries
28002
in the output, then your version of @command{gawk} does not support
28003
arbitrary precision arithmetic.
28004
28005
Additionally,
28006
there are a few elements available in the @code{PROCINFO} array
28007
to provide information about the MPFR and GMP libraries.
28008
@xref{Auto-set}, for more information.
28009
28010
@ignore
28011
Even if you aren't interested in arbitrary precision arithmetic, you
28012
may still benefit from knowing about how @command{gawk} handles numbers
28013
in general, and the limitations of doing arithmetic with ordinary
28014
@command{gawk} numbers.
28015
@end ignore
28016
28017
28018
@node Arbitrary Precision Floats
28019
@section Arbitrary Precision Floating-point Arithmetic with @command{gawk}
28020
28021
@command{gawk} uses the GNU MPFR library
28022
for arbitrary precision floating-point arithmetic. The MPFR library
28023
provides precise control over precisions and rounding modes, and gives
28024
correctly rounded, reproducible, platform-independent results. With one
28025
of the command-line options @option{--bignum} or @option{-M},
28026
all floating-point arithmetic operators and numeric functions can yield
28027
results to any desired precision level supported by MPFR.
28028
Two built-in variables, @code{PREC} and @code{ROUNDMODE},
28029
provide control over the working precision and the rounding mode
28030
(@pxref{Setting Precision}, and
28031
@pxref{Setting Rounding Mode}).
28032
The precision and the rounding mode are set globally for every operation
28033
to follow.
28034
28035
The default working precision for arbitrary precision floating-point values is
28036
53 bits, and the default value for @code{ROUNDMODE} is @code{"N"},
28037
which selects the IEEE-754 @code{roundTiesToEven} rounding mode
28038
(@pxref{Rounding Mode}).@footnote{The
28039
default precision is 53 bits, since according to the MPFR documentation,
28040
the library should be able to exactly reproduce all computations with
28041
double-precision machine floating-point numbers (@code{double} type
28042
in C), except the default exponent range is much wider and subnormal
28043
numbers are not implemented.}
28044
@command{gawk} uses the default exponent range in MPFR
28045
@iftex
28046
(@math{emax = 2^{30} - 1, emin = -emax})
28047
@end iftex
28048
@ifnottex
28049
(@var{emax} = 2^30 @minus{} 1, @var{emin} = @minus{}@var{emax})
28050
@end ifnottex
28051
for all floating-point contexts.
28052
There is no explicit mechanism to adjust the exponent range.
28053
MPFR does not implement subnormal numbers by default,
28054
and this behavior cannot be changed in @command{gawk}.
28055
28056
@quotation NOTE
28057
When emulating an IEEE-754 format (@pxref{Setting Precision}),
28058
@command{gawk} internally adjusts the exponent range
28059
to the value defined for the format and also performs computations needed for
28060
gradual underflow (subnormal numbers).
28061
@end quotation
28062
28063
@quotation NOTE
28064
MPFR numbers are variable-size entities, consuming only as much space as
28065
needed to store the significant digits. Since the performance using MPFR
28066
numbers pales in comparison to doing arithmetic using the underlying machine
28067
types, you should consider using only as much precision as needed by
28068
your program.
28069
@end quotation
28070
28071
@menu
28072
* Setting Precision:: Setting the working precision.
28073
* Setting Rounding Mode:: Setting the rounding mode.
28074
* Floating-point Constants:: Representing floating-point constants.
28075
* Changing Precision:: Changing the precision of a number.
28076
* Exact Arithmetic:: Exact arithmetic with floating-point numbers.
28077
@end menu
28078
28079
@node Setting Precision
28080
@subsection Setting the Working Precision
28081
@cindex @code{PREC} variable
29366
@node Setting precision
29367
@subsection Setting The Precision
28082
29368
28083
29369
@command{gawk} uses a global working precision; it does not keep track of
28084
29370
the precision or accuracy of individual numbers. Performing an arithmetic
28085
29371
operation or calling a built-in function rounds the result to the current
28086
working precision. The default working precision is 53 bits, which can be
28087
modified using the built-in variable @code{PREC}. You can also set the
28088
value to one of the pre-defined case-insensitive strings
29372
working precision. The default working precision is 53 bits, which you can
29373
modify using the built-in variable @code{PREC}. You can also set the
29374
value to one of the predefined case-insensitive strings
28089
29375
shown in @ref{table-predefined-precision-strings},
28090
to emulate an IEEE-754 binary format.
29376
to emulate an IEEE 754 binary format.
28091
29377
28092
29378
@float Table,table-predefined-precision-strings
28093
@caption{Predefined precision strings for @code{PREC}}
29379
@caption{Predefined Precision Strings For @code{PREC}}
28094
29380
@multitable {@code{"double"}} {12345678901234567890123456789012345}
28095
@headitem @code{PREC} @tab IEEE-754 Binary Format
29381
@headitem @code{PREC} @tab IEEE 754 Binary Format
28096
29382
@item @code{"half"} @tab 16-bit half-precision.
28097
29383
@item @code{"single"} @tab Basic 32-bit single precision.
28098
29384
@item @code{"double"} @tab Basic 64-bit double precision.
28111
29397
@print{} 0
28112
29398
@end example
28113
29399
28114
Binary and decimal precisions are related approximately, according to the
28115
formula:
28116
28117
@iftex
28118
@math{prec = 3.322 @cdot dps}
28119
@end iftex
28120
@ifnottex
28121
@var{prec} = 3.322 * @var{dps}
28122
@end ifnottex
28123
28124
@noindent
28125
Here, @var{prec} denotes the binary precision
28126
(measured in bits) and @var{dps} (short for decimal places)
28127
is the decimal digits. We can easily calculate how many decimal
28128
digits the 53-bit significand of an IEEE double is equivalent to:
28129
53 / 3.322 which is equal to about 15.95.
28130
But what does 15.95 digits actually mean? It depends whether you are
28131
concerned about how many digits you can rely on, or how many digits
28132
you need.
28133
28134
It is important to know how many bits it takes to uniquely identify
28135
a double-precision value (the C type @code{double}). If you want to
28136
convert from @code{double} to decimal and back to @code{double} (e.g.,
28137
saving a @code{double} representing an intermediate result to a file, and
28138
later reading it back to restart the computation), then a few more decimal
28139
digits are required. 17 digits is generally enough for a @code{double}.
28140
28141
It can also be important to know what decimal numbers can be uniquely
28142
represented with a @code{double}. If you want to convert
28143
from decimal to @code{double} and back again, 15 digits is the most that
28144
you can get. Stated differently, you should not present
28145
the numbers from your floating-point computations with more than 15
28146
significant digits in them.
28147
28148
Conversely, it takes a precision of 332 bits to hold an approximation
28149
of the constant @value{PI} that is accurate to 100 decimal places.
28150
28151
You should always add some extra bits in order to avoid the confusing round-off
28152
issues that occur because numbers are stored internally in binary.
28153
28154
@node Setting Rounding Mode
28155
@subsection Setting the Rounding Mode
28156
@cindex @code{ROUNDMODE} variable
29400
@quotation CAUTION
29401
Be wary of floating-point constants! When reading a floating-point
29402
constant from program source code, @command{gawk} uses the default
29403
precision (that of a C @code{double}), unless overridden by an assignment
29404
to the special variable @code{PREC} on the command line, to store it
29405
internally as a MPFR number. Changing the precision using @code{PREC}
29406
in the program text does @emph{not} change the precision of a constant.
29407
29408
If you need to represent a floating-point constant at a higher precision
29409
than the default and cannot use a command line assignment to @code{PREC},
29410
you should either specify the constant as a string, or as a rational
29411
number, whenever possible. The following example illustrates the
29412
differences among various ways to print a floating-point constant:
29413
@end quotation
29414
29415
@example
29416
$ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 0.1) @}'}
29417
@print{} 0.1000000000000000055511151
29418
$ @kbd{gawk -M -v PREC=113 'BEGIN @{ printf("%0.25f\n", 0.1) @}'}
29419
@print{} 0.1000000000000000000000000
29420
$ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", "0.1") @}'}
29421
@print{} 0.1000000000000000000000000
29422
$ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 1/10) @}'}
29423
@print{} 0.1000000000000000000000000
29424
@end example
29425
29426
@node Setting the rounding mode
29427
@subsection Setting The Rounding Mode
28157
29428
28158
29429
The @code{ROUNDMODE} variable provides
28159
29430
program level control over the rounding mode.
28172
29443
@end multitable
28173
29444
@end float
28174
29445
28175
@code{ROUNDMODE} has the default value @code{"N"},
28176
which selects the IEEE-754 rounding mode @code{roundTiesToEven}.
28177
In @ref{table-gawk-rounding-modes}, @code{"A"} is listed to select the IEEE-754 mode
28178
@code{roundTiesToAway}. This is only available
28179
if your version of the MPFR library supports it; otherwise setting
28180
@code{ROUNDMODE} to this value has no effect. @xref{Rounding Mode},
28181
for the meanings of the various rounding modes.
28182
28183
Here is an example of how to change the default rounding behavior of
28184
@code{printf}'s output:
28185
28186
@example
28187
$ @kbd{gawk -M -v ROUNDMODE="Z" 'BEGIN @{ printf("%.2f\n", 1.378) @}'}
28188
@print{} 1.37
28189
@end example
28190
28191
@node Floating-point Constants
28192
@subsection Representing Floating-point Constants
28193
@cindex constants, floating-point
28194
28195
Be wary of floating-point constants! When reading a floating-point constant
28196
from program source code, @command{gawk} uses the default precision,
28197
unless overridden
28198
by an assignment to the special variable @code{PREC} on the command
28199
line, to store it internally as a MPFR number.
28200
Changing the precision using @code{PREC} in the program text does
28201
@emph{not} change the precision of a constant. If you need to
28202
represent a floating-point constant at a higher precision than the
28203
default and cannot use a command line assignment to @code{PREC},
28204
you should either specify the constant as a string, or
28205
as a rational number, whenever possible. The following example
28206
illustrates the differences among various ways to
28207
print a floating-point constant:
28208
28209
@example
28210
$ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 0.1) @}'}
28211
@print{} 0.1000000000000000055511151
28212
$ @kbd{gawk -M -v PREC=113 'BEGIN @{ printf("%0.25f\n", 0.1) @}'}
28213
@print{} 0.1000000000000000000000000
28214
$ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", "0.1") @}'}
28215
@print{} 0.1000000000000000000000000
28216
$ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 1/10) @}'}
28217
@print{} 0.1000000000000000000000000
28218
@end example
28219
28220
In the first case, the number is stored with the default precision of 53 bits.
28221
28222
@node Changing Precision
28223
@subsection Changing the Precision of a Number
28224
28225
@cindex Laurie, Dirk
28226
@quotation
28227
@i{The point is that in any variable-precision package,
28228
a decision is made on how to treat numbers given as data,
28229
or arising in intermediate results, which are represented in
28230
floating-point format to a precision lower than working precision.
28231
Do we promote them to full membership of the high-precision club,
28232
or do we treat them and all their associates as second-class citizens?
28233
Sometimes the first course is proper, sometimes the second, and it takes
28234
careful analysis to tell which.}@footnote{Dirk Laurie.
28235
@cite{Variable-precision Arithmetic Considered Perilous --- A Detective Story}.
28236
Electronic Transactions on Numerical Analysis. Volume 28, pp. 168-173, 2008.}
28237
@author Dirk Laurie
28238
@end quotation
28239
28240
@command{gawk} does not implicitly modify the precision of any previously
28241
computed results when the working precision is changed with an assignment
28242
to @code{PREC}. The precision of a number is always the one that was
28243
used at the time of its creation, and there is no way for the user
28244
to explicitly change it afterwards. However, since the result of a
28245
floating-point arithmetic operation is always an arbitrary precision
28246
floating-point value---with a precision set by the value of @code{PREC}---one of the
28247
following workarounds effectively accomplishes the desired behavior:
28248
28249
@example
28250
x = x + 0.0
28251
@end example
28252
28253
@noindent
28254
or:
28255
28256
@example
28257
x += 0.0
28258
@end example
28259
28260
@node Exact Arithmetic
28261
@subsection Exact Arithmetic with Floating-point Numbers
28262
28263
@quotation CAUTION
28264
Never depend on the exactness of floating-point arithmetic,
28265
even for apparently simple expressions!
28266
@end quotation
28267
28268
Can arbitrary precision arithmetic give exact results? There are
28269
no easy answers. The standard rules of algebra often do not apply
28270
when using floating-point arithmetic.
28271
Among other things, the distributive and associative laws
28272
do not hold completely, and order of operation may be important
28273
for your computation. Rounding error, cumulative precision loss
28274
and underflow are often troublesome.
28275
28276
When @command{gawk} tests the expressions @samp{0.1 + 12.2} and @samp{12.3}
28277
for equality
28278
using the machine double precision arithmetic, it decides that they
28279
are not equal!
28280
(@xref{Floating-point Programming}.)
28281
You can get the result you want by increasing the precision;
28282
56 bits in this case will get the job done:
28283
28284
@example
28285
$ @kbd{gawk -M -v PREC=56 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'}
28286
@print{} 1
28287
@end example
28288
28289
If adding more bits is good, perhaps adding even more bits of
28290
precision is better?
28291
Here is what happens if we use an even larger value of @code{PREC}:
28292
28293
@example
28294
$ @kbd{gawk -M -v PREC=201 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'}
28295
@print{} 0
28296
@end example
28297
28298
This is not a bug in @command{gawk} or in the MPFR library.
28299
It is easy to forget that the finite number of bits used to store the value
28300
is often just an approximation after proper rounding.
28301
The test for equality succeeds if and only if @emph{all} bits in the two operands
28302
are exactly the same. Since this is not necessarily true after floating-point
28303
computations with a particular precision and effective rounding rule,
28304
a straight test for equality may not work.
28305
28306
So, don't assume that floating-point values can be compared for equality.
28307
You should also exercise caution when using other forms of comparisons.
28308
The standard way to compare between floating-point numbers is to determine
28309
how much error (or @dfn{tolerance}) you will allow in a comparison and
28310
check to see if one value is within this error range of the other.
28311
28312
In applications where 15 or fewer decimal places suffice,
28313
hardware double precision arithmetic can be adequate, and is usually much faster.
28314
But you do need to keep in mind that every floating-point operation
28315
can suffer a new rounding error with catastrophic consequences as illustrated
28316
by our earlier attempt to compute the value of the constant @value{PI}
28317
(@pxref{Floating-point Programming}).
28318
Extra precision can greatly enhance the stability and the accuracy
28319
of your computation in such cases.
28320
28321
Repeated addition is not necessarily equivalent to multiplication
28322
in floating-point arithmetic. In the example in
28323
@ref{Floating-point Programming}:
28324
28325
@example
28326
$ @kbd{gawk 'BEGIN @{}
28327
> @kbd{for (d = 1.1; d <= 1.5; d += 0.1) # loop five times (?)}
28328
> @kbd{i++}
28329
> @kbd{print i}
28330
> @kbd{@}'}
28331
@print{} 4
28332
@end example
28333
28334
@noindent
28335
you may or may not succeed in getting the correct result by choosing
28336
an arbitrarily large value for @code{PREC}. Reformulation of
28337
the problem at hand is often the correct approach in such situations.
29446
@code{ROUNDMODE} has the default value @code{"N"}, which
29447
selects the IEEE 754 rounding mode @code{roundTiesToEven}.
29448
In @ref{table-gawk-rounding-modes}, the value @code{"A"} selects
29449
@code{roundTiesToAway}. This is only available if your version of the
29450
MPFR library supports it; otherwise setting @code{ROUNDMODE} to @code{"A"}
29451
has no effect.
29452
29453
The default mode @code{roundTiesToEven} is the most preferred,
29454
but the least intuitive. This method does the obvious thing for most values,
29455
by rounding them up or down to the nearest digit.
29456
For example, rounding 1.132 to two digits yields 1.13,
29457
and rounding 1.157 yields 1.16.
29458
29459
However, when it comes to rounding a value that is exactly halfway between,
29460
things do not work the way you probably learned in school.
29461
In this case, the number is rounded to the nearest even digit.
29462
So rounding 0.125 to two digits rounds down to 0.12,
29463
but rounding 0.6875 to three digits rounds up to 0.688.
29464
You probably have already encountered this rounding mode when
29465
using @code{printf} to format floating-point numbers.
29466
For example:
29467
29468
@example
29469
BEGIN @{
29470
x = -4.5
29471
for (i = 1; i < 10; i++) @{
29472
x += 1.0
29473
printf("%4.1f => %2.0f\n", x, x)
29474
@}
29475
@}
29476
@end example
29477
29478
@noindent
29479
produces the following output when run on the author's system:@footnote{It
29480
is possible for the output to be completely different if the
29481
C library in your system does not use the IEEE 754 even-rounding
29482
rule to round halfway cases for @code{printf}.}
29483
29484
@example
29485
-3.5 => -4
29486
-2.5 => -2
29487
-1.5 => -2
29488
-0.5 => 0
29489
0.5 => 0
29490
1.5 => 2
29491
2.5 => 2
29492
3.5 => 4
29493
4.5 => 4
29494
@end example
29495
29496
The theory behind @code{roundTiesToEven} is that it more or less evenly
29497
distributes upward and downward rounds of exact halves, which might
29498
cause any accumulating round-off error to cancel itself out. This is the
29499
default rounding mode for IEEE 754 computing functions and operators.
29500
29501
The other rounding modes are rarely used. Round toward positive infinity
29502
(@code{roundTowardPositive}) and round toward negative infinity
29503
(@code{roundTowardNegative}) are often used to implement interval
29504
arithmetic, where you adjust the rounding mode to calculate upper and
29505
lower bounds for the range of output. The @code{roundTowardZero} mode can
29506
be used for converting floating-point numbers to integers. The rounding
29507
mode @code{roundTiesToAway} rounds the result to the nearest number and
29508
selects the number with the larger magnitude if a tie occurs.
29509
29510
Some numerical analysts will tell you that your choice of rounding
29511
style has tremendous impact on the final outcome, and advise you to
29512
wait until final output for any rounding. Instead, you can often avoid
29513
round-off error problems by setting the precision initially to some
29514
value sufficiently larger than the final desired precision, so that
29515
the accumulation of round-off error does not influence the outcome.
29516
If you suspect that results from your computation are sensitive to
29517
accumulation of round-off error, look for a significant difference in
29518
output when you change the rounding mode to be sure.
28338
29519
28339
29520
@node Arbitrary Precision Integers
28340
29521
@section Arbitrary Precision Integer Arithmetic with @command{gawk}
28341
@cindex integer, arbitrary precision
29522
@cindex integers, arbitrary precision
29523
@cindex arbitrary precision integers
28342
29524
28343
If one of the options @option{--bignum} or @option{-M} is specified,
28344
@command{gawk} performs all
28345
integer arithmetic using GMP arbitrary precision integers.
28346
Any number that looks like an integer in a program source or data file
28347
is stored as an arbitrary precision integer.
28348
The size of the integer is limited only by your computer's memory.
28349
The current floating-point context has no effect on operations involving integers.
28350
For example, the following computes
29525
When given one of the options @option{--bignum} or @option{-M},
29526
@command{gawk} performs all integer arithmetic using GMP arbitrary
29527
precision integers. Any number that looks like an integer in a source
29528
or @value{DF} is stored as an arbitrary precision integer. The size
29529
of the integer is limited only by the available memory. For example,
29530
the following computes
28351
29531
@iftex
28352
29532
@math{5^{4^{3^{2}}}},
28353
29533
@end iftex
28354
29534
@ifnottex
29535
@ifnotdocbook
28355
29536
5^4^3^2,
29537
@end ifnotdocbook
28356
29538
@end ifnottex
29539
@docbook
29540
5<superscript>4<superscript>3<superscript>2</superscript></superscript></superscript>, @c
29541
@end docbook
28357
29542
the result of which is beyond the
28358
limits of ordinary @command{gawk} numbers:
29543
limits of ordinary hardware double-precision floating point values:
28359
29544
28360
29545
@example
28361
29546
$ @kbd{gawk -M 'BEGIN @{}
28367
29552
@print{} 62060698786608744707 ... 92256259918212890625
28368
29553
@end example
28369
29554
28370
If you were to compute the same value using arbitrary precision
28371
floating-point values instead, the precision needed for correct output
28372
(using the formula
29555
If instead you were to compute the same value using arbitrary precision
29556
floating-point values, the precision needed for correct output (using
29557
the formula
28373
29558
@iftex
28374
29559
@math{prec = 3.322 @cdot dps}),
28375
29560
would be @math{3.322 @cdot 183231},
28376
29561
@end iftex
28377
29562
@ifnottex
29563
@ifnotdocbook
28378
29564
@samp{prec = 3.322 * dps}),
28379
29565
would be 3.322 x 183231,
29566
@end ifnotdocbook
28380
29567
@end ifnottex
29568
@docbook
29569
<emphasis>prec</emphasis> = 3.322 ⋅ <emphasis>dps</emphasis>),
29570
would be
29571
<emphasis>prec</emphasis> = 3.322 ⋅ 183231, @c
29572
@end docbook
28381
29573
or 608693.
28382
29574
28383
29575
The result from an arithmetic operation with an integer and a floating-point value
28384
29576
is a floating-point value with a precision equal to the working precision.
28385
29577
The following program calculates the eighth term in
28386
29578
Sylvester's sequence@footnote{Weisstein, Eric W.
28387
@cite{Sylvester's Sequence}. From MathWorld---A Wolfram Web Resource.
28388
@url{http://mathworld.wolfram.com/SylvestersSequence.html}}
29579
@cite{Sylvester's Sequence}. From MathWorld---A Wolfram Web Resource
29580
@w{(@url{http://mathworld.wolfram.com/SylvestersSequence.html}).}}
28389
29581
using a recurrence:
28390
29582
28391
29583
@example
28405
29597
@samp{2.0} with an integer, to perform all computations using integer
28406
29598
arithmetic to get the correct output.
28407
29599
28408
It will sometimes be necessary for @command{gawk} to implicitly convert an
28409
arbitrary precision integer into an arbitrary precision floating-point value.
28410
This is primarily because the MPFR library does not always provide the
28411
relevant interface to process arbitrary precision integers or mixed-mode
28412
numbers as needed by an operation or function.
28413
In such a case, the precision is set to the minimum value necessary
28414
for exact conversion, and the working precision is not used for this purpose.
28415
If this is not what you need or want, you can employ a subterfuge
28416
like this:
29600
Sometimes @command{gawk} must implicitly convert an arbitrary precision
29601
integer into an arbitrary precision floating-point value. This is
29602
primarily because the MPFR library does not always provide the relevant
29603
interface to process arbitrary precision integers or mixed-mode numbers
29604
as needed by an operation or function. In such a case, the precision is
29605
set to the minimum value necessary for exact conversion, and the working
29606
precision is not used for this purpose. If this is not what you need or
29607
want, you can employ a subterfuge, and convert the integer to floating
29608
point first, like this:
28417
29609
28418
29610
@example
28419
29611
gawk -M 'BEGIN @{ n = 13; print (n + 0.0) % 2.0 @}'
28426
29618
gawk -M 'BEGIN @{ n = 13.0; print n % 2.0 @}'
28427
29619
@end example
28428
29620
28429
Note that for the particular example above, there is likely best
29621
Note that for the particular example above, it is likely best
28430
29622
to just use the following:
28431
29623
28432
29624
@example
28433
29625
gawk -M 'BEGIN @{ n = 13; print n % 2 @}'
28434
29626
@end example
28435
29627
29628
@node POSIX Floating Point Problems
29629
@section Standards Versus Existing Practice
29630
29631
Historically, @command{awk} has converted any non-numeric looking string
29632
to the numeric value zero, when required. Furthermore, the original
29633
definition of the language and the original POSIX standards specified that
29634
@command{awk} only understands decimal numbers (base 10), and not octal
29635
(base 8) or hexadecimal numbers (base 16).
29636
29637
Changes in the language of the
29638
2001 and 2004 POSIX standards can be interpreted to imply that @command{awk}
29639
should support additional features. These features are:
29640
29641
@itemize @value{BULLET}
29642
@item
29643
Interpretation of floating point data values specified in hexadecimal
29644
notation (e.g., @code{0xDEADBEEF}). (Note: data values, @emph{not}
29645
source code constants.)
29646
29647
@item
29648
Support for the special IEEE 754 floating point values ``Not A Number''
29649
(NaN), positive Infinity (``inf'') and negative Infinity (``@minus{}inf'').
29650
In particular, the format for these values is as specified by the ISO 1999
29651
C standard, which ignores case and can allow implementation-dependent additional
29652
characters after the @samp{nan} and allow either @samp{inf} or @samp{infinity}.
29653
@end itemize
29654
29655
The first problem is that both of these are clear changes to historical
29656
practice:
29657
29658
@itemize @value{BULLET}
29659
@item
29660
The @command{gawk} maintainer feels that supporting hexadecimal floating
29661
point values, in particular, is ugly, and was never intended by the
29662
original designers to be part of the language.
29663
29664
@item
29665
Allowing completely alphabetic strings to have valid numeric
29666
values is also a very severe departure from historical practice.
29667
@end itemize
29668
29669
The second problem is that the @code{gawk} maintainer feels that this
29670
interpretation of the standard, which requires a certain amount of
29671
``language lawyering'' to arrive at in the first place, was not even
29672
intended by the standard developers. In other words, ``we see how you
29673
got where you are, but we don't think that that's where you want to be.''
29674
29675
Recognizing the above issues, but attempting to provide compatibility
29676
with the earlier versions of the standard,
29677
the 2008 POSIX standard added explicit wording to allow, but not require,
29678
that @command{awk} support hexadecimal floating point values and
29679
special values for ``Not A Number'' and infinity.
29680
29681
Although the @command{gawk} maintainer continues to feel that
29682
providing those features is inadvisable,
29683
nevertheless, on systems that support IEEE floating point, it seems
29684
reasonable to provide @emph{some} way to support NaN and Infinity values.
29685
The solution implemented in @command{gawk} is as follows:
29686
29687
@itemize @value{BULLET}
29688
@item
29689
With the @option{--posix} command-line option, @command{gawk} becomes
29690
``hands off.'' String values are passed directly to the system library's
29691
@code{strtod()} function, and if it successfully returns a numeric value,
29692
that is what's used.@footnote{You asked for it, you got it.}
29693
By definition, the results are not portable across
29694
different systems. They are also a little surprising:
29695
29696
@example
29697
$ @kbd{echo nanny | gawk --posix '@{ print $1 + 0 @}'}
29698
@print{} nan
29699
$ @kbd{echo 0xDeadBeef | gawk --posix '@{ print $1 + 0 @}'}
29700
@print{} 3735928559
29701
@end example
29702
29703
@item
29704
Without @option{--posix}, @command{gawk} interprets the four strings
29705
@samp{+inf},
29706
@samp{-inf},
29707
@samp{+nan},
29708
and
29709
@samp{-nan}
29710
specially, producing the corresponding special numeric values.
29711
The leading sign acts a signal to @command{gawk} (and the user)
29712
that the value is really numeric. Hexadecimal floating point is
29713
not supported (unless you also use @option{--non-decimal-data},
29714
which is @emph{not} recommended). For example:
29715
29716
@example
29717
$ @kbd{echo nanny | gawk '@{ print $1 + 0 @}'}
29718
@print{} 0
29719
$ @kbd{echo +nan | gawk '@{ print $1 + 0 @}'}
29720
@print{} nan
29721
$ @kbd{echo 0xDeadBeef | gawk '@{ print $1 + 0 @}'}
29722
@print{} 0
29723
@end example
29724
29725
@command{gawk} ignores case in the four special values.
29726
Thus @samp{+nan} and @samp{+NaN} are the same.
29727
@end itemize
29728
29729
@node Floating point summary
29730
@section Summary
29731
29732
@itemize @value{BULLET}
29733
@item
29734
Most computer arithmetic is done using either integers or floating-point
29735
values. The default for @command{awk} is to use double-precision
29736
floating-point values.
29737
29738
@item
29739
In the 1980's, Barbie mistakenly said ``Math class is tough!''
29740
While math isn't tough, floating-point arithmetic isn't the same
29741
as pencil and paper math, and care must be taken:
29742
29743
@c nested list
29744
@itemize @value{MINUS}
29745
@item
29746
Not all numbers can be represented exactly.
29747
29748
@item
29749
Comparing values should use a delta, instead of being done directly
29750
with @samp{==} and @samp{!=}.
29751
29752
@item
29753
Errors accumulate.
29754
29755
@item
29756
Operations are not always truly associative or distributive.
29757
@end itemize
29758
29759
@item
29760
Increasing the accuracy can help, but it is not a panacea.
29761
29762
@item
29763
Often, increasing the accuracy and then rounding to the desired
29764
number of digits produces reasonable results.
29765
29766
@item
29767
Use either @option{-M} or @option{--bignum} to enable MPFR
29768
arithmetic. Use @code{PREC} to set the precision in bits, and
29769
@code{ROUNDMODE} to set the IEEE 754 rounding mode.
29770
29771
@item
29772
With @option{-M} or @option{--bignum}, @command{gawk} performs
29773
arbitrary precision integer arithmetic using the GMP library.
29774
This is faster and more space efficient than using MPFR for
29775
the same calculations.
29776
29777
@item
29778
There are several ``dark corners'' with respect to floating-point
29779
numbers where @command{gawk} disagrees with the POSIX standard.
29780
It pays to be aware of them.
29781
29782
@item
29783
Overall, there is no need to be unduly suspicious about the results from
29784
floating-point arithmetic. The lesson to remember is that floating-point
29785
arithmetic is always more complex than arithmetic using pencil and
29786
paper. In order to take advantage of the power of computer floating-point,
29787
you need to know its limitations and work within them. For most casual
29788
use of floating-point arithmetic, you will often get the expected result
29789
if you simply round the display of your final results to the correct number
29790
of significant decimal digits.
29791
29792
@item
29793
As general advice, avoid presenting numerical data in a manner that
29794
implies better precision than is actually the case.
29795
29796
@end itemize
29797
28436
29798
@node Dynamic Extensions
28437
29799
@chapter Writing Extensions for @command{gawk}
29800
@cindex dynamically loaded extensions
28438
29801
28439
29802
It is possible to add new functions written in C or C++ to @command{gawk} using
28440
29803
dynamically loaded libraries. This facility is available on systems
28464
29827
* Extension Samples:: The sample extensions that ship with
28465
29828
@code{gawk}.
28466
29829
* gawkextlib:: The @code{gawkextlib} project.
29830
* Extension summary:: Extension summary.
29831
* Extension Exercises:: Exercises.
28467
29832
@end menu
28468
29833
28469
29834
@node Extension Intro
28470
29835
@section Introduction
28471
29836
29837
@cindex plug-in
28472
29838
An @dfn{extension} (sometimes called a @dfn{plug-in}) is a piece of
28473
29839
external compiled code that @command{gawk} can load at runtime to
28474
29840
provide additional functionality, over and above the built-in capabilities
28488
29854
them, and presents a small sample extension. In addition, it documents
28489
29855
the sample extensions included in the @command{gawk} distribution,
28490
29856
and describes the @code{gawkextlib} project.
29857
@ifclear FOR_PRINT
28491
29858
@xref{Extension Design}, for a discussion of the extension mechanism
28492
29859
goals and design.
29860
@end ifclear
29861
@ifset FOR_PRINT
29862
See @uref{http://www.gnu.org/software/gawk/manual/html_node/Extension-Design.html}
29863
for a discussion of the extension mechanism
29864
goals and design.
29865
@end ifset
28493
29866
28494
29867
@node Plugin License
28495
29868
@section Extension Licensing
28514
29887
@command{gawk} and an extension is two-way. First, when an extension
28515
29888
is loaded, it is passed a pointer to a @code{struct} whose fields are
28516
29889
function pointers.
28517
This is shown in @ref{load-extension}.
29890
@ifnotdocbook
29891
This is shown in @ref{figure-load-extension}.
29892
@end ifnotdocbook
29893
@ifdocbook
29894
This is shown in @inlineraw{docbook, <xref linkend="figure-load-extension"/>}.
29895
@end ifdocbook
28518
29896
28519
@float Figure,load-extension
29897
@ifnotdocbook
29898
@float Figure,figure-load-extension
28520
29899
@caption{Loading The Extension}
28521
29900
@c FIXME: One day, it should not be necessary to have two cases,
28522
29901
@c but rather just the one without the "txt" final argument.
28523
29902
@c This applies to the other figures as well.
28524
29903
@ifinfo
28525
@center @image{api-figure1, , , Loading the extension, txt}
29904
@center @image{api-figure1, , , Loading The Extension, txt}
28526
29905
@end ifinfo
28527
29906
@ifnotinfo
28528
@center @image{api-figure1, , , Loading the extension}
29907
@center @image{api-figure1, , , Loading The Extension}
28529
29908
@end ifnotinfo
28530
29909
@end float
29910
@end ifnotdocbook
29911
29912
@docbook
29913
<figure id="figure-load-extension" float="0">
29914
<title>Loading The Extension</title>
29915
<mediaobject>
29916
<imageobject role="web"><imagedata fileref="api-figure1.png" format="PNG"/></imageobject>
29917
</mediaobject>
29918
</figure>
29919
@end docbook
28531
29920
28532
29921
The extension can call functions inside @command{gawk} through these
28533
29922
function pointers, at runtime, without needing (link-time) access
28534
29923
to @command{gawk}'s symbols. One of these function pointers is to a
28535
29924
function for ``registering'' new built-in functions.
28536
This is shown in @ref{load-new-function}.
29925
@ifnotdocbook
29926
This is shown in @ref{figure-load-new-function}.
29927
@end ifnotdocbook
29928
@ifdocbook
29929
This is shown in @inlineraw{docbook, <xref linkend="figure-load-new-function"/>}.
29930
@end ifdocbook
28537
29931
28538
@float Figure,load-new-function
29932
@ifnotdocbook
29933
@float Figure,figure-load-new-function
28539
29934
@caption{Loading The New Function}
28540
29935
@ifinfo
28541
@center @image{api-figure2, , , Loading the new function, txt}
29936
@center @image{api-figure2, , , Loading The New Function, txt}
28542
29937
@end ifinfo
28543
29938
@ifnotinfo
28544
@center @image{api-figure2, , , Loading the new function}
29939
@center @image{api-figure2, , , Loading The New Function}
28545
29940
@end ifnotinfo
28546
29941
@end float
29942
@end ifnotdocbook
29943
29944
@docbook
29945
<figure id="figure-load-new-function" float="0">
29946
<title>Loading The New Function</title>
29947
<mediaobject>
29948
<imageobject role="web"><imagedata fileref="api-figure2.png" format="PNG"/></imageobject>
29949
</mediaobject>
29950
</figure>
29951
@end docbook
28547
29952
28548
29953
In the other direction, the extension registers its new functions
28549
29954
with @command{gawk} by passing function pointers to the functions that
28550
29955
provide the new feature (@code{do_chdir()}, for example). @command{gawk}
28551
29956
associates the function pointer with a name and can then call it, using a
28552
29957
defined calling convention.
28553
This is shown in @ref{call-new-function}.
29958
@ifnotdocbook
29959
This is shown in @ref{figure-call-new-function}.
29960
@end ifnotdocbook
29961
@ifdocbook
29962
This is shown in @inlineraw{docbook, <xref linkend="figure-call-new-function"/>}.
29963
@end ifdocbook
28554
29964
28555
@float Figure,call-new-function
29965
@ifnotdocbook
29966
@float Figure,figure-call-new-function
28556
29967
@caption{Calling The New Function}
28557
29968
@ifinfo
28558
29969
@center @image{api-figure3, , , Calling the new function, txt}
28561
29972
@center @image{api-figure3, , , Calling the new function}
28562
29973
@end ifnotinfo
28563
29974
@end float
29975
@end ifnotdocbook
29976
29977
@docbook
29978
<figure id="figure-call-new-function" float="0">
29979
<title>Calling The New Function</title>
29980
<mediaobject>
29981
<imageobject role="web"><imagedata fileref="api-figure3.png" format="PNG"/></imageobject>
29982
</mediaobject>
29983
</figure>
29984
@end docbook
28564
29985
28565
29986
The @code{do_@var{xxx}()} function, in turn, then uses the function
28566
29987
pointers in the API @code{struct} to do its work, such as updating
28567
29988
variables or arrays, printing messages, setting @code{ERRNO}, and so on.
28568
29989
28569
Convenience macros in the @file{gawkapi.h} header file make calling
28570
through the function pointers look like regular function calls so that
28571
extension code is quite readable and understandable.
29990
Convenience macros make calling through the function pointers look
29991
like regular function calls so that extension code is quite readable
29992
and understandable.
28572
29993
28573
29994
Although all of this sounds somewhat complicated, the result is that
28574
29995
extension code is quite straightforward to write and to read. You can
28577
29998
28578
29999
Some other bits and pieces:
28579
30000
28580
@itemize @bullet
30001
@itemize @value{BULLET}
28581
30002
@item
28582
30003
The API provides access to @command{gawk}'s @code{do_@var{xxx}} values,
28583
30004
reflecting command line options, like @code{do_lint}, @code{do_profiling}
28597
30018
28598
30019
@node Extension API Description
28599
30020
@section API Description
30021
@cindex extension API
28600
30022
30023
C or C++ code for an extension must include the header file
30024
@file{gawkapi.h}, which declares the functions and defines the data
30025
types used to communicate with @command{gawk}.
28601
30026
This (rather large) @value{SECTION} describes the API in detail.
28602
30027
28603
30028
@menu
28604
30029
* Extension API Functions Introduction:: Introduction to the API functions.
28605
30030
* General Data Types:: The data types.
28606
30031
* Requesting Values:: How to get a value.
30032
* Memory Allocation Functions:: Functions for allocating memory.
28607
30033
* Constructor Functions:: Functions for creating values.
28608
30034
* Registration Functions:: Functions to register things with
28609
30035
@command{gawk}.
28625
30051
28626
30052
API function pointers are provided for the following kinds of operations:
28627
30053
28628
@itemize @bullet
30054
@itemize @value{BULLET}
28629
30055
@item
28630
Registrations functions. You may register:
28631
@itemize @minus
30056
Registration functions. You may register:
30057
@itemize @value{MINUS}
28632
30058
@item
28633
30059
extension functions,
28634
30060
@item
28659
30085
or changing one.
28660
30086
28661
30087
@item
30088
Allocating, reallocating, and releasing memory.
30089
30090
@item
28662
30091
Creating and releasing cached values; this provides an
28663
30092
efficient way to use values for multiple variables and
28664
30093
can be a big performance win.
28666
30095
@item
28667
30096
Manipulating arrays:
28668
30097
28669
@itemize @minus
30098
@itemize @value{MINUS}
28670
30099
@item
28671
30100
Retrieving, adding, deleting, and modifying elements
28672
30101
28686
30115
28687
30116
Some points about using the API:
28688
30117
28689
@itemize @bullet
30118
@itemize @value{BULLET}
28690
30119
@item
28691
30120
The following types and/or macros and/or functions are referenced
28692
30121
in @file{gawkapi.h}. For correct use, you must therefore include the
28695
30124
@multitable {@code{memset()}, @code{memcpy()}} {@code{<sys/types.h>}}
28696
30125
@headitem C Entity @tab Header File
28697
30126
@item @code{EOF} @tab @code{<stdio.h>}
30127
@item Values for @code{errno} @tab @code{<errno.h>}
28698
30128
@item @code{FILE} @tab @code{<stdio.h>}
28699
30129
@item @code{NULL} @tab @code{<stddef.h>}
28700
@item @code{malloc()} @tab @code{<stdlib.h>}
28701
30130
@item @code{memcpy()} @tab @code{<string.h>}
28702
30131
@item @code{memset()} @tab @code{<string.h>}
28703
@item @code{realloc()} @tab @code{<stdlib.h>}
28704
30132
@item @code{size_t} @tab @code{<sys/types.h>}
28705
30133
@item @code{struct stat} @tab @code{<sys/stat.h>}
28706
30134
@end multitable
28712
30140
a portability hodge-podge as can be seen in some parts of
28713
30141
the @command{gawk} source code.
28714
30142
28715
To pass reasonable integer values for @code{ERRNO}, you will also need to
28716
include @code{<errno.h>}.
28717
28718
30143
@item
28719
30144
The @file{gawkapi.h} file may be included more than once without ill effect.
28720
30145
Doing so, however, is poor coding practice.
28730
30155
All pointers filled in by @command{gawk} are to memory
28731
30156
managed by @command{gawk} and should be treated by the extension as
28732
30157
read-only. Memory for @emph{all} strings passed into @command{gawk}
28733
from the extension @emph{must} come from @code{malloc()} and is managed
28734
by @command{gawk} from then on.
30158
from the extension @emph{must} come from calling the API-provided function
30159
pointers @code{api_malloc()}, @code{api_calloc()} or @code{api_realloc()},
30160
and is managed by @command{gawk} from then on.
28735
30161
28736
30162
@item
28737
30163
The API defines several simple @code{struct}s that map values as seen
28738
30164
from @command{awk}. A value can be a @code{double}, a string, or an
28739
30165
array (as in multidimensional arrays, or when creating a new array).
28740
String values maintain both pointer and length since embedded @code{NUL}
30166
String values maintain both pointer and length since embedded @sc{nul}
28741
30167
characters are allowed.
28742
30168
28743
30169
@quotation NOTE
28771
30197
@node General Data Types
28772
30198
@subsection General Purpose Data Types
28773
30199
30200
@cindex Robbins, Arnold
30201
@cindex Ramey, Chet
28774
30202
@quotation
28775
30203
@i{I have a true love/hate relationship with unions.}
28776
30204
@author Arnold Robbins
30205
@end quotation
28777
30206
30207
@quotation
28778
30208
@i{That's the thing about unions: the compiler will arrange things so they
28779
30209
can accommodate both love and hate.}
28780
30210
@author Chet Ramey
28797
30227
while allowing @command{gawk} to use them as it needs to.
28798
30228
28799
30229
@item typedef enum awk_bool @{
28800
@item @ @ @ @ awk_false = 0,
28801
@item @ @ @ @ awk_true
28802
@item @} awk_bool_t;
30230
@itemx @ @ @ @ awk_false = 0,
30231
@itemx @ @ @ @ awk_true
30232
@itemx @} awk_bool_t;
28803
30233
A simple boolean type.
28804
30234
28805
30235
@item typedef struct awk_string @{
28809
30239
This represents a mutable string. @command{gawk}
28810
30240
owns the memory pointed to if it supplied
28811
30241
the value. Otherwise, it takes ownership of the memory pointed to.
28812
@strong{Such memory must come from @code{malloc()}!}
30242
@strong{Such memory must come from calling the API-provided function
30243
pointers @code{api_malloc()}, @code{api_calloc()}, or @code{api_realloc()}!}
28813
30244
28814
30245
As mentioned earlier, strings are maintained using the current
28815
30246
multibyte encoding.
28864
30295
indicates what is in the @code{union}.
28865
30296
28866
30297
Representing numbers is easy---the API uses a C @code{double}. Strings
28867
require more work. Since @command{gawk} allows embedded @code{NUL} bytes
30298
require more work. Since @command{gawk} allows embedded @sc{nul} bytes
28868
30299
in string values, a string must be represented as a pair containing a
28869
30300
data-pointer and length. This is the @code{awk_string_t} type.
28870
30301
28894
30325
can obtain a @dfn{scalar cookie}@footnote{See
28895
30326
@uref{http://catb.org/jargon/html/C/cookie.html, the ``cookie'' entry in the Jargon file} for a
28896
30327
definition of @dfn{cookie}, and @uref{http://catb.org/jargon/html/M/magic-cookie.html,
28897
the ``magic cookie'' entry in the Jargon file} for a nice example. See
28898
also the entry for ``Cookie'' in the @ref{Glossary}.}
30328
the ``magic cookie'' entry in the Jargon file} for a nice example.
30329
@ifclear FOR_PRINT
30330
See also the entry for ``Cookie'' in the @ref{Glossary}.
30331
@end ifclear
30332
}
28899
30333
object for that variable, and then use
28900
30334
the cookie for getting the variable's value or for changing the variable's
28901
30335
value.
28925
30359
value type, as appropriate. This behavior is summarized in
28926
30360
@ref{table-value-types-returned}.
28927
30361
30362
@c FIXME: Try to do this with spans...
30363
30364
@float Table,table-value-types-returned
30365
@caption{API Value Types Returned}
30366
@docbook
30367
<informaltable>
30368
<tgroup cols="2">
30369
<colspec colwidth="50*"/><colspec colwidth="50*"/>
30370
<thead>
30371
<row><entry></entry><entry><para>Type of Actual Value:</para></entry></row>
30372
</thead>
30373
<tbody>
30374
<row><entry></entry><entry></entry></row>
30375
</tbody>
30376
</tgroup>
30377
<tgroup cols="6">
30378
<colspec colwidth="16.6*"/>
30379
<colspec colwidth="16.6*"/>
30380
<colspec colwidth="19.8*"/>
30381
<colspec colwidth="15*"/>
30382
<colspec colwidth="15*"/>
30383
<colspec colwidth="16.6*"/>
30384
<thead>
30385
<row>
30386
<entry></entry>
30387
<entry></entry>
30388
<entry><para>String</para></entry>
30389
<entry><para>Number</para></entry>
30390
<entry><para>Array</para></entry>
30391
<entry><para>Undefined</para></entry>
30392
</row>
30393
</thead>
30394
<tbody>
30395
<row>
30396
<entry></entry>
30397
<entry><para><emphasis role="bold">String</emphasis></para></entry>
30398
<entry><para>String</para></entry>
30399
<entry><para>String</para></entry>
30400
<entry><para>false</para></entry>
30401
<entry><para>false</para></entry>
30402
</row>
30403
<row>
30404
<entry></entry>
30405
<entry><para><emphasis role="bold">Number</emphasis></para></entry>
30406
<entry><para>Number if can be converted, else false</para></entry>
30407
<entry><para>Number</para></entry>
30408
<entry><para>false</para></entry>
30409
<entry><para>false</para></entry>
30410
</row>
30411
<row>
30412
<entry><para><emphasis role="bold">Type</emphasis></para></entry>
30413
<entry><para><emphasis role="bold">Array</emphasis></para></entry>
30414
<entry><para>false</para></entry>
30415
<entry><para>false</para></entry>
30416
<entry><para>Array</para></entry>
30417
<entry><para>false</para></entry>
30418
</row>
30419
<row>
30420
<entry><para><emphasis role="bold">Requested:</emphasis></para></entry>
30421
<entry><para><emphasis role="bold">Scalar</emphasis></para></entry>
30422
<entry><para>Scalar</para></entry>
30423
<entry><para>Scalar</para></entry>
30424
<entry><para>false</para></entry>
30425
<entry><para>false</para></entry>
30426
</row>
30427
<row>
30428
<entry></entry>
30429
<entry><para><emphasis role="bold">Undefined</emphasis></para></entry>
30430
<entry><para>String</para></entry>
30431
<entry><para>Number</para></entry>
30432
<entry><para>Array</para></entry>
30433
<entry><para>Undefined</para></entry>
30434
</row>
30435
<row>
30436
<entry></entry>
30437
<entry><para><emphasis role="bold">Value Cookie</emphasis></para></entry>
30438
<entry><para>false</para></entry>
30439
<entry><para>false</para></entry>
30440
<entry><para>false</para>
30441
</entry><entry><para>false</para></entry>
30442
</row>
30443
</tbody>
30444
</tgroup>
30445
</informaltable>
30446
@end docbook
30447
28928
30448
@ifnotplaintext
28929
@float Table,table-value-types-returned
28930
@caption{Value Types Returned}
30449
@ifnotdocbook
28931
30450
@multitable @columnfractions .50 .50
28932
30451
@headitem @tab Type of Actual Value:
28933
30452
@end multitable
28940
30459
@item @tab @b{Undefined} @tab String @tab Number @tab Array @tab Undefined
28941
30460
@item @tab @b{Value Cookie} @tab false @tab false @tab false @tab false
28942
30461
@end multitable
28943
@end float
30462
@end ifnotdocbook
28944
30463
@end ifnotplaintext
28945
30464
@ifplaintext
28946
@float Table,table-value-types-returned
28947
@caption{Value Types Returned}
28948
30465
@example
28949
30466
+-------------------------------------------------+
28950
30467
| Type of Actual Value: |
28968
30485
| | Cookie | | | | |
28969
30486
+-----------+-----------+------------+------------+-----------+-----------+
28970
30487
@end example
30488
@end ifplaintext
28971
30489
@end float
28972
@end ifplaintext
30490
30491
@node Memory Allocation Functions
30492
@subsection Memory Allocation Functions and Convenience Macros
30493
@cindex allocating memory for extensions
30494
@cindex extensions, allocating memory
30495
30496
The API provides a number of @dfn{memory allocation} functions for
30497
allocating memory that can be passed to @command{gawk}, as well as a number of
30498
convenience macros.
30499
30500
@table @code
30501
@item void *gawk_malloc(size_t size);
30502
Call @command{gawk}-provided @code{api_malloc()} to allocate storage that may
30503
be passed to @command{gawk}.
30504
30505
@item void *gawk_calloc(size_t nmemb, size_t size);
30506
Call @command{gawk}-provided @code{api_calloc()} to allocate storage that may
30507
be passed to @command{gawk}.
30508
30509
@item void *gawk_realloc(void *ptr, size_t size);
30510
Call @command{gawk}-provided @code{api_realloc()} to allocate storage that may
30511
be passed to @command{gawk}.
30512
30513
@item void gawk_free(void *ptr);
30514
Call @command{gawk}-provided @code{api_free()} to release storage that was
30515
allocated with @code{gawk_malloc()}, @code{gawk_calloc()} or @code{gawk_realloc()}.
30516
@end table
30517
30518
The API has to provide these functions because it is possible
30519
for an extension to be compiled and linked against a different
30520
version of the C library than was used for the @command{gawk}
30521
executable.@footnote{This is more common on MS-Windows systems, but
30522
can happen on Unix-like systems as well.} If @command{gawk} were
30523
to use its version of @code{free()} when the memory came from an
30524
unrelated version of @code{malloc()}, unexpected behavior would
30525
likely result.
30526
30527
Two convenience macros may be used for allocating storage
30528
from the API-provided function pointers @code{api_malloc()} and
30529
@code{api_realloc()}. If the allocation fails, they cause @command{gawk}
30530
to exit with a fatal error message. They should be used as if they were
30531
procedure calls that do not return a value.
30532
30533
@table @code
30534
@item #define emalloc(pointer, type, size, message) @dots{}
30535
The arguments to this macro are as follows:
30536
30537
@c nested table
30538
@table @code
30539
@item pointer
30540
The pointer variable to point at the allocated storage.
30541
30542
@item type
30543
The type of the pointer variable, used to create a cast for the call to @code{api_malloc()}.
30544
30545
@item size
30546
The total number of bytes to be allocated.
30547
30548
@item message
30549
A message to be prefixed to the fatal error message. Typically this is the name
30550
of the function using the macro.
30551
@end table
30552
30553
@noindent
30554
For example, you might allocate a string value like so:
30555
30556
@example
30557
awk_value_t result;
30558
char *message;
30559
const char greet[] = "Don't Panic!";
30560
30561
emalloc(message, char *, sizeof(greet), "myfunc");
30562
strcpy(message, greet);
30563
make_malloced_string(message, strlen(message), & result);
30564
@end example
30565
30566
@item #define erealloc(pointer, type, size, message) @dots{}
30567
This is like @code{emalloc()}, but it calls @code{api_realloc()},
30568
instead of @code{api_malloc()}.
30569
The arguments are the same as for the @code{emalloc()} macro.
30570
@end table
28973
30571
28974
30572
@node Constructor Functions
28975
@subsection Constructor Functions and Convenience Macros
30573
@subsection Constructor Functions
28976
30574
28977
30575
The API provides a number of @dfn{constructor} functions for creating
28978
30576
string and numeric values, as well as a number of convenience macros.
28991
30589
@itemx make_malloced_string(const char *string, size_t length, awk_value_t *result)
28992
30590
This function creates a string value in the @code{awk_value_t} variable
28993
30591
pointed to by @code{result}. It expects @code{string} to be a @samp{char *}
28994
value pointing to data previously obtained from @code{malloc()}. The idea here
30592
value pointing to data previously obtained from the api-provided functions @code{api_malloc()}, @code{api_calloc()} or @code{api_realloc()}. The idea here
28995
30593
is that the data is passed directly to @command{gawk}, which assumes
28996
30594
responsibility for it. It returns @code{result}.
28997
30595
29007
30605
pointed to by @code{result}.
29008
30606
@end table
29009
30607
29010
Two convenience macros may be used for allocating storage from @code{malloc()}
29011
and @code{realloc()}. If the allocation fails, they cause @command{gawk} to
29012
exit with a fatal error message. They should be used as if they were
29013
procedure calls that do not return a value.
29014
29015
@table @code
29016
@item #define emalloc(pointer, type, size, message) @dots{}
29017
The arguments to this macro are as follows:
29018
@c nested table
29019
@table @code
29020
@item pointer
29021
The pointer variable to point at the allocated storage.
29022
29023
@item type
29024
The type of the pointer variable, used to create a cast for the call to @code{malloc()}.
29025
29026
@item size
29027
The total number of bytes to be allocated.
29028
29029
@item message
29030
A message to be prefixed to the fatal error message. Typically this is the name
29031
of the function using the macro.
29032
@end table
29033
29034
@noindent
29035
For example, you might allocate a string value like so:
29036
29037
@example
29038
awk_value_t result;
29039
char *message;
29040
const char greet[] = "Don't Panic!";
29041
29042
emalloc(message, char *, sizeof(greet), "myfunc");
29043
strcpy(message, greet);
29044
make_malloced_string(message, strlen(message), & result);
29045
@end example
29046
29047
@item #define erealloc(pointer, type, size, message) @dots{}
29048
This is like @code{emalloc()}, but it calls @code{realloc()},
29049
instead of @code{malloc()}.
29050
The arguments are the same as for the @code{emalloc()} macro.
29051
@end table
29052
29053
30608
@node Registration Functions
29054
30609
@subsection Registration Functions
30610
@cindex register extension
30611
@cindex extension registration
29055
30612
29056
30613
This @value{SECTION} describes the API functions for
29057
30614
registering parts of your extension with @command{gawk}.
29096
30653
This is a pointer to the C function that provides the desired
29097
30654
functionality.
29098
30655
The function must fill in the result with either a number
29099
or a string. @command{awk} takes ownership of any string memory.
29100
As mentioned earlier, string memory @strong{must} come from @code{malloc()}.
30656
or a string. @command{gawk} takes ownership of any string memory.
30657
As mentioned earlier, string memory @strong{must} come from the api-provided functions @code{api_malloc()}, @code{api_calloc()} or @code{api_realloc()}.
29101
30658
29102
30659
The @code{num_actual_args} argument tells the C function how many
29103
30660
actual parameters were passed from the calling @command{awk} code.
29128
30685
29129
30686
An @dfn{exit callback} function is a function that
29130
30687
@command{gawk} calls before it exits.
29131
Such functions are useful if you have general ``clean up'' tasks
30688
Such functions are useful if you have general ``cleanup'' tasks
29132
30689
that should be performed in your extension (such as closing data
29133
30690
base connections or other resource deallocations).
29134
30691
You can register such
29138
30695
@item void awk_atexit(void (*funcp)(void *data, int exit_status),
29139
30696
@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ void *arg0);
29140
30697
The parameters are:
30698
29141
30699
@c nested table
29142
30700
@table @code
29143
30701
@item funcp
29173
30731
29174
30732
@node Input Parsers
29175
30733
@subsubsection Customized Input Parsers
30734
@cindex customized input parser
29176
30735
29177
30736
By default, @command{gawk} reads text files as its input. It uses the value
29178
30737
of @code{RS} to find the end of the record, and then uses @code{FS}
29230
30789
A pointer to your @code{@var{XXX}_take_control_of()} function.
29231
30790
29232
30791
@item awk_const struct input_parser *awk_const next;
29233
This pointer is used by @command{gawk}.
29234
The extension cannot modify it.
30792
This is for use by @command{gawk};
30793
therefore it is marked @code{awk_const} so that the extension cannot
30794
modify it.
29235
30795
@end table
29236
30796
29237
30797
The steps are as follows:
29278
30838
@code{INVALID_HANDLE}. Otherwise, it will.
29279
30839
29280
30840
@item struct stat sbuf;
29281
If file descriptor is valid, then @command{gawk} will have filled
30841
If the file descriptor is valid, then @command{gawk} will have filled
29282
30842
in this structure via a call to the @code{fstat()} system call.
29283
30843
@end table
29284
30844
29420
30980
29421
30981
@node Output Wrappers
29422
30982
@subsubsection Customized Output Wrappers
30983
@cindex customized output wrapper
29423
30984
30985
@cindex output wrapper
29424
30986
An @dfn{output wrapper} is the mirror image of an input parser.
29425
30987
It allows an extension to take over the output to a file opened
29426
30988
with the @samp{>} or @samp{>>} I/O redirection operators (@pxref{Redirection}).
29457
31019
29458
31020
@item awk_const struct output_wrapper *awk_const next;
29459
31021
This is for use by @command{gawk};
29460
therefore they are marked @code{awk_const} so that the extension cannot
29461
modify them.
31022
therefore it is marked @code{awk_const} so that the extension cannot
31023
modify it.
29462
31024
@end table
29463
31025
29464
31026
The @code{awk_output_buf_t} structure looks like this:
29520
31082
upon the @code{name} and @code{mode} fields, and any additional state
29521
31083
(such as @command{awk} variable values) that is appropriate.
29522
31084
29523
When @command{gawk} calls @code{@var{XXX}_take_control_of()}, it should fill
31085
When @command{gawk} calls @code{@var{XXX}_take_control_of()}, that function should fill
29524
31086
in the other fields, as appropriate, except for @code{fp}, which it should just
29525
31087
use normally.
29526
31088
29534
31096
29535
31097
@node Two-way processors
29536
31098
@subsubsection Customized Two-way Processors
31099
@cindex customized two-way processor
29537
31100
29538
31101
A @dfn{two-way processor} combines an input parser and an output wrapper for
29539
31102
two-way I/O with the @samp{|&} operator (@pxref{Redirection}). It makes identical
29560
31123
The name of the two-way processor.
29561
31124
29562
31125
@item awk_bool_t (*can_take_two_way)(const char *name);
29563
This function returns true if it wants to take over two-way I/O for this filename.
31126
This function returns true if it wants to take over two-way I/O for this @value{FN}.
29564
31127
It should not change any state (variable
29565
31128
values, etc.) within @command{gawk}.
29566
31129
29573
31136
29574
31137
@item awk_const struct two_way_processor *awk_const next;
29575
31138
This is for use by @command{gawk};
29576
therefore they are marked @code{awk_const} so that the extension cannot
29577
modify them.
31139
therefore it is marked @code{awk_const} so that the extension cannot
31140
modify it.
29578
31141
@end table
29579
31142
29580
31143
As with the input parser and output processor, you provide
29591
31154
29592
31155
@node Printing Messages
29593
31156
@subsection Printing Messages
31157
@cindex printing messages from extensions
31158
@cindex messages from extensions
29594
31159
29595
31160
You can print different kinds of warning messages from your
29596
31161
extension, as described below. Note that for these functions,
29664
31229
29665
31230
@node Symbol Table Access
29666
31231
@subsection Symbol Table Access
31232
@cindex accessing global variables from extensions
29667
31233
29668
31234
Two sets of routines provide access to global variables, and one set
29669
31235
allows you to create and release cached values.
29709
31275
However, with the exception of the @code{PROCINFO} array, an extension
29710
31276
cannot change any of those variables.
29711
31277
31278
@quotation NOTE
31279
It is possible for the lookup of @code{PROCINFO} to fail. This happens if
31280
the @command{awk} program being run does not reference @code{PROCINFO};
31281
in this case @command{gawk} doesn't bother to create the array and
31282
populate it.
31283
@end quotation
31284
29712
31285
@node Symbol table by cookie
29713
31286
@subsubsection Variable Access and Update by Cookie
29714
31287
29730
31303
29731
31304
@item awk_bool_t sym_update_scalar(awk_scalar_t cookie, awk_value_t *value);
29732
31305
Update the value associated with a scalar cookie. Return false if
29733
the new value is not one of @code{AWK_STRING} or @code{AWK_NUMBER}.
31306
the new value is not of type @code{AWK_STRING} or @code{AWK_NUMBER}.
29734
31307
Here too, the built-in variables may not be updated.
29735
31308
@end table
29736
31309
29835
31408
or @code{sym_update_scalar()}, as you like.
29836
31409
29837
31410
However, you can understand the point of cached values if you remember that
29838
@emph{every} string value's storage @emph{must} come from @code{malloc()}.
31411
@emph{every} string value's storage @emph{must} come from @code{api_malloc()}, @code{api_calloc()} or @code{api_realloc()}.
29839
31412
If you have 20 variables, all of which have the same string value, you
29840
31413
must create 20 identical copies of the string.@footnote{Numeric values
29841
31414
are clearly less problematic, requiring only a C @code{double} to store.}
29848
31421
@item awk_bool_t create_value(awk_value_t *value, awk_value_cookie_t *result);
29849
31422
Create a cached string or numeric value from @code{value} for efficient later
29850
31423
assignment.
29851
Only @code{AWK_NUMBER} and @code{AWK_STRING} values are allowed. Any other type
31424
Only values of type @code{AWK_NUMBER} and @code{AWK_STRING} are allowed. Any other type
29852
31425
is rejected. While @code{AWK_UNDEFINED} could be allowed, doing so would
29853
31426
result in inferior performance.
29854
31427
29909
31482
are all the others be changed too?''
29910
31483
29911
31484
That's a great question. The answer is that no, it's not a problem.
29912
Internally, @command{gawk} uses reference-counted strings. This means
31485
Internally, @command{gawk} uses @dfn{reference-counted strings}. This means
29913
31486
that many variables can share the same string value, and @command{gawk}
29914
31487
keeps track of the usage. When a variable's value changes, @command{gawk}
29915
31488
simply decrements the reference count on the old value and updates
29916
31489
the variable to use the new value.
29917
31490
29918
Finally, as part of your clean up action (@pxref{Exit Callback Functions})
31491
Finally, as part of your cleanup action (@pxref{Exit Callback Functions})
29919
31492
you should release any cached values that you created, using
29920
31493
@code{release_value()}.
29921
31494
29922
31495
@node Array Manipulation
29923
31496
@subsection Array Manipulation
31497
@cindex array manipulation in extensions
29924
31498
29925
31499
The primary data structure@footnote{Okay, the only data structure.} in @command{awk}
29926
31500
is the associative array (@pxref{Arrays}).
30032
31606
(@pxref{Conversion}); thus using integral values is safest.
30033
31607
30034
31608
As with @emph{all} strings passed into @code{gawk} from an extension,
30035
the string value of @code{index} must come from @code{malloc()}, and
31609
the string value of @code{index} must come from the API-provided functions @code{api_malloc()}, @code{api_calloc()} or @code{api_realloc()} and
30036
31610
@command{gawk} releases the storage.
30037
31611
30038
31612
@item awk_bool_t set_array_element(awk_array_t a_cookie,
30040
31614
@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ const@ awk_value_t *const value);
30041
31615
In the array represented by @code{a_cookie}, create or modify
30042
31616
the element whose index is given by @code{index}.
30043
The @code{ARGV} and @code{ENVIRON} arrays may not be changed.
31617
The @code{ARGV} and @code{ENVIRON} arrays may not be changed,
31618
although the @code{PROCINFO} array can be.
30044
31619
30045
31620
@item awk_bool_t set_array_element_by_elem(awk_array_t a_cookie,
30046
31621
@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_element_t element);
30311
31886
Thus, the correct way to build an array is to work ``top down.'' Create
30312
31887
the array, and immediately install it in @command{gawk}'s symbol table
30313
31888
using @code{sym_update()}, or install it as an element in a previously
30314
existing array using @code{set_element()}. We show example code shortly.
31889
existing array using @code{set_array_element()}. We show example code shortly.
30315
31890
30316
31891
@item
30317
31892
Due to gawk internals, after using @code{sym_update()} to install an array
30337
31912
@end enumerate
30338
31913
30339
31914
The following C code is a simple test extension to create an array
30340
with two regular elements and with a subarray. The leading @samp{#include}
31915
with two regular elements and with a subarray. The leading @code{#include}
30341
31916
directives and boilerplate variable declarations are omitted for brevity.
30342
31917
The first step is to create a new array and then install it
30343
31918
in the symbol table:
30500
32075
30501
32076
@node Extension Versioning
30502
32077
@subsubsection API Version Constants and Variables
32078
@cindex API version
32079
@cindex extension API version
30503
32080
30504
32081
The API provides both a ``major'' and a ``minor'' version number.
30505
32082
The API versions are available at compile time as constants:
30553
32130
30554
32131
@node Extension API Informational Variables
30555
32132
@subsubsection Informational Variables
32133
@cindex API informational variables
32134
@cindex extension API informational variables
30556
32135
30557
32136
The API provides access to several variables that describe
30558
32137
whether the corresponding command-line options were enabled when
30559
32138
@command{gawk} was invoked. The variables are:
30560
32139
30561
32140
@table @code
32141
@item do_debug
32142
This variable is true if @command{gawk} was invoked with @option{--debug} option.
32143
30562
32144
@item do_lint
30563
32145
This variable is true if @command{gawk} was invoked with @option{--lint} option
30564
32146
(@pxref{Options}).
30565
32147
30566
@item do_traditional
30567
This variable is true if @command{gawk} was invoked with @option{--traditional} option.
32148
@item do_mpfr
32149
This variable is true if @command{gawk} was invoked with @option{--bignum} option.
30568
32150
30569
32151
@item do_profile
30570
32152
This variable is true if @command{gawk} was invoked with @option{--profile} option.
30572
32154
@item do_sandbox
30573
32155
This variable is true if @command{gawk} was invoked with @option{--sandbox} option.
30574
32156
30575
@item do_debug
30576
This variable is true if @command{gawk} was invoked with @option{--debug} option.
30577
30578
@item do_mpfr
30579
This variable is true if @command{gawk} was invoked with @option{--bignum} option.
32157
@item do_traditional
32158
This variable is true if @command{gawk} was invoked with @option{--traditional} option.
30580
32159
@end table
30581
32160
30582
32161
The value of @code{do_lint} can change if @command{awk} code
30627
32206
30628
32207
@table @code
30629
32208
@item int plugin_is_GPL_compatible;
30630
This asserts that the extension is compatible with the GNU GPL
30631
(@pxref{Copying}). If your extension does not have this, @command{gawk}
32209
This asserts that the extension is compatible with
32210
@ifclear FOR_PRINT
32211
the GNU GPL (@pxref{Copying}).
32212
@end ifclear
32213
@ifset FOR_PRINT
32214
the GNU GPL.
32215
@end ifset
32216
If your extension does not have this, @command{gawk}
30632
32217
will not load it (@pxref{Plugin License}).
30633
32218
30634
32219
@item static gawk_api_t *const api;
30652
32237
It can then be looped over for multiple calls to
30653
32238
@code{add_ext_func()}.
30654
32239
32240
@c Use @var{OR} for docbook
30655
32241
@item static awk_bool_t (*init_func)(void) = NULL;
30656
@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @r{OR}
32242
@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @var{OR}
30657
32243
@itemx static awk_bool_t init_my_module(void) @{ @dots{} @}
30658
32244
@itemx static awk_bool_t (*init_func)(void) = init_my_module;
30659
32245
If you need to do some initialization work, you should define a
30698
32284
30699
32285
@node Finding Extensions
30700
32286
@section How @command{gawk} Finds Extensions
32287
@cindex extension search path
32288
@cindex finding extensions
30701
32289
30702
32290
Compiled extensions have to be installed in a directory where
30703
32291
@command{gawk} can find them. If @command{gawk} is configured and
30708
32296
30709
32297
@node Extension Example
30710
32298
@section Example: Some File Functions
32299
@cindex extension example
30711
32300
30712
32301
@quotation
30713
32302
@i{No matter where you go, there you are.}
30889
32478
to make use of the API macros and boilerplate code
30890
32479
(@pxref{Extension API Boilerplate}).
30891
32480
30892
@c break line for page breaking
30893
32481
@example
30894
32482
#ifdef HAVE_CONFIG_H
30895
32483
#include <config.h>
30976
32564
that turns a numeric mode into a printable representation
30977
32565
(e.g., 644 becomes @samp{-rw-r--r--}). This is omitted here for brevity:
30978
32566
30979
@c break line for page breaking
30980
32567
@example
30981
32568
/* format_mode --- turn a stat mode field into something readable */
30982
32569
31166
32753
awk_array_t array;
31167
32754
int ret;
31168
32755
struct stat sbuf;
31169
/* default is stat() */
32756
/* default is lstat() */
31170
32757
int (*statfunc)(const char *path, struct stat *sbuf) = lstat;
31171
32758
31172
32759
assert(result != NULL);
31250
32837
static awk_ext_func_t func_table[] = @{
31251
32838
@{ "chdir", do_chdir, 1 @},
31252
32839
@{ "stat", do_stat, 2 @},
32840
#ifndef __MINGW32__
31253
32841
@{ "fts", do_fts, 3 @},
32842
#endif
31254
32843
@};
31255
32844
@end example
31256
32845
31264
32853
dl_load_func(func_table, filefuncs, "")
31265
32854
@end example
31266
32855
31267
And that's it! As an exercise, consider adding functions to
31268
implement system calls such as @code{chown()}, @code{chmod()},
31269
and @code{umask()}.
32856
And that's it!
31270
32857
31271
32858
@node Using Internal File Ops
31272
32859
@subsection Integrating The Extensions
31278
32865
a file named @file{filefuncs.c}, and @var{idir} is the location
31279
32866
of the @file{gawkapi.h} header file,
31280
32867
the following steps@footnote{In practice, you would probably want to
31281
use the GNU Autotools---Automake, Autoconf, Libtool, and Gettext---to
32868
use the GNU Autotools---Automake, Autoconf, Libtool, and @command{gettext}---to
31282
32869
configure and build your libraries. Instructions for doing so are beyond
31283
32870
the scope of this @value{DOCUMENT}. @xref{gawkextlib}, for WWW links to
31284
32871
the tools.} create a GNU/Linux shared library:
31320
32907
@end example
31321
32908
31322
32909
The @env{AWKLIBPATH} environment variable tells
31323
@command{gawk} where to find shared libraries (@pxref{Finding Extensions}).
32910
@command{gawk} where to find extensions (@pxref{Finding Extensions}).
31324
32911
We set it to the current directory and run the program:
31325
32912
31326
32913
@example
31352
32939
31353
32940
@node Extension Samples
31354
32941
@section The Sample Extensions In The @command{gawk} Distribution
32942
@cindex extensions distributed with @command{gawk}
31355
32943
31356
32944
This @value{SECTION} provides brief overviews of the sample extensions
31357
32945
that come in the @command{gawk} distribution. Some of them are intended
31382
32970
The @code{filefuncs} extension provides three different functions, as follows:
31383
32971
The usage is:
31384
32972
31385
@table @code
32973
@table @asis
31386
32974
@item @@load "filefuncs"
31387
32975
This is how you load the extension.
31388
32976
31389
@cindex @code{chdir} extension function
31390
@item result = chdir("/some/directory")
32977
@cindex @code{chdir()} extension function
32978
@item @code{result = chdir("/some/directory")}
31391
32979
The @code{chdir()} function is a direct hook to the @code{chdir()}
31392
32980
system call to change the current directory. It returns zero
31393
32981
upon success or less than zero upon error. In the latter case it updates
31394
32982
@code{ERRNO}.
31395
32983
31396
@cindex @code{stat} extension function
31397
@item result = stat("/some/path", statdata [, follow])
32984
@cindex @code{stat()} extension function
32985
@item @code{result = stat("/some/path", statdata} [@code{, follow}]@code{)}
31398
32986
The @code{stat()} function provides a hook into the
31399
32987
@code{stat()} system call.
31400
32988
It returns zero upon success or less than zero upon error.
31407
32995
When the call is successful, @code{stat()} fills the @code{statdata}
31408
32996
array with information retrieved from the filesystem, as follows:
31409
32997
31410
@c nested table
31411
@multitable @columnfractions .25 .60
31412
@item @code{statdata["name"]} @tab
31413
The name of the file.
31414
31415
@item @code{statdata["dev"]} @tab
31416
Corresponds to the @code{st_dev} field in the @code{struct stat}.
31417
31418
@item @code{statdata["ino"]} @tab
31419
Corresponds to the @code{st_ino} field in the @code{struct stat}.
31420
31421
@item @code{statdata["mode"]} @tab
31422
Corresponds to the @code{st_mode} field in the @code{struct stat}.
31423
31424
@item @code{statdata["nlink"]} @tab
31425
Corresponds to the @code{st_nlink} field in the @code{struct stat}.
31426
31427
@item @code{statdata["uid"]} @tab
31428
Corresponds to the @code{st_uid} field in the @code{struct stat}.
31429
31430
@item @code{statdata["gid"]} @tab
31431
Corresponds to the @code{st_gid} field in the @code{struct stat}.
31432
31433
@item @code{statdata["size"]} @tab
31434
Corresponds to the @code{st_size} field in the @code{struct stat}.
31435
31436
@item @code{statdata["atime"]} @tab
31437
Corresponds to the @code{st_atime} field in the @code{struct stat}.
31438
31439
@item @code{statdata["mtime"]} @tab
31440
Corresponds to the @code{st_mtime} field in the @code{struct stat}.
31441
31442
@item @code{statdata["ctime"]} @tab
31443
Corresponds to the @code{st_ctime} field in the @code{struct stat}.
31444
31445
@item @code{statdata["rdev"]} @tab
31446
Corresponds to the @code{st_rdev} field in the @code{struct stat}.
31447
This element is only present for device files.
31448
31449
@item @code{statdata["major"]} @tab
31450
Corresponds to the @code{st_major} field in the @code{struct stat}.
31451
This element is only present for device files.
31452
31453
@item @code{statdata["minor"]} @tab
31454
Corresponds to the @code{st_minor} field in the @code{struct stat}.
31455
This element is only present for device files.
31456
31457
@item @code{statdata["blksize"]} @tab
31458
Corresponds to the @code{st_blksize} field in the @code{struct stat},
31459
if this field is present on your system.
31460
(It is present on all modern systems that we know of.)
31461
31462
@item @code{statdata["pmode"]} @tab
31463
A human-readable version of the mode value, such as printed by
31464
@command{ls}. For example, @code{"-rwxr-xr-x"}.
31465
31466
@item @code{statdata["linkval"]} @tab
31467
If the named file is a symbolic link, this element will exist
31468
and its value is the value of the symbolic link (where the
31469
symbolic link points to).
31470
31471
@item @code{statdata["type"]} @tab
31472
The type of the file as a string. One of
32998
@multitable @columnfractions .15 .50 .20
32999
@headitem Subscript @tab Field in @code{struct stat} @tab File type
33000
@item @code{"name"} @tab The @value{FN} @tab All
33001
@item @code{"dev"} @tab @code{st_dev} @tab All
33002
@item @code{"ino"} @tab @code{st_ino} @tab All
33003
@item @code{"mode"} @tab @code{st_mode} @tab All
33004
@item @code{"nlink"} @tab @code{st_nlink} @tab All
33005
@item @code{"uid"} @tab @code{st_uid} @tab All
33006
@item @code{"gid"} @tab @code{st_gid} @tab All
33007
@item @code{"size"} @tab @code{st_size} @tab All
33008
@item @code{"atime"} @tab @code{st_atime} @tab All
33009
@item @code{"mtime"} @tab @code{st_mtime} @tab All
33010
@item @code{"ctime"} @tab @code{st_ctime} @tab All
33011
@item @code{"rdev"} @tab @code{st_rdev} @tab Device files
33012
@item @code{"major"} @tab @code{st_major} @tab Device files
33013
@item @code{"minor"} @tab @code{st_minor} @tab Device files
33014
@item @code{"blksize"} @tab @code{st_blksize} @tab All
33015
@item @code{"pmode"} @tab A human-readable version of the mode value, such as printed by
33016
@command{ls}. For example, @code{"-rwxr-xr-x"} @tab All
33017
@item @code{"linkval"} @tab The value of the symbolic link @tab Symbolic links
33018
@item @code{"type"} @tab The type of the file as a string. One of
31473
33019
@code{"file"},
31474
33020
@code{"blockdev"},
31475
33021
@code{"chardev"},
31480
33026
@code{"door"},
31481
33027
or
31482
33028
@code{"unknown"}.
31483
Not all systems support all file types.
33029
Not all systems support all file types. @tab All
31484
33030
@end multitable
31485
33031
31486
@cindex @code{fts} extension function
31487
@item flags = or(FTS_PHYSICAL, ...)
31488
@itemx result = fts(pathlist, flags, filedata)
33032
@cindex @code{fts()} extension function
33033
@item @code{flags = or(FTS_PHYSICAL, ...)}
33034
@itemx @code{result = fts(pathlist, flags, filedata)}
31489
33035
Walk the file trees provided in @code{pathlist} and fill in the
31490
33036
@code{filedata} array as described below. @code{flags} is the bitwise
31491
33037
OR of several predefined constant values, also described below.
31502
33048
31503
33049
@table @code
31504
33050
@item pathlist
31505
An array of filenames. The element values are used; the index values are ignored.
33051
An array of @value{FN}s. The element values are used; the index values are ignored.
31506
33052
31507
33053
@item flags
31508
33054
This should be the bitwise OR of one or more of the following
31604
33150
@node Extension Sample Fnmatch
31605
33151
@subsection Interface To @code{fnmatch()}
31606
33152
31607
@cindex @code{fnmatch} extension function
31608
33153
This extension provides an interface to the C library
31609
33154
@code{fnmatch()} function. The usage is:
31610
33155
31611
@example
31612
@@load "fnmatch"
31613
31614
result = fnmatch(pattern, string, flags)
31615
@end example
31616
31617
The @code{fnmatch} extension adds a single function named
31618
@code{fnmatch()}, one constant (@code{FNM_NOMATCH}), and an array of
31619
flag values named @code{FNM}.
33156
@table @code
33157
@item @@load "fnmatch"
33158
This is how you load the extension.
33159
33160
@cindex @code{fnmatch()} extension function
33161
@item result = fnmatch(pattern, string, flags)
33162
The return value is zero on success, @code{FNM_NOMATCH}
33163
if the string did not match the pattern, or
33164
a different non-zero value if an error occurred.
33165
@end table
33166
33167
Besides the @code{fnmatch()} function, the @code{fnmatch} extension
33168
adds one constant (@code{FNM_NOMATCH}), and an array of flag values
33169
named @code{FNM}.
31620
33170
31621
33171
The arguments to @code{fnmatch()} are:
31622
33172
31623
33173
@table @code
31624
33174
@item pattern
31625
The filename wildcard to match.
33175
The @value{FN} wildcard to match.
31626
33176
31627
33177
@item string
31628
The filename string.
33178
The @value{FN} string.
31629
33179
31630
33180
@item flag
31631
33181
Either zero, or the bitwise OR of one or more of the
31632
33182
flags in the @code{FNM} array.
31633
33183
@end table
31634
33184
31635
The return value is zero on success, @code{FNM_NOMATCH}
31636
if the string did not match the pattern, or
31637
a different non-zero value if an error occurred.
31638
31639
33185
The flags are follows:
31640
33186
31641
33187
@multitable @columnfractions .25 .75
31642
@item @code{FNM["CASEFOLD"]} @tab
31643
Corresponds to the @code{FNM_CASEFOLD} flag as defined in @code{fnmatch()}.
31644
31645
@item @code{FNM["FILE_NAME"]} @tab
31646
Corresponds to the @code{FNM_FILE_NAME} flag as defined in @code{fnmatch()}.
31647
31648
@item @code{FNM["LEADING_DIR"]} @tab
31649
Corresponds to the @code{FNM_LEADING_DIR} flag as defined in @code{fnmatch()}.
31650
31651
@item @code{FNM["NOESCAPE"]} @tab
31652
Corresponds to the @code{FNM_NOESCAPE} flag as defined in @code{fnmatch()}.
31653
31654
@item @code{FNM["PATHNAME"]} @tab
31655
Corresponds to the @code{FNM_PATHNAME} flag as defined in @code{fnmatch()}.
31656
31657
@item @code{FNM["PERIOD"]} @tab
31658
Corresponds to the @code{FNM_PERIOD} flag as defined in @code{fnmatch()}.
33188
@headitem Array element @tab Corresponding flag defined by @code{fnmatch()}
33189
@item @code{FNM["CASEFOLD"]} @tab @code{FNM_CASEFOLD}
33190
@item @code{FNM["FILE_NAME"]} @tab @code{FNM_FILE_NAME}
33191
@item @code{FNM["LEADING_DIR"]} @tab @code{FNM_LEADING_DIR}
33192
@item @code{FNM["NOESCAPE"]} @tab @code{FNM_NOESCAPE}
33193
@item @code{FNM["PATHNAME"]} @tab @code{FNM_PATHNAME}
33194
@item @code{FNM["PERIOD"]} @tab @code{FNM_PERIOD}
31659
33195
@end multitable
31660
33196
31661
33197
Here is an example:
31677
33213
@item @@load "fork"
31678
33214
This is how you load the extension.
31679
33215
31680
@cindex @code{fork} extension function
33216
@cindex @code{fork()} extension function
31681
33217
@item pid = fork()
31682
This function creates a new process. The return value is the zero in the
31683
child and the process-id number of the child in the parent, or @minus{}1
33218
This function creates a new process. The return value is zero in the
33219
child and the process-ID number of the child in the parent, or @minus{}1
31684
33220
upon error. In the latter case, @code{ERRNO} indicates the problem.
31685
33221
In the child, @code{PROCINFO["pid"]} and @code{PROCINFO["ppid"]} are
31686
33222
updated to reflect the correct values.
31687
33223
31688
@cindex @code{waitpid} extension function
33224
@cindex @code{waitpid()} extension function
31689
33225
@item ret = waitpid(pid)
31690
This function takes a numeric argument, which is the process-id to
33226
This function takes a numeric argument, which is the process-ID to
31691
33227
wait for. The return value is that of the
31692
33228
@code{waitpid()} system call.
31693
33229
31694
@cindex @code{wait} extension function
33230
@cindex @code{wait()} extension function
31695
33231
@item ret = wait()
31696
33232
This function waits for the first child to die.
31697
33233
The return value is that of the
31746
33282
and permissions as the original. After the file has been processed,
31747
33283
the extension restores standard output to its original destination.
31748
33284
If @code{INPLACE_SUFFIX} is not an empty string, the original file is
31749
linked to a backup filename created by appending that suffix. Finally,
31750
the temporary file is renamed to the original filename.
33285
linked to a backup @value{FN} created by appending that suffix. Finally,
33286
the temporary file is renamed to the original @value{FN}.
31751
33287
31752
33288
If any error occurs, the extension issues a fatal error to terminate
31753
33289
processing immediately without damaging the original file.
31765
33301
> @kbd{@{ print @}' file1 file2 file3}
31766
33302
@end example
31767
33303
31768
We leave it as an exercise to write a wrapper script that presents an
31769
interface similar to @samp{sed -i}.
31770
31771
33304
@node Extension Sample Ord
31772
33305
@subsection Character and Numeric values: @code{ord()} and @code{chr()}
31773
33306
31778
33311
@item @@load "ordchr"
31779
33312
This is how you load the extension.
31780
33313
31781
@cindex @code{ord} extension function
33314
@cindex @code{ord()} extension function
31782
33315
@item number = ord(string)
31783
33316
Return the numeric value of the first character in @code{string}.
31784
33317
31785
@cindex @code{chr} extension function
33318
@cindex @code{chr()} extension function
31786
33319
@item char = chr(number)
31787
33320
Return a string whose first character is that represented by @code{number}.
31788
33321
@end table
31813
33346
they are read, with each entry returned as a record.
31814
33347
31815
33348
The record consists of three fields. The first two are the inode number and the
31816
filename, separated by a forward slash character.
33349
@value{FN}, separated by a forward slash character.
31817
33350
On systems where the directory entry contains the file type, the record
31818
33351
has a third field (also separated by a slash) which is a single letter
31819
indicating the type of the file:
33352
indicating the type of the file. The letters are file types are shown
33353
in @ref{table-readdir-file-types}.
31820
33354
33355
@float Table,table-readdir-file-types
33356
@caption{File Types Returned By @code{readdir()}}
31821
33357
@multitable @columnfractions .1 .9
31822
33358
@headitem Letter @tab File Type
31823
33359
@item @code{b} @tab Block device
31829
33365
@item @code{s} @tab Socket
31830
33366
@item @code{u} @tab Anything else (unknown)
31831
33367
@end multitable
33368
@end float
31832
33369
31833
33370
On systems without the file type information, the third field is always
31834
33371
@samp{u}.
31863
33400
31864
33401
BEGIN @{
31865
33402
REVOUT = 1
31866
print "hello, world" > "/dev/stdout"
33403
print "don't panic" > "/dev/stdout"
31867
33404
@}
31868
33405
@end example
31869
33406
31870
33407
The output from this program is:
31871
@samp{dlrow ,olleh}.
33408
@samp{cinap t'nod}.
31872
33409
31873
33410
@node Extension Sample Rev2way
31874
33411
@subsection Two-Way I/O Example
31885
33422
31886
33423
BEGIN @{
31887
33424
cmd = "/magic/mirror"
31888
print "hello, world" |& cmd
33425
print "don't panic" |& cmd
31889
33426
cmd |& getline result
31890
33427
print result
31891
33428
close(cmd)
31892
33429
@}
31893
33430
@end example
31894
33431
33432
The output from this program
33433
@ifnotinfo
33434
also is:
33435
@end ifnotinfo
33436
@ifinfo
33437
is:
33438
@end ifinfo
33439
@samp{cinap t'nod}.
33440
31895
33441
@node Extension Sample Read write array
31896
33442
@subsection Dumping and Restoring An Array
31897
33443
31899
33445
named @code{writea()} and @code{reada()}, as follows:
31900
33446
31901
33447
@table @code
31902
@cindex @code{writea} extension function
33448
@cindex @code{writea()} extension function
31903
33449
@item ret = writea(file, array)
31904
33450
This function takes a string argument, which is the name of the file
31905
to which dump the array, and the array itself as the second argument.
31906
@code{writea()} understands multidimensional arrays. It returns one on
33451
to which to dump the array, and the array itself as the second argument.
33452
@code{writea()} understands arrays of arrays. It returns one on
31907
33453
success, or zero upon failure.
31908
33454
31909
@cindex @code{reada} extension function
33455
@cindex @code{reada()} extension function
31910
33456
@item ret = reada(file, array)
31911
33457
@code{reada()} is the inverse of @code{writea()};
31912
33458
it reads the file named as its first argument, filling in
31943
33489
@subsection Reading An Entire File
31944
33490
31945
33491
The @code{readfile} extension adds a single function
31946
named @code{readfile()}:
33492
named @code{readfile()}, and an input parser:
31947
33493
31948
33494
@table @code
31949
33495
@item @@load "readfile"
31950
33496
This is how you load the extension.
31951
33497
31952
@cindex @code{readfile} extension function
33498
@cindex @code{readfile()} extension function
31953
33499
@item result = readfile("/some/path")
31954
33500
The argument is the name of the file to read. The return value is a
31955
33501
string containing the entire contents of the requested file. Upon error,
31956
33502
the function returns the empty string and sets @code{ERRNO}.
33503
33504
@item BEGIN @{ PROCINFO["readfile"] = 1 @}
33505
In addition, the extension adds an input parser that is activated if
33506
@code{PROCINFO["readfile"]} exists.
33507
When activated, each input file is returned in its entirety as @code{$0}.
33508
@code{RT} is set to the null string.
31957
33509
@end table
31958
33510
31959
33511
Here is an example:
31982
33534
@node Extension Sample Time
31983
33535
@subsection Extension Time Functions
31984
33536
31985
These functions can be used either by invoking @command{gawk}
31986
with a command-line argument of @samp{-l time} or by
31987
inserting @samp{@@load "time"} in your script.
33537
The @code{time} extension adds two functions, named @code{gettimeofday()}
33538
and @code{sleep()}, as follows:
31988
33539
31989
33540
@table @code
31990
33541
@item @@load "time"
31991
33542
This is how you load the extension.
31992
33543
31993
@cindex @code{gettimeofday} extension function
33544
@cindex @code{gettimeofday()} extension function
31994
33545
@item the_time = gettimeofday()
31995
33546
Return the time in seconds that has elapsed since 1970-01-01 UTC as a
31996
33547
floating point value. If the time is unavailable on this platform, return
31997
33548
@minus{}1 and set @code{ERRNO}. The returned time should have sub-second
31998
33549
precision, but the actual precision may vary based on the platform.
31999
33550
If the standard C @code{gettimeofday()} system call is available on this
32000
platform, then it simply returns the value. Otherwise, if on Windows,
33551
platform, then it simply returns the value. Otherwise, if on MS-Windows,
32001
33552
it tries to use @code{GetSystemTimeAsFileTime()}.
32002
33553
32003
@cindex @code{sleep} extension function
33554
@cindex @code{sleep()} extension function
32004
33555
@item result = sleep(@var{seconds})
32005
33556
Attempt to sleep for @var{seconds} seconds. If @var{seconds} is negative,
32006
33557
or the attempt to sleep fails, return @minus{}1 and set @code{ERRNO}.
32012
33563
32013
33564
@node gawkextlib
32014
33565
@section The @code{gawkextlib} Project
33566
@cindex @code{gawkextlib}
33567
@cindex extensions, where to find
32015
33568
32016
33569
@cindex @code{gawkextlib} project
32017
33570
The @uref{http://sourceforge.net/projects/gawkextlib/, @code{gawkextlib}}
32019
33572
processing XML files. This is the evolution of the original @command{xgawk}
32020
33573
(XML @command{gawk}) project.
32021
33574
32022
As of this writing, there are four extensions:
33575
As of this writing, there are five extensions:
32023
33576
32024
@itemize @bullet
33577
@itemize @value{BULLET}
32025
33578
@item
32026
33579
XML parser extension, using the @uref{http://expat.sourceforge.net, Expat}
32027
33580
XML parsing library.
32028
33581
32029
33582
@item
33583
PDF extension.
33584
33585
@item
32030
33586
PostgreSQL extension.
32031
33587
32032
33588
@item
32042
33598
Time}) was originally from this project but has been moved in to the
32043
33599
main @command{gawk} distribution.
32044
33600
33601
@cindex @command{git} utility
32045
33602
You can check out the code for the @code{gawkextlib} project
32046
using the @uref{http://git-scm.com, GIT} distributed source
33603
using the @uref{http://git-scm.com, Git} distributed source
32047
33604
code control system. The command is as follows:
32048
33605
32049
33606
@example
32059
33616
@uref{http://www.gnu.org/software/automake, Automake},
32060
33617
@uref{http://www.gnu.org/software/libtool, Libtool},
32061
33618
and
32062
@uref{http://www.gnu.org/software/gettext, Gettext}).
33619
@uref{http://www.gnu.org/software/gettext, GNU @command{gettext}}).
32063
33620
32064
33621
The simple recipe for building and testing @code{gawkextlib} is as follows.
32065
33622
First, build and install @command{gawk}:
32093
33650
@code{gawkextlib} project.
32094
33651
See the project's web site for more information.
32095
33652
32096
@iftex
32097
@part Part IV:@* Appendices
32098
@end iftex
32099
32100
@ignore
33653
@node Extension summary
33654
@section Summary
33655
33656
@itemize @value{BULLET}
33657
@item
33658
You can write extensions (sometimes called plug-ins) for @command{gawk}
33659
in C or C++ using the Application Programming Interface (API) defined
33660
by the @command{gawk} developers.
33661
33662
@item
33663
Extensions must have a license compatible with the GNU General Public
33664
License (GPL), and they must assert that fact by declaring a variable
33665
named @code{plugin_is_GPL_compatible}.
33666
33667
@item
33668
Communication between @command{gawk} and an extension is two-way.
33669
@command{gawk} passes a @code{struct} to the extension which contains
33670
various data fields and function pointers. The extension can then call
33671
into @command{gawk} via the supplied function pointers to accomplish
33672
certain tasks.
33673
33674
@item
33675
One of these tasks is to ``register'' the name and implementation of
33676
a new @command{awk}-level function with @command{gawk}. The implementation
33677
takes the form of a C function pointer with a defined signature.
33678
By convention, implementation functions are named @code{do_@var{XXXX}()}
33679
for some @command{awk}-level function @code{@var{XXXX}()}.
33680
33681
@item
33682
The API is defined in a header file named @file{gawkpi.h}. You must include
33683
a number of standard header files @emph{before} including it in your source file.
33684
33685
@item
33686
API function pointers are provided for the following kinds of operations:
33687
33688
@itemize @value{BULLET}
33689
@item
33690
Registration functions. You may register
33691
extension functions,
33692
exit callbacks,
33693
a version string,
33694
input parsers,
33695
output wrappers,
33696
and two-way processors.
33697
33698
@item
33699
Printing fatal, warning, and ``lint'' warning messages.
33700
33701
@item
33702
Updating @code{ERRNO}, or unsetting it.
33703
33704
@item
33705
Accessing parameters, including converting an undefined parameter into
33706
an array.
33707
33708
@item
33709
Symbol table access: retrieving a global variable, creating one,
33710
or changing one.
33711
33712
@item
33713
Allocating, reallocating, and releasing memory.
33714
33715
@item
33716
Creating and releasing cached values; this provides an
33717
efficient way to use values for multiple variables and
33718
can be a big performance win.
33719
33720
@item
33721
Manipulating arrays:
33722
retrieving, adding, deleting, and modifying elements;
33723
getting the count of elements in an array;
33724
creating a new array;
33725
clearing an array;
33726
and
33727
flattening an array for easy C style looping over all its indices and elements
33728
@end itemize
33729
33730
@item
33731
The API defines a number of standard data types for representing
33732
@command{awk} values, array elements, and arrays.
33733
33734
@item
33735
The API provide convenience functions for constructing values.
33736
It also provides memory management functions to ensure compatibility
33737
between memory allocated by @command{gawk} and memory allocated by an
33738
extension.
33739
33740
@item
33741
@emph{All} memory passed from @command{gawk} to an extension must be
33742
treated as read-only by the extension.
33743
33744
@item
33745
@emph{All} memory passed from an extension to @command{gawk} must come from
33746
the API's memory allocation functions. @command{gawk} takes responsibility for
33747
the memory and will release it when appropriate.
33748
33749
@item
33750
The API provides information about the running version of @command{gawk} so
33751
that an extension can make sure it is compatible with the @command{gawk}
33752
that loaded it.
33753
33754
@item
33755
It is easiest to start a new extension by copying the boilerplate code
33756
described in this @value{CHAPTER}. Macros in the @file{gawkapi.h} make
33757
this easier to do.
33758
33759
@item
33760
The @command{gawk} distribution includes a number of small but useful
33761
sample extensions. The @code{gawkextlib} project includes several more,
33762
larger, extensions. If you wish to write an extension and contribute it
33763
to the community of @command{gawk} users, the @code{gawkextlib} project
33764
should be the place to do so.
33765
33766
@end itemize
33767
33768
@node Extension Exercises
33769
@section Exercises
33770
33771
@enumerate
33772
@item
33773
Add functions to implement system calls such as @code{chown()},
33774
@code{chmod()}, and @code{umask()} to the file operations extension
33775
presented in @ref{Internal File Ops}.
33776
33777
@item
33778
(Hard.)
33779
How would you provide namespaces in @command{gawk}, so that the
33780
names of functions in different extensions don't conflict with each other?
33781
If you come up with a really good scheme, contact the @command{gawk}
33782
maintainer to tell him about it.
33783
33784
@item
33785
Write a wrapper script that provides an interface similar to
33786
@samp{sed -i} for the ``inplace'' extension presented in
33787
@ref{Extension Sample Inplace}.
33788
33789
@end enumerate
33790
33791
@ifnotinfo
33792
@part @value{PART4}Appendices
33793
@end ifnotinfo
33794
32101
33795
@ifdocbook
32102
33796
32103
@part Part IV:@* Appendices
32104
32105
Part IV provides the appendices, the Glossary, and two licenses that cover
32106
the @command{gawk} source code and this @value{DOCUMENT}, respectively.
32107
It contains the following appendices:
32108
32109
@itemize @bullet
33797
@ifclear FOR_PRINT
33798
Part IV contains the appendixes (including the two licenses that cover
33799
the @command{gawk} source code and this @value{DOCUMENT}, respectively)
33800
and the Glossary:
33801
@end ifclear
33802
33803
@ifset FOR_PRINT
33804
Part IV contains two appendixes:
33805
@end ifset
33806
33807
@itemize @value{BULLET}
32110
33808
@item
32111
33809
@ref{Language History}.
32112
33810
32113
33811
@item
32114
33812
@ref{Installation}.
32115
33813
33814
@ifclear FOR_PRINT
32116
33815
@item
32117
33816
@ref{Notes}.
32118
33817
32127
33826
32128
33827
@item
32129
33828
@ref{GNU Free Documentation License}.
33829
@end ifclear
32130
33830
@end itemize
32131
33831
@end ifdocbook
32132
@end ignore
32133
33832
32134
33833
@node Language History
32135
33834
@appendix The Evolution of the @command{awk} Language
32136
33835
32137
This @value{DOCUMENT} describes the GNU implementation of @command{awk}, which follows
32138
the POSIX specification.
32139
Many long-time @command{awk} users learned @command{awk} programming
32140
with the original @command{awk} implementation in Version 7 Unix.
32141
(This implementation was the basis for @command{awk} in Berkeley Unix,
32142
through 4.3-Reno. Subsequent versions of Berkeley Unix, and some systems
32143
derived from 4.4BSD-Lite, use various versions of @command{gawk}
32144
for their @command{awk}.)
32145
This @value{CHAPTER} briefly describes the
32146
evolution of the @command{awk} language, with cross-references to other parts
32147
of the @value{DOCUMENT} where you can find more information.
33836
This @value{DOCUMENT} describes the GNU implementation of @command{awk},
33837
which follows the POSIX specification. Many long-time @command{awk}
33838
users learned @command{awk} programming with the original @command{awk}
33839
implementation in Version 7 Unix. (This implementation was the basis for
33840
@command{awk} in Berkeley Unix, through 4.3-Reno. Subsequent versions
33841
of Berkeley Unix, and some systems derived from 4.4BSD-Lite, used various
33842
versions of @command{gawk} for their @command{awk}.) This @value{CHAPTER}
33843
briefly describes the evolution of the @command{awk} language, with
33844
cross-references to other parts of the @value{DOCUMENT} where you can
33845
find more information.
33846
33847
@ifset FOR_PRINT
33848
To save space, we have omitted
33849
information on the history of features in @command{gawk} from this
33850
edition. You can find it in the
33851
@uref{http://www.gnu.org/software/gawk/manual/html_node/Feature-History.html,
33852
online documentation}.
33853
@end ifset
32148
33854
32149
33855
@menu
32150
33856
* V7/SVR3.1:: The major changes between V7 and System V
32156
33862
@command{awk}.
32157
33863
* POSIX/GNU:: The extensions in @command{gawk} not in POSIX
32158
33864
@command{awk}.
33865
* Feature History:: The history of the features in @command{gawk}.
32159
33866
* Common Extensions:: Common Extensions Summary.
32160
33867
* Ranges and Locales:: How locales used to affect regexp ranges.
32161
33868
* Contributors:: The major contributors to @command{gawk}.
33869
* History summary:: History summary.
32162
33870
@end menu
32163
33871
32164
33872
@node V7/SVR3.1
32173
33881
System V Release 3.1 (1987). This @value{SECTION} summarizes the changes, with
32174
33882
cross-references to further details:
32175
33883
32176
@itemize @bullet
33884
@itemize @value{BULLET}
32177
33885
@item
32178
33886
The requirement for @samp{;} to separate rules on a line
32179
33887
(@pxref{Statements/Lines}).
32264
33972
The System V Release 4 (1989) version of Unix @command{awk} added these features
32265
33973
(some of which originated in @command{gawk}):
32266
33974
32267
@itemize @bullet
33975
@itemize @value{BULLET}
32268
33976
@item
32269
33977
The @code{ENVIRON} array (@pxref{Built-in Variables}).
32270
33978
@c gawk and MKS awk
32324
34032
The POSIX Command Language and Utilities standard for @command{awk} (1992)
32325
34033
introduced the following changes into the language:
32326
34034
32327
@itemize @bullet
34035
@itemize @value{BULLET}
32328
34036
@item
32329
34037
The use of @option{-W} for implementation-specific options
32330
34038
(@pxref{Options}).
32349
34057
In 2012, a number of extensions that had been commonly available for
32350
34058
many years were finally added to POSIX. They are:
32351
34059
32352
@itemize @bullet
34060
@itemize @value{BULLET}
32353
34061
@item
32354
34062
The @code{fflush()} built-in function for flushing buffered output
32355
34063
(@pxref{I/O Functions}).
32386
34094
This @value{SECTION} describes common extensions that
32387
34095
originally appeared in his version of @command{awk}.
32388
34096
32389
@itemize @bullet
34097
@itemize @value{BULLET}
32390
34098
@item
32391
34099
The @samp{**} and @samp{**=} operators
32392
34100
(@pxref{Arithmetic Ops}
32404
34112
@ignore
32405
34113
@item
32406
34114
The @code{SYMTAB} array, that allows access to @command{awk}'s internal symbol
32407
table. This feature is not documented, largely because
34115
table. This feature was never documented for his @command{awk}, largely because
32408
34116
it is somewhat shakily implemented. For instance, you cannot access arrays
32409
34117
or array elements through it.
32410
34118
@end ignore
32431
34139
summarizes the additional features over POSIX @command{awk} that are
32432
34140
in the current version of @command{gawk}.
32433
34141
32434
@itemize @bullet
34142
@itemize @value{BULLET}
32435
34143
32436
34144
@item
32437
34145
Additional built-in variables:
32438
34146
32439
@itemize @minus
34147
@itemize @value{MINUS}
32440
34148
@item
32441
34149
The
32442
34150
@code{ARGIND}
32457
34165
@item
32458
34166
Special files in I/O redirections:
32459
34167
32460
@itemize @minus{}
34168
@itemize @value{MINUS}
32461
34169
@item
32462
34170
The @file{/dev/stdin}, @file{/dev/stdout}, @file{/dev/stderr} and
32463
@file{/dev/fd/@var{N}} special file names
34171
@file{/dev/fd/@var{N}} special @value{FN}s
32464
34172
(@pxref{Special Files}).
32465
34173
32466
34174
@item
32473
34181
@item
32474
34182
Changes and/or additions to the language:
32475
34183
32476
@itemize @minus{}
34184
@itemize @value{MINUS}
32477
34185
@item
32478
34186
The @samp{\x} escape sequence
32479
34187
(@pxref{Escape Sequences}).
32512
34220
@item
32513
34221
New keywords:
32514
34222
32515
@itemize @minus{}
34223
@itemize @value{MINUS}
32516
34224
@item
32517
34225
The @code{BEGINFILE} and @code{ENDFILE} special patterns.
32518
34226
(@pxref{BEGINFILE/ENDFILE}).
32533
34241
@item
32534
34242
Changes to standard @command{awk} functions:
32535
34243
32536
@itemize @minus
34244
@itemize @value{MINUS}
32537
34245
@item
32538
34246
The optional second argument to @code{close()} that allows closing one end
32539
34247
of a two-way pipe to a coprocess
32566
34274
@item
32567
34275
Additional functions only in @command{gawk}:
32568
34276
32569
@itemize @minus
34277
@itemize @value{MINUS}
32570
34278
@item
32571
34279
The
32572
34280
@code{and()},
32609
34317
@item
32610
34318
Changes and/or additions in the command-line options:
32611
34319
32612
@itemize @minus
34320
@itemize @value{MINUS}
32613
34321
@item
32614
34322
The @env{AWKPATH} environment variable for specifying a path search for
32615
34323
the @option{-f} command-line option
32684
34392
32685
34393
@item
32686
34394
Support for the following obsolete systems was removed from the code
32687
and the documentation for @command{gawk} version 4.0:
34395
and the documentation for @command{gawk} @value{PVERSION} 4.0:
32688
34396
32689
34397
@c nested table
32690
@itemize @minus
34398
@itemize @value{MINUS}
32691
34399
@item
32692
34400
Amiga
32693
34401
32721
34429
@item
32722
34430
Prestandard VAX C compiler for VAX/VMS
32723
34431
34432
@item
34433
GCC for VAX and Alpha has not been tested for a while.
34434
32724
34435
@end itemize
32725
34436
32726
34437
@end itemize
32731
34442
@c ENDOFRANGE exgnot
32732
34443
@c ENDOFRANGE posnot
32733
34444
34445
@c This does not need to be in the formal book.
34446
@ifclear FOR_PRINT
34447
@node Feature History
34448
@appendixsec History of @command{gawk} Features
34449
34450
@ignore
34451
See the thread:
34452
https://groups.google.com/forum/#!topic/comp.lang.awk/SAUiRuff30c
34453
This motivated me to add this section.
34454
@end ignore
34455
34456
@ignore
34457
I've tried to follow this general order, esp.@: for the 3.0 and 3.1 sections:
34458
variables
34459
special files
34460
language changes (e.g., hex constants)
34461
differences in standard awk functions
34462
new gawk functions
34463
new keywords
34464
new command-line options
34465
behavioral changes
34466
new ports
34467
Within each category, be alphabetical.
34468
@end ignore
34469
34470
This @value{SECTION} describes the features in @command{gawk}
34471
over and above those in POSIX @command{awk},
34472
in the order they were added to @command{gawk}.
34473
34474
Version 2.10 of @command{gawk} introduced the following features:
34475
34476
@itemize @value{BULLET}
34477
@item
34478
The @env{AWKPATH} environment variable for specifying a path search for
34479
the @option{-f} command-line option
34480
(@pxref{Options}).
34481
34482
@item
34483
The @code{IGNORECASE} variable and its effects
34484
(@pxref{Case-sensitivity}).
34485
34486
@item
34487
The @file{/dev/stdin}, @file{/dev/stdout}, @file{/dev/stderr} and
34488
@file{/dev/fd/@var{N}} special @value{FN}s
34489
(@pxref{Special Files}).
34490
@end itemize
34491
34492
Version 2.13 of @command{gawk} introduced the following features:
34493
34494
@itemize @value{BULLET}
34495
@item
34496
The @code{FIELDWIDTHS} variable and its effects
34497
(@pxref{Constant Size}).
34498
34499
@item
34500
The @code{systime()} and @code{strftime()} built-in functions for obtaining
34501
and printing timestamps
34502
(@pxref{Time Functions}).
34503
34504
@item
34505
Additional command-line options
34506
(@pxref{Options}):
34507
34508
@itemize @value{MINUS}
34509
@item
34510
The @option{-W lint} option to provide error and portability checking
34511
for both the source code and at runtime.
34512
34513
@item
34514
The @option{-W compat} option to turn off the GNU extensions.
34515
34516
@item
34517
The @option{-W posix} option for full POSIX compliance.
34518
@end itemize
34519
@end itemize
34520
34521
Version 2.14 of @command{gawk} introduced the following feature:
34522
34523
@itemize @value{BULLET}
34524
@item
34525
The @code{next file} statement for skipping to the next @value{DF}
34526
(@pxref{Nextfile Statement}).
34527
@end itemize
34528
34529
Version 2.15 of @command{gawk} introduced the following features:
34530
34531
@itemize @value{BULLET}
34532
@item
34533
New variables (@pxref{Built-in Variables}):
34534
34535
@itemize @value{MINUS}
34536
@item
34537
@code{ARGIND}, which tracks the movement of @code{FILENAME}
34538
through @code{ARGV}.
34539
34540
@item
34541
@code{ERRNO}, which contains the system error message when
34542
@code{getline} returns @minus{}1 or @code{close()} fails.
34543
@end itemize
34544
34545
@item
34546
The @file{/dev/pid}, @file{/dev/ppid}, @file{/dev/pgrpid}, and
34547
@file{/dev/user} special @value{FN}s. These have since been removed.
34548
34549
@item
34550
The ability to delete all of an array at once with @samp{delete @var{array}}
34551
(@pxref{Delete}).
34552
34553
@item
34554
Command line option changes
34555
(@pxref{Options}):
34556
34557
@itemize @value{MINUS}
34558
@item
34559
The ability to use GNU-style long-named options that start with @option{--}.
34560
34561
@item
34562
The @option{--source} option for mixing command-line and library-file
34563
source code.
34564
@end itemize
34565
@end itemize
34566
34567
Version 3.0 of @command{gawk} introduced the following features:
34568
34569
@itemize @value{BULLET}
34570
@item
34571
New or changed variables:
34572
34573
@itemize @value{MINUS}
34574
@item
34575
@code{IGNORECASE} changed, now applying to string comparison as well
34576
as regexp operations
34577
(@pxref{Case-sensitivity}).
34578
34579
@item
34580
@code{RT}, which contains the input text that matched @code{RS}
34581
(@pxref{Records}).
34582
@end itemize
34583
34584
@item
34585
Full support for both POSIX and GNU regexps
34586
(@pxref{Regexp}).
34587
34588
@item
34589
The @code{gensub()} function for more powerful text manipulation
34590
(@pxref{String Functions}).
34591
34592
@item
34593
The @code{strftime()} function acquired a default time format,
34594
allowing it to be called with no arguments
34595
(@pxref{Time Functions}).
34596
34597
@item
34598
The ability for @code{FS} and for the third
34599
argument to @code{split()} to be null strings
34600
(@pxref{Single Character Fields}).
34601
34602
@item
34603
The ability for @code{RS} to be a regexp
34604
(@pxref{Records}).
34605
34606
@item
34607
The @code{next file} statement became @code{nextfile}
34608
(@pxref{Nextfile Statement}).
34609
34610
@item
34611
The @code{fflush()} function from
34612
Brian Kernighan's @command{awk}
34613
(then at Bell Laboratories;
34614
@pxref{I/O Functions}).
34615
34616
@item
34617
New command line options:
34618
34619
@itemize @value{MINUS}
34620
@item
34621
The @option{--lint-old} option to
34622
warn about constructs that are not available in
34623
the original Version 7 Unix version of @command{awk}
34624
(@pxref{V7/SVR3.1}).
34625
34626
@item
34627
The @option{-m} option from Brian Kernighan's @command{awk}. (He was
34628
still at Bell Laboratories at the time.) This was later removed from
34629
both his @command{awk} and from @command{gawk}.
34630
34631
@item
34632
The @option{--re-interval} option to provide interval expressions in regexps
34633
(@pxref{Regexp Operators}).
34634
34635
@item
34636
The @option{--traditional} option was added as a better name for
34637
@option{--compat} (@pxref{Options}).
34638
@end itemize
34639
34640
@item
34641
The use of GNU Autoconf to control the configuration process
34642
(@pxref{Quick Installation}).
34643
34644
@item
34645
Amiga support.
34646
This has since been removed.
34647
34648
@end itemize
34649
34650
Version 3.1 of @command{gawk} introduced the following features:
34651
34652
@itemize @value{BULLET}
34653
@item
34654
New variables
34655
(@pxref{Built-in Variables}):
34656
34657
@itemize @value{MINUS}
34658
@item
34659
@code{BINMODE}, for non-POSIX systems,
34660
which allows binary I/O for input and/or output files
34661
(@pxref{PC Using}).
34662
34663
@item
34664
@code{LINT}, which dynamically controls lint warnings.
34665
34666
@item
34667
@code{PROCINFO}, an array for providing process-related information.
34668
34669
@item
34670
@code{TEXTDOMAIN}, for setting an application's internationalization text domain
34671
(@pxref{Internationalization}).
34672
@end itemize
34673
34674
@item
34675
The ability to use octal and hexadecimal constants in @command{awk}
34676
program source code
34677
(@pxref{Nondecimal-numbers}).
34678
34679
@item
34680
The @samp{|&} operator for two-way I/O to a coprocess
34681
(@pxref{Two-way I/O}).
34682
34683
@item
34684
The @file{/inet} special files for TCP/IP networking using @samp{|&}
34685
(@pxref{TCP/IP Networking}).
34686
34687
@item
34688
The optional second argument to @code{close()} that allows closing one end
34689
of a two-way pipe to a coprocess
34690
(@pxref{Two-way I/O}).
34691
34692
@item
34693
The optional third argument to the @code{match()} function
34694
for capturing text-matching subexpressions within a regexp
34695
(@pxref{String Functions}).
34696
34697
@item
34698
Positional specifiers in @code{printf} formats for
34699
making translations easier
34700
(@pxref{Printf Ordering}).
34701
34702
@item
34703
A number of new built-in functions:
34704
34705
@itemize @value{MINUS}
34706
@item
34707
The @code{asort()} and @code{asorti()} functions for sorting arrays
34708
(@pxref{Array Sorting}).
34709
34710
@item
34711
The @code{bindtextdomain()}, @code{dcgettext()} and @code{dcngettext()} functions
34712
for internationalization
34713
(@pxref{Programmer i18n}).
34714
34715
@item
34716
The @code{extension()} function and the ability to add
34717
new built-in functions dynamically
34718
(@pxref{Dynamic Extensions}).
34719
34720
@item
34721
The @code{mktime()} function for creating timestamps
34722
(@pxref{Time Functions}).
34723
34724
@item
34725
The @code{and()}, @code{or()}, @code{xor()}, @code{compl()},
34726
@code{lshift()}, @code{rshift()}, and @code{strtonum()} functions
34727
(@pxref{Bitwise Functions}).
34728
@end itemize
34729
34730
@item
34731
@cindex @code{next file} statement
34732
The support for @samp{next file} as two words was removed completely
34733
(@pxref{Nextfile Statement}).
34734
34735
@item
34736
Additional command-line options
34737
(@pxref{Options}):
34738
34739
@itemize @value{MINUS}
34740
@item
34741
The @option{--dump-variables} option to print a list of all global variables.
34742
34743
@item
34744
The @option{--exec} option, for use in CGI scripts.
34745
34746
@item
34747
The @option{--gen-po} command-line option and the use of a leading
34748
underscore to mark strings that should be translated
34749
(@pxref{String Extraction}).
34750
34751
@item
34752
The @option{--non-decimal-data} option to allow non-decimal
34753
input data
34754
(@pxref{Nondecimal Data}).
34755
34756
@item
34757
The @option{--profile} option and @command{pgawk}, the
34758
profiling version of @command{gawk}, for producing execution
34759
profiles of @command{awk} programs
34760
(@pxref{Profiling}).
34761
34762
@item
34763
The @option{--use-lc-numeric} option to force @command{gawk}
34764
to use the locale's decimal point for parsing input data
34765
(@pxref{Conversion}).
34766
@end itemize
34767
34768
@item
34769
The use of GNU Automake to help in standardizing the configuration process
34770
(@pxref{Quick Installation}).
34771
34772
@item
34773
The use of GNU @command{gettext} for @command{gawk}'s own message output
34774
(@pxref{Gawk I18N}).
34775
34776
@item
34777
BeOS support. This was later removed.
34778
34779
@item
34780
Tandem support. This was later removed.
34781
34782
@item
34783
The Atari port became officially unsupported and was
34784
later removed entirely.
34785
34786
@item
34787
The source code changed to use ISO C standard-style function definitions.
34788
34789
@item
34790
POSIX compliance for @code{sub()} and @code{gsub()}
34791
(@pxref{Gory Details}).
34792
34793
@item
34794
The @code{length()} function was extended to accept an array argument
34795
and return the number of elements in the array
34796
(@pxref{String Functions}).
34797
34798
@item
34799
The @code{strftime()} function acquired a third argument to
34800
enable printing times as UTC
34801
(@pxref{Time Functions}).
34802
@end itemize
34803
34804
Version 4.0 of @command{gawk} introduced the following features:
34805
34806
@itemize @value{BULLET}
34807
34808
@item
34809
Variable additions:
34810
34811
@itemize @value{MINUS}
34812
@item
34813
@code{FPAT}, which allows you to specify a regexp that matches
34814
the fields, instead of matching the field separator
34815
(@pxref{Splitting By Content}).
34816
34817
@item
34818
If @code{PROCINFO["sorted_in"]} exists, @samp{for(iggy in foo)} loops sort the
34819
indices before looping over them. The value of this element
34820
provides control over how the indices are sorted before the loop
34821
traversal starts
34822
(@pxref{Controlling Scanning}).
34823
34824
@item
34825
@code{PROCINFO["strftime"]}, which holds
34826
the default format for @code{strftime()}
34827
(@pxref{Time Functions}).
34828
@end itemize
34829
34830
@item
34831
The special files @file{/dev/pid}, @file{/dev/ppid}, @file{/dev/pgrpid}
34832
and @file{/dev/user} were removed.
34833
34834
@item
34835
Support for IPv6 was added via the @file{/inet6} special file.
34836
@file{/inet4} forces IPv4 and @file{/inet} chooses the system
34837
default, which is probably IPv4
34838
(@pxref{TCP/IP Networking}).
34839
34840
@item
34841
The use of @samp{\s} and @samp{\S} escape sequences in regular expressions
34842
(@pxref{GNU Regexp Operators}).
34843
34844
@item
34845
Interval expressions became part of default regular expressions
34846
(@pxref{Regexp Operators}).
34847
34848
@item
34849
POSIX character classes work even with @option{--traditional}
34850
(@pxref{Regexp Operators}).
34851
34852
@item
34853
@code{break} and @code{continue} became invalid outside a loop,
34854
even with @option{--traditional}
34855
(@pxref{Break Statement}, and also see
34856
@ref{Continue Statement}).
34857
34858
@item
34859
@code{fflush()}, @code{nextfile}, and @samp{delete @var{array}}
34860
are allowed if @option{--posix} or @option{--traditional}, since they
34861
are all now part of POSIX.
34862
34863
@item
34864
An optional third argument to
34865
@code{asort()} and @code{asorti()}, specifying how to sort
34866
(@pxref{String Functions}).
34867
34868
@item
34869
The behavior of @code{fflush()} changed to match Brian Kernighan's @command{awk}
34870
and for POSIX; now both @samp{fflush()} and @samp{fflush("")}
34871
flush all open output redirections
34872
(@pxref{I/O Functions}).
34873
34874
@item
34875
The @code{isarray()}
34876
function which distinguishes if an item is an array
34877
or not, to make it possible to traverse arrays of arrays
34878
(@pxref{Type Functions}).
34879
34880
@item
34881
The @code{patsplit()}
34882
function which gives the same capability as @code{FPAT}, for splitting
34883
(@pxref{String Functions}).
34884
34885
@item
34886
An optional fourth argument to the @code{split()} function,
34887
which is an array to hold the values of the separators
34888
(@pxref{String Functions}).
34889
34890
@item
34891
Arrays of arrays
34892
(@pxref{Arrays of Arrays}).
34893
34894
@item
34895
The @code{BEGINFILE} and @code{ENDFILE} special patterns
34896
(@pxref{BEGINFILE/ENDFILE}).
34897
34898
@item
34899
Indirect function calls
34900
(@pxref{Indirect Calls}).
34901
34902
@item
34903
@code{switch} / @code{case} are enabled by default
34904
(@pxref{Switch Statement}).
34905
34906
@item
34907
Command line option changes
34908
(@pxref{Options}):
34909
34910
@itemize @value{MINUS}
34911
@item
34912
The @option{-b} and @option{--characters-as-bytes} options
34913
which prevent @command{gawk} from treating input as a multibyte string.
34914
34915
@item
34916
The redundant @option{--compat}, @option{--copyleft}, and @option{--usage}
34917
long options were removed.
34918
34919
@item
34920
The @option{--gen-po} option was finally renamed to the correct @option{--gen-pot}.
34921
34922
@item
34923
The @option{--sandbox} option which disables certain features.
34924
34925
@item
34926
All long options acquired corresponding short options, for use in @samp{#!} scripts.
34927
@end itemize
34928
34929
@item
34930
Directories named on the command line now produce a warning, not a fatal
34931
error, unless @option{--posix} or @option{--traditional} are used
34932
(@pxref{Command line directories}).
34933
34934
@item
34935
The @command{gawk} internals were rewritten, bringing the @command{dgawk}
34936
debugger and possibly improved performance
34937
(@pxref{Debugger}).
34938
34939
@item
34940
Per the GNU Coding Standards, dynamic extensions must now define
34941
a global symbol indicating that they are GPL-compatible
34942
(@pxref{Plugin License}).
34943
34944
@item
34945
In POSIX mode, string comparisons use @code{strcoll()} / @code{wcscoll()}
34946
(@pxref{POSIX String Comparison}).
34947
34948
@item
34949
The option for raw sockets was removed, since it was never implemented
34950
(@pxref{TCP/IP Networking}).
34951
34952
@item
34953
Ranges of the form @samp{[d-h]} are treated as if they were in the
34954
C locale, no matter what kind of regexp is being used, and even if
34955
@option{--posix}
34956
(@pxref{Ranges and Locales}).
34957
34958
@item
34959
Support was removed for the following systems:
34960
34961
@itemize @value{MINUS}
34962
@item
34963
Atari
34964
34965
@item
34966
Amiga
34967
34968
@item
34969
BeOS
34970
34971
@item
34972
Cray
34973
34974
@item
34975
MIPS RiscOS
34976
34977
@item
34978
MS-DOS with Microsoft Compiler
34979
34980
@item
34981
MS-Windows with Microsoft Compiler
34982
34983
@item
34984
NeXT
34985
34986
@item
34987
SunOS 3.x, Sun 386 (Road Runner)
34988
34989
@item
34990
Tandem (non-POSIX)
34991
34992
@item
34993
Prestandard VAX C compiler for VAX/VMS
34994
@end itemize
34995
@end itemize
34996
34997
Version 4.1 of @command{gawk} introduced the following features:
34998
34999
@itemize @value{BULLET}
35000
35001
@item
35002
Three new arrays:
35003
@code{SYMTAB}, @code{FUNCTAB}, and @code{PROCINFO["identifiers"]}
35004
(@pxref{Auto-set}).
35005
35006
@item
35007
The three executables @command{gawk}, @command{pgawk}, and @command{dgawk}, were merged into
35008
one, named just @command{gawk}. As a result the command line options changed.
35009
35010
@item
35011
Command line option changes
35012
(@pxref{Options}):
35013
35014
@itemize @value{MINUS}
35015
@item
35016
The @option{-D} option invokes the debugger.
35017
35018
@item
35019
The @option{-i} and @option{--include} options
35020
load @command{awk} library files.
35021
35022
@item
35023
The @option{-l} and @option{--load} options load compiled dynamic extensions.
35024
35025
@item
35026
The @option{-M} and @option{--bignum} options enable MPFR.
35027
35028
@item
35029
The @option{-o} only does pretty-printing.
35030
35031
@item
35032
The @option{-p} option is used for profiling.
35033
35034
@item
35035
The @option{-R} option was removed.
35036
@end itemize
35037
35038
@item
35039
Support for high precision arithmetic with MPFR.
35040
(@pxref{Arbitrary Precision Arithmetic}).
35041
35042
@item
35043
The @code{and()}, @code{or()} and @code{xor()} functions
35044
changed to allow any number of arguments,
35045
with a minimum of two
35046
(@pxref{Bitwise Functions}).
35047
35048
@item
35049
The dynamic extension interface was completely redone
35050
(@pxref{Dynamic Extensions}).
35051
35052
@end itemize
35053
35054
@c XXX ADD MORE STUFF HERE
35055
@end ifclear
35056
32734
35057
@node Common Extensions
32735
35058
@appendixsec Common Extensions Summary
32736
35059
32744
35067
@multitable {@file{/dev/stderr} special file} {BWK Awk} {Mawk} {GNU Awk}
32745
35068
@headitem Feature @tab BWK Awk @tab Mawk @tab GNU Awk
32746
35069
@item @samp{\x} Escape sequence @tab X @tab X @tab X
32747
@item @code{RS} as regexp @tab @tab X @tab X
32748
35070
@item @code{FS} as null string @tab X @tab X @tab X
32749
@item @file{/dev/stdin} special file @tab X @tab @tab X
35071
@item @file{/dev/stdin} special file @tab X @tab X @tab X
32750
35072
@item @file{/dev/stdout} special file @tab X @tab X @tab X
32751
35073
@item @file{/dev/stderr} special file @tab X @tab X @tab X
35074
@item @code{delete} without subscript @tab X @tab X @tab X
35075
@item @code{fflush()} function @tab X @tab X @tab X
35076
@item @code{length()} of an array @tab X @tab X @tab X
35077
@item @code{nextfile} statement @tab X @tab X @tab X
32752
35078
@item @code{**} and @code{**=} operators @tab X @tab @tab X
32753
@item @code{fflush()} function @tab X @tab X @tab X
32754
35079
@item @code{func} keyword @tab X @tab @tab X
32755
@item @code{nextfile} statement @tab X @tab X @tab X
32756
@item @code{delete} without subscript @tab X @tab X @tab X
32757
@item @code{length()} of an array @tab X @tab @tab X
32758
35080
@item @code{BINMODE} variable @tab @tab X @tab X
35081
@item @code{RS} as regexp @tab @tab X @tab X
32759
35082
@item Time related functions @tab @tab X @tab X
32760
35083
@end multitable
32761
35084
32775
35098
the first character in the range and the last character in the range,
32776
35099
inclusive. Ordering was based on the numeric value of each character
32777
35100
in the machine's native character set. Thus, on ASCII-based systems,
32778
@code{[a-z]} matched all the lowercase letters, and only the lowercase
35101
@samp{[a-z]} matched all the lowercase letters, and only the lowercase
32779
35102
letters, since the numeric values for the letters from @samp{a} through
32780
35103
@samp{z} were contiguous. (On an EBCDIC system, the range @samp{[a-z]}
32781
35104
includes additional, non-alphabetic characters as well.)
32786
35109
that @samp{[A-Z]} was the ``correct'' way to match uppercase letters.
32787
35110
And indeed, this was true.@footnote{And Life was good.}
32788
35111
32789
The 1993 POSIX standard introduced the idea of locales (@pxref{Locales}).
35112
The 1992 POSIX standard introduced the idea of locales (@pxref{Locales}).
32790
35113
Since many locales include other letters besides the plain twenty-six
32791
35114
letters of the American English alphabet, the POSIX standard added
32792
35115
character classes (@pxref{Bracket Expressions}) as a way to match
32825
35148
This result is due to the locale setting (and thus you may not see
32826
35149
it on your system).
32827
35150
35151
@cindex Unicode
32828
35152
Similar considerations apply to other ranges. For example, @samp{["-/]}
32829
35153
is perfectly valid in ASCII, but is not valid in many Unicode locales,
32830
such as @samp{en_US.UTF-8}.
35154
such as @code{en_US.UTF-8}.
32831
35155
32832
35156
Early versions of @command{gawk} used regexp matching code that was not
32833
35157
locale aware, so ranges had their traditional interpretation.
32836
35160
the problems began; especially as both GNU/Linux and commercial Unix
32837
35161
vendors started implementing non-ASCII locales, @emph{and making them
32838
35162
the default}. Perhaps the most frequently asked question became something
32839
like ``why does @code{[A-Z]} match lowercase letters?!?''
35163
like ``why does @samp{[A-Z]} match lowercase letters?!?''
32840
35164
35165
@cindex Berry, Karl
32841
35166
This situation existed for close to 10 years, if not more, and
32842
35167
the @command{gawk} maintainer grew weary of trying to explain that
32843
35168
@command{gawk} was being nicely standards-compliant, and that the issue
32844
was in the user's locale. During the development of version 4.0,
35169
was in the user's locale. During the development of @value{PVERSION} 4.0,
32845
35170
he modified @command{gawk} to always treat ranges in the original,
32846
35171
pre-POSIX fashion, unless @option{--posix} was used (@pxref{Options}).@footnote{And
32847
thus was born the Campain for Rational Range Interpretation (or RRI). A number
32848
of GNU tools, such as @command{grep} and @command{sed}, have either
32849
implemented this change, or will soon. Thanks to Karl Berry for coining the phrase
32850
``Rational Range Interpretation.''}
35172
thus was born the Campaign for Rational Range Interpretation (or
35173
RRI). A number of GNU tools have either implemented this change,
35174
or will soon. Thanks to Karl Berry for coining the phrase ``Rational
35175
Range Interpretation.''}
32851
35176
32852
35177
Fortunately, shortly before the final release of @command{gawk} 4.0,
32853
35178
the maintainer learned that the 2008 standard had changed the
32860
35185
By using this lovely technical term, the standard gives license
32861
35186
to implementors to implement ranges in whatever way they choose.
32862
35187
The @command{gawk} maintainer chose to apply the pre-POSIX meaning in all
32863
cases: the default regexp matching; with @option{--traditional}, and with
35188
cases: the default regexp matching; with @option{--traditional} and with
32864
35189
@option{--posix}; in all cases, @command{gawk} remains POSIX compliant.
32865
35190
32866
35191
@node Contributors
32874
35199
This @value{SECTION} names the major contributors to @command{gawk}
32875
35200
and/or this @value{DOCUMENT}, in approximate chronological order:
32876
35201
32877
@itemize @bullet
35202
@itemize @value{BULLET}
32878
35203
@item
32879
35204
@cindex Aho, Alfred
32880
35205
@cindex Weinberger, Peter
32954
35279
Michal Jaegermann
32955
35280
provided the port to Atari systems and its documentation.
32956
35281
(This port is no longer supported.)
32957
He continues to provide portability checking with DEC Alpha
32958
systems, and has done a lot of work to make sure @command{gawk}
35282
He continues to provide portability checking,
35283
and has done a lot of work to make sure @command{gawk}
32959
35284
works on non-32-bit systems.
32960
35285
32961
35286
@item
33026
35351
@cindex Peters, Arno
33027
35352
Arno Peters
33028
35353
did the initial work to convert @command{gawk} to use
33029
GNU Automake and GNU @code{gettext}.
35354
GNU Automake and GNU @command{gettext}.
33030
35355
33031
35356
@item
33032
35357
@cindex Broder, Alan J.@:
33056
35381
(This is no longer supported)
33057
35382
33058
35383
@item
35384
@cindex Wallin, Anders
35385
Anders Wallin helped keep the VMS port going for several years.
35386
35387
@item
35388
@cindex Gordon, Assaf
35389
Assaf Gordon contributed the code to implement the
35390
@option{--sandbox} option.
35391
35392
@item
33059
35393
@cindex Haque, John
33060
35394
John Haque made the following contributions:
33061
35395
33062
@itemize @minus
35396
@itemize @value{MINUS}
33063
35397
@item
33064
35398
The modifications to convert @command{gawk}
33065
35399
into a byte-code interpreter, including the debugger.
33066
35400
33067
35401
@item
33068
The addition of true multidimensional arrays.
33069
@ref{Arrays of Arrays}.
35402
The addition of true arrays of arrays.
33070
35403
33071
35404
@item
33072
35405
The additional modifications for support of arbitrary precision arithmetic.
33087
35420
with Pat Rankin.
33088
35421
@end itemize
33089
35422
35423
@cindex Papadopoulos, Panos
35424
@item
35425
Panos Papadopoulos contributed the original text for @ref{Include Files}.
35426
33090
35427
@item
33091
35428
@cindex Yawitz, Efraim
33092
35429
Efraim Yawitz contributed the original text for @ref{Debugger}.
33099
35436
the rest of the development team.
33100
35437
33101
35438
@item
35439
@cindex Colombo, Antonio
35440
Antonio Giovanni Colombo rewrote a number of examples in the early
35441
chapters that were severely dated, for which I am incredibly grateful.
35442
35443
@item
33102
35444
@cindex Robbins, Arnold
33103
35445
Arnold Robbins
33104
35446
has been working on @command{gawk} since 1988, at first
33105
35447
helping David Trueman, and as the primary maintainer since around 1994.
33106
35448
@end itemize
33107
35449
35450
@node History summary
35451
@appendixsec Summary
35452
35453
@itemize @value{BULLET}
35454
@item
35455
The @command{awk} language has evolved over time. The first release
35456
was with V7 Unix circa 1978. In 1987 for System V Release 3.1,
35457
major additions, including user-defined functions, were made to the language.
35458
Additional changes were made for System V Release 4, in 1989.
35459
Since then, further minor changes happen under the auspices of the
35460
POSIX standard.
35461
35462
@item
35463
Brian Kernighan's @command{awk} provides a small number of extensions
35464
that are implemented in common with other versions of @command{awk}.
35465
35466
@item
35467
@command{gawk} provides a large number of extensions over POSIX @command{awk}.
35468
They can be disabled with either the @option{--traditional} or @option{--posix}
35469
options.
35470
35471
@item
35472
The interaction of POSIX locales and regexp matching in @command{gawk} has been confusing over
35473
the years. Today, @command{gawk} implements Rational Range Interpretation, where
35474
ranges of the form @samp{[a-z]} match @emph{only} the characters numerically between
35475
@samp{a} through @samp{z} in the machine's native character set. Usually this is ASCII
35476
but it can be EBCDIC on IBM S/390 systems.
35477
35478
@item
35479
Many people have contributed to @command{gawk} development over the years.
35480
We hope that the list provided in this @value{CHAPTER} is complete and gives
35481
the appropriate credit where credit is due.
35482
35483
@end itemize
35484
33108
35485
@node Installation
33109
35486
@appendix Installing @command{gawk}
33110
35487
33111
35488
@c last two commas are part of see also
33112
@cindex operating systems, See Also GNU/Linux, PC operating systems, Unix
35489
@cindex operating systems, See Also GNU/Linux@comma{} PC operating systems@comma{} Unix
33113
35490
@c STARTOFRANGE gligawk
33114
35491
@cindex @command{gawk}, installing
33115
35492
@c STARTOFRANGE ingawk
33130
35507
* Bugs:: Reporting Problems and Bugs.
33131
35508
* Other Versions:: Other freely available @command{awk}
33132
35509
implementations.
35510
* Installation summary:: Summary of installation.
33133
35511
@end menu
33134
35512
33135
35513
@node Gawk Distribution
33149
35527
@node Getting
33150
35528
@appendixsubsec Getting the @command{gawk} Distribution
33151
35529
@cindex @command{gawk}, source code@comma{} obtaining
33152
There are three ways to get GNU software:
35530
There are two ways to get GNU software:
33153
35531
33154
@itemize @bullet
35532
@itemize @value{BULLET}
33155
35533
@item
33156
35534
Copy it from someone else who already has it.
33157
35535
33190
35568
pipeline to produce the @command{gawk} distribution:
33191
35569
33192
35570
@example
33193
# Under System V, add 'o' to the tar options
33194
35571
gzip -d -c gawk-@value{VERSION}.@value{PATCHLEVEL}.tar.gz | tar -xvpf -
33195
35572
@end example
33196
35573
33206
35583
creates a directory named @file{gawk-@value{VERSION}.@value{PATCHLEVEL}}
33207
35584
in the current directory.
33208
35585
33209
The distribution file name is of the form
35586
The distribution @value{FN} is of the form
33210
35587
@file{gawk-@var{V}.@var{R}.@var{P}.tar.gz}.
33211
35588
The @var{V} represents the major version of @command{gawk},
33212
35589
the @var{R} represents the current release of version @var{V}, and
33345
35722
33346
35723
@item Makefile.am
33347
35724
@itemx */Makefile.am
33348
Files used by the GNU @command{automake} software for generating
33349
the @file{Makefile.in} files used by @command{autoconf} and
35725
Files used by the GNU Automake software for generating
35726
the @file{Makefile.in} files used by Autoconf and
33350
35727
@command{configure}.
33351
35728
33352
35729
@item Makefile.in
33389
35766
The rest of the programs in this @value{DOCUMENT} are available in appropriate
33390
35767
subdirectories of @file{awklib/eg}.
33391
35768
35769
@item extension/*
35770
The source code, manual pages, and infrastructure files for
35771
the sample extensions included with @command{gawk}.
35772
@xref{Dynamic Extensions}, for more information.
35773
33392
35774
@item posix/*
33393
35775
Files needed for building @command{gawk} on POSIX-compliant systems.
33394
35776
33395
35777
@item pc/*
33396
Files needed for building @command{gawk} under MS-Windows and OS/2
35778
Files needed for building @command{gawk} under MS-Windows
35779
@ifclear FOR_PRINT
35780
and OS/2
35781
@end ifclear
33397
35782
(@pxref{PC Installation}, for details).
33398
35783
33399
35784
@item vms/*
33400
Files needed for building @command{gawk} under VMS
35785
Files needed for building @command{gawk} under Vax/VMS and OpenVMS
33401
35786
(@pxref{VMS Installation}, for details).
33402
35787
33403
35788
@item test/*
33434
35819
@command{gawk} is configured
33435
35820
automatically for your system by running the @command{configure} program.
33436
35821
This program is a Bourne shell script that is generated automatically using
33437
GNU @command{autoconf}.
35822
GNU Autoconf.
33438
35823
@ifnotinfo
33439
(The @command{autoconf} software is
35824
(The Autoconf software is
33440
35825
described fully in
33441
35826
@cite{Autoconf---Generating Automatic Configuration Scripts},
33442
35827
which can be found online at
33444
35829
the Free Software Foundation's web site}.)
33445
35830
@end ifnotinfo
33446
35831
@ifinfo
33447
(The @command{autoconf} software is described fully starting with
35832
(The Autoconf software is described fully starting with
33448
35833
@inforef{Top, , Autoconf, autoconf,Autoconf---Generating Automatic Configuration Scripts}.)
33449
35834
@end ifinfo
33450
35835
33492
35877
33493
35878
Of course, once you've built @command{gawk}, it is likely that you will
33494
35879
wish to install it. To do so, you need to run the command @samp{make
33495
check}, as a user with the appropriate permissions. How to do this
35880
install}, as a user with the appropriate permissions. How to do this
33496
35881
varies by system, but on many systems you can use the @command{sudo}
33497
35882
command to do so. The command then becomes @samp{sudo make install}. It
33498
35883
is likely that you will be asked for your password, and you will have
33509
35894
33510
35895
@table @code
33511
35896
33512
@cindex @code{--disable-extensions} configuration option
35897
@cindex @option{--disable-extensions} configuration option
33513
35898
@cindex configuration option, @code{--disable-extensions}
33514
35899
@item --disable-extensions
33515
35900
Disable configuring and building the sample extensions in the
33517
35902
The default action is to dynamically check if the extensions
33518
35903
can be configured and compiled.
33519
35904
33520
@cindex @code{--disable-lint} configuration option
35905
@cindex @option{--disable-lint} configuration option
33521
35906
@cindex configuration option, @code{--disable-lint}
33522
35907
@item --disable-lint
33523
35908
Disable all lint checking within @code{gawk}. The
33537
35922
Using this option will cause some of the tests in the test suite
33538
35923
to fail. This option may be removed at a later date.
33539
35924
33540
@cindex @code{--disable-nls} configuration option
35925
@cindex @option{--disable-nls} configuration option
33541
35926
@cindex configuration option, @code{--disable-nls}
33542
35927
@item --disable-nls
33543
35928
Disable all message-translation facilities.
33544
35929
This is usually not desirable, but it may bring you some slight performance
33545
35930
improvement.
33546
35931
33547
@cindex @code{--with-whiny-user-strftime} configuration option
35932
@cindex @option{--with-whiny-user-strftime} configuration option
33548
35933
@cindex configuration option, @code{--with-whiny-user-strftime}
33549
35934
@item --with-whiny-user-strftime
33550
Force use of the included version of the @code{strftime()}
35935
Force use of the included version of the C @code{strftime()}
33551
35936
function for deficient systems.
33552
35937
@end table
33553
35938
33594
35979
@file{config.h}.
33595
35980
33596
35981
It is also possible that the @command{configure} program generated by
33597
@command{autoconf} will not work on your system in some other fashion.
35982
Autoconf will not work on your system in some other fashion.
33598
35983
If you do have a problem, the file @file{configure.ac} is the input for
33599
@command{autoconf}. You may be able to change this file and generate a
35984
Autoconf. You may be able to change this file and generate a
33600
35985
new version of @command{configure} that works on your system
33601
35986
(@pxref{Bugs},
33602
35987
for information on how to report problems in configuring @command{gawk}).
33624
36009
@cindex PC operating systems@comma{} @command{gawk} on, installing
33625
36010
@cindex operating systems, PC@comma{} @command{gawk} on, installing
33626
36011
This @value{SECTION} covers installation and usage of @command{gawk} on x86 machines
36012
@ifclear FOR_PRINT
33627
36013
running MS-DOS, any version of MS-Windows, or OS/2.
36014
@end ifclear
36015
@ifset FOR_PRINT
36016
running MS-DOS and any version of MS-Windows.
36017
@end ifset
33628
36018
In this @value{SECTION}, the term ``Windows32''
33629
refers to any of Microsoft Windows-95/98/ME/NT/2000/XP/Vista/7.
36019
refers to any of Microsoft Windows-95/98/ME/NT/2000/XP/Vista/7/8.
33630
36020
33631
The limitations of MS-DOS (and MS-DOS shells under Windows32 or OS/2) has meant
33632
that various ``DOS extenders'' are often used with programs such as
33633
@command{gawk}. The varying capabilities of Microsoft Windows 3.1
33634
and Windows32 can add to the confusion. For an overview of the
33635
considerations, please refer to @file{README_d/README.pc} in the
33636
distribution.
36021
The limitations of MS-DOS (and MS-DOS shells under the other operating
36022
systems) has meant that various ``DOS extenders'' are often used with
36023
programs such as @command{gawk}. The varying capabilities of Microsoft
36024
Windows 3.1 and Windows32 can add to the confusion. For an overview
36025
of the considerations, please refer to @file{README_d/README.pc} in
36026
the distribution.
33637
36027
33638
36028
@menu
33639
36029
* PC Binary Installation:: Installing a prepared distribution.
33647
36037
* MSYS:: Using @command{gawk} In The MSYS Environment.
33648
36038
@end menu
33649
36039
36040
@ifclear FOR_PRINT
33650
36041
@node PC Binary Installation
33651
36042
@appendixsubsubsec Installing a Prepared Distribution for PC Systems
33652
36043
33685
36076
33686
36077
The binary distribution may contain a separate file containing additional
33687
36078
or more detailed installation instructions.
36079
@end ifclear
33688
36080
33689
36081
@node PC Compiling
33690
36082
@appendixsubsubsec Compiling @command{gawk} for PC Operating Systems
33691
36083
36084
@ifclear FOR_PRINT
33692
36085
@command{gawk} can be compiled for MS-DOS, Windows32, and OS/2 using the GNU
33693
development tools from DJ Delorie (DJGPP: MS-DOS only) or Eberhard
33694
Mattes (EMX: MS-DOS, Windows32 and OS/2). The file
36086
development tools from DJ Delorie (DJGPP: MS-DOS only), MinGW (Windows32) or Eberhard
36087
Mattes (EMX: MS-DOS, Windows32 and OS/2).
36088
@end ifclear
36089
@ifset FOR_PRINT
36090
@command{gawk} can be compiled for MS-DOS and Windows32 using the GNU
36091
development tools from DJ Delorie (DJGPP: MS-DOS only) or MinGW (Windows32).
36092
@end ifset
36093
The file
33695
36094
@file{README_d/README.pc} in the @command{gawk} distribution contains
33696
36095
additional notes, and @file{pc/Makefile} contains important information on
33697
36096
compilation options.
33713
36112
@uref{ftp://ftp.delorie.com/pub/djgpp/current/v2gnu/}.) To build a
33714
36113
native MS-Windows binary of @command{gawk}, type @samp{make mingw32}.
33715
36114
36115
@ifclear FOR_PRINT
33716
36116
@cindex compiling @command{gawk} with EMX for OS/2
33717
36117
The 32 bit EMX version of @command{gawk} works ``out of the box'' under OS/2.
33718
36118
However, it is highly recommended to use GCC 2.95.3 for the compilation.
33747
36147
@end ignore
33748
36148
33749
36149
@ignore
33750
The internal @code{gettext} library tends to be problematic. It is therefore recommended
36150
The internal @command{gettext} library tends to be problematic. It is therefore recommended
33751
36151
to use either an external one (@option{--without-included-gettext}) or to disable
33752
36152
NLS entirely (@option{--disable-nls}).
33753
36153
@end ignore
33784
36184
the Makefiles of this package. If you encounter any problems with
33785
36185
@command{make}, try GNU Make 3.79.1 or later versions. You should
33786
36186
find the latest version on
33787
@uref{ftp://hobbes.nmsu.edu/pub/os2/}.
36187
@uref{ftp://hobbes.nmsu.edu/pub/os2/}.@footnote{As of May, 2014,
36188
this site is still there, but the author could not find a package
36189
for GNU Make.}
33788
36190
@end quotation
36191
@end ifclear
33789
36192
33790
36193
@node PC Testing
33791
36194
@appendixsubsubsec Testing @command{gawk} on PC Operating Systems
33797
36200
Alternatively, run @command{make check CMP="diff -a"} to use GNU @command{diff}
33798
36201
in text mode instead of @command{cmp} to compare the resulting files.
33799
36202
36203
@ifclear FOR_PRINT
33800
36204
Most
33801
36205
of the tests work properly with Stewartson's shell along with the
33802
36206
companion utilities or appropriate GNU utilities. However, some editing of
33809
36213
@code{fork()}/@code{execl()} to start child processes.
33810
36214
Also the @code{mbfw1} and @code{mbprintf1} tests fail because the needed
33811
36215
multibyte functionality is not available.
33812
36216
@end ifclear
33813
36217
33814
36218
@node PC Using
33815
36219
@appendixsubsubsec Using @command{gawk} on PC Operating Systems
33818
36222
@c STARTOFRANGE pcgawon
33819
36223
@cindex PC operating systems, @command{gawk} on
33820
36224
33821
With the exception of the Cygwin environment,
33822
the @samp{|&} operator and TCP/IP networking
33823
(@pxref{TCP/IP Networking})
33824
are not supported for MS-DOS or MS-Windows. EMX (OS/2 only) does support
33825
at least the @samp{|&} operator.
36225
Under MS-DOS and MS-Windows, the Cygwin and MinGW environments support
36226
both the @samp{|&} operator and TCP/IP networking
36227
(@pxref{TCP/IP Networking}).
36228
@ifclear FOR_PRINT
36229
EMX (OS/2 only) supports at least the @samp{|&} operator.
36230
@end ifclear
33826
36231
33827
36232
@cindex search paths
33828
36233
@cindex search paths, for source files
33829
@cindex @command{gawk}, OS/2 version of
33830
36234
@cindex @command{gawk}, MS-DOS version of
33831
36235
@cindex @command{gawk}, MS-Windows version of
33832
36236
@cindex @code{;} (semicolon), @code{AWKPATH} variable and
33837
36241
semicolons (rather than colons) separate elements in the @env{AWKPATH}
33838
36242
variable. If @env{AWKPATH} is not set or is empty, then the default
33839
36243
search path for MS-Windows and MS-DOS versions is
33840
@code{@w{".;c:/lib/awk;c:/gnu/lib/awk"}}.
36244
@samp{@w{.;c:/lib/awk;c:/gnu/lib/awk}}.
33841
36245
36246
@ifclear FOR_PRINT
36247
@cindex @command{gawk}, OS/2 version of
33842
36248
@cindex @code{UNIXROOT} variable, on OS/2 systems
33843
36249
The search path for OS/2 (32 bit, EMX) is determined by the prefix directory
33844
36250
(most likely @file{/usr} or @file{c:/usr}) that has been specified as an option of
33845
the @command{configure} script like it is the case for the Unix versions.
36251
the @command{configure} script as is the case for the Unix versions.
33846
36252
If @file{c:/usr} is the prefix directory then the default search path contains @file{.}
33847
36253
and @file{c:/usr/share/awk}.
33848
36254
Additionally, to support binary distributions of @command{gawk} for OS/2
33849
systems whose drive @samp{c:} might not support long file names or might not exist
36255
systems whose drive @samp{c:} might not support long @value{FN}s or might not exist
33850
36256
at all, there is a special environment variable. If @env{UNIXROOT} specifies
33851
36257
a drive then this specific drive is also searched for program files.
33852
36258
E.g., if @env{UNIXROOT} is set to @file{e:} the complete default search path is
33853
@code{@w{".;c:/usr/share/awk;e:/usr/share/awk"}}.
36259
@samp{@w{.;c:/usr/share/awk;e:/usr/share/awk}}.
33854
36260
33855
36261
An @command{sh}-like shell (as opposed to @command{command.com} under MS-DOS
33856
36262
or @command{cmd.exe} under MS-Windows or OS/2) may be useful for @command{awk} programming.
33857
36263
The DJGPP collection of tools includes an MS-DOS port of Bash,
33858
36264
and several shells are available for OS/2, including @command{ksh}.
36265
@end ifclear
36266
@ifset FOR_PRINT
36267
An @command{sh}-like shell (as opposed to @command{command.com} under MS-DOS
36268
or @command{cmd.exe} under MS-Windows) may be useful for @command{awk} programming.
36269
The DJGPP collection of tools includes an MS-DOS port of Bash.
36270
@end ifset
33859
36271
33860
36272
@cindex common extensions, @code{BINMODE} variable
33861
36273
@cindex extensions, common@comma{} @code{BINMODE} variable
33862
36274
@cindex differences in @command{awk} and @command{gawk}, @code{BINMODE} variable
33863
36275
@cindex @code{BINMODE} variable
33864
Under MS-Windows, OS/2 and MS-DOS, @command{gawk} (and many other text programs) silently
33865
translate end-of-line @code{"\r\n"} to @code{"\n"} on input and @code{"\n"}
33866
to @code{"\r\n"} on output. A special @code{BINMODE} variable @value{COMMONEXT}
36276
@ifclear FOR_PRINT
36277
Under MS-Windows, OS/2 and MS-DOS,
36278
@end ifclear
36279
@ifset FOR_PRINT
36280
Under MS-Windows and MS-DOS,
36281
@end ifset
36282
@command{gawk} (and many other text programs) silently
36283
translate end-of-line @samp{\r\n} to @samp{\n} on input and @samp{\n}
36284
to @samp{\r\n} on output. A special @code{BINMODE} variable @value{COMMONEXT}
33867
36285
allows control over these translations and is interpreted as follows:
33868
36286
33869
@itemize @bullet
36287
@itemize @value{BULLET}
33870
36288
@item
33871
36289
If @code{BINMODE} is @code{"r"}, or one,
33872
36290
then
33904
36322
@command{mawk} adds a @samp{-W BINMODE=@var{N}} option and an environment
33905
36323
variable that can set @code{BINMODE}, @code{RS}, and @code{ORS}. The
33906
36324
files @file{binmode[1-3].awk} (under @file{gnu/lib/awk} in some of the
33907
prepared distributions) have been chosen to match @command{mawk}'s @samp{-W
36325
prepared binary distributions) have been chosen to match @command{mawk}'s @samp{-W
33908
36326
BINMODE=@var{N}} option. These can be changed or discarded; in particular,
33909
36327
the setting of @code{RS} giving the fewest ``surprises'' is open to debate.
33910
36328
@command{mawk} uses @samp{RS = "\r\n"} if binary mode is set on read, which is
33952
36370
33953
36371
@command{gawk} can be built and used ``out of the box'' under MS-Windows
33954
36372
if you are using the @uref{http://www.cygwin.com, Cygwin environment}.
33955
This environment provides an excellent simulation of Unix, using the
36373
This environment provides an excellent simulation of GNU/Linux, using the
33956
36374
GNU tools, such as Bash, the GNU Compiler Collection (GCC), GNU Make,
33957
36375
and other GNU programs. Compilation and installation for Cygwin is the
33958
36376
same as for a Unix system:
33968
36386
step on Cygwin takes considerably longer. However, it does finish,
33969
36387
and then the @samp{make} proceeds as usual.
33970
36388
33971
@quotation NOTE
33972
The @samp{|&} operator and TCP/IP networking
33973
(@pxref{TCP/IP Networking})
33974
are fully supported in the Cygwin environment. This is not true
33975
for any other environment on MS-Windows.
33976
@end quotation
33977
33978
36389
@node MSYS
33979
36390
@appendixsubsubsec Using @command{gawk} In The MSYS Environment
33980
36391
33987
36398
translation of @code{"\r\n"}, since it won't. Caveat Emptor!
33988
36399
33989
36400
@node VMS Installation
33990
@appendixsubsec How to Compile and Install @command{gawk} on VMS
36401
@appendixsubsec How to Compile and Install @command{gawk} on Vax/VMS and OpenVMS
33991
36402
33992
36403
@c based on material from Pat Rankin <rankin@eql.caltech.edu>
33993
36404
@c now rankin@pactechdata.com
34000
36411
34001
36412
@menu
34002
36413
* VMS Compilation:: How to compile @command{gawk} under VMS.
36414
* VMS Dynamic Extensions:: Compiling @command{gawk} dynamic extensions on
36415
VMS.
34003
36416
* VMS Installation Details:: How to install @command{gawk} under VMS.
34004
36417
* VMS Running:: How to run @command{gawk} under VMS.
36418
* VMS GNV:: The VMS GNV Project.
34005
36419
* VMS Old Gawk:: An old version comes with some VMS systems.
34006
36420
@end menu
34007
36421
34009
36423
@appendixsubsubsec Compiling @command{gawk} on VMS
34010
36424
@cindex compiling @command{gawk} for VMS
34011
36425
34012
To compile @command{gawk} under VMS, there is a @code{DCL} command procedure that
34013
issues all the necessary @code{CC} and @code{LINK} commands. There is
34014
also a @file{Makefile} for use with the @code{MMS} utility. From the source
34015
directory, use either:
34016
34017
@example
34018
$ @kbd{@@[.VMS]VMSBUILD.COM}
34019
@end example
34020
34021
@noindent
34022
or:
34023
34024
@example
34025
$ @kbd{MMS/DESCRIPTION=[.VMS]DESCRIP.MMS GAWK}
34026
@end example
34027
34028
Older versions of @command{gawk} could be built with VAX C or
34029
GNU C on VAX/VMS, as well as with DEC C, but that is no longer
34030
supported. DEC C (also briefly known as ``Compaq C'' and now known
34031
as ``HP C,'' but referred to here as ``DEC C'') is required. Both
34032
@code{VMSBUILD.COM} and @code{DESCRIP.MMS} contain some obsolete support
34033
for the older compilers but are set up to use DEC C by default.
34034
34035
@command{gawk} has been tested under Alpha/VMS 7.3-1 using Compaq C V6.4,
34036
and on Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS 8.3.@footnote{The IA64
34037
architecture is also known as ``Itanium.''}
36426
To compile @command{gawk} under VMS, there is a @code{DCL} command procedure
36427
that issues all the necessary @code{CC} and @code{LINK} commands. There is
36428
also a @file{Makefile} for use with the @code{MMS} and @code{MMK} utilities.
36429
From the source directory, use either:
36430
36431
@example
36432
$ @kbd{@@[.vms]vmsbuild.com}
36433
@end example
36434
36435
@noindent
36436
or:
36437
36438
@example
36439
$ @kbd{MMS/DESCRIPTION=[.vms]descrip.mms gawk}
36440
@end example
36441
36442
@noindent
36443
or:
36444
36445
@example
36446
$ @kbd{MMK/DESCRIPTION=[.vms]descrip.mms gawk}
36447
@end example
36448
36449
@command{MMK} is an open source, free, near-clone of @command{MMS} and
36450
can better handle ODS-5 volumes with upper- and lowercase @value{FN}s.
36451
@command{MMK} is available from @uref{https://github.com/endlesssoftware/mmk}.
36452
36453
With ODS-5 volumes and extended parsing enabled, the case of the target
36454
parameter may need to be exact.
36455
36456
@command{gawk} has been tested under VAX/VMS 7.3 and Alpha/VMS 7.3-1
36457
using Compaq C V6.4, and Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS 8.3.
36458
The most recent builds used HP C V7.3 on Alpha VMS 8.3 and both
36459
Alpha and IA64 VMS 8.4 used HP C 7.3.@footnote{The IA64 architecture
36460
is also known as ``Itanium.''}
36461
36462
@xref{VMS GNV}, for information on building
36463
@command{gawk} as a PCSI kit that is compatible with the GNV product.
36464
36465
@node VMS Dynamic Extensions
36466
@appendixsubsubsec Compiling @command{gawk} Dynamic Extensions on VMS
36467
36468
The extensions that have been ported to VMS can be built using one of
36469
the following commands.
36470
36471
@example
36472
$ @kbd{MMS/DESCRIPTION=[.vms]descrip.mms extensions}
36473
@end example
36474
36475
@noindent
36476
or:
36477
36478
@example
36479
$ @kbd{MMK/DESCRIPTION=[.vms]descrip.mms extensions}
36480
@end example
36481
36482
@command{gawk} uses @code{AWKLIBPATH} as either an environment variable
36483
or a logical name to find the dynamic extensions.
36484
36485
Dynamic extensions need to be compiled with the same compiler options for
36486
floating point, pointer size, and symbol name handling as were used
36487
to compile @command{gawk} itself.
36488
Alpha and Itanium should use IEEE floating point. The pointer size is 32 bits,
36489
and the symbol name handling should be exact case with CRC shortening for
36490
symbols longer than 32 bits.
36491
36492
For Alpha and Itanium:
36493
36494
@example
36495
/name=(as_is,short)
36496
/float=ieee/ieee_mode=denorm_results
36497
@end example
36498
36499
For VAX:
36500
36501
@example
36502
/name=(as_is,short)
36503
@end example
36504
36505
Compile time macros need to be defined before the first VMS-supplied
36506
header file is included.
36507
36508
@example
36509
#if (__CRTL_VER >= 70200000) && !defined (__VAX)
36510
#define _LARGEFILE 1
36511
#endif
36512
36513
#ifndef __VAX
36514
#ifdef __CRTL_VER
36515
#if __CRTL_VER >= 80200000
36516
#define _USE_STD_STAT 1
36517
#endif
36518
#endif
36519
#endif
36520
@end example
34038
36521
34039
36522
@node VMS Installation Details
34040
36523
@appendixsubsubsec Installing @command{gawk} on VMS
34041
36524
34042
To install @command{gawk}, all you need is a ``foreign'' command, which is
34043
a @code{DCL} symbol whose value begins with a dollar sign. For example:
36525
To use @command{gawk}, all you need is a ``foreign'' command, which is a
36526
@code{DCL} symbol whose value begins with a dollar sign. For example:
34044
36527
34045
36528
@example
34046
$ @kbd{GAWK :== $disk1:[gnubin]GAWK}
36529
$ @kbd{GAWK :== $disk1:[gnubin]gawk}
34047
36530
@end example
34048
36531
34049
36532
@noindent
34055
36538
@file{sylogin.com} procedure, which allows all users
34056
36539
to run @command{gawk}.
34057
36540
34058
Optionally, the help entry can be loaded into a VMS help library:
34059
34060
@example
34061
$ @kbd{LIBRARY/HELP SYS$HELP:HELPLIB [.VMS]GAWK.HLP}
36541
If your @command{gawk} was installed by a PCSI kit into the
36542
@file{GNV$GNU:} directory tree, the program will be known as
36543
@file{GNV$GNU:[bin]gnv$gawk.exe} and the help file will be
36544
@file{GNV$GNU:[vms_help]gawk.hlp}.
36545
36546
The PCSI kit also installs a @file{GNV$GNU:[vms_bin]gawk_verb.cld} file
36547
which can be used to add @command{gawk} and @command{awk} as DCL commands.
36548
36549
For just the current process you can use:
36550
36551
@example
36552
$ @kbd{set command gnv$gnu:[vms_bin]gawk_verb.cld}
36553
@end example
36554
36555
Or the system manager can use @file{GNV$GNU:[vms_bin]gawk_verb.cld} to
36556
add the @command{gawk} and @command{awk} to the system wide @samp{DCLTABLES}.
36557
36558
The DCL syntax is documented in the @file{gawk.hlp} file.
36559
36560
Optionally, the @file{gawk.hlp} entry can be loaded into a VMS help library:
36561
36562
@example
36563
$ @kbd{LIBRARY/HELP sys$help:helplib [.vms]gawk.hlp}
34062
36564
@end example
34063
36565
34064
36566
@noindent
34076
36578
34077
36579
The logical name @samp{AWK_LIBRARY} can designate a default location
34078
36580
for @command{awk} program files. For the @option{-f} option, if the specified
34079
file name has no device or directory path information in it, @command{gawk}
36581
@value{FN} has no device or directory path information in it, @command{gawk}
34080
36582
looks in the current directory first, then in the directory specified
34081
36583
by the translation of @samp{AWK_LIBRARY} if the file is not found.
34082
36584
If, after searching in both directories, the file still is not found,
34083
@command{gawk} appends the suffix @samp{.awk} to the filename and retries
36585
@command{gawk} appends the suffix @samp{.awk} to the @value{FN} and retries
34084
36586
the file search. If @samp{AWK_LIBRARY} has no definition, a default value
34085
36587
of @samp{SYS$LIBRARY:} is used for it.
34086
36588
34109
36611
single parameter (as in the quoted string program above), the command
34110
36612
becomes ambiguous. To work around this, the normally optional @option{--}
34111
36613
flag is required to force Unix-style parsing rather than @code{DCL} parsing. If any
34112
other dash-type options (or multiple parameters such as data files to
36614
other dash-type options (or multiple parameters such as @value{DF}s to
34113
36615
process) are present, there is no ambiguity and @option{--} can be omitted.
34114
36616
36617
@cindex exit status, of VMS
36618
The @code{exit} value is a Unix-style value and is encoded to a VMS exit
36619
status value when the program exits.
36620
36621
The VMS severity bits will be set based on the @code{exit} value.
36622
A failure is indicated by 1 and VMS sets the @code{ERROR} status.
36623
A fatal error is indicated by 2 and VMS will set the @code{FATAL} status.
36624
All other values will have the @code{SUCCESS} status. The exit value is
36625
encoded to comply with VMS coding standards and will have the
36626
@code{C_FACILITY_NO} of @code{0x350000} with the constant @code{0xA000}
36627
added to the number shifted over by 3 bits to make room for the severity codes.
36628
36629
To extract the actual @command{gawk} exit code from the VMS status use:
36630
36631
@example
36632
unix_status = (vms_status .and. &x7f8) / 8
36633
@end example
36634
36635
@noindent
36636
A C program that uses @code{exec()} to call @command{gawk} will get the original
36637
Unix-style exit value.
36638
36639
Older versions of @command{gawk} treated a Unix exit code 0 as 1, a failure
36640
as 2, a fatal error as 4, and passed all the other numbers through.
36641
This violated the VMS exit status coding requirements.
36642
36643
@cindex floating-point, VAX/VMS
36644
VAX/VMS floating point uses unbiased rounding. @xref{Round Function}.
36645
36646
VMS reports time values in GMT unless one of the @code{SYS$TIMEZONE_RULE}
36647
or @code{TZ} logical names is set. Older versions of VMS, such as VAX/VMS
36648
7.3 do not set these logical names.
36649
34115
36650
@c @cindex directory search
34116
36651
@c @cindex path, search
34117
36652
@cindex search paths
34123
36658
When defining it, the value should be quoted so that it retains a single
34124
36659
translation and not a multitranslation @code{RMS} searchlist.
34125
36660
36661
@node VMS GNV
36662
@appendixsubsubsec The VMS GNV Project
36663
36664
The VMS GNV package provides a build environment similar to POSIX with ports
36665
of a collection of open source tools. The @command{gawk} found in the GNV
36666
base kit is an older port. Currently the GNV project is being reorganized
36667
to supply individual PCSI packages for each component.
36668
See @w{@uref{https://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/}.}
36669
36670
The normal build procedure for @command{gawk} produces a program that
36671
is suitable for use with GNV.
36672
36673
The @file{vms/gawk_build_steps.txt} in the source documents the procedure
36674
for building a VMS PCSI kit that is compatible with GNV.
36675
34126
36676
@ignore
34127
36677
@c The VMS POSIX product, also known as POSIX for OpenVMS, is long defunct
34128
36678
@c and building gawk for it has not been tested in many years, but these
34170
36720
$ @kbd{gawk :== $sys$common:[syshlp.examples.tcpip.snmp]gawk.exe}
34171
36721
@end example
34172
36722
34173
This is apparently version 2.15.6, which is extremely old. We
36723
This is apparently @value{PVERSION} 2.15.6, which is extremely old. We
34174
36724
recommend compiling and using the current version.
34175
36725
34176
36726
@c ENDOFRANGE opgawx
34199
36749
to do something or not, report that too; it's a bug in the documentation!
34200
36750
34201
36751
Before reporting a bug or trying to fix it yourself, try to isolate it
34202
to the smallest possible @command{awk} program and input data file that
34203
reproduces the problem. Then send us the program and data file,
36752
to the smallest possible @command{awk} program and input @value{DF} that
36753
reproduces the problem. Then send us the program and @value{DF},
34204
36754
some idea of what kind of Unix system you're using,
34205
36755
the compiler you used to compile @command{gawk}, and the exact results
34206
36756
@command{gawk} gave you. Also say what you expected to occur; this helps
34216
36766
@EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org}.
34217
36767
34218
36768
@cindex Robbins, Arnold
34219
Using this address automatically sends a copy of your
34220
mail to me. If necessary, I can be reached directly at
36769
The @command{gawk} maintainers subscribe to this address and
36770
thus they will receive your bug report.
36771
If necessary, the primary maintainer can be reached directly at
34221
36772
@EMAIL{arnold@@skeeve.com,arnold at skeeve dot com}.
34222
36773
The bug reporting address is preferred since the
34223
36774
email list is archived at the GNU Project.
34224
@emph{All email should be in English, since that is my native language.}
36775
@emph{All email should be in English. This is the only language
36776
understood in common by all the maintainers.}
34225
36777
34226
36778
@cindex @code{comp.lang.awk} newsgroup
34227
36779
@quotation CAUTION
34254
36806
34255
36807
If you find bugs in one of the non-Unix ports of @command{gawk}, please send
34256
36808
an electronic mail message to the person who maintains that port. They
34257
are named in the following list, as well as in the @file{README} file in the @command{gawk}
34258
distribution. Information in the @file{README} file should be considered
34259
authoritative if it conflicts with this @value{DOCUMENT}.
36809
are named in the following list, as well as in the @file{README} file
36810
in the @command{gawk} distribution. Information in the @file{README}
36811
file should be considered authoritative if it conflicts with this
36812
@value{DOCUMENT}.
34260
36813
34261
36814
The people maintaining the non-Unix ports of @command{gawk} are
34262
36815
as follows:
34263
36816
34264
@multitable {MS-Windows with MINGW} {123456789012345678901234567890123456789001234567890}
36817
@c put the index entries outside the table, for docbook
34265
36818
@cindex Deifik, Scott
36819
@cindex Zaretskii, Eli
36820
@cindex Buening, Andreas
36821
@cindex Rankin, Pat
36822
@cindex Malmberg, John
36823
@cindex Pitts, Dave
36824
@multitable {MS-Windows with MinGW} {123456789012345678901234567890123456789001234567890}
34266
36825
@item MS-DOS with DJGPP @tab Scott Deifik, @EMAIL{scottd.mail@@sbcglobal.net,scottd dot mail at sbcglobal dot net}.
34267
36826
34268
@cindex Zaretskii, Eli
34269
@item MS-Windows with MINGW @tab Eli Zaretskii, @EMAIL{eliz@@gnu.org,eliz at gnu dot org}.
36827
@item MS-Windows with MinGW @tab Eli Zaretskii, @EMAIL{eliz@@gnu.org,eliz at gnu dot org}.
34270
36828
34271
@cindex Buening, Andreas
36829
@c Leave this in the print version on purpose.
36830
@c OS/2 is not mentioned anywhere else in the print version though.
34272
36831
@item OS/2 @tab Andreas Buening, @EMAIL{andreas.buening@@nexgo.de,andreas dot buening at nexgo dot de}.
34273
36832
34274
@cindex Rankin, Pat
34275
@item VMS @tab Pat Rankin, @EMAIL{r.pat.rankin@@gmail.com,r.pat.rankin at gmail.com}
36833
@item VMS @tab Pat Rankin, @EMAIL{r.pat.rankin@@gmail.com,r.pat.rankin at gmail.com}, and
36834
John Malmberg, @EMAIL{wb8tyw@@qsl.net,wb8tyw at qsl.net}.
34276
36835
34277
@cindex Pitts, Dave
34278
36836
@item z/OS (OS/390) @tab Dave Pitts, @EMAIL{dpitts@@cozx.com,dpitts at cozx dot com}.
34279
36837
@end multitable
34280
36838
34281
36839
If your bug is also reproducible under Unix, please send a copy of your
34282
report to the @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org} email list as well.
36840
report to the @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org} email
36841
list as well.
34283
36842
@c ENDOFRANGE dbugg
34284
36843
@c ENDOFRANGE tblgawb
34285
36844
34308
36867
@cindex Kernighan, Brian
34309
36868
@cindex source code, Brian Kernighan's @command{awk}
34310
36869
@cindex @command{awk}, versions of, See Also Brian Kernighan's @command{awk}
34311
@cindex Brian Kernighan's @command{awk}
36870
@cindex Brian Kernighan's @command{awk}, source code
34312
36871
@item Unix @command{awk}
34313
36872
Brian Kernighan, one of the original designers of Unix @command{awk},
34314
36873
has made his implementation of
34328
36887
@uref{http://www.cs.princeton.edu/~bwk/btl.mirror/awk.zip}
34329
36888
@end table
34330
36889
36890
@cindex @command{git} utility
34331
36891
You can also retrieve it from Git Hub:
34332
36892
34333
36893
@example
34347
36907
for a list of extensions in this @command{awk} that are not in POSIX @command{awk}.
34348
36908
34349
36909
@cindex Brennan, Michael
34350
@cindex @command{mawk} program
36910
@cindex @command{mawk} utility
34351
36911
@cindex source code, @command{mawk}
34352
36912
@item @command{mawk}
34353
36913
Michael Brennan wrote an independent implementation of @command{awk},
34354
called @command{mawk}. It is available under the GPL
34355
(@pxref{Copying}),
36914
called @command{mawk}. It is available under the
36915
@ifclear FOR_PRINT
36916
GPL (@pxref{Copying}),
36917
@end ifclear
36918
@ifset FOR_PRINT
36919
GPL,
36920
@end ifset
34356
36921
just as @command{gawk} is.
34357
36922
34358
36923
The original distribution site for the @command{mawk} source code
34393
36958
The project seems to be frozen; no new code changes have been made
34394
36959
since approximately 2003.
34395
36960
34396
@cindex Beebe, Nelson
36961
@cindex Beebe, Nelson H.F.@:
34397
36962
@cindex @command{pawk} (profiling version of Brian Kernighan's @command{awk})
34398
36963
@cindex source code, @command{pawk}
34399
36964
@item @command{pawk}
34421
36986
@cindex Solaris, POSIX-compliant @command{awk}
34422
36987
@cindex source code, Solaris @command{awk}
34423
36988
@item The OpenSolaris POSIX @command{awk}
34424
The version of @command{awk} in @file{/usr/xpg4/bin} on Solaris is
34425
more-or-less POSIX-compliant. It is based on the @command{awk} from
34426
Mortice Kern Systems for PCs.
34427
This author was able to make it compile and work under GNU/Linux
36989
The versions of @command{awk} in @file{/usr/xpg4/bin} and
36990
@file{/usr/xpg6/bin} on Solaris are more-or-less POSIX-compliant.
36991
They are based on the @command{awk} from Mortice Kern Systems for PCs.
36992
This author was able to make this code compile and work under GNU/Linux
34428
36993
with 1--2 hours of work. Making it more generally portable (using
34429
36994
GNU Autoconf and/or Automake) would take more work, and this
34430
36995
has not been done, at least to our knowledge.
34456
37021
@uref{http://repo.hu/projects/libmawk/}.
34457
37022
34458
37023
@item @code{pawk}
37024
@cindex source code, @command{pawk} (Python version)
34459
37025
@cindex @code{pawk}, @command{awk}-like facilities for Python
34460
37026
This is a Python module that claims to bring @command{awk}-like
34461
37027
features to Python. See @uref{https://github.com/alecthomas/pawk}
34478
37044
See @uref{http://www.quiktrim.org/QTawk.html} for more information,
34479
37045
including the manual and a download link.
34480
37046
37047
The project may also be frozen; no new code changes have been made
37048
since approximately 2008.
37049
34481
37050
@item Other Versions
34482
37051
See also the @uref{http://en.wikipedia.org/wiki/Awk_language#Versions_and_implementations,
34483
37052
Wikipedia article}, for information on additional versions.
34484
37053
34485
37054
@end table
37055
@c ENDOFRANGE awkim
37056
37057
@node Installation summary
37058
@appendixsec Summary
37059
37060
@itemize @value{BULLET}
37061
@item
37062
The @command{gawk} distribution is available from GNU project's main
37063
distribution site, @code{ftp.gnu.org}. The canonical build recipe is:
37064
37065
@example
37066
wget http://ftp.gnu.org/gnu/gawk/gawk-@value{VERSION}.@value{PATCHLEVEL}.tar.gz
37067
tar -xvpzf gawk-@value{VERSION}.@value{PATCHLEVEL}.tar.gz
37068
cd gawk-@value{VERSION}.@value{PATCHLEVEL}
37069
./configure && make && make check
37070
@end example
37071
37072
@item
37073
@command{gawk} may be built on non-POSIX systems as well. The currently
37074
supported systems are MS-Windows using DJGPP, MSYS, MinGW and Cygwin,
37075
@ifclear FOR_PRINT
37076
OS/2 using EMX,
37077
@end ifclear
37078
and both Vax/VMS and OpenVMS.
37079
Instructions for each system are included in this @value{CHAPTER}.
37080
37081
@item
37082
Bug reports should be sent via email to @email{bug-gawk@@gnu.org}.
37083
Bug reports should be in English, and should include the version of @command{gawk},
37084
how it was compiled, and a short program and @value{DF} which demonstrate
37085
the problem.
37086
37087
@item
37088
There are a number of other freely available @command{awk}
37089
implementations. Many are POSIX compliant; others are less so.
37090
37091
@end itemize
37092
34486
37093
@c ENDOFRANGE gligawk
34487
37094
@c ENDOFRANGE ingawk
34488
@c ENDOFRANGE awkim
34489
37095
37096
@ifclear FOR_PRINT
34490
37097
@node Notes
34491
37098
@appendix Implementation Notes
34492
37099
@c STARTOFRANGE gawii
34506
37113
* Implementation Limitations:: Some limitations of the implementation.
34507
37114
* Extension Design:: Design notes about the extension API.
34508
37115
* Old Extension Mechanism:: Some compatibility for old extensions.
37116
* Notes summary:: Summary of implementation notes.
34509
37117
@end menu
34510
37118
34511
37119
@node Compatibility Mode
34526
37134
@table @code
34527
37135
@item -Y
34528
37136
@itemx --parsedebug
34529
Prints out the parse stack information as the program is being parsed.
37137
Print out the parse stack information as the program is being parsed.
34530
37138
@end table
34531
37139
34532
37140
This option is intended only for serious @command{gawk} developers
34550
37158
@command{gawk}.
34551
37159
* New Ports:: Porting @command{gawk} to a new operating
34552
37160
system.
34553
* Derived Files:: Why derived files are kept in the
34554
@command{git} repository.
37161
* Derived Files:: Why derived files are kept in the Git
37162
repository.
34555
37163
@end menu
34556
37164
34557
37165
@node Accessing The Source
34561
37169
@ref{Gawk Distribution}, describes how to get and build the formal,
34562
37170
released versions of @command{gawk}.
34563
37171
37172
@cindex @command{git} utility
34564
37173
However, if you want to modify @command{gawk} and contribute back your
34565
37174
changes, you will probably wish to work with the development version.
34566
37175
To do so, you will need to access the @command{gawk} source code
34574
37183
@end example
34575
37184
34576
37185
@noindent
34577
This will clone the @command{gawk} repository. If you are behind a
34578
firewall that will not allow you to use the Git native protocol, you
37186
This clones the @command{gawk} repository. If you are behind a
37187
firewall that does not allow you to use the Git native protocol, you
34579
37188
can still access the repository using:
34580
37189
34581
37190
@example
34603
37212
You are free to add any new features you like to @command{gawk}.
34604
37213
However, if you want your changes to be incorporated into the @command{gawk}
34605
37214
distribution, there are several steps that you need to take in order to
34606
make it possible to include your changes:
37215
make it possible to include them:
34607
37216
34608
37217
@enumerate 1
34609
37218
@item
34625
37234
@item
34626
37235
Get the latest version.
34627
37236
It is much easier for me to integrate changes if they are relative to
34628
the most recent distributed version of @command{gawk}. If your version of
34629
@command{gawk} is very old, I may not be able to integrate them at all.
37237
the most recent distributed version of @command{gawk}, or better yet,
37238
relative to the latest code in the Git repository. If your version of
37239
@command{gawk} is very old, I may not be able to integrate your changes at all.
34630
37240
(@xref{Getting},
34631
37241
for information on getting the latest version of @command{gawk}.)
34632
37242
34633
37243
@item
34634
37244
@ifnotinfo
34635
Follow the @cite{GNU Coding Standards}.
37245
Follow the @uref{http://www.gnu.org/prep/standards/, @cite{GNU Coding Standards}}.
34636
37246
@end ifnotinfo
34637
37247
@ifinfo
34638
37248
See @inforef{Top, , Version, standards, GNU Coding Standards}.
34653
37263
of braces and the use of TABs. In brief, the coding rules for @command{gawk}
34654
37264
are as follows:
34655
37265
34656
@itemize @bullet
37266
@itemize @value{BULLET}
34657
37267
@item
34658
37268
Use ANSI/ISO style (prototype) function headers when defining functions.
34659
37269
34736
37346
34737
37347
You will also have to sign paperwork for your documentation changes.
34738
37348
37349
@cindex @command{git} utility
34739
37350
@item
34740
37351
Submit changes as unified diffs.
34741
37352
Use @samp{diff -u -r -N} to compare
34756
37367
Include an entry for the @file{ChangeLog} file with your submission.
34757
37368
This helps further minimize the amount of work I have to do,
34758
37369
making it easier for me to accept patches.
37370
It is simplest if you just make this part of your diff.
34759
37371
@end enumerate
34760
37372
34761
37373
Although this sounds like a lot of work, please remember that while you
34791
37403
In order for the FSF to distribute your code, you must either place
34792
37404
your code in the public domain and submit a signed statement to that
34793
37405
effect, or assign the copyright in your code to the FSF.
34794
@ifinfo
34795
37406
Both of these actions are easy to do and @emph{many} people have done so
34796
37407
already. If you have questions, please contact me, or
34797
37408
@email{gnu@@gnu.org}.
34798
@end ifinfo
34799
37409
34800
37410
@item
34801
37411
When doing a port, bear in mind that your code must coexist peacefully
34815
37425
people. Thus, you should not change them
34816
37426
unless it is for a very good reason; i.e., changes are not out of the
34817
37427
question, but changes to these files are scrutinized extra carefully.
34818
The files are @file{dfa.c}, @file{dfa.h}, @file{getopt1.c}, @file{getopt.c},
34819
@file{getopt.h}, @file{install-sh}, @file{mkinstalldirs}, @file{regcomp.c},
34820
@file{regex.c}, @file{regexec.c}, @file{regexex.c}, @file{regex.h},
34821
@file{regex_internal.c}, and @file{regex_internal.h}.
37428
The files are
37429
@file{dfa.c},
37430
@file{dfa.h},
37431
@file{getopt.c},
37432
@file{getopt.h},
37433
@file{getopt1.c},
37434
@file{getopt_int.h},
37435
@file{gettext.h},
37436
@file{regcomp.c},
37437
@file{regex.c},
37438
@file{regex.h},
37439
@file{regex_internal.c},
37440
@file{regex_internal.h},
37441
and
37442
@file{regexec.c}.
37443
37444
@item
37445
A number of other files are provided by the GNU
37446
Autotools (Autoconf, Automake, and GNU @command{gettext}).
37447
You should not change them either, unless it is for a very
37448
good reason. The files are
37449
@file{ABOUT-NLS},
37450
@file{config.guess},
37451
@file{config.rpath},
37452
@file{config.sub},
37453
@file{depcomp},
37454
@file{INSTALL},
37455
@file{install-sh},
37456
@file{missing},
37457
@file{mkinstalldirs},
37458
@file{xalloc.h},
37459
and
37460
@file{ylwrap}.
34822
37461
34823
37462
@item
34824
37463
Be willing to continue to maintain the port.
34869
37508
coding style and brace layout that suits your taste.
34870
37509
34871
37510
@node Derived Files
34872
@appendixsubsec Why Generated Files Are Kept In @command{git}
37511
@appendixsubsec Why Generated Files Are Kept In Git
34873
37512
37513
@c STARTOFRANGE gawkgit
37514
@cindex Git, use of for @command{gawk} source code
34874
37515
@c From emails written March 22, 2012, to the gawk developers list.
34875
37516
34876
If you look at the @command{gawk} source in the @command{git}
37517
If you look at the @command{gawk} source in the Git
34877
37518
repository, you will notice that it includes files that are automatically
34878
37519
generated by GNU infrastructure tools, such as @file{Makefile.in} from
34879
@command{automake} and even @file{configure} from @command{autoconf}.
37520
Automake and even @file{configure} from Autoconf.
34880
37521
34881
37522
This is different from many Free Software projects that do not store
34882
37523
the derived files, because that keeps the repository less cluttered,
34902
37543
@emph{it} would build?)
34903
37544
34904
37545
If the repository has all the generated files, then it's easy to just check
34905
them out and build. (Or @emph{easier}, depending upon how far back we go.
34906
@code{:-)})
37546
them out and build. (Or @emph{easier}, depending upon how far back we go.)
34907
37547
34908
37548
And that brings us to the second (and stronger) reason why all the files
34909
really need to be in @command{git}. It boils down to who do you cater
37549
really need to be in Git. It boils down to who do you cater
34910
37550
to---the @command{gawk} developer(s), or the user who just wants to check
34911
37551
out a version and try it out?
34912
37552
34915
37555
world to just clone the repository, check out the branch of interest and
34916
37556
build it. Without their having to have the correct version(s) of the
34917
37557
autotools.@footnote{There is one GNU program that is (in our opinion)
34918
severely difficult to bootstrap from the @command{git} repository. For
34919
example, on the author's old (but still working) PowerPC macintosh with
37558
severely difficult to bootstrap from the Git repository. For
37559
example, on the author's old (but still working) PowerPC Macintosh with
34920
37560
Mac OS X 10.5, it was necessary to bootstrap a ton of software, starting
34921
with @command{git} itself, in order to try to work with the latest code.
37561
with Git itself, in order to try to work with the latest code.
34922
37562
It's not pleasant, and especially on older systems, it's a big waste
34923
37563
of time.
34924
37564
34941
37581
34942
37582
Further, the @command{gawk} maintainer would argue that it's also
34943
37583
important for the @command{gawk} developers. When he tried to check out
34944
the @code{xgawk} branch@footnote{A branch created by one of the other
37584
the @code{xgawk} branch@footnote{A branch (since removed) created by one of the other
34945
37585
developers that did not include the generated files.} to build it, he
34946
37586
couldn't. (No @file{ltmain.sh} file, and he had no idea how to create it,
34947
37587
and that was not the only problem.)
34948
37588
34949
37589
He felt @emph{extremely} frustrated. With respect to that branch,
34950
37590
the maintainer is no different than Jane User who wants to try to build
34951
@code{gawk-4.0-stable} or @code{master} from the repository.
37591
@code{gawk-4.1-stable} or @code{master} from the repository.
34952
37592
34953
37593
Thus, the maintainer thinks that it's not just important, but critical,
34954
37594
that for any given branch, the above incantation @emph{just works}.
34968
37608
34969
37609
@item
34970
37610
He is really good at @samp{git diff x y > /tmp/diff1 ; gvim /tmp/diff1} to
34971
remove the diffs that aren't of interest in order to review code. @code{:-)}
37611
remove the diffs that aren't of interest in order to review code.
34972
37612
@end enumerate
34973
37613
34974
37614
@item
34975
37615
It would certainly help if everyone used the same versions of the GNU tools
34976
37616
as he does, which in general are the latest released versions of
34977
@command{automake},
34978
@command{autoconf},
37617
Automake,
37618
Autoconf,
34979
37619
@command{bison},
34980
37620
and
34981
@command{gettext}.
37621
GNU @command{gettext}.
34982
37622
34983
37623
@ignore
34984
If it would help if I sent out an "I just upgraded to version x.y
34985
of tool Z" kind of message to this list, I can do that. Up until
37624
If it would help if I sent out an ``I just upgraded to version x.y
37625
of tool Z'' kind of message to this list, I can do that. Up until
34986
37626
now it hasn't been a real issue since I'm the only one who's been
34987
37627
dorking with the configuration machinery.
34988
37628
@end ignore
34989
37629
34990
@enumerate A
34991
@item
37630
@c @enumerate A
37631
@c @item
34992
37632
Installing from source is quite easy. It's how the maintainer worked for years
34993
under Fedora.
37633
(and still works).
34994
37634
He had @file{/usr/local/bin} at the front of his @env{PATH} and just did:
34995
37635
34996
37636
@example
35001
37641
make install # as root
35002
37642
@end example
35003
37643
35004
@item
37644
@c @item
37645
@ignore
35005
37646
These days the maintainer uses Ubuntu 12.04 which is medium current, but
35006
he is already doing the above for @command{autoconf}, @command{automake}
35007
and @command{bison}.
37647
he is already doing the above for Automake, Autoconf, and @command{bison}.
37648
@end ignore
35008
37649
35009
37650
@ignore
35010
37651
(C. Rant: Recent Linux versions with GNOME 3 really suck. What
35012
37653
me to Ubuntu, but Ubuntu 11.04 and 11.10 are totally unusable from
35013
37654
a UI perspective. Bleah.)
35014
37655
@end ignore
35015
@end enumerate
37656
@c @end enumerate
35016
37657
35017
37658
@ignore
35018
37659
@item
35028
37669
Most of the above was originally written by the maintainer to other
35029
37670
@command{gawk} developers. It raised the objection from one of
35030
37671
the developers ``@dots{} that anybody pulling down the source from
35031
@command{git} is not an end user.''
37672
Git is not an end user.''
35032
37673
35033
37674
However, this is not true. There are ``power @command{awk} users''
35034
37675
who can build @command{gawk} (using the magic incantation shown previously)
35038
37679
It was then suggested that there be a @command{cron} job to create
35039
37680
nightly tarballs of ``the source.'' Here, the problem is that there
35040
37681
are source trees, corresponding to the various branches! So,
35041
nightly tar balls aren't the answer, especially as the repository can go
37682
nightly tarballs aren't the answer, especially as the repository can go
35042
37683
for weeks without significant change being introduced.
35043
37684
35044
Fortunately, the @command{git} server can meet this need. For any given
37685
Fortunately, the Git server can meet this need. For any given
35045
37686
branch named @var{branchname}, use:
35046
37687
35047
37688
@example
35050
37691
35051
37692
@noindent
35052
37693
to retrieve a snapshot of the given branch.
35053
37694
@c ENDOFRANGE gawkgit
35054
37695
35055
37696
@node Future Extensions
35056
37697
@appendixsec Probable Future Extensions
35094
37735
@quotation
35095
37736
@i{AWK is a language similar to PERL, only considerably more elegant.}
35096
37737
@author Arnold Robbins
37738
@end quotation
35097
37739
37740
@quotation
35098
37741
@i{Hey!}
35099
37742
@author Larry Wall
35100
37743
@end quotation
35101
37744
35102
The @file{TODO} file in the @command{gawk} Git repository lists possible
35103
future enhancements. Some of these relate to the source code, and others
35104
to possible new features. Please see that file for the list.
37745
The @file{TODO} file in the @code{master} branch of the @command{gawk}
37746
Git repository lists possible future enhancements. Some of these relate
37747
to the source code, and others to possible new features. Please see
37748
that file for the list.
35105
37749
@xref{Additions},
35106
37750
if you are interested in tackling any of the projects listed there.
35107
37751
35115
37759
@multitable @columnfractions .40 .60
35116
37760
@headitem Item @tab Limit
35117
37761
@item Characters in a character class @tab 2^(number of bits per byte)
35118
@item Length of input record @tab @code{MAX_INT }
37762
@item Length of input record @tab @code{MAX_INT}
35119
37763
@item Length of output record @tab Unlimited
35120
37764
@item Length of source line @tab Unlimited
35121
37765
@item Number of fields in a record @tab @code{MAX_LONG}
35124
37768
@item Number of input records total @tab @code{MAX_LONG}
35125
37769
@item Number of pipe redirections @tab min(number of processes per user, number of open files)
35126
37770
@item Numeric values @tab Double-precision floating point (if not using MPFR)
35127
@item Size of a field @tab @code{MAX_INT }
35128
@item Size of a literal string @tab @code{MAX_INT }
35129
@item Size of a printf string @tab @code{MAX_INT }
37771
@item Size of a field @tab @code{MAX_INT}
37772
@item Size of a literal string @tab @code{MAX_INT}
37773
@item Size of a printf string @tab @code{MAX_INT}
35130
37774
@end multitable
35131
37775
35132
37776
@node Extension Design
35161
37805
35162
37806
The old extension mechanism had several problems:
35163
37807
35164
@itemize @bullet
37808
@itemize @value{BULLET}
35165
37809
@item
35166
37810
It depended heavily upon @command{gawk} internals. Any time the
35167
37811
@code{NODE} structure@footnote{A critical central data structure
35173
37817
@item
35174
37818
Being able to call into @command{gawk} from an extension required linker
35175
37819
facilities that are common on Unix-derived systems but that did
35176
not work on Windows systems; users wanting extensions on Windows
35177
had to statically link them into @command{gawk}, even though Windows supports
37820
not work on MS-Windows systems; users wanting extensions on MS-Windows
37821
had to statically link them into @command{gawk}, even though MS-Windows supports
35178
37822
dynamic loading of shared objects.
35179
37823
35180
37824
@item
35197
37841
35198
37842
Some goals for the new API were:
35199
37843
35200
@itemize @bullet
37844
@itemize @value{BULLET}
35201
37845
@item
35202
37846
The API should be independent of @command{gawk} internals. Changes in
35203
37847
@command{gawk} internals should not be visible to the writer of an
35212
37856
same ``appearance'' to @command{awk}-level code as @command{awk}
35213
37857
functions do. This means that extensions should have:
35214
37858
35215
@itemize @minus
37859
@itemize @value{MINUS}
35216
37860
@item
35217
37861
The ability to access function parameters.
35218
37862
35228
37872
35229
37873
@item
35230
37874
The ability to create arrays (including @command{gawk}'s true
35231
multidimensional arrays).
37875
arrays of arrays).
35232
37876
@end itemize
35233
37877
@end itemize
35234
37878
35235
37879
Some additional important goals were:
35236
37880
35237
@itemize @bullet
37881
@itemize @value{BULLET}
35238
37882
@item
35239
37883
The API should use only features in ISO C 90, so that extensions
35240
37884
can be written using the widest range of C and C++ compilers. The header
35249
37893
symbols@footnote{The @dfn{symbols} are the variables and functions
35250
37894
defined inside @command{gawk}. Access to these symbols by code
35251
37895
external to @command{gawk} loaded dynamically at runtime is
35252
problematic on Windows.} by the compile-time or dynamic linker,
35253
in order to enable creation of extensions that also work on Windows.
37896
problematic on MS-Windows.} by the compile-time or dynamic linker,
37897
in order to enable creation of extensions that also work on MS-Windows.
35254
37898
@end itemize
35255
37899
35256
37900
During development, it became clear that there were other features
35257
37901
that should be available to extensions, which were also subsequently
35258
37902
provided:
35259
37903
35260
@itemize @bullet
37904
@itemize @value{BULLET}
35261
37905
@item
35262
37906
Extensions should have the ability to hook into @command{gawk}'s
35263
37907
I/O redirection mechanism. In particular, the @command{xgawk}
35268
37912
35269
37913
@item
35270
37914
An extension should be able to provide a ``call back'' function
35271
to perform clean up actions when @command{gawk} exits.
37915
to perform cleanup actions when @command{gawk} exits.
35272
37916
35273
37917
@item
35274
37918
An extension should be able to provide a version string so that
35338
37982
35339
37983
The API can later be expanded, in two ways:
35340
37984
35341
@itemize @bullet
37985
@itemize @value{BULLET}
35342
37986
@item
35343
37987
@command{gawk} passes an ``extension id'' into the extension when it
35344
37988
first loads the extension. The extension then passes this id back
35361
38005
35362
38006
@ref{Dynamic Extensions}, describes the supported API and mechanisms
35363
38007
for writing extensions for @command{gawk}. This API was introduced
35364
in version 4.1. However, for many years @command{gawk}
38008
in @value{PVERSION} 4.1. However, for many years @command{gawk}
35365
38009
provided an extension mechanism that required knowledge of @command{gawk}
35366
38010
internals and that was not as well designed.
35367
38011
35368
In order to provide a transition period, @command{gawk} version
35369
4.1 continues to support the original extension mechanism.
38012
In order to provide a transition period, @command{gawk} @value{PVERSION} 4.1
38013
continues to support the original extension mechanism.
35370
38014
This will be true for the life of exactly one major release. This support
35371
38015
will be withdrawn, and removed from the source code, at the next major
35372
38016
release.
35392
38036
convert any old extensions that you may have to use the new API
35393
38037
described in @ref{Dynamic Extensions}.
35394
38038
38039
@node Notes summary
38040
@appendixsec Summary
38041
38042
@itemize @value{BULLET}
38043
@item
38044
@command{gawk}'s extensions can be disabled with either the
38045
@option{--traditional} option or with the @option{--posix} option.
38046
The @option{--parsedebug} option is available if @command{gawk} is
38047
compiled with @samp{-DDEBUG}.
38048
38049
@item
38050
The source code for @command{gawk} is maintained in a publicly
38051
accessable Git repository. Anyone may check it out and view the source.
38052
38053
@item
38054
Contributions to @command{gawk} are welcome. Following the steps
38055
outlined in this @value{CHAPTER} will make it easier to integrate
38056
your contributions into the code base.
38057
This applies both to new feature contributions and to ports to
38058
additional operating systems.
38059
38060
@item
38061
@command{gawk} has some limits---generally those that are imposed by
38062
the machine architecture.
38063
38064
@item
38065
The extension API design was intended to solve a number of problems
38066
with the previous extension mechanism, enable features needed by
38067
the @code{xgawk} project, and provide binary compatibility going forward.
38068
38069
@item
38070
The previous extension mechanism is still supported in @value{PVERSION} 4.1
38071
of @command{gawk}, but it @emph{will} be removed in the next major release.
38072
38073
@end itemize
38074
35395
38075
@c ENDOFRANGE impis
35396
38076
@c ENDOFRANGE gawii
35397
38077
35419
38099
35420
38100
@cindex processing data
35421
38101
At the most basic level, the job of a program is to process
35422
some input data and produce results. See @ref{figure-general-flow}.
38102
some input data and produce results.
38103
@ifnotdocbook
38104
See @ref{figure-general-flow}.
38105
@end ifnotdocbook
38106
@ifdocbook
38107
See @inlineraw{docbook, <xref linkend="figure-general-flow"/>}.
38108
@end ifdocbook
35423
38109
38110
@ifnotdocbook
35424
38111
@float Figure,figure-general-flow
35425
38112
@caption{General Program Flow}
35426
38113
@ifinfo
35430
38117
@center @image{general-program, , , General program flow}
35431
38118
@end ifnotinfo
35432
38119
@end float
38120
@end ifnotdocbook
38121
38122
@docbook
38123
<figure id="figure-general-flow" float="0">
38124
<title>General Program Flow</title>
38125
<mediaobject>
38126
<imageobject role="web"><imagedata fileref="general-program.png" format="PNG"/></imageobject>
38127
</mediaobject>
38128
</figure>
38129
@end docbook
35433
38130
35434
38131
@cindex compiled programs
35435
38132
@cindex interpreted programs
35445
38142
35446
38143
@cindex programming, basic steps
35447
38144
When you write a program, it usually consists
35448
of the following, very basic set of steps, as shown
35449
in @ref{figure-process-flow}:
38145
of the following, very basic set of steps,
38146
@ifnotdocbook
38147
as shown in @ref{figure-process-flow}:
38148
@end ifnotdocbook
38149
@ifdocbook
38150
as shown in @inlineraw{docbook, <xref linkend="figure-process-flow"/>}:
38151
@end ifdocbook
35450
38152
38153
@ifnotdocbook
35451
38154
@float Figure,figure-process-flow
35452
38155
@caption{Basic Program Steps}
35453
38156
@ifinfo
35457
38160
@center @image{process-flow, , , Basic Program Stages}
35458
38161
@end ifnotinfo
35459
38162
@end float
38163
@end ifnotdocbook
38164
38165
@docbook
38166
<figure id="figure-process-flow" float="0">
38167
<title>Basic Program Stages</title>
38168
<mediaobject>
38169
<imageobject role="web"><imagedata fileref="process-flow.png" format="PNG"/></imageobject>
38170
</mediaobject>
38171
</figure>
38172
@end docbook
35460
38173
35461
38174
@table @asis
35462
38175
@item Initialization
35552
38265
referred to as @dfn{scalar} values.
35553
38266
Groups of values, such as arrays, are not scalars.
35554
38267
35555
@ref{General Arithmetic}, provided a basic introduction to numeric
38268
@ref{Computer Arithmetic}, provided a basic introduction to numeric
35556
38269
types (integer and floating-point) and how they are used in a computer.
35557
38270
Please review that information, including a number of caveats that
35558
38271
were presented.
35568
38281
35569
38282
Humans are used to working in decimal; i.e., base 10. In base 10,
35570
38283
numbers go from 0 to 9, and then ``roll over'' into the next
35571
column. (Remember grade school? 42 is 4 times 10 plus 2.)
38284
column. (Remember grade school? 42 = 4 x 10 + 2.)
35572
38285
35573
38286
There are other number bases though. Computers commonly use base 2
35574
38287
or @dfn{binary}, base 8 or @dfn{octal}, and base 16 or @dfn{hexadecimal}.
35575
38288
In binary, each column represents two times the value in the column to
35576
38289
its right. Each column may contain either a 0 or a 1.
35577
Thus, binary 1010 represents 1 times 8, plus 0 times 4, plus 1 times 2,
35578
plus 0 times 1, or decimal 10.
38290
Thus, binary 1010 represents (1 x 8) + (0 x 4) + (1 x 2)
38291
+ (0 x 1), or decimal 10.
35579
38292
Octal and hexadecimal are discussed more in
35580
38293
@ref{Nondecimal-numbers}.
35581
38294
35612
38325
@item Action
35613
38326
A series of @command{awk} statements attached to a rule. If the rule's
35614
38327
pattern matches an input record, @command{awk} executes the
35615
rule's action. Actions are always enclosed in curly braces.
38328
rule's action. Actions are always enclosed in braces.
35616
38329
(@xref{Action Overview}.)
35617
38330
35618
38331
@cindex Spencer, Henry
35627
38340
You can get it from @uref{http://awk.info/?awk100/aaa}.
35628
38341
35629
38342
@cindex Ada programming language
35630
@cindex Programming languages, Ada
38343
@cindex programming languages, Ada
35631
38344
@item Ada
35632
38345
A programming language originally defined by the U.S.@: Department of
35633
38346
Defense for embedded programming. It was designed to enforce good
35695
38408
@end ifinfo
35696
38409
See also ``Bourne Shell.''
35697
38410
35698
@item BBS
35699
See ``Bulletin Board System.''
35700
35701
38411
@item Bit
35702
38412
Short for ``Binary Digit.''
35703
38413
All values in computer memory ultimately reduce to binary digits: values
35720
38430
35721
38431
@item Bourne Shell
35722
38432
The standard shell (@file{/bin/sh}) on Unix and Unix-like systems,
35723
originally written by Steven R.@: Bourne.
38433
originally written by Steven R.@: Bourne at Bell Laboratories.
35724
38434
Many shells (Bash, @command{ksh}, @command{pdksh}, @command{zsh}) are
35725
38435
generally upwardly compatible with the Bourne shell.
35726
38436
35770
38480
(@xref{Built-in Variables}.)
35771
38481
35772
38482
@item Braces
35773
See ``Curly Braces.''
35774
35775
@item Bulletin Board System
35776
A computer system allowing users to log in and read and/or leave messages
35777
for other users of the system, much like leaving paper notes on a bulletin
35778
board.
38483
The characters @samp{@{} and @samp{@}}. Braces are used in
38484
@command{awk} for delimiting actions, compound statements, and function
38485
bodies.
35779
38486
35780
38487
@item C
35781
38488
The system programming language that most GNU software is written in. The
35800
38507
Standard Code for Information Interchange). Many European
35801
38508
countries use an extension of ASCII known as ISO-8859-1 (ISO Latin-1).
35802
38509
The @uref{http://www.unicode.org, Unicode character set} is
35803
becoming increasingly popular and standard, and is particularly
38510
increasingly popular and standard, and is particularly
35804
38511
widely used on GNU/Linux systems.
35805
38512
38513
@cindex Kernighan, Brian
38514
@cindex Bentley, Jon
35806
38515
@cindex @command{chem} utility
35807
38516
@item CHEM
35808
38517
A preprocessor for @command{pic} that reads descriptions of molecules
35811
38520
by Brian Kernighan and Jon Bentley, and is available from
35812
38521
@uref{http://netlib.sandia.gov/netlib/typesetting/chem.gz}.
35813
38522
38523
@cindex McIlroy, Doug
35814
38524
@cindex cookie
35815
38525
@item Cookie
35816
38526
A peculiar goodie, token, saying or remembrance
35817
produced by or presented to a program. (With thanks to Doug McIlroy.)
38527
produced by or presented to a program. (With thanks to Professor Doug McIlroy.)
35818
38528
@ignore
35819
38529
From: Doug McIlroy <doug@cs.dartmouth.edu>
35820
38530
Date: Sat, 13 Oct 2012 19:55:25 -0400
35892
38602
(@xref{Typing and Comparison}.)
35893
38603
35894
38604
@item Curly Braces
35895
The characters @samp{@{} and @samp{@}}. Curly braces are used in
35896
@command{awk} for delimiting actions, compound statements, and function
35897
bodies.
38605
See ``Braces.''
35898
38606
35899
38607
@cindex dark corner
35900
38608
@item Dark Corner
35939
38647
(@xref{Computed Regexps}.)
35940
38648
35941
38649
@item Environment
35942
A collection of strings, of the form @var{name@code{=}val}, that each
38650
A collection of strings, of the form @samp{@var{name}=@var{val}}, that each
35943
38651
program has available to it. Users generally place values into the
35944
38652
environment in order to provide information to various programs. Typical
35945
38653
examples are the environment variables @env{HOME} and @env{PATH}.
35993
38701
See also ``Double Precision'' and ``Single Precision.''
35994
38702
35995
38703
@item Format
35996
Format strings are used to control the appearance of output in the
35997
@code{strftime()} and @code{sprintf()} functions, and are used in the
38704
Format strings control the appearance of output in the
38705
@code{strftime()} and @code{sprintf()} functions, and in the
35998
38706
@code{printf} statement as well. Also, data conversions from numbers to strings
35999
38707
are controlled by the format strings contained in the built-in variables
36000
38708
@code{CONVFMT} and @code{OFMT}. (@xref{Control Letters}.)
36063
38771
@code{A}--@code{F}, with @samp{A}
36064
38772
representing 10, @samp{B} representing 11, and so on, up to @samp{F} for 15.
36065
38773
Hexadecimal numbers are written in C using a leading @samp{0x},
36066
to indicate their base. Thus, @code{0x12} is 18 (1 times 16 plus 2).
38774
to indicate their base. Thus, @code{0x12} is 18 ((1 x 16) + 2).
36067
38775
@xref{Nondecimal-numbers}.
36068
38776
36069
38777
@item I/O
36108
38816
three-letter acronym.
36109
38817
36110
38818
@cindex Java programming language
36111
@cindex Programming languages, Java
38819
@cindex programming languages, Java
36112
38820
@item Java
36113
38821
A modern programming language originally developed by Sun Microsystems
36114
38822
(now Oracle) supporting Object-Oriented programming. Although usually
36137
38845
@code{function},
36138
38846
@code{func},
36139
38847
@code{if},
38848
@code{next},
36140
38849
@code{nextfile},
36141
@code{next},
36142
38850
@code{switch},
36143
38851
and
36144
38852
@code{while}.
36199
38907
@item Octal
36200
38908
Base-eight notation, where the digits are @code{0}--@code{7}.
36201
38909
Octal numbers are written in C using a leading @samp{0},
36202
to indicate their base. Thus, @code{013} is 11 (one times 8 plus 3).
38910
to indicate their base. Thus, @code{013} is 11 ((1 x 8) + 3).
36203
38911
@xref{Nondecimal-numbers}.
36204
38912
36205
@cindex P1003.1 POSIX standard
36206
@item P1003.1
36207
See ``POSIX.''
36208
36209
38913
@item Pattern
36210
38914
Patterns tell @command{awk} which input records are interesting to which
36211
38915
rules.
36246
38950
36247
38951
@item Recursion
36248
38952
When a function calls itself, either directly or indirectly.
36249
As long as this is not clear, refer to the entry for ``recursion.''
36250
38953
If this is clear, stop, and proceed to the next entry.
38954
Otherwise, refer to the entry for ``recursion.''
36251
38955
36252
38956
@item Redirection
36253
38957
Redirection means performing input from something other than the standard input
36326
39030
An internal representation of numbers that can have fractional parts.
36327
39031
Single precision numbers keep track of fewer digits than do double precision
36328
39032
numbers, but operations on them are sometimes less expensive in terms of CPU time.
36329
This is the type used by some very old versions of @command{awk} to store
39033
This is the type used by some ancient versions of @command{awk} to store
36330
39034
numeric values. It is the C type @code{float}.
36331
39035
36332
39036
@item Space
36333
39037
The character generated by hitting the space bar on the keyboard.
36334
39038
36335
39039
@item Special File
36336
A file name interpreted internally by @command{gawk}, instead of being handed
39040
A @value{FN} interpreted internally by @command{gawk}, instead of being handed
36337
39041
directly to the underlying operating system---for example, @file{/dev/stderr}.
36338
39042
(@xref{Special Files}.)
36339
39043
36363
39067
A value in the ``seconds since the epoch'' format used by Unix
36364
39068
and POSIX systems. Used for the @command{gawk} functions
36365
39069
@code{mktime()}, @code{strftime()}, and @code{systime()}.
36366
See also ``Epoch'' and ``UTC.''
39070
See also ``Epoch,'' ``GMT,'' and ``UTC.''
36367
39071
36368
39072
@cindex Linux
36369
39073
@cindex GNU/Linux
36395
39099
@c The GNU General Public License.
36396
39100
@node Copying
36397
39101
@unnumbered GNU General Public License
39102
@ifnotdocbook
36398
39103
@center Version 3, 29 June 2007
39104
@end ifnotdocbook
39105
@docbook
39106
<subtitle>Version 3, 29 June 2007</subtitle>
39107
@end docbook
36399
39108
36400
39109
@c This file is intended to be included within another document,
36401
39110
@c hence no sectioning command or @node.
37120
39829
@c The GNU Free Documentation License.
37121
39830
@node GNU Free Documentation License
37122
39831
@unnumbered GNU Free Documentation License
39832
@ifnotdocbook
39833
@center Version 1.3, 3 November 2008
39834
@end ifnotdocbook
39835
39836
@docbook
39837
<subtitle>Version 1.3, 3 November 2008</subtitle>
39838
@end docbook
39839
37123
39840
@cindex FDL (Free Documentation License)
37124
39841
@cindex Free Documentation License (FDL)
37125
39842
@cindex GNU Free Documentation License
37126
@center Version 1.3, 3 November 2008
37127
39843
37128
39844
@c This file is intended to be included within another document,
37129
39845
@c hence no sectioning command or @node.
37624
40340
free software license, such as the GNU General Public License,
37625
40341
to permit their use in free software.
37626
40342
37627
@c Local Variables:
37628
@c ispell-local-pdict: "ispell-dict"
37629
@c End:
40343
@end ifclear
37630
40344
40345
@ifnotdocbook
37631
40346
@node Index
37632
40347
@unnumbered Index
40348
@end ifnotdocbook
37633
40349
@printindex cp
37634
40350
37635
40351
@bye
37737
40453
37738
40454
Suggestions:
37739
40455
------------
37740
% Next edition:
37741
% 1. Standardize the error messages from the functions and programs
37742
% in the two sample code chapters.
37743
% 2. Nuke the BBS stuff and use something that won't be obsolete
37744
% 3. Turn the advanced notes into sidebars by using @cartouche
37745
40456
37746
40457
Better sidebars can almost sort of be done with:
37747
40458
37774
40485
37775
40486
which sorta sucks.
37776
40487
40488
TODO:
40489
-----
Loggerhead is a web-based interface for Breezy
Version: 2.0.1