« back to all changes in this revision
Viewing changes to pod/perlrecharclass.pod
- browse files at revision 1.1.10
- compare with another revision
- download diff
- download tarball
- view history from revision 1.1.10
- Committer: Bazaar Package Importer
- Author(s): Niko Tyni
- Date: 2011-02-06 11:31:38 UTC
- mto: (8.2.12 experimental) (1.1.12)
- mto: This revision was merged to the branch mainline in revision 46.
- Revision ID: james.westby@ubuntu.com-20110206113138-lzpm3g6rur7i3eyp
Tags: upstream-5.12.3
Import upstream version 5.12.3
- files added:
- cpan/CGI/t/headers.t
- files modified:
- Cross/config.sh-arm-linux
added
removed
pod/perlrecharclass.pod
9
9
is found in L<perlre>.
10
10
11
11
This manual page discusses the syntax and use of character
12
classes in Perl Regular Expressions.
12
classes in Perl regular expressions.
13
13
14
A character class is a way of denoting a set of characters,
14
A character class is a way of denoting a set of characters
15
15
in such a way that one character of the set is matched.
16
It's important to remember that matching a character class
16
It's important to remember that: matching a character class
17
17
consumes exactly one character in the source string. (The source
18
18
string is the string the regular expression is matched against.)
19
19
20
20
There are three types of character classes in Perl regular
21
expressions: the dot, backslashed sequences, and the form enclosed in square
21
expressions: the dot, backslash sequences, and the form enclosed in square
22
22
brackets. Keep in mind, though, that often the term "character class" is used
23
to mean just the bracketed form. This is true in other Perl documentation.
23
to mean just the bracketed form. Certainly, most Perl documentation does that.
24
24
25
25
=head2 The dot
26
26
27
27
The dot (or period), C<.> is probably the most used, and certainly
28
28
the most well-known character class. By default, a dot matches any
29
29
character, except for the newline. The default can be changed to
30
add matching the newline with the I<single line> modifier: either
31
for the entire regular expression using the C</s> modifier, or
32
locally using C<(?s)>.
30
add matching the newline by using the I<single line> modifier: either
31
for the entire regular expression with the C</s> modifier, or
32
locally with C<(?s)>. (The experimental C<\N> backslash sequence, described
33
below, matches any character except newline without regard to the
34
I<single line> modifier.)
33
35
34
36
Here are some examples:
35
37
41
43
"\n" =~ /(?s:.)/ # Match (local 'single line' modifier)
42
44
"ab" =~ /^.$/ # No match (dot matches one character)
43
45
44
=head2 Backslashed sequences
46
=head2 Backslash sequences
45
47
X<\w> X<\W> X<\s> X<\S> X<\d> X<\D> X<\p> X<\P>
46
48
X<\N> X<\v> X<\V> X<\h> X<\H>
47
49
X<word> X<whitespace>
48
50
49
Perl regular expressions contain many backslashed sequences that
50
constitute a character class. That is, they will match a single
51
character, if that character belongs to a specific set of characters
52
(defined by the sequence). A backslashed sequence is a sequence of
53
characters starting with a backslash. Not all backslashed sequences
54
are character classes; for a full list, see L<perlrebackslash>.
55
56
Here's a list of the backslashed sequences that are character classes. They
57
are discussed in more detail below.
58
59
\d Match a digit character.
60
\D Match a non-digit character.
51
A backslash sequence is a sequence of characters, the first one of which is a
52
backslash. Perl ascribes special meaning to many such sequences, and some of
53
these are character classes. That is, they match a single character each,
54
provided that the character belongs to the specific set of characters defined
55
by the sequence.
56
57
Here's a list of the backslash sequences that are character classes. They
58
are discussed in more detail below. (For the backslash sequences that aren't
59
character classes, see L<perlrebackslash>.)
60
61
\d Match a decimal digit character.
62
\D Match a non-decimal-digit character.
61
63
\w Match a "word" character.
62
64
\W Match a non-"word" character.
63
65
\s Match a whitespace character.
64
66
\S Match a non-whitespace character.
65
67
\h Match a horizontal whitespace character.
66
68
\H Match a character that isn't horizontal whitespace.
67
\N Match a character that isn't newline. Experimental.
68
69
\v Match a vertical whitespace character.
69
70
\V Match a character that isn't vertical whitespace.
70
\pP, \p{Prop} Match a character matching a Unicode property.
71
\PP, \P{Prop} Match a character that doesn't match a Unicode property.
71
\N Match a character that isn't a newline. Experimental.
72
\pP, \p{Prop} Match a character that has the given Unicode property.
73
\PP, \P{Prop} Match a character that doesn't have the Unicode property
72
74
73
75
=head3 Digits
74
76
75
C<\d> matches a single character that is considered to be a I<digit>. What is
76
considered a digit depends on the internal encoding of the source string and
77
the locale that is in effect. If the source string is in UTF-8 format, C<\d>
78
not only matches the digits '0' - '9', but also Arabic, Devanagari and digits
79
from other languages. Otherwise, if there is a locale in effect, it will match
80
whatever characters the locale considers digits. Without a locale, C<\d>
81
matches the digits '0' to '9'. See L</Locale, EBCDIC, Unicode and UTF-8>.
77
C<\d> matches a single character that is considered to be a decimal I<digit>.
78
What is considered a decimal digit depends on the internal encoding of the
79
source string and the locale that is in effect. If the source string is in
80
UTF-8 format, C<\d> not only matches the digits '0' - '9', but also Arabic,
81
Devanagari and digits from other languages. Otherwise, if there is a locale in
82
effect, it will match whatever characters the locale considers decimal digits.
83
Without a locale, C<\d> matches just the digits '0' to '9'.
84
See L</Locale, EBCDIC, Unicode and UTF-8>.
85
86
Unicode digits may cause some confusion, and some security issues. In UTF-8
87
strings, C<\d> matches the same characters matched by
88
C<\p{General_Category=Decimal_Number}>, or synonymously,
89
C<\p{General_Category=Digit}>. Starting with Unicode version 4.1, this is the
90
same set of characters matched by C<\p{Numeric_Type=Decimal}>.
91
92
But Unicode also has a different property with a similar name,
93
C<\p{Numeric_Type=Digit}>, which matches a completely different set of
94
characters. These characters are things such as subscripts.
95
96
The design intent is for C<\d> to match all the digits (and no other characters)
97
that can be used with "normal" big-endian positional decimal syntax, whereby a
98
sequence of such digits {N0, N1, N2, ...Nn} has the numeric value (...(N0 * 10
99
+ N1) * 10 + N2) * 10 ... + Nn). In Unicode 5.2, the Tamil digits (U+0BE6 -
100
U+0BEF) can also legally be used in old-style Tamil numbers in which they would
101
appear no more than one in a row, separated by characters that mean "times 10",
102
"times 100", etc. (See L<http://www.unicode.org/notes/tn21>.)
103
104
Some of the non-European digits that C<\d> matches look like European ones, but
105
have different values. For example, BENGALI DIGIT FOUR (U+09A) looks very much
106
like an ASCII DIGIT EIGHT (U+0038).
107
108
It may be useful for security purposes for an application to require that all
109
digits in a row be from the same script. See L<Unicode::UCD/charscript()>.
82
110
83
111
Any character that isn't matched by C<\d> will be matched by C<\D>.
84
112
85
113
=head3 Word characters
86
114
87
115
A C<\w> matches a single alphanumeric character (an alphabetic character, or a
88
decimal digit) or an underscore (C<_>), not a whole word. Use C<\w+> to match
89
a string of Perl-identifier characters (which isn't the same as matching an
90
English word). What is considered a word character depends on the internal
116
decimal digit) or an underscore (C<_>), not a whole word. To match a whole
117
word, use C<\w+>. This isn't the same thing as matching an English word, but
118
is the same as a string of Perl-identifier characters. What is considered a
119
word character depends on the internal
91
120
encoding of the string and the locale or EBCDIC code page that is in effect. If
92
121
it's in UTF-8 format, C<\w> matches those characters that are considered word
93
122
characters in the Unicode database. That is, it not only matches ASCII letters,
97
126
C<\w> matches the ASCII letters, digits and the underscore.
98
127
See L</Locale, EBCDIC, Unicode and UTF-8>.
99
128
129
There are a number of security issues with the full Unicode list of word
130
characters. See L<http://unicode.org/reports/tr36>.
131
132
Also, for a somewhat finer-grained set of characters that are in programming
133
language identifiers beyond the ASCII range, you may wish to instead use the
134
more customized Unicode properties, "ID_Start", ID_Continue", "XID_Start", and
135
"XID_Continue". See L<http://unicode.org/reports/tr31>.
136
100
137
Any character that isn't matched by C<\w> will be matched by C<\W>.
101
138
102
139
=head3 Whitespace
103
140
104
C<\s> matches any single character that is considered whitespace. In the ASCII
105
range, C<\s> matches the horizontal tab (C<\t>), the new line (C<\n>), the form
106
feed (C<\f>), the carriage return (C<\r>), and the space. (The vertical tab,
107
C<\cK> is not matched by C<\s>.) The exact set of characters matched by C<\s>
108
depends on whether the source string is in UTF-8 format and the locale or
109
EBCDIC code page that is in effect. If it's in UTF-8 format, C<\s> matches what
110
is considered whitespace in the Unicode database; the complete list is in the
111
table below. Otherwise, if there is a locale or EBCDIC code page in effect,
112
C<\s> matches whatever is considered whitespace by the current locale or EBCDIC
113
code page. Without a locale or EBCDIC code page, C<\s> matches the five
114
characters mentioned in the beginning of this paragraph. Perhaps the most
115
notable possible surprise is that C<\s> matches a non-breaking space only if
116
the non-breaking space is in a UTF-8 encoded string or the locale or EBCDIC
117
code page that is in effect has that character.
141
C<\s> matches any single character that is considered whitespace. The exact
142
set of characters matched by C<\s> depends on whether the source string is in
143
UTF-8 format and the locale or EBCDIC code page that is in effect. If it's in
144
UTF-8 format, C<\s> matches what is considered whitespace in the Unicode
145
database; the complete list is in the table below. Otherwise, if there is a
146
locale or EBCDIC code page in effect, C<\s> matches whatever is considered
147
whitespace by the current locale or EBCDIC code page. Without a locale or
148
EBCDIC code page, C<\s> matches the horizontal tab (C<\t>), the newline
149
(C<\n>), the form feed (C<\f>), the carriage return (C<\r>), and the space.
150
(Note that it doesn't match the vertical tab, C<\cK>.) Perhaps the most notable
151
possible surprise is that C<\s> matches a non-breaking space only if the
152
non-breaking space is in a UTF-8 encoded string or the locale or EBCDIC code
153
page that is in effect has that character.
118
154
See L</Locale, EBCDIC, Unicode and UTF-8>.
119
155
120
156
Any character that isn't matched by C<\s> will be matched by C<\S>.
121
157
122
158
C<\h> will match any character that is considered horizontal whitespace;
123
this includes the space and the tab characters and 17 other characters that are
124
listed in the table below. C<\H> will match any character
159
this includes the space and the tab characters and a number other characters,
160
all of which are listed in the table below. C<\H> will match any character
125
161
that is not considered horizontal whitespace.
126
162
127
C<\N> is new in 5.12, and is experimental. It, like the dot, will match any
128
character that is not a newline. The difference is that C<\N> will not be
129
influenced by the single line C</s> regular expression modifier. Note that
130
there is a second meaning of C<\N> when of the form C<\N{...}>. This form is
131
for named characters. See L<charnames> for those. If C<\N> is followed by an
132
opening brace and something that is not a quantifier, perl will assume that a
133
character name is coming, and not this meaning of C<\N>. For example, C<\N{3}>
134
means to match 3 non-newlines; C<\N{5,}> means to match 5 or more non-newlines,
135
but C<\N{4F}> and C<\N{F4}> are not legal quantifiers, and will cause perl to
136
look for characters named C<4F> or C<F4>, respectively (and won't find them,
137
thus raising an error, unless they have been defined using custom names).
138
139
163
C<\v> will match any character that is considered vertical whitespace;
140
this includes the carriage return and line feed characters (newline) plus 5
141
other characters listed in the table below.
164
this includes the carriage return and line feed characters (newline) plus several
165
other characters, all listed in the table below.
142
166
C<\V> will match any character that is not considered vertical whitespace.
143
167
144
168
C<\R> matches anything that can be considered a newline under Unicode
156
180
vertical tab (C<"\x0b">) is not matched by C<\s>, it is however considered
157
181
vertical whitespace. Furthermore, if the source string is not in UTF-8 format,
158
182
and any locale or EBCDIC code page that is in effect doesn't include them, the
159
next line (C<"\x85">) and the no-break space (C<"\xA0">) characters are not
160
matched by C<\s>, but are by C<\v> and C<\h> respectively. If the source
161
string is in UTF-8 format, both the next line and the no-break space are
162
matched by C<\s>.
183
next line (ASCII-platform C<"\x85">) and the no-break space (ASCII-platform
184
C<"\xA0">) characters are not matched by C<\s>, but are by C<\v> and C<\h>
185
respectively. If the source string is in UTF-8 format, both the next line and
186
the no-break space are matched by C<\s>.
163
187
164
188
The following table is a complete listing of characters matched by
165
189
C<\s>, C<\h> and C<\v> as of Unicode 5.2.
209
233
complete numbers or words. To match a number (that consists of integers),
210
234
use C<\d+>; to match a word, use C<\w+>.
211
235
236
=head3 \N
237
238
C<\N> is new in 5.12, and is experimental. It, like the dot, will match any
239
character that is not a newline. The difference is that C<\N> is not influenced
240
by the I<single line> regular expression modifier (see L</The dot> above). Note
241
that the form C<\N{...}> may mean something completely different. When the
242
C<{...}> is a L<quantifier|perlre/Quantifiers>, it means to match a non-newline
243
character that many times. For example, C<\N{3}> means to match 3
244
non-newlines; C<\N{5,}> means to match 5 or more non-newlines. But if C<{...}>
245
is not a legal quantifier, it is presumed to be a named character. See
246
L<charnames> for those. For example, none of C<\N{COLON}>, C<\N{4F}>, and
247
C<\N{F4}> contain legal quantifiers, so Perl will try to find characters whose
248
names are, respectively, C<COLON>, C<4F>, and C<F4>.
212
249
213
250
=head3 Unicode Properties
214
251
263
300
=head2 Bracketed Character Classes
264
301
265
302
The third form of character class you can use in Perl regular expressions
266
is the bracketed form. In its simplest form, it lists the characters
303
is the bracketed character class. In its simplest form, it lists the characters
267
304
that may be matched, surrounded by square brackets, like this: C<[aeiou]>.
268
305
This matches one of C<a>, C<e>, C<i>, C<o> or C<u>. Like the other
269
306
character classes, exactly one character will be matched. To match
270
307
a longer string consisting of characters mentioned in the character
271
class, follow the character class with a quantifier. For instance,
272
C<[aeiou]+> matches a string of one or more lowercase ASCII vowels.
308
class, follow the character class with a L<quantifier|perlre/Quantifiers>. For
309
instance, C<[aeiou]+> matches a string of one or more lowercase English vowels.
273
310
274
311
Repeating a character in a character class has no
275
312
effect; it's considered to be in the set only once.
297
334
case the backslash may be omitted.
298
335
299
336
The sequence C<\b> is special inside a bracketed character class. While
300
outside the character class C<\b> is an assertion indicating a point
337
outside the character class, C<\b> is an assertion indicating a point
301
338
that does not have either two word characters or two non-word characters
302
339
on either side, inside a bracketed character class, C<\b> matches a
303
340
backspace character.
320
357
Also, a backslash followed by two or three octal digits is considered an octal
321
358
number.
322
359
323
A C<[> is not special inside a character class, unless it's the start
324
of a POSIX character class (see below). It normally does not need escaping.
360
A C<[> is not special inside a character class, unless it's the start of a
361
POSIX character class (see L</POSIX Character Classes> below). It normally does
362
not need escaping.
325
363
326
A C<]> is normally either the end of a POSIX character class (see below), or it
327
signals the end of the bracketed character class. If you want to include a
328
C<]> in the set of characters, you must generally escape it.
364
A C<]> is normally either the end of a POSIX character class (see
365
L</POSIX Character Classes> below), or it signals the end of the bracketed
366
character class. If you want to include a C<]> in the set of characters, you
367
must generally escape it.
329
368
However, if the C<]> is the I<first> (or the second if the first
330
369
character is a caret) character of a bracketed character class, it
331
370
does not denote the end of the class (as you cannot have an empty class)
362
401
If a hyphen in a character class cannot syntactically be part of a range, for
363
402
instance because it is the first or the last character of the character class,
364
403
or if it immediately follows a range, the hyphen isn't special, and will be
365
considered a character that may be matched literally. You have to escape the
404
considered a character that is to be matched literally. You have to escape the
366
405
hyphen with a backslash if you want to have a hyphen in your set of characters
367
406
to be matched, and its position in the class is such that it could be
368
407
considered part of a range.
403
442
You can put any backslash sequence character class (with the exception of
404
443
C<\N>) inside a bracketed character class, and it will act just
405
444
as if you put all the characters matched by the backslash sequence inside the
406
character class. For instance, C<[a-f\d]> will match any digit, or any of the
407
lowercase letters between 'a' and 'f' inclusive.
408
409
C<\N> within a bracketed character class must be of the forms C<\N{I<name>}> or
410
C<\N{U+I<wide hex char>}> for the same reason that a dot C<.> inside a
411
bracketed character class loses its special meaning: it matches nearly
412
anything, which generally isn't what you want to happen.
445
character class. For instance, C<[a-f\d]> will match any decimal digit, or any
446
of the lowercase letters between 'a' and 'f' inclusive.
447
448
C<\N> within a bracketed character class must be of the forms C<\N{I<name>}>
449
or C<\N{U+I<wide hex char>}>, and NOT be the form that matches non-newlines,
450
for the same reason that a dot C<.> inside a bracketed character class loses
451
its special meaning: it matches nearly anything, which generally isn't what you
452
want to happen.
453
413
454
414
455
Examples:
415
456
419
460
# character, nor a parenthesis.
420
461
421
462
Backslash sequence character classes cannot form one of the endpoints
422
of a range.
423
424
=head3 Posix Character Classes
463
of a range. Thus, you can't say:
464
465
/[\p{Thai}-\d]/ # Wrong!
466
467
=head3 POSIX Character Classes
425
468
X<character class> X<\p> X<\p{}>
426
469
X<alpha> X<alnum> X<ascii> X<blank> X<cntrl> X<digit> X<graph>
427
470
X<lower> X<print> X<punct> X<space> X<upper> X<word> X<xdigit>
428
471
429
Posix character classes have the form C<[:class:]>, where I<class> is
430
name, and the C<[:> and C<:]> delimiters. Posix character classes only appear
472
POSIX character classes have the form C<[:class:]>, where I<class> is
473
name, and the C<[:> and C<:]> delimiters. POSIX character classes only appear
431
474
I<inside> bracketed character classes, and are a convenient and descriptive
432
475
way of listing a group of characters, though they currently suffer from
433
portability issues (see below and L<Locale, EBCDIC, Unicode and UTF-8>). Be
434
careful about the syntax,
476
portability issues (see below and L<Locale, EBCDIC, Unicode and UTF-8>).
477
478
Be careful about the syntax,
435
479
436
480
# Correct:
437
481
$string =~ /[[:alpha:]]/
441
485
442
486
The latter pattern would be a character class consisting of a colon,
443
487
and the letters C<a>, C<l>, C<p> and C<h>.
444
These character classes can be part of a larger bracketed character class. For
488
POSIX character classes can be part of a larger bracketed character class. For
445
489
example,
446
490
447
491
[01[:alpha:]%]
471
515
between POSIX character classes and these counterparts.
472
516
473
517
One counterpart, in the column labelled "ASCII-range Unicode" in
474
the table will only match characters in the ASCII range. (On EBCDIC platforms,
475
they match those characters which have ASCII equivalents.)
518
the table, will only match characters in the ASCII character set.
476
519
477
520
The other counterpart, in the column labelled "Full-range Unicode", matches any
478
521
appropriate characters in the full Unicode character set. For example,
490
533
the string is in UTF-8 format or not, or whether the platform is EBCDIC or not.
491
534
In contrast, the POSIX character classes are affected. If the source string is
492
535
in UTF-8 format, the POSIX classes (with the exception of C<[[:punct:]]>, see
493
Note [5]) behave like their "Full-range" Unicode counterparts. If the source
494
string is not in UTF-8 format, and no locale is in effect, and the platform is
495
not EBCDIC, all the POSIX classes behave like their ASCII-range counterparts.
496
Otherwise, they behave based on the rules of the locale or EBCDIC code page.
536
Note [5] below) behave like their "Full-range" Unicode counterparts. If the
537
source string is not in UTF-8 format, and no locale is in effect, and the
538
platform is not EBCDIC, all the POSIX classes behave like their ASCII-range
539
counterparts. Otherwise, they behave based on the rules of the locale or
540
EBCDIC code page.
541
497
542
It is proposed to change this behavior in a future release of Perl so that the
498
543
the UTF8ness of the source string will be irrelevant to the behavior of the
499
544
POSIX character classes. This means they will always behave in strict
537
582
538
583
On EBCDIC platforms, it is likely that the code page will define C<[[:cntrl:]]>
539
584
to be the EBCDIC equivalents of the ASCII controls, plus the controls
540
that in Unicode have ordinals from 128 through 139.
585
that in Unicode have ordinals from 128 through 159.
541
586
542
587
=item [3]
543
588
549
594
All printable characters, which is the set of all the graphical characters
550
595
plus whitespace characters that are not also controls.
551
596
552
=item [5]
597
=item [5] (punct)
553
598
554
599
C<\p{PosixPunct}> and C<[[:punct:]]> in the ASCII range match all the
555
600
non-controls, non-alphanumeric, non-space characters:
556
601
C<[-!"#$%&'()*+,./:;<=E<gt>?@[\\\]^_`{|}~]> (although if a locale is in effect,
557
602
it could alter the behavior of C<[[:punct:]]>).
558
603
559
When the matching string is in UTF-8 format, C<[[:punct:]]> matches the above
560
set, plus what C<\p{Punct}> matches. This is different than strictly matching
561
according to C<\p{Punct}>, because the above set includes characters that aren't
562
considered punctuation by Unicode, but rather "symbols". Another way to say it
563
is that for a UTF-8 string, C<[[:punct:]]> matches all the characters that
564
Unicode considers to be punctuation, plus all the ASCII-range characters that
565
Unicode considers to be symbols.
604
C<\p{Punct}> matches a somewhat different set in the ASCII range, namely
605
C<[-!"#%&'()*,./:;?@[\\\]_{}]>. That is, it is missing C<[$+E<lt>=E<gt>^`|~]>.
606
This is because Unicode splits what POSIX considers to be punctuation into two
607
categories, Punctuation and Symbols.
608
609
When the matching string is in UTF-8 format, C<[[:punct:]]> matches what it
610
matches in the ASCII range, plus what C<\p{Punct}> matches. This is different
611
than strictly matching according to C<\p{Punct}>. Another way to say it is that
612
for a UTF-8 string, C<[[:punct:]]> matches all the characters that Unicode
613
considers to be punctuation, plus all the ASCII-range characters that Unicode
614
considers to be symbols.
566
615
567
616
=item [6]
568
617
624
673
classes match according to the Unicode properties. If the source string
625
674
isn't, then the character classes match according to whatever locale or EBCDIC
626
675
code page is in effect. If there is no locale nor EBCDIC, they match the ASCII
627
defaults (52 letters, 10 digits and underscore for C<\w>; 0 to 9 for C<\d>;
676
defaults (0 to 9 for C<\d>; 52 letters, 10 digits and underscore for C<\w>;
628
677
etc.).
629
678
630
679
This usually means that if you are matching against characters whose C<ord()>
632
681
or not depending on the current locale or EBCDIC code page, and whether the
633
682
source string is in UTF-8 format. The string will be in UTF-8 format if it
634
683
contains characters whose C<ord()> value exceeds 255. But a string may be in
635
UTF-8 format without it having such characters. See L<perluniprops/The
684
UTF-8 format without it having such characters. See L<perlunicode/The
636
685
"Unicode Bug">.
637
686
638
687
For portability reasons, it may be better to not use C<\w>, C<\d>, C<\s>
Loggerhead is a web-based interface for Breezy
Version: 2.0.1