2
:mod:`re` --- Regular expression operations
3
===========================================
6
:synopsis: Regular expression operations.
7
.. moduleauthor:: Fredrik Lundh <fredrik@pythonware.com>
8
.. sectionauthor:: Andrew M. Kuchling <amk@amk.ca>
13
This module provides regular expression matching operations similar to
14
those found in Perl. The :mod:`re` module is always available.
16
Both patterns and strings to be searched can be Unicode strings as well as
17
8-bit strings. However, Unicode strings and 8-bit strings cannot be mixed:
18
that is, you cannot match an Unicode string with a byte pattern or
19
vice-versa; similarly, when asking for a substitution, the replacement
20
string must be of the same type as both the pattern and the search string.
22
Regular expressions use the backslash character (``'\'``) to indicate
23
special forms or to allow special characters to be used without invoking
24
their special meaning. This collides with Python's usage of the same
25
character for the same purpose in string literals; for example, to match
26
a literal backslash, one might have to write ``'\\\\'`` as the pattern
27
string, because the regular expression must be ``\\``, and each
28
backslash must be expressed as ``\\`` inside a regular Python string
31
The solution is to use Python's raw string notation for regular expression
32
patterns; backslashes are not handled in any special way in a string literal
33
prefixed with ``'r'``. So ``r"\n"`` is a two-character string containing
34
``'\'`` and ``'n'``, while ``"\n"`` is a one-character string containing a
35
newline. Usually patterns will be expressed in Python code using this raw
38
It is important to note that most regular expression operations are available as
39
module-level functions and :class:`RegexObject` methods. The functions are
40
shortcuts that don't require you to compile a regex object first, but miss some
41
fine-tuning parameters.
45
Mastering Regular Expressions
46
Book on regular expressions by Jeffrey Friedl, published by O'Reilly. The
47
second edition of the book no longer covers Python at all, but the first
48
edition covered writing good regular expression patterns in great detail.
50
`Kodos <http://kodos.sf.net/>`_
51
is a graphical regular expression debugger written in Python.
56
Regular Expression Syntax
57
-------------------------
59
A regular expression (or RE) specifies a set of strings that matches it; the
60
functions in this module let you check if a particular string matches a given
61
regular expression (or if a given regular expression matches a particular
62
string, which comes down to the same thing).
64
Regular expressions can be concatenated to form new regular expressions; if *A*
65
and *B* are both regular expressions, then *AB* is also a regular expression.
66
In general, if a string *p* matches *A* and another string *q* matches *B*, the
67
string *pq* will match AB. This holds unless *A* or *B* contain low precedence
68
operations; boundary conditions between *A* and *B*; or have numbered group
69
references. Thus, complex expressions can easily be constructed from simpler
70
primitive expressions like the ones described here. For details of the theory
71
and implementation of regular expressions, consult the Friedl book referenced
72
above, or almost any textbook about compiler construction.
74
A brief explanation of the format of regular expressions follows. For further
75
information and a gentler presentation, consult the :ref:`regex-howto`.
77
Regular expressions can contain both special and ordinary characters. Most
78
ordinary characters, like ``'A'``, ``'a'``, or ``'0'``, are the simplest regular
79
expressions; they simply match themselves. You can concatenate ordinary
80
characters, so ``last`` matches the string ``'last'``. (In the rest of this
81
section, we'll write RE's in ``this special style``, usually without quotes, and
82
strings to be matched ``'in single quotes'``.)
84
Some characters, like ``'|'`` or ``'('``, are special. Special
85
characters either stand for classes of ordinary characters, or affect
86
how the regular expressions around them are interpreted. Regular
87
expression pattern strings may not contain null bytes, but can specify
88
the null byte using the ``\number`` notation, e.g., ``'\x00'``.
91
The special characters are:
94
(Dot.) In the default mode, this matches any character except a newline. If
95
the :const:`DOTALL` flag has been specified, this matches any character
99
(Caret.) Matches the start of the string, and in :const:`MULTILINE` mode also
100
matches immediately after each newline.
103
Matches the end of the string or just before the newline at the end of the
104
string, and in :const:`MULTILINE` mode also matches before a newline. ``foo``
105
matches both 'foo' and 'foobar', while the regular expression ``foo$`` matches
106
only 'foo'. More interestingly, searching for ``foo.$`` in ``'foo1\nfoo2\n'``
107
matches 'foo2' normally, but 'foo1' in :const:`MULTILINE` mode; searching for
108
a single ``$`` in ``'foo\n'`` will find two (empty) matches: one just before
109
the newline, and one at the end of the string.
112
Causes the resulting RE to match 0 or more repetitions of the preceding RE, as
113
many repetitions as are possible. ``ab*`` will match 'a', 'ab', or 'a' followed
114
by any number of 'b's.
117
Causes the resulting RE to match 1 or more repetitions of the preceding RE.
118
``ab+`` will match 'a' followed by any non-zero number of 'b's; it will not
122
Causes the resulting RE to match 0 or 1 repetitions of the preceding RE.
123
``ab?`` will match either 'a' or 'ab'.
125
``*?``, ``+?``, ``??``
126
The ``'*'``, ``'+'``, and ``'?'`` qualifiers are all :dfn:`greedy`; they match
127
as much text as possible. Sometimes this behaviour isn't desired; if the RE
128
``<.*>`` is matched against ``'<H1>title</H1>'``, it will match the entire
129
string, and not just ``'<H1>'``. Adding ``'?'`` after the qualifier makes it
130
perform the match in :dfn:`non-greedy` or :dfn:`minimal` fashion; as *few*
131
characters as possible will be matched. Using ``.*?`` in the previous
132
expression will match only ``'<H1>'``.
135
Specifies that exactly *m* copies of the previous RE should be matched; fewer
136
matches cause the entire RE not to match. For example, ``a{6}`` will match
137
exactly six ``'a'`` characters, but not five.
140
Causes the resulting RE to match from *m* to *n* repetitions of the preceding
141
RE, attempting to match as many repetitions as possible. For example,
142
``a{3,5}`` will match from 3 to 5 ``'a'`` characters. Omitting *m* specifies a
143
lower bound of zero, and omitting *n* specifies an infinite upper bound. As an
144
example, ``a{4,}b`` will match ``aaaab`` or a thousand ``'a'`` characters
145
followed by a ``b``, but not ``aaab``. The comma may not be omitted or the
146
modifier would be confused with the previously described form.
149
Causes the resulting RE to match from *m* to *n* repetitions of the preceding
150
RE, attempting to match as *few* repetitions as possible. This is the
151
non-greedy version of the previous qualifier. For example, on the
152
6-character string ``'aaaaaa'``, ``a{3,5}`` will match 5 ``'a'`` characters,
153
while ``a{3,5}?`` will only match 3 characters.
156
Either escapes special characters (permitting you to match characters like
157
``'*'``, ``'?'``, and so forth), or signals a special sequence; special
158
sequences are discussed below.
160
If you're not using a raw string to express the pattern, remember that Python
161
also uses the backslash as an escape sequence in string literals; if the escape
162
sequence isn't recognized by Python's parser, the backslash and subsequent
163
character are included in the resulting string. However, if Python would
164
recognize the resulting sequence, the backslash should be repeated twice. This
165
is complicated and hard to understand, so it's highly recommended that you use
166
raw strings for all but the simplest expressions.
169
Used to indicate a set of characters. Characters can be listed individually, or
170
a range of characters can be indicated by giving two characters and separating
171
them by a ``'-'``. Special characters are not active inside sets. For example,
172
``[akm$]`` will match any of the characters ``'a'``, ``'k'``,
173
``'m'``, or ``'$'``; ``[a-z]`` will match any lowercase letter, and
174
``[a-zA-Z0-9]`` matches any letter or digit. Character classes such
175
as ``\w`` or ``\S`` (defined below) are also acceptable inside a
176
range, although the characters they match depends on whether
177
:const:`ASCII` or :const:`LOCALE` mode is in force. If you want to
178
include a ``']'`` or a ``'-'`` inside a set, precede it with a
179
backslash, or place it as the first character. The pattern ``[]]``
180
will match ``']'``, for example.
182
You can match the characters not within a range by :dfn:`complementing` the set.
183
This is indicated by including a ``'^'`` as the first character of the set;
184
``'^'`` elsewhere will simply match the ``'^'`` character. For example,
185
``[^5]`` will match any character except ``'5'``, and ``[^^]`` will match any
186
character except ``'^'``.
188
Note that inside ``[]`` the special forms and special characters lose
189
their meanings and only the syntaxes described here are valid. For
190
example, ``+``, ``*``, ``(``, ``)``, and so on are treated as
191
literals inside ``[]``, and backreferences cannot be used inside
195
``A|B``, where A and B can be arbitrary REs, creates a regular expression that
196
will match either A or B. An arbitrary number of REs can be separated by the
197
``'|'`` in this way. This can be used inside groups (see below) as well. As
198
the target string is scanned, REs separated by ``'|'`` are tried from left to
199
right. When one pattern completely matches, that branch is accepted. This means
200
that once ``A`` matches, ``B`` will not be tested further, even if it would
201
produce a longer overall match. In other words, the ``'|'`` operator is never
202
greedy. To match a literal ``'|'``, use ``\|``, or enclose it inside a
203
character class, as in ``[|]``.
206
Matches whatever regular expression is inside the parentheses, and indicates the
207
start and end of a group; the contents of a group can be retrieved after a match
208
has been performed, and can be matched later in the string with the ``\number``
209
special sequence, described below. To match the literals ``'('`` or ``')'``,
210
use ``\(`` or ``\)``, or enclose them inside a character class: ``[(] [)]``.
213
This is an extension notation (a ``'?'`` following a ``'('`` is not meaningful
214
otherwise). The first character after the ``'?'`` determines what the meaning
215
and further syntax of the construct is. Extensions usually do not create a new
216
group; ``(?P<name>...)`` is the only exception to this rule. Following are the
217
currently supported extensions.
220
(One or more letters from the set ``'a'``, ``'i'``, ``'L'``, ``'m'``,
221
``'s'``, ``'u'``, ``'x'``.) The group matches the empty string; the
222
letters set the corresponding flags: :const:`re.a` (ASCII-only matching),
223
:const:`re.I` (ignore case), :const:`re.L` (locale dependent),
224
:const:`re.M` (multi-line), :const:`re.S` (dot matches all),
225
and :const:`re.X` (verbose), for the entire regular expression. (The
226
flags are described in :ref:`contents-of-module-re`.) This
227
is useful if you wish to include the flags as part of the regular
228
expression, instead of passing a *flag* argument to the
229
:func:`compile` function.
231
Note that the ``(?x)`` flag changes how the expression is parsed. It should be
232
used first in the expression string, or after one or more whitespace characters.
233
If there are non-whitespace characters before the flag, the results are
237
A non-grouping version of regular parentheses. Matches whatever regular
238
expression is inside the parentheses, but the substring matched by the group
239
*cannot* be retrieved after performing a match or referenced later in the
243
Similar to regular parentheses, but the substring matched by the group is
244
accessible via the symbolic group name *name*. Group names must be valid Python
245
identifiers, and each group name must be defined only once within a regular
246
expression. A symbolic group is also a numbered group, just as if the group
247
were not named. So the group named 'id' in the example below can also be
248
referenced as the numbered group 1.
250
For example, if the pattern is ``(?P<id>[a-zA-Z_]\w*)``, the group can be
251
referenced by its name in arguments to methods of match objects, such as
252
``m.group('id')`` or ``m.end('id')``, and also by name in pattern text (for
253
example, ``(?P=id)``) and replacement text (such as ``\g<id>``).
256
Matches whatever text was matched by the earlier group named *name*.
259
A comment; the contents of the parentheses are simply ignored.
262
Matches if ``...`` matches next, but doesn't consume any of the string. This is
263
called a lookahead assertion. For example, ``Isaac (?=Asimov)`` will match
264
``'Isaac '`` only if it's followed by ``'Asimov'``.
267
Matches if ``...`` doesn't match next. This is a negative lookahead assertion.
268
For example, ``Isaac (?!Asimov)`` will match ``'Isaac '`` only if it's *not*
269
followed by ``'Asimov'``.
272
Matches if the current position in the string is preceded by a match for ``...``
273
that ends at the current position. This is called a :dfn:`positive lookbehind
274
assertion`. ``(?<=abc)def`` will find a match in ``abcdef``, since the
275
lookbehind will back up 3 characters and check if the contained pattern matches.
276
The contained pattern must only match strings of some fixed length, meaning that
277
``abc`` or ``a|b`` are allowed, but ``a*`` and ``a{3,4}`` are not. Note that
278
patterns which start with positive lookbehind assertions will never match at the
279
beginning of the string being searched; you will most likely want to use the
280
:func:`search` function rather than the :func:`match` function:
283
>>> m = re.search('(?<=abc)def', 'abcdef')
287
This example looks for a word following a hyphen:
289
>>> m = re.search('(?<=-)\w+', 'spam-egg')
294
Matches if the current position in the string is not preceded by a match for
295
``...``. This is called a :dfn:`negative lookbehind assertion`. Similar to
296
positive lookbehind assertions, the contained pattern must only match strings of
297
some fixed length. Patterns which start with negative lookbehind assertions may
298
match at the beginning of the string being searched.
300
``(?(id/name)yes-pattern|no-pattern)``
301
Will try to match with ``yes-pattern`` if the group with given *id* or *name*
302
exists, and with ``no-pattern`` if it doesn't. ``no-pattern`` is optional and
303
can be omitted. For example, ``(<)?(\w+@\w+(?:\.\w+)+)(?(1)>)`` is a poor email
304
matching pattern, which will match with ``'<user@host.com>'`` as well as
305
``'user@host.com'``, but not with ``'<user@host.com'``.
308
The special sequences consist of ``'\'`` and a character from the list below.
309
If the ordinary character is not on the list, then the resulting RE will match
310
the second character. For example, ``\$`` matches the character ``'$'``.
313
Matches the contents of the group of the same number. Groups are numbered
314
starting from 1. For example, ``(.+) \1`` matches ``'the the'`` or ``'55 55'``,
315
but not ``'the end'`` (note the space after the group). This special sequence
316
can only be used to match one of the first 99 groups. If the first digit of
317
*number* is 0, or *number* is 3 octal digits long, it will not be interpreted as
318
a group match, but as the character with octal value *number*. Inside the
319
``'['`` and ``']'`` of a character class, all numeric escapes are treated as
323
Matches only at the start of the string.
326
Matches the empty string, but only at the beginning or end of a word.
327
A word is defined as a sequence of Unicode alphanumeric or underscore
328
characters, so the end of a word is indicated by whitespace or a
329
non-alphanumeric, non-underscore Unicode character. Note that
330
formally, ``\b`` is defined as the boundary between a ``\w`` and a
331
``\W`` character (or vice versa). By default Unicode alphanumerics
332
are the ones used, but this can be changed by using the :const:`ASCII`
333
flag. Inside a character range, ``\b`` represents the backspace
334
character, for compatibility with Python's string literals.
337
Matches the empty string, but only when it is *not* at the beginning or end of a
338
word. This is just the opposite of ``\b``, so word characters are
339
Unicode alphanumerics or the underscore, although this can be changed
340
by using the :const:`ASCII` flag.
343
For Unicode (str) patterns:
344
Matches any Unicode digit (which includes ``[0-9]``, and also many
345
other digit characters). If the :const:`ASCII` flag is used only
346
``[0-9]`` is matched (but the flag affects the entire regular
347
expression, so in such cases using an explicit ``[0-9]`` may be a
349
For 8-bit (bytes) patterns:
350
Matches any decimal digit; this is equivalent to ``[0-9]``.
353
Matches any character which is not a Unicode decimal digit. This is
354
the opposite of ``\d``. If the :const:`ASCII` flag is used this
355
becomes the equivalent of ``[^0-9]`` (but the flag affects the entire
356
regular expression, so in such cases using an explicit ``[^0-9]`` may
360
For Unicode (str) patterns:
361
Matches Unicode whitespace characters (which includes
362
``[ \t\n\r\f\v]``, and also many other characters, for example the
363
non-breaking spaces mandated by typography rules in many
364
languages). If the :const:`ASCII` flag is used, only
365
``[ \t\n\r\f\v]`` is matched (but the flag affects the entire
366
regular expression, so in such cases using an explicit
367
``[ \t\n\r\f\v]`` may be a better choice).
369
For 8-bit (bytes) patterns:
370
Matches characters considered whitespace in the ASCII character set;
371
this is equivalent to ``[ \t\n\r\f\v]``.
374
Matches any character which is not a Unicode whitespace character. This is
375
the opposite of ``\s``. If the :const:`ASCII` flag is used this
376
becomes the equivalent of ``[^ \t\n\r\f\v]`` (but the flag affects the entire
377
regular expression, so in such cases using an explicit ``[^ \t\n\r\f\v]`` may
381
For Unicode (str) patterns:
382
Matches Unicode word characters; this includes most characters
383
that can be part of a word in any language, as well as numbers and
384
the underscore. If the :const:`ASCII` flag is used, only
385
``[a-zA-Z0-9_]`` is matched (but the flag affects the entire
386
regular expression, so in such cases using an explicit
387
``[a-zA-Z0-9_]`` may be a better choice).
388
For 8-bit (bytes) patterns:
389
Matches characters considered alphanumeric in the ASCII character set;
390
this is equivalent to ``[a-zA-Z0-9_]``.
393
Matches any character which is not a Unicode word character. This is
394
the opposite of ``\w``. If the :const:`ASCII` flag is used this
395
becomes the equivalent of ``[^a-zA-Z0-9_]`` (but the flag affects the
396
entire regular expression, so in such cases using an explicit
397
``[^a-zA-Z0-9_]`` may be a better choice).
400
Matches only at the end of the string.
402
Most of the standard escapes supported by Python string literals are also
403
accepted by the regular expression parser::
409
Octal escapes are included in a limited form: If the first digit is a 0, or if
410
there are three octal digits, it is considered an octal escape. Otherwise, it is
411
a group reference. As for string literals, octal escapes are always at most
412
three digits in length.
415
.. _matching-searching:
417
Matching vs Searching
418
---------------------
420
.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
423
Python offers two different primitive operations based on regular expressions:
424
**match** checks for a match only at the beginning of the string, while
425
**search** checks for a match anywhere in the string (this is what Perl does
428
Note that match may differ from search even when using a regular expression
429
beginning with ``'^'``: ``'^'`` matches only at the start of the string, or in
430
:const:`MULTILINE` mode also immediately following a newline. The "match"
431
operation succeeds only if the pattern matches at the start of the string
432
regardless of mode, or at the starting position given by the optional *pos*
433
argument regardless of whether a newline precedes it.
435
>>> re.match("c", "abcdef") # No match
436
>>> re.search("c", "abcdef") # Match
437
<_sre.SRE_Match object at ...>
440
.. _contents-of-module-re:
445
The module defines several functions, constants, and an exception. Some of the
446
functions are simplified versions of the full featured methods for compiled
447
regular expressions. Most non-trivial applications always use the compiled
451
.. function:: compile(pattern[, flags])
453
Compile a regular expression pattern into a regular expression object, which
454
can be used for matching using its :func:`match` and :func:`search` methods,
457
The expression's behaviour can be modified by specifying a *flags* value.
458
Values can be any of the following variables, combined using bitwise OR (the
463
prog = re.compile(pattern)
464
result = prog.match(string)
468
result = re.match(pattern, string)
470
but using :func:`compile` and saving the resulting regular expression object
471
for reuse is more efficient when the expression will be used several times
476
The compiled versions of the most recent patterns passed to
477
:func:`re.match`, :func:`re.search` or :func:`re.compile` are cached, so
478
programs that use only a few regular expressions at a time needn't worry
479
about compiling regular expressions.
485
Make ``\w``, ``\W``, ``\b``, ``\B``, ``\s`` and ``\S`` perform ASCII-only
486
matching instead of full Unicode matching. This is only meaningful for
487
Unicode patterns, and is ignored for byte patterns.
489
Note that for backward compatibility, the :const:`re.U` flag still
490
exists (as well as its synonym :const:`re.UNICODE` and its embedded
491
counterpart ``(?u)``), but these are redundant in Python 3.0 since
492
matches are Unicode by default for strings (and Unicode matching
493
isn't allowed for bytes).
499
Perform case-insensitive matching; expressions like ``[A-Z]`` will match
500
lowercase letters, too. This is not affected by the current locale
501
and works for Unicode characters as expected.
507
Make ``\w``, ``\W``, ``\b``, ``\B``, ``\s`` and ``\S`` dependent on the
508
current locale. The use of this flag is discouraged as the locale mechanism
509
is very unreliable, and it only handles one "culture" at a time anyway;
510
you should use Unicode matching instead, which is the default in Python 3.0
511
for Unicode (str) patterns.
517
When specified, the pattern character ``'^'`` matches at the beginning of the
518
string and at the beginning of each line (immediately following each newline);
519
and the pattern character ``'$'`` matches at the end of the string and at the
520
end of each line (immediately preceding each newline). By default, ``'^'``
521
matches only at the beginning of the string, and ``'$'`` only at the end of the
522
string and immediately before the newline (if any) at the end of the string.
528
Make the ``'.'`` special character match any character at all, including a
529
newline; without this flag, ``'.'`` will match anything *except* a newline.
535
This flag allows you to write regular expressions that look nicer. Whitespace
536
within the pattern is ignored, except when in a character class or preceded by
537
an unescaped backslash, and, when a line contains a ``'#'`` neither in a
538
character class or preceded by an unescaped backslash, all characters from the
539
leftmost such ``'#'`` through the end of the line are ignored.
541
That means that the two following regular expression objects that match a
542
decimal number are functionally equal::
544
a = re.compile(r"""\d + # the integral part
545
\. # the decimal point
546
\d * # some fractional digits""", re.X)
547
b = re.compile(r"\d+\.\d*")
552
.. function:: search(pattern, string[, flags])
554
Scan through *string* looking for a location where the regular expression
555
*pattern* produces a match, and return a corresponding :class:`MatchObject`
556
instance. Return ``None`` if no position in the string matches the pattern; note
557
that this is different from finding a zero-length match at some point in the
561
.. function:: match(pattern, string[, flags])
563
If zero or more characters at the beginning of *string* match the regular
564
expression *pattern*, return a corresponding :class:`MatchObject` instance.
565
Return ``None`` if the string does not match the pattern; note that this is
566
different from a zero-length match.
570
If you want to locate a match anywhere in *string*, use :meth:`search`
574
.. function:: split(pattern, string[, maxsplit=0, flags=0])
576
Split *string* by the occurrences of *pattern*. If capturing parentheses are
577
used in *pattern*, then the text of all groups in the pattern are also returned
578
as part of the resulting list. If *maxsplit* is nonzero, at most *maxsplit*
579
splits occur, and the remainder of the string is returned as the final element
582
>>> re.split('\W+', 'Words, words, words.')
583
['Words', 'words', 'words', '']
584
>>> re.split('(\W+)', 'Words, words, words.')
585
['Words', ', ', 'words', ', ', 'words', '.', '']
586
>>> re.split('\W+', 'Words, words, words.', 1)
587
['Words', 'words, words.']
588
>>> re.split('[a-f]+', '0a3B9', flags=re.IGNORECASE)
591
If there are capturing groups in the separator and it matches at the start of
592
the string, the result will start with an empty string. The same holds for
593
the end of the string:
595
>>> re.split('(\W+)', '...words, words...')
596
['', '...', 'words', ', ', 'words', '...', '']
598
That way, separator components are always found at the same relative
599
indices within the result list (e.g., if there's one capturing group
600
in the separator, the 0th, the 2nd and so forth).
602
Note that *split* will never split a string on an empty pattern match.
605
>>> re.split('x*', 'foo')
607
>>> re.split("(?m)^$", "foo\n\nbar\n")
610
.. versionchanged:: 2.7,3.1
611
Added the optional flags argument.
614
.. function:: findall(pattern, string[, flags])
616
Return all non-overlapping matches of *pattern* in *string*, as a list of
617
strings. The *string* is scanned left-to-right, and matches are returned in
618
the order found. If one or more groups are present in the pattern, return a
619
list of groups; this will be a list of tuples if the pattern has more than
620
one group. Empty matches are included in the result unless they touch the
621
beginning of another match.
624
.. function:: finditer(pattern, string[, flags])
626
Return an :term:`iterator` yielding :class:`MatchObject` instances over all
627
non-overlapping matches for the RE *pattern* in *string*. The *string* is
628
scanned left-to-right, and matches are returned in the order found. Empty
629
matches are included in the result unless they touch the beginning of another
633
.. function:: sub(pattern, repl, string[, count, flags])
635
Return the string obtained by replacing the leftmost non-overlapping occurrences
636
of *pattern* in *string* by the replacement *repl*. If the pattern isn't found,
637
*string* is returned unchanged. *repl* can be a string or a function; if it is
638
a string, any backslash escapes in it are processed. That is, ``\n`` is
639
converted to a single newline character, ``\r`` is converted to a linefeed, and
640
so forth. Unknown escapes such as ``\j`` are left alone. Backreferences, such
641
as ``\6``, are replaced with the substring matched by group 6 in the pattern.
644
>>> re.sub(r'def\s+([a-zA-Z_][a-zA-Z_0-9]*)\s*\(\s*\):',
645
... r'static PyObject*\npy_\1(void)\n{',
647
'static PyObject*\npy_myfunc(void)\n{'
649
If *repl* is a function, it is called for every non-overlapping occurrence of
650
*pattern*. The function takes a single match object argument, and returns the
651
replacement string. For example:
653
>>> def dashrepl(matchobj):
654
... if matchobj.group(0) == '-': return ' '
656
>>> re.sub('-{1,2}', dashrepl, 'pro----gram-files')
658
>>> re.sub(r'\sAND\s', ' & ', 'Baked Beans And Spam', flags=re.IGNORECASE)
661
The pattern may be a string or an RE object; if you need to specify regular
662
expression flags, you must use a RE object, or use embedded modifiers in a
663
pattern; for example, ``sub("(?i)b+", "x", "bbbb BBBB")`` returns ``'x x'``.
665
The optional argument *count* is the maximum number of pattern occurrences to be
666
replaced; *count* must be a non-negative integer. If omitted or zero, all
667
occurrences will be replaced. Empty matches for the pattern are replaced only
668
when not adjacent to a previous match, so ``sub('x*', '-', 'abc')`` returns
671
In addition to character escapes and backreferences as described above,
672
``\g<name>`` will use the substring matched by the group named ``name``, as
673
defined by the ``(?P<name>...)`` syntax. ``\g<number>`` uses the corresponding
674
group number; ``\g<2>`` is therefore equivalent to ``\2``, but isn't ambiguous
675
in a replacement such as ``\g<2>0``. ``\20`` would be interpreted as a
676
reference to group 20, not a reference to group 2 followed by the literal
677
character ``'0'``. The backreference ``\g<0>`` substitutes in the entire
678
substring matched by the RE.
680
.. versionchanged:: 2.7,3.1
681
Added the optional flags argument.
684
.. function:: subn(pattern, repl, string[, count, flags])
686
Perform the same operation as :func:`sub`, but return a tuple ``(new_string,
687
number_of_subs_made)``.
689
.. versionchanged:: 2.7,3.1
690
Added the optional flags argument.
693
.. function:: escape(string)
695
Return *string* with all non-alphanumerics backslashed; this is useful if you
696
want to match an arbitrary literal string that may have regular expression
697
metacharacters in it.
702
Exception raised when a string passed to one of the functions here is not a
703
valid regular expression (for example, it might contain unmatched parentheses)
704
or when some other error occurs during compilation or matching. It is never an
705
error if a string contains no match for a pattern.
710
Regular Expression Objects
711
--------------------------
713
Compiled regular expression objects support the following methods and
717
.. method:: RegexObject.match(string[, pos[, endpos]])
719
If zero or more characters at the beginning of *string* match this regular
720
expression, return a corresponding :class:`MatchObject` instance. Return
721
``None`` if the string does not match the pattern; note that this is different
722
from a zero-length match.
726
If you want to locate a match anywhere in *string*, use :meth:`search`
729
The optional second parameter *pos* gives an index in the string where the
730
search is to start; it defaults to ``0``. This is not completely equivalent to
731
slicing the string; the ``'^'`` pattern character matches at the real beginning
732
of the string and at positions just after a newline, but not necessarily at the
733
index where the search is to start.
735
The optional parameter *endpos* limits how far the string will be searched; it
736
will be as if the string is *endpos* characters long, so only the characters
737
from *pos* to ``endpos - 1`` will be searched for a match. If *endpos* is less
738
than *pos*, no match will be found, otherwise, if *rx* is a compiled regular
739
expression object, ``rx.match(string, 0, 50)`` is equivalent to
740
``rx.match(string[:50], 0)``.
742
>>> pattern = re.compile("o")
743
>>> pattern.match("dog") # No match as "o" is not at the start of "dog."
744
>>> pattern.match("dog", 1) # Match as "o" is the 2nd character of "dog".
745
<_sre.SRE_Match object at ...>
748
.. method:: RegexObject.search(string[, pos[, endpos]])
750
Scan through *string* looking for a location where this regular expression
751
produces a match, and return a corresponding :class:`MatchObject` instance.
752
Return ``None`` if no position in the string matches the pattern; note that this
753
is different from finding a zero-length match at some point in the string.
755
The optional *pos* and *endpos* parameters have the same meaning as for the
756
:meth:`match` method.
759
.. method:: RegexObject.split(string[, maxsplit=0])
761
Identical to the :func:`split` function, using the compiled pattern.
764
.. method:: RegexObject.findall(string[, pos[, endpos]])
766
Identical to the :func:`findall` function, using the compiled pattern.
769
.. method:: RegexObject.finditer(string[, pos[, endpos]])
771
Identical to the :func:`finditer` function, using the compiled pattern.
774
.. method:: RegexObject.sub(repl, string[, count=0])
776
Identical to the :func:`sub` function, using the compiled pattern.
779
.. method:: RegexObject.subn(repl, string[, count=0])
781
Identical to the :func:`subn` function, using the compiled pattern.
784
.. attribute:: RegexObject.flags
786
The flags argument used when the RE object was compiled, or ``0`` if no flags
790
.. attribute:: RegexObject.groups
792
The number of capturing groups in the pattern.
795
.. attribute:: RegexObject.groupindex
797
A dictionary mapping any symbolic group names defined by ``(?P<id>)`` to group
798
numbers. The dictionary is empty if no symbolic groups were used in the
802
.. attribute:: RegexObject.pattern
804
The pattern string from which the RE object was compiled.
812
Match objects always have a boolean value of :const:`True`, so that you can test
813
whether e.g. :func:`match` resulted in a match with a simple if statement. They
814
support the following methods and attributes:
817
.. method:: MatchObject.expand(template)
819
Return the string obtained by doing backslash substitution on the template
820
string *template*, as done by the :meth:`sub` method. Escapes such as ``\n`` are
821
converted to the appropriate characters, and numeric backreferences (``\1``,
822
``\2``) and named backreferences (``\g<1>``, ``\g<name>``) are replaced by the
823
contents of the corresponding group.
826
.. method:: MatchObject.group([group1, ...])
828
Returns one or more subgroups of the match. If there is a single argument, the
829
result is a single string; if there are multiple arguments, the result is a
830
tuple with one item per argument. Without arguments, *group1* defaults to zero
831
(the whole match is returned). If a *groupN* argument is zero, the corresponding
832
return value is the entire matching string; if it is in the inclusive range
833
[1..99], it is the string matching the corresponding parenthesized group. If a
834
group number is negative or larger than the number of groups defined in the
835
pattern, an :exc:`IndexError` exception is raised. If a group is contained in a
836
part of the pattern that did not match, the corresponding result is ``None``.
837
If a group is contained in a part of the pattern that matched multiple times,
838
the last match is returned.
840
>>> m = re.match(r"(\w+) (\w+)", "Isaac Newton, physicist")
841
>>> m.group(0) # The entire match
843
>>> m.group(1) # The first parenthesized subgroup.
845
>>> m.group(2) # The second parenthesized subgroup.
847
>>> m.group(1, 2) # Multiple arguments give us a tuple.
850
If the regular expression uses the ``(?P<name>...)`` syntax, the *groupN*
851
arguments may also be strings identifying groups by their group name. If a
852
string argument is not used as a group name in the pattern, an :exc:`IndexError`
855
A moderately complicated example:
857
>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
858
>>> m.group('first_name')
860
>>> m.group('last_name')
863
Named groups can also be referred to by their index:
870
If a group matches multiple times, only the last match is accessible:
872
>>> m = re.match(r"(..)+", "a1b2c3") # Matches 3 times.
873
>>> m.group(1) # Returns only the last match.
877
.. method:: MatchObject.groups([default])
879
Return a tuple containing all the subgroups of the match, from 1 up to however
880
many groups are in the pattern. The *default* argument is used for groups that
881
did not participate in the match; it defaults to ``None``.
885
>>> m = re.match(r"(\d+)\.(\d+)", "24.1632")
889
If we make the decimal place and everything after it optional, not all groups
890
might participate in the match. These groups will default to ``None`` unless
891
the *default* argument is given:
893
>>> m = re.match(r"(\d+)\.?(\d+)?", "24")
894
>>> m.groups() # Second group defaults to None.
896
>>> m.groups('0') # Now, the second group defaults to '0'.
900
.. method:: MatchObject.groupdict([default])
902
Return a dictionary containing all the *named* subgroups of the match, keyed by
903
the subgroup name. The *default* argument is used for groups that did not
904
participate in the match; it defaults to ``None``. For example:
906
>>> m = re.match(r"(?P<first_name>\w+) (?P<last_name>\w+)", "Malcom Reynolds")
908
{'first_name': 'Malcom', 'last_name': 'Reynolds'}
911
.. method:: MatchObject.start([group])
912
MatchObject.end([group])
914
Return the indices of the start and end of the substring matched by *group*;
915
*group* defaults to zero (meaning the whole matched substring). Return ``-1`` if
916
*group* exists but did not contribute to the match. For a match object *m*, and
917
a group *g* that did contribute to the match, the substring matched by group *g*
918
(equivalent to ``m.group(g)``) is ::
920
m.string[m.start(g):m.end(g)]
922
Note that ``m.start(group)`` will equal ``m.end(group)`` if *group* matched a
923
null string. For example, after ``m = re.search('b(c?)', 'cba')``,
924
``m.start(0)`` is 1, ``m.end(0)`` is 2, ``m.start(1)`` and ``m.end(1)`` are both
925
2, and ``m.start(2)`` raises an :exc:`IndexError` exception.
927
An example that will remove *remove_this* from email addresses:
929
>>> email = "tony@tiremove_thisger.net"
930
>>> m = re.search("remove_this", email)
931
>>> email[:m.start()] + email[m.end():]
935
.. method:: MatchObject.span([group])
937
For :class:`MatchObject` *m*, return the 2-tuple ``(m.start(group),
938
m.end(group))``. Note that if *group* did not contribute to the match, this is
939
``(-1, -1)``. *group* defaults to zero, the entire match.
942
.. attribute:: MatchObject.pos
944
The value of *pos* which was passed to the :func:`search` or :func:`match`
945
method of the :class:`RegexObject`. This is the index into the string at which
946
the RE engine started looking for a match.
949
.. attribute:: MatchObject.endpos
951
The value of *endpos* which was passed to the :func:`search` or :func:`match`
952
method of the :class:`RegexObject`. This is the index into the string beyond
953
which the RE engine will not go.
956
.. attribute:: MatchObject.lastindex
958
The integer index of the last matched capturing group, or ``None`` if no group
959
was matched at all. For example, the expressions ``(a)b``, ``((a)(b))``, and
960
``((ab))`` will have ``lastindex == 1`` if applied to the string ``'ab'``, while
961
the expression ``(a)(b)`` will have ``lastindex == 2``, if applied to the same
965
.. attribute:: MatchObject.lastgroup
967
The name of the last matched capturing group, or ``None`` if the group didn't
968
have a name, or if no group was matched at all.
971
.. attribute:: MatchObject.re
973
The regular expression object whose :meth:`match` or :meth:`search` method
974
produced this :class:`MatchObject` instance.
977
.. attribute:: MatchObject.string
979
The string passed to :func:`match` or :func:`search`.
989
In this example, we'll use the following helper function to display match
990
objects a little more gracefully:
994
def displaymatch(match):
997
return '<Match: %r, groups=%r>' % (match.group(), match.groups())
999
Suppose you are writing a poker program where a player's hand is represented as
1000
a 5-character string with each character representing a card, "a" for ace, "k"
1001
for king, "q" for queen, j for jack, "0" for 10, and "1" through "9"
1002
representing the card with that value.
1004
To see if a given string is a valid hand, one could do the following:
1006
>>> valid = re.compile(r"[0-9akqj]{5}$")
1007
>>> displaymatch(valid.match("ak05q")) # Valid.
1008
"<Match: 'ak05q', groups=()>"
1009
>>> displaymatch(valid.match("ak05e")) # Invalid.
1010
>>> displaymatch(valid.match("ak0")) # Invalid.
1011
>>> displaymatch(valid.match("727ak")) # Valid.
1012
"<Match: '727ak', groups=()>"
1014
That last hand, ``"727ak"``, contained a pair, or two of the same valued cards.
1015
To match this with a regular expression, one could use backreferences as such:
1017
>>> pair = re.compile(r".*(.).*\1")
1018
>>> displaymatch(pair.match("717ak")) # Pair of 7s.
1019
"<Match: '717', groups=('7',)>"
1020
>>> displaymatch(pair.match("718ak")) # No pairs.
1021
>>> displaymatch(pair.match("354aa")) # Pair of aces.
1022
"<Match: '354aa', groups=('a',)>"
1024
To find out what card the pair consists of, one could use the :func:`group`
1025
method of :class:`MatchObject` in the following manner:
1029
>>> pair.match("717ak").group(1)
1032
# Error because re.match() returns None, which doesn't have a group() method:
1033
>>> pair.match("718ak").group(1)
1034
Traceback (most recent call last):
1035
File "<pyshell#23>", line 1, in <module>
1036
re.match(r".*(.).*\1", "718ak").group(1)
1037
AttributeError: 'NoneType' object has no attribute 'group'
1039
>>> pair.match("354aa").group(1)
1046
.. index:: single: scanf()
1048
Python does not currently have an equivalent to :cfunc:`scanf`. Regular
1049
expressions are generally more powerful, though also more verbose, than
1050
:cfunc:`scanf` format strings. The table below offers some more-or-less
1051
equivalent mappings between :cfunc:`scanf` format tokens and regular
1054
+--------------------------------+---------------------------------------------+
1055
| :cfunc:`scanf` Token | Regular Expression |
1056
+================================+=============================================+
1058
+--------------------------------+---------------------------------------------+
1059
| ``%5c`` | ``.{5}`` |
1060
+--------------------------------+---------------------------------------------+
1061
| ``%d`` | ``[-+]?\d+`` |
1062
+--------------------------------+---------------------------------------------+
1063
| ``%e``, ``%E``, ``%f``, ``%g`` | ``[-+]?(\d+(\.\d*)?|\.\d+)([eE][-+]?\d+)?`` |
1064
+--------------------------------+---------------------------------------------+
1065
| ``%i`` | ``[-+]?(0[xX][\dA-Fa-f]+|0[0-7]*|\d+)`` |
1066
+--------------------------------+---------------------------------------------+
1067
| ``%o`` | ``0[0-7]*`` |
1068
+--------------------------------+---------------------------------------------+
1069
| ``%s`` | ``\S+`` |
1070
+--------------------------------+---------------------------------------------+
1071
| ``%u`` | ``\d+`` |
1072
+--------------------------------+---------------------------------------------+
1073
| ``%x``, ``%X`` | ``0[xX][\dA-Fa-f]+`` |
1074
+--------------------------------+---------------------------------------------+
1076
To extract the filename and numbers from a string like ::
1078
/usr/sbin/sendmail - 0 errors, 4 warnings
1080
you would use a :cfunc:`scanf` format like ::
1082
%s - %d errors, %d warnings
1084
The equivalent regular expression would be ::
1086
(\S+) - (\d+) errors, (\d+) warnings
1092
If you create regular expressions that require the engine to perform a lot of
1093
recursion, you may encounter a :exc:`RuntimeError` exception with the message
1094
``maximum recursion limit`` exceeded. For example, ::
1096
>>> s = 'Begin ' + 1000*'a very long string ' + 'end'
1097
>>> re.match('Begin (\w| )*? end', s).end()
1098
Traceback (most recent call last):
1099
File "<stdin>", line 1, in ?
1100
File "/usr/local/lib/python2.5/re.py", line 132, in match
1101
return _compile(pattern, flags).match(string)
1102
RuntimeError: maximum recursion limit exceeded
1104
You can often restructure your regular expression to avoid recursion.
1106
Simple uses of the ``*?`` pattern are special-cased to avoid recursion. Thus,
1107
the above regular expression can avoid recursion by being recast as ``Begin
1108
[a-zA-Z0-9_ ]*?end``. As a further benefit, such regular expressions will run
1109
faster than their recursive equivalents.
1112
search() vs. match()
1113
^^^^^^^^^^^^^^^^^^^^
1115
In a nutshell, :func:`match` only attempts to match a pattern at the beginning
1116
of a string where :func:`search` will match a pattern anywhere in a string.
1119
>>> re.match("o", "dog") # No match as "o" is not the first letter of "dog".
1120
>>> re.search("o", "dog") # Match as search() looks everywhere in the string.
1121
<_sre.SRE_Match object at ...>
1125
The following applies only to regular expression objects like those created
1126
with ``re.compile("pattern")``, not the primitives ``re.match(pattern,
1127
string)`` or ``re.search(pattern, string)``.
1129
:func:`match` has an optional second parameter that gives an index in the string
1130
where the search is to start::
1132
>>> pattern = re.compile("o")
1133
>>> pattern.match("dog") # No match as "o" is not at the start of "dog."
1135
# Equivalent to the above expression as 0 is the default starting index:
1136
>>> pattern.match("dog", 0)
1138
# Match as "o" is the 2nd character of "dog" (index 0 is the first):
1139
>>> pattern.match("dog", 1)
1140
<_sre.SRE_Match object at ...>
1141
>>> pattern.match("dog", 2) # No match as "o" is not the 3rd character of "dog."
1147
:func:`split` splits a string into a list delimited by the passed pattern. The
1148
method is invaluable for converting textual data into data structures that can be
1149
easily read and modified by Python as demonstrated in the following example that
1150
creates a phonebook.
1152
First, here is the input. Normally it may come from a file, here we are using
1153
triple-quoted string syntax:
1155
>>> input = """Ross McFluff: 834.345.1254 155 Elm Street
1157
... Ronald Heathmore: 892.345.3428 436 Finley Avenue
1158
... Frank Burger: 925.541.7625 662 South Dogwood Way
1161
... Heather Albrecht: 548.326.4584 919 Park Place"""
1163
The entries are separated by one or more newlines. Now we convert the string
1164
into a list with each nonempty line having its own entry:
1167
:options: +NORMALIZE_WHITESPACE
1169
>>> entries = re.split("\n+", input)
1171
['Ross McFluff: 834.345.1254 155 Elm Street',
1172
'Ronald Heathmore: 892.345.3428 436 Finley Avenue',
1173
'Frank Burger: 925.541.7625 662 South Dogwood Way',
1174
'Heather Albrecht: 548.326.4584 919 Park Place']
1176
Finally, split each entry into a list with first name, last name, telephone
1177
number, and address. We use the ``maxsplit`` parameter of :func:`split`
1178
because the address has spaces, our splitting pattern, in it:
1181
:options: +NORMALIZE_WHITESPACE
1183
>>> [re.split(":? ", entry, 3) for entry in entries]
1184
[['Ross', 'McFluff', '834.345.1254', '155 Elm Street'],
1185
['Ronald', 'Heathmore', '892.345.3428', '436 Finley Avenue'],
1186
['Frank', 'Burger', '925.541.7625', '662 South Dogwood Way'],
1187
['Heather', 'Albrecht', '548.326.4584', '919 Park Place']]
1189
The ``:?`` pattern matches the colon after the last name, so that it does not
1190
occur in the result list. With a ``maxsplit`` of ``4``, we could separate the
1191
house number from the street name:
1194
:options: +NORMALIZE_WHITESPACE
1196
>>> [re.split(":? ", entry, 4) for entry in entries]
1197
[['Ross', 'McFluff', '834.345.1254', '155', 'Elm Street'],
1198
['Ronald', 'Heathmore', '892.345.3428', '436', 'Finley Avenue'],
1199
['Frank', 'Burger', '925.541.7625', '662', 'South Dogwood Way'],
1200
['Heather', 'Albrecht', '548.326.4584', '919', 'Park Place']]
1206
:func:`sub` replaces every occurrence of a pattern with a string or the
1207
result of a function. This example demonstrates using :func:`sub` with
1208
a function to "munge" text, or randomize the order of all the characters
1209
in each word of a sentence except for the first and last characters::
1212
... inner_word = list(m.group(2))
1213
... random.shuffle(inner_word)
1214
... return m.group(1) + "".join(inner_word) + m.group(3)
1215
>>> text = "Professor Abdolmalek, please report your absences promptly."
1216
>>> re.sub("(\w)(\w+)(\w)", repl, text)
1217
'Poefsrosr Aealmlobdk, pslaee reorpt your abnseces plmrptoy.'
1218
>>> re.sub("(\w)(\w+)(\w)", repl, text)
1219
'Pofsroser Aodlambelk, plasee reoprt yuor asnebces potlmrpy.'
1225
:func:`findall` matches *all* occurrences of a pattern, not just the first
1226
one as :func:`search` does. For example, if one was a writer and wanted to
1227
find all of the adverbs in some text, he or she might use :func:`findall` in
1228
the following manner:
1230
>>> text = "He was carefully disguised but captured quickly by police."
1231
>>> re.findall(r"\w+ly", text)
1232
['carefully', 'quickly']
1235
Finding all Adverbs and their Positions
1236
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
1238
If one wants more information about all matches of a pattern than the matched
1239
text, :func:`finditer` is useful as it provides instances of
1240
:class:`MatchObject` instead of strings. Continuing with the previous example,
1241
if one was a writer who wanted to find all of the adverbs *and their positions*
1242
in some text, he or she would use :func:`finditer` in the following manner:
1244
>>> text = "He was carefully disguised but captured quickly by police."
1245
>>> for m in re.finditer(r"\w+ly", text):
1246
... print('%02d-%02d: %s' % (m.start(), m.end(), m.group(0)))
1254
Raw string notation (``r"text"``) keeps regular expressions sane. Without it,
1255
every backslash (``'\'``) in a regular expression would have to be prefixed with
1256
another one to escape it. For example, the two following lines of code are
1257
functionally identical:
1259
>>> re.match(r"\W(.)\1\W", " ff ")
1260
<_sre.SRE_Match object at ...>
1261
>>> re.match("\\W(.)\\1\\W", " ff ")
1262
<_sre.SRE_Match object at ...>
1264
When one wants to match a literal backslash, it must be escaped in the regular
1265
expression. With raw string notation, this means ``r"\\"``. Without raw string
1266
notation, one must use ``"\\\\"``, making the following lines of code
1267
functionally identical:
1269
>>> re.match(r"\\", r"\\")
1270
<_sre.SRE_Match object at ...>
1271
>>> re.match("\\\\", r"\\")
1272
<_sre.SRE_Match object at ...>
added
removed
Doc/library/re.rst
