A regular expression is a special sequence of characters that helps you match or find other strings or sets of strings, using a specialized syntax held in a pattern. Regular expressions are widely used in UNIX world.

The module re provides full support for Perl-like regular expressions in Python. The re module raises the exception re.error if an error occurs while compiling or using a regular expression.

We would cover two important functions, which would be used to handle regular expressions. But a small thing first: There are various characters, which would have special meaning when they are used in regular expression. To avoid any confusion while dealing with regular expressions, we would use Raw Strings as r'expression'.

The match Function

This function attempts to match RE pattern to string with optional flags.

Here is the syntax for this function:

re.match(pattern, string, flags=0)

Here is the description of the parameters:

	Parameter			Description
	pattern			This is the regular expression to be matched.
	string			This is the string, which would be searched to match the
				pattern at the beginning of string.
	flags			You can specify different flags using bitwise OR (|). These
				are modifiers, which are listed in the table below.
	The re.match function			returns  a match object  on  success, none on  failure.  We
usegroup(num) or groups() function of match object to get matched expression.
	Match	Object		Description
	Methods

group(num=0)	This method returns entire match (or specific subgroup num)
groups()	This  method  returns  all  matching  subgroups  in  a  tuple
	(empty if there weren't any)

Example

#!/usr/bin/python import re

line = "Cats are smarter than dogs"

matchObj = re.match( r'(.*) are (.*?) .*', line, re.M|re.I)

if matchObj:

print "matchObj.group() : ", matchObj.group() print "matchObj.group(1) : ", matchObj.group(1) print "matchObj.group(2) : ", matchObj.group(2)

else:

print "No match!!"

When the above code is executed, it produces following result:

matchObj.group() :     Cats are smarter than dogs

matchObj.group(1) :     Cats

matchObj.group(2) :     smarter

The search Function

This function searches for first occurrence of RE pattern within string with optional flags.

Here is the syntax for this function:

A regular expression is a special sequence of characters that helps you match or find other strings or sets of strings, using a specialized syntax held in a pattern. Regular expressions are widely used in UNIX world.

The module re provides full support for Perl-like regular expressions in Python. The re module raises the exception re.error if an error occurs while compiling or using a regular expression.

We would cover two important functions, which would be used to handle regular expressions. But a small thing first: There are various characters, which would have special meaning when they are used in regular expression. To avoid any confusion while dealing with regular expressions, we would use Raw Strings as r'expression'.

The match Function

This function attempts to match RE pattern to string with optional flags.

Here is the syntax for this function:

re.match(pattern, string, flags=0)

Here is the description of the parameters:

	Parameter			Description
	pattern			This is the regular expression to be matched.
	string			This is the string, which would be searched to match the
				pattern at the beginning of string.
	flags			You can specify different flags using bitwise OR (|). These
				are modifiers, which are listed in the table below.
	The re.match function			returns  a match object  on  success, none on  failure.  We
usegroup(num) or groups() function of match object to get matched expression.
	Match	Object		Description
	Methods

 

group(num=0)	This method returns entire match (or specific subgroup num)
groups()	This  method  returns  all  matching  subgroups  in  a  tuple
	(empty if there weren't any)

Example

#!/usr/bin/python import re

line = "Cats are smarter than dogs"

matchObj = re.match( r'(.*) are (.*?) .*', line, re.M|re.I)

if matchObj:

print "matchObj.group() : ", matchObj.group() print "matchObj.group(1) : ", matchObj.group(1) print "matchObj.group(2) : ", matchObj.group(2)

else:

print "No match!!"

When the above code is executed, it produces following result:

matchObj.group() :     Cats are smarter than dogs

matchObj.group(1) :     Cats

matchObj.group(2) :     smarter

The search Function

this function searches for first occurrence of RE pattern within string with optional flags.

Here is the syntax for this function:

Python

re.search(pattern, string, flags=0)

Here is the description of the parameters:

	Parameter		Description
	pattern		This is the regular expression to be matched.
	string		This is the string, which is searched to match the pattern
			anywhere in the string.
	flags		You can specify different flags using bitwise OR (|). These
			are modifiers, which are listed in the table below.

The re.search function returns a match object on success, none on failure. We use group(num) or groups() function of match object to get matched expression.

 

Match	Object	Description
Methods
group(num=0)		This method returns entire match (or specific subgroup num).
groups()		This method returns all matching subgroups in a tuple (empty if
		there weren't any).

Example

#!/usr/bin/python import re

line = "Cats are smarter than dogs";

searchObj = re.search( r'(.*) are (.*?) .*', line, re.M|re.I)

if searchObj:

print "searchObj.group() : ", searchObj.group()

print "searchObj.group(1) : ", searchObj.group(1)

print "searchObj.group(2) : ", searchObj.group(2)

else:

print "Nothing found!!"

When the above code is executed, it produces following result:

matchObj.group() :     Cats are smarter than dogs

matchObj.group(1) :     Cats

matchObj.group(2) :     smarter

Matching Versus Searching

Python offers two different primitive operations based on regular expressions: match checks for a match only at the beginning of the string, while search checks for a match anywhere in the string (this is what Perl does by default).

Example

#!/usr/bin/python import re

line = "Cats are smarter than dogs";

matchObj = re.match( r'dogs', line, re.M|re.I) if matchObj:

print "match --> matchObj.group() : ", matchObj.group() else:

print "No match!!"

searchObj = re.search( r'dogs', line, re.M|re.I) if searchObj:

print "search --> searchObj.group() : ", searchObj.group()

else:

print "Nothing found!!"

When the above code is executed, it produces the following result:

No match!!

search --> matchObj.group() :     dogs

Search and Replace

One of the most important re methods that use regular expressions is sub.

Syntax

re.sub(pattern, repl, string, max=0)

This method replaces all occurrences of the RE pattern in string with repl, substituting all occurrences unless max provided. This method returns modified string.

Example

#!/usr/bin/python import re

phone = "2004-959-559 # This is Phone Number"

• Delete Python-style comments num = re.sub(r'#.*$', "", phone) print "Phone Num : ", num
• Remove anything other than digits num = re.sub(r'\D', "", phone)print "Phone Num : ", num

When the above code is executed, it produces the following result:

Phone Num :     2004-959-559

Phone Num :     2004959559

Regular-Expression Modifiers: Option Flags

Regular expression literals may include an optional modifier to control various aspects of matching. The modifiers are specified as an optional flag. You can provide multiple modifiers using exclusive OR (|), as shown previously and may be represented by one of these:

Modifier	Description
re.I	Performs case-insensitive matching.
re.L	Interprets words according to the current locale. This interpretation
	affects the alphabetic group (\w and \W), as well as word boundary
	behavior (\b and \B).
re.M	Makes $ match the end of a line (not just the end of the string) and
	makes ^ match the start of any line (not just the start of the string).
re.S	Makes a period (dot) match any character, including a newline.
re.U	Interprets letters according to the Unicode character set. This flag
	affects the behavior of \w, \W, \b, \B.
re.X	Permits  "cuter"  regular  expression  syntax.  It  ignores  whitespace
	(except inside a set [] or when escaped by a backslash) and treats
	unescaped # as a comment marker.

Regular-Expression Patterns

Except for control characters, (+ ? . * ^ $ ( ) [ ] { } | \), all characters match themselves. You can escape a control character by preceding it with a backslash.

Following table lists the regular expression syntax that is available in Python:

Pattern	Description
^	Matches beginning of line.
$	Matches end of line.
	allows it to match newline as well.
[...]	Matches any single character in brackets.
[^...]	Matches any single character not in brackets
re*	Matches 0 or more occurrences of preceding expression.
re+	Matches 1 or more occurrence of preceding expression.
re?	Matches 0 or 1 occurrence of preceding expression.
re{ n}	Matches  exactly  n  number  of  occurrences  of  preceding
	expression.
re{ n,}	Matches n or more occurrences of preceding expression.
re{ n, m}	Matches at least n and at most m occurrences of preceding
	expression.
a| b	Matches either a or b.
(re)	Groups regular expressions and remembers matched text.
(?imx)	Temporarily  toggles  on  i,  m,  or  x  options  within  a  regular
	expression. If in parentheses, only that area is affected.
(?-imx)	Temporarily  toggles  off  i,  m,  or  x  options  within  a  regular
	expression. If in parentheses, only that area is affected.

(?: re)	Groups regular expressions without remembering matched text.
(?imx: re)	Temporarily toggles on i, m, or x options within parentheses.
(?-imx: re)	Temporarily toggles off i, m, or x options within parentheses.
(?#...)	Comment.
(?= re)	Specifies position using a pattern. Doesn't have a range.
(?! re)	Specifies position using pattern negation. Does not have a range.
(?> re)	Matches independent pattern without backtracking.
\w	Matches word characters.
\W	Matches non-word characters.
\s	Matches whitespace. Equivalent to [\t\n\r\f].
\S	Matches non-whitespace.
\d	Matches digits. Equivalent to [0-9].
\D	Matches non-digits.
\A	Matches beginning of string.
\Z	Matches end of string. If a newline exists, it matches just before
	newline.
\z	Matches end of string.
\G	Matches point where last match finished.
\b	Matches  word  boundaries  when  outside  brackets.  Matches
	backspace (0x08) when inside brackets.

\B	Matches non-word boundaries.
\n, \t, etc.	Matches newlines, carriage returns, tabs, etc.
\1...\9	Matches nth grouped subexpression.
\10	Matches  nth  grouped  subexpression  if  it  matched  already.
	Otherwise refers to the octal representation of a character code.

Regular-Expression Examples

Literal characters

Example                          Description

python

Match "python".

Character classes

Example	Description
[Pp]ython	Match "Python" or "python"
rub[ye]	Match "ruby" or "rube"
[aeiou]	Match any one lowercase vowel
[0-9]	Match any digit; same as [0123456789]
[a-z]	Match any lowercase ASCII letter
[A-Z]	Match any uppercase ASCII letter
[a-zA-Z0-9]	Match any of the above
[^aeiou]	Match anything other than a lowercase vowel

[^0-9]

Match anything other than a digit

Special Character Classes

Example	Description
.	Match any character except newline
\d	Match a digit: [0-9]
\D	Match a non-digit: [^0-9]
\s	Match a whitespace character: [ \t\r\n\f]
\S	Match non-whitespace: [^ \t\r\n\f]
\w	Match a single word character: [A-Za-z0-9_]
\W	Match a non-word character: [^A-Za-z0-9_]

Repetition Cases

Example	Description
ruby?	Match "rub" or "ruby": the y is optional.
ruby*	Match "rub" plus 0 or more ys.
ruby+	Match "rub" plus 1 or more ys.
\d{3}	Match exactly 3 digits.
\d{3,}	Match 3 or more digits.
\d{3,5}	Match 3, 4, or 5 digits.

Nongreedy repetition

This matches the smallest number of repetitions:

Example	Description
<.*>	Greedy repetition: matches "<python>perl>".
<.*?>	Nongreedy: matches "<python>" in "<python>perl>".

Grouping with Parentheses

Example	Description
\D\d+	No group: + repeats \d.
(\D\d)+	Grouped: + repeats \D\d pair.
([Pp]ython(, )?)+	Match "Python", "Python, python, python", etc.

Backreferences

This matches a previously matched group again:

Example                           Description

([Pp])ython&\1ails    Match python&pails or Python&Pails.

(['"])[^\1]*\1               Single or double-quoted string. \1 matches whatever the 1st group matched. \2 matches whatever the 2nd group matched, etc.

Alternatives

Example                          Description

python|perl

Match "python" or "perl".

rub(y|le))	Match "ruby" or "ruble".
Python(!+|\?)	"Python" followed by one or more ! or one ?

Anchors

This needs to specify match position.

Example	Description
^Python	Match "Python" at the start of a string or internal line.
Python$	Match "Python" at the end of a string or line.
\APython	Match "Python" at the start of a string.
Python\Z	Match "Python" at the end of a string.
\bPython\b	Match "Python" at a word boundary.
\brub\B	\B is non-word boundary: match "rub" in "rube" and "ruby" but
	not alone.
Python(?=!)	Match "Python", if followed by an exclamation point.
Python(?!!)	Match "Python", if not followed by an exclamation point.

Special Syntax with Parentheses

Example	Description
R(?#comment)	Matches "R". All the rest is a comment.
R(?i)uby	Case-insensitive while matching "uby".
R(?i:uby)	Same as above

rub(?:y|le))

Group only without creating \1 back reference.

 

 

 

#########################################################

Python RegEx

---

A RegEx, or Regular Expression, is a sequence of characters that forms a search pattern.

RegEx can be used to check if a string contains the specified search pattern.

---

RegEx Module

Python has a built-in package called re, which can be used to work with Regular Expressions.

Import the re module:

---

RegEx in Python

When you have imported the re module, you can start using regular expressions:

---

RegEx Functions

The re module offers a set of functions that allows us to search a string for a match:

Function	Description
findall	Returns a list containing all matches
search	Returns a Match object if there is a match anywhere in the string
split	Returns a list where the string has been split at each match
sub	Replaces one or many matches with a string

---

---

Metacharacters

Metacharacters are characters with a special meaning:

Character	Description	Example	Try it
[]	A set of characters	"[a-m]"	Try it »
\	Signals a special sequence (can also be used to escape special characters)	"\d"	Try it »
.	Any character (except newline character)	"he..o"	Try it »
^	Starts with	"^hello"	Try it »
$	Ends with	"world$"	Try it »
*	Zero or more occurrences	"aix*"	Try it »
+	One or more occurrences	"aix+"	Try it »
{}	Exactly the specified number of occurrences	"al{2}"	Try it »
|	Either or	"falls|stays"	Try it »
()	Capture and group

---

Special Sequences

A special sequence is a \ followed by one of the characters in the list below, and has a special meaning:

Character	Description	Example	Try it
\A	Returns a match if the specified characters are at the beginning of the string	"\AThe"	Try it »
\b	Returns a match where the specified characters are at the beginning or at the end of a word	r"\bain" r"ain\b"	Try it » Try it »
\B	Returns a match where the specified characters are present, but NOT at the beginning (or at the end) of a word	r"\Bain" r"ain\B"	Try it » Try it »
\d	Returns a match where the string contains digits (numbers from 0-9)	"\d"	Try it »
\D	Returns a match where the string DOES NOT contain digits	"\D"	Try it »
\s	Returns a match where the string contains a white space character	"\s"	Try it »
\S	Returns a match where the string DOES NOT contain a white space character	"\S"	Try it »
\w	Returns a match where the string contains any word characters (characters from a to Z, digits from 0-9, and the underscore _ character)	"\w"	Try it »
\W	Returns a match where the string DOES NOT contain any word characters	"\W"	Try it »
\Z	Returns a match if the specified characters are at the end of the string	"Spain\Z"	Try it »

---

Sets

A set is a set of characters inside a pair of square brackets [] with a special meaning:

Set	Description	Try it
[arn]	Returns a match where one of the specified characters (a, r, or n) are present	Try it »
[a-n]	Returns a match for any lower case character, alphabetically between a and n	Try it »
[^arn]	Returns a match for any character EXCEPT a, r, and n	Try it »
[0123]	Returns a match where any of the specified digits (0, 1, 2, or 3) are present	Try it »
[0-9]	Returns a match for any digit between 0 and 9	Try it »
[0-5][0-9]	Returns a match for any two-digit numbers from 00 and 59	Try it »
[a-zA-Z]	Returns a match for any character alphabetically between a and z, lower case OR upper case	Try it »
[+]	In sets, +, *, ., |, (), $,{} has no special meaning, so [+] means: return a match for any + character in the string	Try it »

---

The findall() Function

The findall() function returns a list containing all matches.

The list contains the matches in the order they are found.

If no matches are found, an empty list is returned:

---

The search() Function

The search() function searches the string for a match, and returns a Match object if there is a match.

If there is more than one match, only the first occurrence of the match will be returned:

If no matches are found, the value None is returned:

---

The split() Function

The split() function returns a list where the string has been split at each match:

You can control the number of occurrences by specifying the maxsplit parameter:

---

The sub() Function

The sub() function replaces the matches with the text of your choice:

You can control the number of replacements by specifying the count parameter:

---

Match Object

A Match Object is an object containing information about the search and the result.

The Match object has properties and methods used to retrieve information about the search, and the result:

.span() returns a tuple containing the start-, and end positions of the match.
.string returns the string passed into the function
.group() returns the part of the string where there was a match