Python Regex Tutorial: The re Module with Practical Examples

Imagine you are handed 3,000 delivery addresses typed by customers of an Indore sweet shop. Some write "Indore 452001", some write "Indore-452010", some put the PIN code first, and a few forget it. Your manager wants a column with just the PIN codes.

You could scan each line by eye. Or you could describe what a PIN code looks like, "exactly six digits, standing on their own", and let the computer find every one in a second.

That description is a regular expression. The rule "six digits standing alone" becomes the pattern \b\d{6}\b. The addresses are the text you search. Each PIN code found is a match.

A regular expression in Python is a pattern, written in a compact syntax and processed by the standard library's re module, that describes a set of strings so you can search, extract, validate, split or replace text that fits the pattern.

This lesson follows Python decorators and builds on the string methods from Python strings. Examples target Python 3.12+ and pandas 2.x.

What is a regular expression in Python?

The re module ships with Python, so there is nothing to install. You give it a pattern and a piece of text, and it tells you where the pattern matches.

import re

addresses = [
    "12 Sapna Sangeeta Road, Indore 452001",
    "MG Road, Bhopal - 462001",
    "452010 Vijay Nagar, Indore",
    "Rajwada, Indore (PIN not given)",
]
for a in addresses:
    m = re.search(r"\b\d{6}\b", a)
    print(m.group() if m else "------", "<-", a)

re.search returns a match object when it finds the pattern and None when it does not, so always check before calling .group(). The r before the string makes it a raw string, which stops Python from treating backslashes as escape characters. Use raw strings for every pattern.

Regex syntax: the building blocks

A pattern is built from ordinary characters, which match themselves, and special symbols. The table lists the ones you will use in nearly every data task.

Capital letters reverse the shorthand classes: \D is any non-digit, \W any non-word character and \S any non-space.

import re

text = "Order 1041 paid Rs 1,450 on 2026-11-03; order 1042 paid Rs 980."
print(re.findall(r"\d+", text))
print(re.findall(r"\b\d{4}\b", text))
print(re.findall(r"Rs [\d,]+", text))
print(re.findall(r"[Oo]rder \d+", text))
print(re.findall(r"\d{4}-\d{2}-\d{2}", text))

Notice how \d+ splits "1,450" into two numbers, because the comma is not a digit. The pattern Rs [\d,]+ fixes that by allowing digits and commas together.

Read a pattern left to right, one piece at a time. Each group in brackets captures its own part of the match.

search, match, fullmatch, findall and finditer

The re module has several finding functions. They differ in where they look and how many matches they return.

import re

text = "PIN 452001, alt 452010"
pat = r"\d{6}"
print("match:    ", re.match(pat, text))
print("search:   ", re.search(pat, text).group())
print("findall:  ", re.findall(pat, text))
for m in re.finditer(pat, text):
    print("finditer: ", m.group(), "at", m.span())
print("fullmatch:", re.fullmatch(pat, "452001") is not None, re.fullmatch(pat, text))

Use fullmatch for validation (is this whole value a PIN?) and search or findall for extraction.

re.match trips up many beginners. It only checks the beginning, so it returns None here even though two PIN codes exist later in the text.

How do regex groups work?

Round brackets create a group. The whole match is group 0, and each bracket pair is numbered from 1 by its opening bracket. Better still, name the groups with (?P<name>...) so your code reads like the data.

import re

logs = [
    "2026-11-03 09:15 INFO order=1041 amount=1450",
    "2026-11-03 09:42 ERROR order=1042 amount=0",
    "2026-11-04 10:05 INFO order=1043 amount=2300",
]
pat = re.compile(r"(?P<date>\S+) \S+ (?P<level>[A-Z]+) order=(?P<order>\d+) amount=(?P<amount>\d+)")

for line in logs:
    m = pat.search(line)
    print(m.group("level"), m.group("order"), int(m.group("amount")), "|", m.group(1))

print(pat.search(logs[1]).groupdict())

re.compile turns the pattern into a reusable object. Python caches recent patterns anyway, but compiling gives the pattern a name and keeps long expressions in one place.

When a pattern has groups, re.findall returns the groups instead of the whole match: a list of strings for one group, a list of tuples for several. That surprise is worth remembering.

Replacing and splitting text with re.sub and re.split

Cleaning is where regex earns its keep in analytics. re.sub(pattern, replacement, text) replaces every match, and re.split cuts text wherever the pattern appears.

import re

amounts = ["Rs 1,450", "₹ 980.50", "INR2,300", "1450/-"]
print([float(re.sub(r"[^\d.]", "", a)) for a in amounts])

phones = ["+91 98260-12345", "098260 12346", "9826012347"]
print([re.sub(r"\D", "", p)[-10:] for p in phones])

print(re.split(r"\s*[,;|]\s*", "Indore, Bhopal;Ujjain | Dewas"))

masked = re.sub(r"\d(?=\d{4})", "X", "9826012345")
print(masked)

print(re.sub(r"(\d{2})-(\d{2})-(\d{4})", r"\3-\2-\1", "Due 05-12-2026"))

The first line deletes everything that is not a digit or a dot, turning four messy formats into numbers. The masking line uses a lookahead, (?=\d{4}), which means "a digit that still has four digits after it", so only the last four survive. The last line reorders a date using \1, \2 and \3, the captured groups.

Greedy vs lazy matching and regex flags

Quantifiers such as * and + are greedy: they grab as much text as they can. Adding ? after them makes them lazy, so they take as little as possible.

import re

html = "<b>Indore</b> and <b>Bhopal</b>"
print(re.findall(r"<b>.*</b>", html))
print(re.findall(r"<b>.*?</b>", html))
print(re.findall(r"<b>(.*?)</b>", html))

print(re.findall(r"indore", "Indore INDORE indore", flags=re.IGNORECASE))
notes = "total: 1450\nTOTAL: 980\nsubtotal: 300"
print(re.findall(r"^total: (\d+)", notes, flags=re.IGNORECASE | re.MULTILINE))

The greedy version swallows everything from the first <b> to the last </b>. The lazy version stops at the first closing tag. For real HTML, use a parser; regex is fine for quick, known snippets.

re.VERBOSE makes long patterns readable. Here it validates email addresses in a simple, practical way.

import re

email = re.compile(r"""
    ^(?P<user>[\w.+-]+)              # name before the @
    @
    (?P<domain>[\w-]+(?:\.[\w-]+)+)$  # domain with at least one dot
""", re.VERBOSE)

for e in ["riya.sharma@gmail.com", "info@debugshala.com", "bad@site", "two@@x.com"]:
    m = email.match(e)
    print(f"{e:24}", m.group("domain") if m else "invalid")

(?:...) is a non-capturing group: it groups for repetition without creating a numbered group. No regex can fully validate email; for sign-ups, send a confirmation link.

Using regex with pandas string methods

pandas applies the same patterns to an entire column at once through the .str accessor. This is how you will use regex most often as an analyst.

import pandas as pd

df = pd.DataFrame({"remark": [
    "Paid via UPI ref 4521889",
    "COD, call 9826012345",
    "upi ref 7781002, late",
    "Card declined",
]})
df["is_upi"] = df["remark"].str.contains(r"\bupi\b", case=False, regex=True)
df["upi_ref"] = df["remark"].str.extract(r"(?i)ref (\d+)", expand=False)
df["phone"] = df["remark"].str.extract(r"\b(\d{10})\b", expand=False)
df["clean"] = df["remark"].str.replace(r"\d", "#", regex=True)
print(df.to_string(index=False))

str.extract returns the first group of each row and NaN where nothing matched. The inline flag (?i) makes the pattern case-insensitive without a separate argument. The data cleaning with pandas guide uses these methods on a full messy dataset.

Common mistakes with Python regex

• Forgetting the raw string. In "\b" Python sees a backspace character; r"\b" is a word boundary.
• Using re.match to search. It only checks the start of the string. Use re.search.
• Calling .group() on None. Check the result first, or you get AttributeError.
• Forgetting to escape special characters. A literal dot is \. and a literal + is \+. re.escape(text) escapes a whole string for you.
• Greedy .* grabbing too much. Use .*? or a tighter class such as [^<]*.
• Validating with search instead of fullmatch. re.search(r"\d{6}", "abc4520019") succeeds even though the value is not a PIN.

Key takeaways

• The built-in re module handles regex; write patterns as raw strings.
• \d, \w, \s, classes [...] and quantifiers + * ? {n} cover most needs.
• search finds the first match anywhere, findall finds all, fullmatch validates.
• Groups and named groups extract parts; groupdict() gives them as a dict.
• re.sub and re.split clean and cut text, and can reuse groups as \1.
• Add ? for lazy matching; use flags such as re.I, re.M and re.X.
• pandas str.contains, str.extract and str.replace apply regex to whole columns.

Dates are the next messy field you will meet, so continue with Python datetime. If you want a mentor to review your cleaning patterns on real exports, the data analytics classroom course at Debugshala, Indore works through exactly this kind of data.