Exception handling is one of the hallmarks of production-ready code. As a senior developer, I can tell you that the difference between a prototype and a system that runs reliably in production often comes down to how well it handles failure. Defensive programming means anticipating what can go wrong and writing code that degrades gracefully rather than crashing spectacularly.
In Python, errors and exceptions are related but distinct concepts. An error is a broader term that includes syntax mistakes caught before your code runs, while an exception is a runtime event that disrupts the normal flow of execution. Python’s exception handling mechanism gives you fine-grained control over how your program responds to these runtime failures, and mastering it is non-negotiable for writing robust software.
This guide covers exception handling in depth: from the basics of try/except to custom exception hierarchies, exception chaining, context managers, and the best practices that separate junior code from senior code.
Before diving in, it is important to understand the difference between the two main categories of errors you will encounter in Python.
Syntax errors (also called parsing errors) are caught by the Python interpreter before your code executes. These are mistakes in the structure of your code — a missing colon, unmatched parentheses, or incorrect indentation.
# Syntax error — caught before execution
if True
print("missing colon")
# SyntaxError: expected ':'
Runtime exceptions occur while your program is running. The syntax is valid, but something goes wrong during execution — dividing by zero, accessing a missing dictionary key, or trying to open a file that does not exist.
# Runtime exception — code is syntactically valid result = 12 / 0 # Traceback (most recent call last): # File "main.py", line 1, in <module> # result = 12 / 0 # ZeroDivisionError: division by zero
Exception handling deals exclusively with runtime exceptions. Syntax errors must be fixed in your source code before the program can run at all.
The fundamental mechanism for handling exceptions in Python is the try/except block. You place the code that might raise an exception inside the try clause, and the code that handles the exception inside the except clause.
# Without exception handling — program crashes
result = 12 / 0
print(result)
print("This line never executes")
# Traceback (most recent call last):
# File "main.py", line 1, in <module>
# result = 12 / 0
# ZeroDivisionError: division by zero
# With exception handling — program continues
try:
result = 12 / 0
print(result)
except ZeroDivisionError as e:
print(f"Caught an exception: {e}")
print("Program continues normally")
# Output:
# Caught an exception: division by zero
# Program continues normally
The key insight here is that once an exception is caught and handled, execution continues normally after the try/except block. Without the handler, the program terminates immediately.
Always catch the most specific exception possible. This is a principle that separates professional code from amateur code. Each exception type tells you exactly what went wrong, and your handler should respond accordingly.
# Catching specific exception types
def process_data(data):
try:
value = data["price"]
result = 100 / value
formatted = "Total: " + result # TypeError: can't concat str and float
return formatted
except KeyError as e:
print(f"Missing key in data: {e}")
return None
except ZeroDivisionError:
print("Price cannot be zero")
return None
except TypeError as e:
print(f"Type mismatch: {e}")
return None
# Test with different failure scenarios
process_data({}) # Missing key in data: 'price'
process_data({"price": 0}) # Price cannot be zero
process_data({"price": 50}) # Type mismatch: can only concatenate str...
You can also catch multiple exception types in a single except clause using a tuple.
# Grouping related exceptions
try:
value = int(input("Enter a number: "))
result = 100 / value
except (ValueError, ZeroDivisionError) as e:
print(f"Invalid input: {e}")
A try clause can have any number of except clauses to handle different exceptions. Python evaluates them from top to bottom and executes the first matching handler. Only one except block runs per exception.
def safe_list_access(items, index, key):
"""Demonstrate multiple except blocks with different responses."""
try:
item = items[index]
value = item[key]
return int(value)
except IndexError:
print(f"Index {index} is out of range (list has {len(items)} items)")
except KeyError:
print(f"Key '{key}' not found in item at index {index}")
except (ValueError, TypeError) as e:
print(f"Could not convert value to int: {e}")
except Exception as e:
print(f"Unexpected error: {type(e).__name__}: {e}")
return None
data = [{"name": "Alice", "age": "30"}, {"name": "Bob"}]
safe_list_access(data, 0, "age") # Returns 30
safe_list_access(data, 5, "age") # Index 5 is out of range
safe_list_access(data, 1, "age") # Key 'age' not found in item at index 1
safe_list_access(data, 0, "name") # Could not convert value to int: ...
Important: Order your except clauses from most specific to most general. If you put except Exception first, it will catch everything and the more specific handlers below it will never execute.
The else clause runs only when the try block completes without raising any exception. This is useful for code that should only execute on success, and it keeps the try block focused on the operations that might fail.
def divide(a, b):
try:
result = a / b
except ZeroDivisionError:
print("Cannot divide by zero")
except TypeError:
print("Both arguments must be numbers")
else:
# Only runs if no exception was raised
print(f"{a} / {b} = {result}")
return result
return None
divide(10, 3) # 10 / 3 = 3.3333333333333335
divide(10, 0) # Cannot divide by zero
divide(10, "a") # Both arguments must be numbers
A key advantage of the else clause is that exceptions raised inside it are not caught by the preceding except blocks. This gives you a clean separation between “code that might fail” and “code that should only run on success.”
The finally clause executes no matter what — whether an exception was raised, caught, or not. It is the right place for cleanup operations: closing files, releasing database connections, or shutting down network sockets. If you acquire a resource, finally guarantees you release it.
def read_config(filepath):
"""Read a configuration file with guaranteed cleanup."""
f = None
try:
f = open(filepath, "r")
content = f.read()
config = parse_config(content)
return config
except FileNotFoundError:
print(f"Config file not found: {filepath}")
return {}
except PermissionError:
print(f"Permission denied: {filepath}")
return {}
finally:
# This ALWAYS runs, even if we return from try or except
if f is not None:
f.close()
print(f"File handle closed for: {filepath}")
You can combine all four clauses in a single statement: try, except, else, and finally.
def complete_example(filename):
"""Demonstrate the full try/except/else/finally structure."""
f = None
try:
f = open(filename, "r")
data = f.read()
except FileNotFoundError:
print(f"File '{filename}' not found")
data = None
else:
print(f"Successfully read {len(data)} characters")
finally:
if f is not None:
f.close()
print("Cleanup complete")
return data
Use the raise keyword to explicitly throw an exception when your code detects an invalid state. This is how you enforce preconditions, validate inputs, and signal error conditions to calling code.
def set_age(age):
"""Validate and set age with clear error messages."""
if not isinstance(age, int):
raise TypeError(f"Age must be an integer, got {type(age).__name__}")
if age < 0:
raise ValueError(f"Age cannot be negative, got {age}")
if age > 150:
raise ValueError(f"Age {age} is unrealistic")
return age
# Usage
try:
set_age("thirty")
except TypeError as e:
print(e) # Age must be an integer, got str
try:
set_age(-5)
except ValueError as e:
print(e) # Age cannot be negative, got -5
You can also re-raise the current exception using raise without arguments. This is useful when you want to log an exception but still let it propagate up the call stack.
import logging
logger = logging.getLogger(__name__)
def process_payment(amount):
try:
charge_credit_card(amount)
except PaymentError as e:
logger.error(f"Payment failed for amount {amount}: {e}")
raise # Re-raise the same exception for the caller to handle
For any non-trivial application, you should define your own exception classes. Custom exceptions make your error handling more expressive, allow callers to catch domain-specific errors, and keep your API clean.
# Define a custom exception hierarchy for a user registration system
class ApplicationError(Exception):
"""Base exception for the application."""
pass
class ValidationError(ApplicationError):
"""Raised when input validation fails."""
def __init__(self, field, message):
self.field = field
self.message = message
super().__init__(f"Validation error on '{field}': {message}")
class DuplicateUserError(ApplicationError):
"""Raised when attempting to register an existing user."""
def __init__(self, email):
self.email = email
super().__init__(f"User with email '{email}' already exists")
class AuthenticationError(ApplicationError):
"""Raised when authentication fails."""
pass
# Usage in a registration system
def register_user(email, password, age):
if not email or "@" not in email:
raise ValidationError("email", "Must be a valid email address")
if len(password) < 8:
raise ValidationError("password", "Must be at least 8 characters")
if not isinstance(age, int) or age < 13:
raise ValidationError("age", "Must be an integer >= 13")
existing_users = ["alice@example.com", "bob@example.com"]
if email in existing_users:
raise DuplicateUserError(email)
return {"email": email, "status": "registered"}
# Callers can catch at different levels of specificity
try:
user = register_user("alice@example.com", "short", 25)
except ValidationError as e:
print(f"Fix your input - {e.field}: {e.message}")
except DuplicateUserError as e:
print(f"Try a different email: {e.email}")
except ApplicationError as e:
print(f"Something went wrong: {e}")
# Output:
# Fix your input - password: Must be at least 8 characters
Exception chaining lets you indicate that one exception was caused by another. This preserves the full causal chain in the traceback, which is invaluable for debugging. Use raise ... from ... to explicitly set the cause.
class ConfigError(Exception):
"""Raised when configuration loading fails."""
pass
def load_config(path):
"""Load config, converting low-level errors to domain errors."""
try:
with open(path) as f:
import json
return json.load(f)
except FileNotFoundError as e:
raise ConfigError(f"Config file missing: {path}") from e
except json.JSONDecodeError as e:
raise ConfigError(f"Invalid JSON in config: {path}") from e
# When this fails, the traceback shows BOTH exceptions:
# FileNotFoundError: [Errno 2] No such file or directory: '/etc/app/config.json'
#
# The above exception was the direct cause of the following exception:
# ConfigError: Config file missing: /etc/app/config.json
You can also suppress the chained context using raise ... from None when the original exception is not relevant to the caller.
def get_user_age(data):
try:
return int(data["age"])
except (KeyError, ValueError, TypeError):
raise ValidationError("age", "Missing or invalid age") from None
Understanding the built-in exception hierarchy helps you write precise exception handlers. Here is the relevant portion of the class tree.
BaseException
+-- SystemExit
+-- KeyboardInterrupt
+-- GeneratorExit
+-- Exception
+-- ArithmeticError
| +-- ZeroDivisionError
| +-- OverflowError
| +-- FloatingPointError
+-- LookupError
| +-- IndexError
| +-- KeyError
+-- OSError
| +-- FileNotFoundError
| +-- PermissionError
| +-- FileExistsError
| +-- IsADirectoryError
| +-- ConnectionError
| +-- ConnectionRefusedError
| +-- ConnectionResetError
| +-- BrokenPipeError
+-- TypeError
+-- ValueError
| +-- UnicodeError
+-- AttributeError
+-- ImportError
| +-- ModuleNotFoundError
+-- RuntimeError
| +-- NotImplementedError
| +-- RecursionError
+-- StopIteration
+-- Warning
Key takeaway: Never catch BaseException — it includes SystemExit and KeyboardInterrupt, which you almost certainly want to let propagate. If you need a broad catch-all, use Exception instead.
| Exception | When It Occurs | Example |
|---|---|---|
ValueError |
Function receives correct type but invalid value | int("abc") |
TypeError |
Operation applied to wrong type | "hello" + 5 |
KeyError |
Dictionary key not found | d = {}; d["missing"] |
IndexError |
Sequence index out of range | [1, 2, 3][10] |
FileNotFoundError |
File or directory does not exist | open("nope.txt") |
AttributeError |
Object has no such attribute | None.split() |
ZeroDivisionError |
Division or modulo by zero | 1 / 0 |
ImportError |
Module cannot be imported | import nonexistent |
PermissionError |
Insufficient file system permissions | open("/etc/shadow") |
RuntimeError |
Generic error that does not fit elsewhere | Detected during execution |
StopIteration |
Iterator has no more items | next(iter([])) |
NotImplementedError |
Abstract method not overridden | Subclass forgot to implement |
Context managers (the with statement) are the Pythonic way to handle resource cleanup. They guarantee that cleanup code runs even if an exception occurs, eliminating the need for try/finally in many cases.
# Instead of try/finally for file handling...
f = None
try:
f = open("data.txt", "r")
content = f.read()
finally:
if f:
f.close()
# ...use a context manager
with open("data.txt", "r") as f:
content = f.read()
# File is automatically closed, even if an exception occurred
You can write your own context managers using the __enter__ and __exit__ methods or the contextlib module.
import contextlib
class DatabaseConnection:
"""Context manager for database connections with guaranteed cleanup."""
def __init__(self, connection_string):
self.connection_string = connection_string
self.connection = None
def __enter__(self):
print(f"Connecting to {self.connection_string}")
self.connection = self._create_connection()
return self.connection
def __exit__(self, exc_type, exc_val, exc_tb):
if self.connection:
self.connection.close()
print("Connection closed")
if exc_type is not None:
print(f"An error occurred: {exc_val}")
# Return False to propagate the exception, True to suppress it
return False
def _create_connection(self):
# Simulated connection
return type("Conn", (), {"close": lambda s: None, "query": lambda s, sql: []})()
# Usage — connection is always closed, even on error
with DatabaseConnection("postgres://localhost/mydb") as conn:
results = conn.query("SELECT * FROM users")
# Using contextlib for simpler context managers
from contextlib import contextmanager
import time
@contextmanager
def timer(label):
"""Context manager that times a block of code."""
start = time.perf_counter()
try:
yield
except Exception as e:
elapsed = time.perf_counter() - start
print(f"[{label}] Failed after {elapsed:.3f}s: {e}")
raise
else:
elapsed = time.perf_counter() - start
print(f"[{label}] Completed in {elapsed:.3f}s")
with timer("data processing"):
data = [i ** 2 for i in range(1_000_000)]
# Output: [data processing] Completed in 0.085s
def safe_divide(numerator, denominator, default=None):
"""
Safely divide two numbers, returning a default value on failure.
Args:
numerator: The dividend
denominator: The divisor
default: Value to return if division fails (default: None)
Returns:
The quotient, or the default value if division is not possible
"""
try:
return numerator / denominator
except ZeroDivisionError:
return default
except TypeError:
return default
# Usage
print(safe_divide(10, 3)) # 3.3333333333333335
print(safe_divide(10, 0)) # None
print(safe_divide(10, 0, 0)) # 0
print(safe_divide("ten", 2)) # None
import json
import os
def read_json_file(filepath):
"""
Read and parse a JSON file with comprehensive error handling.
Returns:
Parsed JSON data, or None on failure
"""
if not os.path.exists(filepath):
print(f"Error: File does not exist: {filepath}")
return None
try:
with open(filepath, "r", encoding="utf-8") as f:
data = json.load(f)
except PermissionError:
print(f"Error: No permission to read {filepath}")
return None
except json.JSONDecodeError as e:
print(f"Error: Invalid JSON in {filepath} at line {e.lineno}, col {e.colno}")
return None
except UnicodeDecodeError:
print(f"Error: File {filepath} is not valid UTF-8")
return None
except OSError as e:
print(f"Error: Could not read {filepath}: {e}")
return None
else:
print(f"Successfully loaded {filepath}")
return data
import time
import json
import logging
import urllib.request
import urllib.error
logger = logging.getLogger(__name__)
class APIError(Exception):
"""Base exception for API errors."""
def __init__(self, status_code, message):
self.status_code = status_code
self.message = message
super().__init__(f"API error {status_code}: {message}")
class RetryableError(APIError):
"""Raised for transient errors that may succeed on retry."""
pass
def api_call_with_retry(url, max_retries=3, backoff_factor=1.0):
"""
Make an API call with exponential backoff retry logic.
Retries only on transient errors (server errors, timeouts).
Fails fast on client errors (4xx status codes).
"""
last_exception = None
for attempt in range(max_retries):
try:
req = urllib.request.Request(url)
with urllib.request.urlopen(req, timeout=10) as response:
return json.loads(response.read().decode())
except urllib.error.HTTPError as e:
if 400 <= e.code < 500:
# Client errors — do not retry
raise APIError(e.code, f"Client error: {e.reason}") from e
else:
# Server errors — retry
last_exception = RetryableError(e.code, f"Server error: {e.reason}")
except urllib.error.URLError as e:
last_exception = RetryableError(0, f"Connection error: {e.reason}")
except TimeoutError:
last_exception = RetryableError(0, "Request timed out")
# Exponential backoff before retry
if attempt < max_retries - 1:
wait_time = backoff_factor * (2 ** attempt)
logger.warning(
f"Attempt {attempt + 1} failed: {last_exception}. "
f"Retrying in {wait_time:.1f}s..."
)
time.sleep(wait_time)
raise last_exception
import re
class RegistrationError(Exception):
"""Base exception for registration failures."""
pass
class InvalidEmailError(RegistrationError):
def __init__(self, email):
self.email = email
super().__init__(f"Invalid email format: '{email}'")
class WeakPasswordError(RegistrationError):
def __init__(self, reasons):
self.reasons = reasons
super().__init__(f"Weak password: {'; '.join(reasons)}")
class UserExistsError(RegistrationError):
def __init__(self, email):
self.email = email
super().__init__(f"Account already exists for '{email}'")
def validate_email(email):
pattern = r'^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$'
if not re.match(pattern, email):
raise InvalidEmailError(email)
def validate_password(password):
issues = []
if len(password) < 8:
issues.append("must be at least 8 characters")
if not re.search(r'[A-Z]', password):
issues.append("must contain an uppercase letter")
if not re.search(r'[a-z]', password):
issues.append("must contain a lowercase letter")
if not re.search(r'[0-9]', password):
issues.append("must contain a digit")
if issues:
raise WeakPasswordError(issues)
def register_user(email, password):
"""Register a user with full validation."""
# Validate inputs
validate_email(email)
validate_password(password)
# Check for duplicates (simulated)
existing = ["admin@example.com", "user@example.com"]
if email.lower() in existing:
raise UserExistsError(email)
print(f"User {email} registered successfully")
return {"email": email, "status": "active"}
# Demonstration
test_cases = [
("not-an-email", "Password1"),
("valid@email.com", "weak"),
("admin@example.com", "StrongPass1"),
("new@example.com", "StrongPass1"),
]
for email, password in test_cases:
try:
result = register_user(email, password)
print(f" Result: {result}")
except InvalidEmailError as e:
print(f" Email error: {e}")
except WeakPasswordError as e:
print(f" Password error: {e}")
for reason in e.reasons:
print(f" - {reason}")
except UserExistsError as e:
print(f" Duplicate: {e}")
import sqlite3
import logging
logger = logging.getLogger(__name__)
def execute_transaction(db_path, queries):
"""
Execute a series of SQL queries in a single transaction.
Rolls back on any failure and always closes the connection.
"""
conn = None
cursor = None
try:
conn = sqlite3.connect(db_path)
cursor = conn.cursor()
for sql, params in queries:
cursor.execute(sql, params)
conn.commit()
logger.info(f"Transaction committed: {len(queries)} queries executed")
return cursor.lastrowid
except sqlite3.IntegrityError as e:
logger.error(f"Integrity constraint violated: {e}")
if conn:
conn.rollback()
raise
except sqlite3.OperationalError as e:
logger.error(f"Database operational error: {e}")
if conn:
conn.rollback()
raise
except Exception as e:
logger.error(f"Unexpected error during transaction: {e}")
if conn:
conn.rollback()
raise
finally:
if cursor:
cursor.close()
if conn:
conn.close()
logger.debug("Database connection closed")
# Usage
queries = [
("INSERT INTO users (name, email) VALUES (?, ?)", ("Alice", "alice@example.com")),
("INSERT INTO users (name, email) VALUES (?, ?)", ("Bob", "bob@example.com")),
]
try:
execute_transaction("app.db", queries)
except sqlite3.IntegrityError:
print("A user with that email already exists")
These are the mistakes I see most frequently in code reviews. Avoid them.
# BAD — catches everything, including KeyboardInterrupt and SystemExit
try:
do_something()
except:
pass # Silently swallows ALL errors
# GOOD — catch only what you expect
try:
do_something()
except (ValueError, ConnectionError) as e:
logger.error(f"Operation failed: {e}")
# ACCEPTABLE — broad catch with logging (never silently)
try:
do_something()
except Exception as e:
logger.exception(f"Unexpected error: {e}")
raise
# BAD — hides bugs by catching everything
try:
user = get_user(user_id)
send_email(user.email)
update_database(user)
except Exception:
print("Something went wrong")
# GOOD — handle each failure mode specifically
try:
user = get_user(user_id)
except UserNotFoundError:
print(f"No user with ID {user_id}")
else:
try:
send_email(user.email)
except EmailError as e:
logger.warning(f"Failed to send email: {e}")
# Continue anyway — email is not critical
update_database(user)
# BAD — using exceptions as goto statements
def find_item(items, target):
try:
for item in items:
if item == target:
raise StopIteration # Abusing exception for control flow
except StopIteration:
return True
return False
# GOOD — use normal control flow
def find_item(items, target):
for item in items:
if item == target:
return True
return False
# ALSO GOOD — Pythonic
def find_item(items, target):
return target in items
# BAD — silently swallowing errors hides bugs
try:
process_data(data)
except ValueError:
pass # This hides real problems
# GOOD — at minimum, log the error
try:
process_data(data)
except ValueError as e:
logger.warning(f"Skipping invalid data: {e}")
Python follows the EAFP (Easier to Ask Forgiveness than Permission) philosophy rather than LBYL (Look Before You Leap). This means it is more Pythonic to try an operation and handle the exception than to check preconditions beforehand.
# LBYL style (common in other languages)
if "key" in dictionary:
value = dictionary["key"]
else:
value = default_value
# EAFP style (Pythonic)
try:
value = dictionary["key"]
except KeyError:
value = default_value
# Even more Pythonic — use dict.get()
value = dictionary.get("key", default_value)
import logging
logger = logging.getLogger(__name__)
try:
process_order(order)
except Exception as e:
# logger.exception() automatically includes the traceback
logger.exception(f"Failed to process order {order.id}")
raise # Re-raise after logging
# Define a clear hierarchy for your application
class AppError(Exception):
"""Base exception for the application."""
pass
class ServiceError(AppError):
"""Errors from external services."""
pass
class DatabaseError(ServiceError):
"""Database-related errors."""
pass
class CacheError(ServiceError):
"""Cache-related errors."""
pass
class BusinessLogicError(AppError):
"""Business rule violations."""
pass
class InsufficientFundsError(BusinessLogicError):
"""Raised when account balance is too low."""
pass
# Callers can catch at the level of granularity they need:
# - catch InsufficientFundsError for specific handling
# - catch BusinessLogicError for all rule violations
# - catch AppError for any application error
# Prefer context managers over try/finally
# BAD
lock.acquire()
try:
do_work()
finally:
lock.release()
# GOOD
with lock:
do_work()
# BAD — too much code in the try block
try:
data = load_data(path)
processed = transform(data)
result = calculate(processed)
save(result)
except FileNotFoundError:
# Which operation caused this?
handle_error()
# GOOD — narrow try blocks make error sources clear
try:
data = load_data(path)
except FileNotFoundError:
logger.error(f"Data file missing: {path}")
return
processed = transform(data)
result = calculate(processed)
save(result)
except clauses. Catching Exception is acceptable only as a last resort, and always log the error.else for success-only code, finally for guaranteed cleanup.with statement is cleaner and less error-prone.raise ... from ...) to preserve the causal chain when converting low-level exceptions to domain-level ones.