Python Context Managers: Safe Files, Connections and Custom with Blocks

Python course module guide: File handling and resource safety

Python context managers define a reliable boundary around setup and cleanup. The with statement makes resource ownership visible and ensures cleanup is attempted whether the block succeeds, returns early or raises an exception.

Why the with statement matters

Resources such as files, database connections and locks have a lifecycle: acquire, use and release. If cleanup depends on remembering a final method call, an early return or exception can leave the resource open. A context manager places cleanup in the protocol rather than in the programmer’s memory.

from pathlib import Path

path = Path("students.csv")
with path.open("r", encoding="utf-8") as handle:
    first_line = handle.readline()

print(handle.closed)  # True

The Python file handling guide covers modes, encodings, CSV, JSON and common I/O errors. Context managers provide the cleanup boundary around those operations.

The context manager protocol

An object used by with implements __enter__() and __exit__(). Enter returns the value assigned after as. Exit receives exception information and performs cleanup.

class ManagedReport:
    def __init__(self, path):
        self.path = path
        self.handle = None

    def __enter__(self):
        self.handle = open(self.path, "w", encoding="utf-8")
        return self.handle

    def __exit__(self, exc_type, exc_value, traceback):
        if self.handle is not None:
            self.handle.close()
        return False

with ManagedReport("summary.txt") as report:
    report.write("completed\n")

Returning False allows an exception to continue. Returning True suppresses it. Suppression should be rare and deliberate because silent failures are difficult to diagnose.

Create a context manager with contextlib

from contextlib import contextmanager
from time import perf_counter

@contextmanager
def measured_block(label):
    started = perf_counter()
    try:
        yield
    finally:
        elapsed = perf_counter() - started
        print(f"{label}: {elapsed:.4f}s")

with measured_block("daily report"):
    total = sum(range(100_000))

The code before yield performs setup, and the finally block performs cleanup. Use the class form when a reusable object has meaningful state or methods; use @contextmanager for a focused setup/cleanup sequence.

Database transaction boundaries

import sqlite3
from contextlib import closing

# The enrolments table must already exist.
with closing(sqlite3.connect("training.db")) as connection:
    with connection:
        connection.execute(
            "UPDATE enrolments SET status = ? WHERE id = ?",
            ("confirmed", 42),
        )

The sqlite3 connection context manages transaction commit or rollback behaviour; it does not close the connection. The outer closing context explicitly closes it. Do not assume every context manager performs every kind of cleanup. The Python SQLite expense tracker provides a larger CRUD and transaction example.

Control multiple resources

from pathlib import Path

with (
    Path("source.txt").open(encoding="utf-8") as source,
    Path("clean.txt").open("w", encoding="utf-8") as target,
):
    for line in source:
        if line.strip():
            target.write(line)

Grouping resources expresses that they belong to one operation. For a dynamic number of resources, contextlib.ExitStack can manage callbacks and context managers in one stack.

Context managers in API and web code

Network sessions, tracing spans, temporary directories and request-scoped database connections all use similar lifecycle ideas. In the Python API pagination and retries guide, a session is explicitly created and closed. The Flask REST API project uses request-context cleanup for SQLite connections.

Common context-manager mistakes

• Opening a resource outside the with block and losing ownership clarity.
• Using the resource after the block has closed it.
• Returning True from __exit__ without intending to suppress an exception.
• Doing heavy business logic inside setup or cleanup methods.
• Assuming transaction management also closes a connection.
• Testing only the successful path and never checking cleanup after failure.

Practice exercise: safe export

Build a context manager that writes a report to a temporary filename. If the block succeeds, rename the temporary file to its final name. If it fails, remove the incomplete temporary file and allow the original exception to continue. Test both paths and make the cleanup behavior explicit.

Separate releasing a resource from completing an operation

A file may close successfully even though the report written inside it is incomplete. A database transaction may roll back while its connection remains open. Decide which responsibility belongs to each context manager, then state what callers can rely on after leaving the block. The presence of a with statement alone does not make a multi-step business operation atomic.

The SQLite example now has two explicit boundaries: the inner connection context handles the transaction, and the outer closing helper releases the connection. This prevents the earlier transaction example from leaving connection ownership ambiguous. The table must already exist, and updating a nonexistent row still needs a separate affected-row check if the application requires exactly one update.

Test the failure path with observable evidence

Use a temporary directory and a small report rather than testing against an important existing file. In the success case, write two lines, leave the block and verify that the handle is closed and the content is readable. In the failure case, write one line and deliberately raise a ValueError. Catch that error outside the context, then check that the handle closed and that the original exception reached the caller.

These checks answer separate questions: was the resource released, did the error propagate and what happened to partial output? Do not replace them with a single “no exception occurred” assertion. If your manager intentionally suppresses one expected error, add a different error that must still propagate; this catches accidentally broad suppression.

A generator manager must yield once

A function decorated with contextmanager is a generator-based protocol implementation. It yields once to transfer control to the block, then resumes when that block finishes. A normal loop that yields several resources is therefore not a context manager. If code catches an exception around the yield only to log it, it must re-raise when the intended policy is to preserve that failure.

The timing example uses finally so elapsed time is recorded during both normal and exceptional exit. Keep such cleanup small and reliable. If cleanup raises a new error, it can become the visible exception, with the earlier failure available through exception chaining. When diagnosing a failed report, inspect the complete traceback rather than assuming the final line identifies the original business failure.

Plan the safe-export exercise before implementing it

Write down four acceptance cases for the temporary-report exercise. A successful export should publish the new report. A failure during report generation should leave the previous final report intact. A publication failure should be reported clearly. A cleanup failure should be diagnosable without pretending that publication succeeded.

Use a unique temporary filename in the destination directory rather than a predictable shared name. That avoids two simultaneous exercises writing the same staging file and keeps the intended rename on the same filesystem. Treat this as a design constraint, not a blanket durability guarantee: atomic replacement and persistence after a system crash are different properties, and filesystem behaviour matters.

Test using a final report containing “previous version”. Force an error after writing the first temporary line and verify that the final content remains unchanged. Then run the successful path and verify the complete new content. This focuses the exercise on a business promise the reader can observe, not merely on whether a close method was called.

Choose a scope that communicates ownership

When the number of files is dynamic, ExitStack lets you register each successful acquisition for later cleanup. Release happens in reverse registration order. Keep ownership transfer explicit: callers should know whether a function borrows their already-open handle or opens and closes its own resource. Closing a borrowed handle unexpectedly can break otherwise correct calling code.

Create a fresh generator-based manager for each use. Reusing the same instance after it has finished is not the same as calling its factory again. For a portfolio demonstration, show both the normal call pattern and the failure tests, with a short explanation of the intended exception policy.

Additional official documentation: Python contextlib reference; Python with-statement semantics; Python sqlite3 transaction context.

Context manager FAQs

Does with catch exceptions?

The context manager receives exception information during exit. The exception continues unless the manager deliberately suppresses it by returning a true value.

Can one with statement manage several resources?

Yes. Multiple context managers can appear in one statement. They are exited in reverse order, which supports predictable nested cleanup.

Should every class be a context manager?

No. Implement the protocol when an object has a clear acquisition and release lifecycle. Ordinary data objects usually do not need it.

What happens when setup fails?

If __enter__ raises an exception, the block never begins and that same manager’s __exit__ is not called. Setup code must therefore clean up anything it acquired before the failure. When several steps are dynamic, ExitStack can register cleanup as each resource succeeds.

Official reference: Python contextlib documentation.