How to Use Python's logging Module Like a Pro — From Beginner to Production Setup
Python's built-in logging module is one of those tools every developer knows about, but few use well. When I started, I used print() everywhere. When I moved to production apps, that approach didn't scale.
Here's what I learned about logging in Python, from basic setup to production-ready patterns.
The Minimum Viable Logger
Instead of print(), start with this:
import logging
logging.basicConfig(level=logging.INFO)
logging.info("Application started")
logging.error("Something went wrong: %s", err)
That's it. You now have timestamps, log levels, and proper output formatting. The %s formatting is intentional — it delays string interpolation until the message is actually logged.
Why Levels Matter
Most beginners use logging.info() for everything. Here's a better mental model:
- DEBUG (10): Detailed information for diagnosing problems. Ship with this off.
- INFO (20): Confirmation that things are working as expected.
- WARNING (30): Something unexpected happened, but the app still works.
- ERROR (40): The software couldn't perform a function.
- CRITICAL (50): A serious error that may prevent the program from continuing.
Set your default level to INFO in production, DEBUG during development.
Rotating File Handlers
If you log to a file, you'll eventually run out of disk space. Use RotatingFileHandler:
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
'app.log', maxBytes=10_000_000, backupCount=5
)
logging.basicConfig(
level=logging.INFO,
handlers=[handler],
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
This keeps 5 backup files of 10MB each. When the current log fills up, it rotates automatically.
Structured Logging with JSON
For production systems, especially if you use log aggregation tools, JSON logs are superior:
import json
class JSONFormatter(logging.Formatter):
def format(self, record):
log_entry = {
"timestamp": self.formatTime(record),
"level": record.levelname,
"logger": record.name,
"message": record.getMessage(),
}
if record.exc_info and record.exc_info[0]:
log_entry["exception"] = self.formatException(record.exc_info)
return json.dumps(log_entry)
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler])
Now your logs parse cleanly in tools like Logstash, Datadog, or Grafana.
Logging Decorators for Function Tracing
Here's a pattern I use for debugging complex flows:
import logging
from functools import wraps
logger = logging.getLogger(__name__)
def log_call(func):
@wraps(func)
def wrapper(*args, **kwargs):
logger.debug("Entering %s", func.__name__)
try:
result = func(*args, **kwargs)
logger.debug("Exiting %s", func.__name__)
return result
except Exception as e:
logger.exception("Error in %s: %s", func.__name__, e)
raise
return wrapper
@log_call
def process_order(order_id: str) -> bool:
# Your business logic here
return True
The Logger Object Pattern
For larger applications, create loggers per module:
# In each module
logger = logging.getLogger(__name__)
# This automatically gives you module-qualified logger names
# like "myapp.services.payment" instead of flat "myapp"
Context Filters for Request Tracing
When debugging, you often need to trace a single request across multiple modules:
import threading
class RequestContextFilter(logging.Filter):
"""Add request_id to every log record."""
_context = threading.local()
@classmethod
def set_request_id(cls, request_id):
cls._context.request_id = request_id
def filter(self, record):
record.request_id = getattr(
self._context, 'request_id', 'N/A'
)
return True
# Usage
handler = logging.StreamHandler()
handler.addFilter(RequestContextFilter())
logging.basicConfig(level=logging.INFO, handlers=[handler])
logging.info("Request started") # Includes request_id field
What I Actually Use in Production
Here's my production logging setup, simplified:
import logging
import sys
from logging.handlers import RotatingFileHandler
def setup_logging(env: str = "development"):
log_format = (
"%(asctime)s | %(levelname)-8s | %(name)s | "
"%(message)s"
)
root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG if env == "development" else logging.INFO)
# Console output
console = logging.StreamHandler(sys.stdout)
console.setFormatter(logging.Formatter(log_format))
root_logger.addHandler(console)
# File output with rotation
if env != "development":
file_handler = RotatingFileHandler(
"logs/app.log", maxBytes=10_000_000, backupCount=5
)
file_handler.setFormatter(logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s'
))
root_logger.addHandler(file_handler)
# Suppress noisy third-party logs
logging.getLogger("urllib3").setLevel(logging.WARNING)
logging.getLogger("httpx").setLevel(logging.WARNING)
Quick Tips
-
Never use
logging.getLogger()without__name__in modules — you lose context. -
Use
logging.exception()in except blocks — it includes the full traceback. -
Don't format strings manually — let the logger do it:
logging.info("User %s logged in", user.id)(faster, and avoids work if the log level is suppressed). - Add a startup log — log your app version, config path, and environment on startup. Saves hours of debugging.
I use these patterns in my own automation scripts. If you're building Python automation tools and want ready-to-run scripts (file organizers, log analyzers, data pipelines), check out my Python Automation Scripts Pack — 50 copy-paste ready scripts with proper logging already built in.
What's your go-to logging pattern? Drop a comment below.
SOCIAL SHARE CARD GENERATOR