---
tags:
- python
- logging
- best-practice
---

# Python logging

Entry = {timestamp} {level} {message}
message = {context}|{detail}
context = {module} {thread id}
detail = everything else in the log entry

Python:

2020-12-07T13:20:00.000Z INFO ezid.util 123|This is a message.

Apache:

2020-12-07T13:20:00.000Z INFO ezid 1234|client-ip method status connection-status bytes-sent path querystring

Goal is for consistent log reporting in python applications.

Requirements:

* easily parsed by tools including grep, splunk, elasticsearch
* Capture information to assist debugging, performance evaluation, application status
* consistency across applications to simplify processing (e.g. apache logs and python logs as well as logs from different python applications)
* Modularization to enable / disable logging of various components at various levels
* Consistency of use of logging levels
* Play well with tools like logrotate
* A log message should appear on a single line in a log file
* Sensitive information should never appear in logs

## Logging levels

`TRACE` Detailed information of state when the message is emitted. Often covers several lines of output. Very verbose. never used in a production environment. Not available in standard library of python, but [simple to add](https://bugs.python.org/msg307164).
`DEBUG` For debugging purposes using development. On a production system this level should be disabled by default and rarely if ever enabled.
`INFO` For reporting interesting but expected events.
`WARNING` For reporting unexpected or unusual events that may warrant some further action (e.g. file system is getting close to full).
`ERROR` An error message is created when something went wrong but is generally recoverable.
`CRITICAL` Something really bad happened and help is needed. the application is in an unstable or failed state. Call Maria at 2am.

## Time stamp
Every log message should include a timestamp. The timestamp should be in ISO-8601 format and expressed in UTC. If UTC os too confusing, then at least log with a timezone aware timstamp.

## Apache Configuration

Using [mod_log_config](http://httpd.apache.org/docs/current/mod/mod_log_config.html)

See also:
* [Load balancer headers](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/x-forwarded-headers.html)

LogFormat "%{%Y-%m-%d}tT%{%T}t.%{msec_frac}t%{%z}t INFO %v %P|%a %m %s%X %T %U %q" common_format
CustomLog ${APACHE_LOG_DIR}/common.log common_format

2020-12-08T15:49:42.456-0500 INFO someserver.com 617149|69.140.62.177 GET 200- 0 /index.html