---
# System prepended metadata

title: Python logging
tags: [logging, python, best-practice]

---

---
tags: 
  - python 
  - logging
  - best-practice
---

# Python logging

```
Entry = {timestamp} {level} {message}
message = {context}|{detail}
context = {module} {thread id}
detail = everything else in the log entry
```

Python:
```
2020-12-07T13:20:00.000Z INFO ezid.util 123|This is a message.
```

Apache:
```
2020-12-07T13:20:00.000Z INFO ezid 1234|client-ip method status connection-status bytes-sent path querystring
```

Goal is for consistent log reporting in python applications.

Requirements:

* easily parsed by tools including grep, splunk, elasticsearch
* Capture information to assist debugging, performance evaluation, application status
* consistency across applications to simplify processing (e.g. apache logs and python logs as well as logs from different python applications)
* Modularization to enable / disable logging of various components at various levels
* Consistency of use of logging levels
* Play well with tools like logrotate
* A log message should appear on a single line in a log file
* Sensitive information should never appear in logs

## Logging levels

`TRACE` Detailed information of state when the message is emitted. Often covers several lines of output. Very verbose. never used in a production environment. Not available in standard library of python, but [simple to add](https://bugs.python.org/msg307164).
`DEBUG` For debugging purposes using development. On a production system this level should be disabled by default and rarely if ever enabled.
`INFO` For reporting interesting but expected events.
`WARNING` For reporting unexpected or unusual events that may warrant some further action (e.g. file system is getting close to full).
`ERROR` An error message is created when something went wrong but is generally recoverable. 
`CRITICAL` Something really bad happened and help is needed. the application is in an unstable or failed state. Call Maria at 2am.

## Time stamp
Every log message should include a timestamp. The timestamp should be in ISO-8601 format and expressed in UTC. If UTC os too confusing, then at least log with a timezone aware timstamp.

## Apache Configuration

Using [mod_log_config](http://httpd.apache.org/docs/current/mod/mod_log_config.html)

See also:
* [Load balancer headers](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/x-forwarded-headers.html)

```
LogFormat "%{%Y-%m-%d}tT%{%T}t.%{msec_frac}t%{%z}t INFO %v %P|%a %m %s%X %T %U %q" common_format
CustomLog ${APACHE_LOG_DIR}/common.log common_format
```

```
2020-12-08T15:49:42.456-0500 INFO someserver.com 617149|69.140.62.177 GET 200- 0 /index.html
```