How docker file log works

tags: IT

Docker generate 2 items when a conatiner is generated.

1. logger
2. copier

1. Logger:

1. Monitoring the size of logs based on config
2. Rotate log file if config stores data into local files.
3. Provide target to copier
4. Provide log() to copier

2. Copier:

1. map the stream(stdio and stderr) to target (could be json, jounrald, or any others)
2. Copy data from stream into target(ex: json file)

3. Copier, logger and stdio stderr

How does filelog work ?

1. dockerd opens a log file at begining, but closes it when the size is full
2. dockerd generate copier gorouting to catch data from stdio and stderr.
3. Check rotation before wrting logs

This means even your file has large size (2GB), it doesn't affect write performance much, why?
The behavior (stdio,stderr –> target) is no difference between a large and small logfile.

Docker logger call stack

https://github.com/moby/moby/blob/master/container/container.go

containerStart
-> InitializeStdio
-> startLogging
-> Generate Copier -> register stdio, stderr and target
-> copier.run()
-> run() : Generate gorouting for stdio and stderr

How the rotation happens on Jsonfile

When copier execute log,which is generated based on log driver configuration, the log checks capacity first and decide to rotate or not.

The flow is :
log
-> WriteLogEntry
-> w.capacity in WriteLogEntry
-> rotate in WriteLogEntry
-> rotate

How docker daemond rotates files.

    err := unlink(lastFile) // remove file
    ...
    ...

	for i := maxFiles - 1; i > 1; i-- {
        ...
        ...
		err := os.Rename(fromPath, toPath)
		logrus.WithError(err).WithField("source", fromPath).WithField("target", toPath).Trace("Rotating log file")
        ...
        ...
	}

Where is STDIO and STDERR File of a Container ?

They are under /run/docker/containerd/<container-id>

# ls /run/docker/containerd/<container-id>
init-stderr  init-stdout

Related docs:

• https://signoz.io/blog/docker-logging/
• https://www.logicmonitor.com/blog/docker-logging-how-do-logs-work-with-docker-containers#docker-daemon-logs
• Docker runtime img