Docker 日志清理和设置

背景

今天服务器 Redis 服务突然报错

MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.

网上搜索到解决方案是修改 redis.conf 配置文件，把 stop-writes-on-bgsave-error 设置为 no 即可。结果在保存配置文件的时候报错，提示磁盘已满。

后来发现罪魁祸首原来是 Docker 日志，有一个日志文件居然占了 23GB！

$ du -h $(find /var/lib/docker/containers/ -name *-json.log)
164M    /var/lib/docker/containers/1111/1111-json.log
60M     /var/lib/docker/containers/2222/2222-json.log
100M    /var/lib/docker/containers/3333/3333-json.log
12M     /var/lib/docker/containers/4444/4444-json.log
52K     /var/lib/docker/containers/5555/5555-json.log
23G     /var/lib/docker/containers/6666/6666-json.log

解决

1、清空日志文件

$ cat /dev/null > /var/lib/docker/containers/xxx/xxx-json.log

直接删除日志文件不能立即释放磁盘空间

2、全局日志设置

Docker 支持全局日志设置，但只对之后新建的容器有效。

$ vim /etc/docker/daemon.json

{
    .
    .
    .
    "log-driver": "json-file",
    "log-opts": {
        "max-size": "100m",
        "max-file": "3"
    }
}