logrotate服务错误:read-only filesystem

  一台服务器配置了 logrotate 来对 tomcat 日志进行切割,手动执行 logrotate /etc/logrotate.conf 的时候是正常,但是systemd timer 触发的 logrotate.service 状态为 failure, 手动执行 systemctl status logrotate.service 可以看到报错信息:logrotate[2870734]: error: error opening /usr/local/tomcat/logs/catalina.out: Read-only file system

问题源头

经过一番权限的检查,甚至给上了 777 权限,问题依旧。
由于手动执行 logrotate 是正常的,这时候想起查看它的 service 配置文件: logrotate.service

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
[Unit]
Description=Rotate log files
Documentation=man:logrotate(8) man:logrotate.conf(5)
ConditionACPower=true

[Service]
Type=oneshot
ExecStart=/usr/sbin/logrotate /etc/logrotate.conf

# performance options
Nice=19
IOSchedulingClass=best-effort
IOSchedulingPriority=7

# hardening options
# details: https://www.freedesktop.org/software/systemd/man/systemd.exec.html
# no ProtectHome for userdir logs
# no PrivateNetwork for mail deliviery
# no ProtectKernelTunables for working SELinux with systemd older than 235
# no MemoryDenyWriteExecute for gzip on i686
PrivateDevices=true
PrivateTmp=true
ProtectControlGroups=true
ProtectKernelModules=true
ProtectSystem=full
RestrictRealtime=true

注意看倒数第二行 ProtectSystem=full 这就是问题所在, 查看官方文档

Takes a boolean argument or the special values “full” or “strict”. If true, mounts the /usr/ and the boot loader directories (/boot and /efi) read-only for processes invoked by this unit. If set to “full”, the /etc/ directory is mounted read-only, too. If set to “strict” the entire file system hierarchy is mounted read-only, except for the API file system subtrees /dev/, /proc/ and /sys/ (protect these directories using PrivateDevices=, ProtectKernelTunables=, ProtectControlGroups=). This setting ensures that any modification of the vendor-supplied operating system (and optionally its configuration, and local mounts) is prohibited for the service. It is recommended to enable this setting for all long-running services, unless they are involved with system updates or need to modify the operating system in other ways. If this option is used, ReadWritePaths= may be used to exclude specific directories from being made read-only. This setting is implied if DynamicUser= is set. This setting cannot ensure protection in all cases. In general it has the same limitations as ReadOnlyPaths=, see below. Defaults to off.

捕捉一下关键点, 当此值为 full 时, /usr, boot, /etc, efi 会挂载为 read-only, 值得一提的是,我们的 tomcat 恰好是安装在 /usr 下的, 出现前面的报错是自然的。

解决问题

通过文档得知,我们可以利用 ReadWritePaths 来指定可读写的路径。
vim /lib/systemd/system/logrotate.service 在末尾加上:

1
ReadWritePaths=/usr/local/tomcat/logs

执行:

1
2
3
systemctl daemon-reload 

systemctl restart logrotate.service

即可。