During the daily operation and maintenance, it is often necessary to manage, clean up and monitor all kinds of logs. In particular, because of the application bug, dozens of G logs can be written within one hour, resulting in full disk and system failure.
nohup.out,access.log,catalina.out
This article briefly introduces the use of Linux's own logrotate to manage various logs in the operating system.
1. Introduction to logrotate
The logrotate utility is designed to simplify the administration of log files on a system which generates a lot of log files. Logrotate allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size.
In order to use it, there are two main areas that need to be modified: one is /etc/logrotate Conf and /etc/logrotate D/ the following documents.
You can either Conf directly defines how to handle your log file. You can also use /logrotate D/ create a new file corresponding to your log to define the log processing behavior. It is recommended to use the directory /logrotate D/ create your own file below to process personalized logs.
logrotate defines how to handle logs, which itself is called periodically by crond.
A production example I used:
/usr/local/nginx/logs/*.log {
create 0644 root root
daily
rotate 2
missingok
copytruncate
ifempty
compress
noolddir
}
Save the above contents to nginxlog file and Directory: /etc/logrotate D/nginxlog
Setting permissions: owner=root group=root mode=0644
Test whether the configuration is correct:
lograte -d /etc/logrotate.d/nginxlog
2. logrotate configuration parameters
Logrotate global configuration file: /etc/logrotate Conf
configuration parameter | Function description |
compress | Compress and dump logs through gzip |
nocompress | Use this parameter when compression is not required |
copytruncate | It is used to back up and truncate the current log files that are still open; It is a method of copying first and then emptying. There is a time difference between copying and emptying, and some log data may be lost. |
nocopytruncate | Backup log files without truncation |
create mode owner group | Dump file to create a new log file using the specified file mode. Specify the attribute to create a new file during rotation, such as create 0777 nobody nobody |
nocreate | Do not create a new log file |
delaycompress | When used with compress, the dumped log file is not compressed until the next dump |
nodelaycompress | Override the delaycompress option, dump and compress at the same time |
errors address | Send the error message during exclusive storage to the specified Email address |
ifempty | Dump even empty files. This is the default option of logrotate. |
notifempty | If it is an empty file, do not dump it |
mail address | Send the dumped log file to the specified E-mail address |
nomail | Do not send log files during dump |
olddir directory | The dumped log file is placed in the specified directory and must be in the same file system as the current log file |
noolddir | The dumped log file and the current log file are placed in the same directory |
prerotate/endscript |
Instructions to be executed before logrotate dump, such as modifying file attributes; These two keywords must be on separate lines; |
postrotate/endscript | Instructions to be executed after logrotate dump, such as restarting (kill -HUP) a service! Must travel independently; |
daily | Specify dump period as per day |
weekly | Specify dump cycle as weekly |
monthly | Specifies that the dump cycle is monthly |
rotate count | Specify the number of dumps before deleting the log file. 0 means no backup, and 5 means 5 backups are reserved |
tabootext [+] list let logrotate | Do not dump files with the specified extension. The default extension is: RPM Orig Rpmsave, V, and ~ |
size | Size the log file is dumped when it reaches the specified size. Size can specify bytes (default) and KB (sizek) or MB (sizem) |
missingok | If the log is lost, continue to scroll to the next log without error |
notifempty | Do not rotate when the log file is empty |
sharedscripts | Run the postrotate script to execute the script once after all logs are rotated. If this is not configured, the script will be executed once after each log rotation |
dateext | Use current date as naming format |
dateformat .%s | It is used in conjunction with dateext and appears immediately on the next line. It defines the file name after file cutting. It must be used in conjunction with dateext. Only the four parameters%y%m%d%s are supported |
Size (or minsize) log size | The log file is dumped when it reaches the specified size. The log size can specify bytes (default) and KB (sizek) or MB(sizem) |
Description:
Dump when the log file > = log size.
The following is the legal format: (the unit case of other formats has not been tried)
size = 5 or size 5 (> = dump after 5 bytes)
size = 100k or size 100k
size = 100M or size 100M
Example:
/home/deploy/apps/production.log {
missingok
copytruncate
rotate 10
notifempty
sharedscripts
dateext
dateformat -%Y-%m-%d-%s
size=10M
postrotate
mv /home/deploy/apps/production.log-* /data1/log/rails
gzip /data1/log/rails/production.log-*
endscript
}
Question: what is the difference between rotate and maxage?
Both of them are used to control how many log files are saved. The difference is that rotate is in number, while maxage is in days. If we rotate the logs by day, the difference between the two is not big.
4. nginx log cutting instance
vim /etc/logrotate.d/nginx #Create nginx log cutting profile
/application/nginx/logs/*.log{
daily
rotate 10
create
dateext
}
logrotate -d /etc/logrotate.d/nginx Commissioning test -d debug
logrotate -d /etc/logrotate.d/nginx Manual cutting log test
ls /application/nginx/logs/ The date indicates the cut log
access.log bbs.log-20180228 error.log www.log
access.log-20180228 blog.log error.log-20180228 www.log-20180228
bbs.log blog.log-20180228 nginx.pid
Configure the effective time of nginx cutting log
# cat /etc/anacrontab #This document has an effective date
# /etc/anacrontab: configuration file for anacron
# See anacron(8) and anacrontab(5) for details.
SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root
# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45
# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22 #The effective time range is from 3:00 to 22:00
#period in days delay in minutes job-identifier command
1 5 cron.daily nice run-parts /etc/cron.daily
7 25 cron.weekly nice run-parts /etc/cron.weekly
@monthly 45 cron.monthly nice run-parts /etc/cron.monthly
In other words, the configured nginx cutting log takes effect between 3:00 a.m. and 22:00 a.m., and the random delay time is 45 minutes
5. Other configuration examples
/var/log/htmlaccess.log {
errors jim
notifempty
nocompress
weekly
prerotate
/usr/bin/chattr -a /var/log/htmlaccess.log
endscript
postrotate
/usr/bin/chattr +a /var/log/htmlaccess.log
endscript
}
Continuous integration system log processing configuration
/var/log/jenkins/jenkins.log /var/log/jenkins/access_log {
compress
dateext
maxage 365 #Retention maximum 365 days
rotate 99 #Retain up to 99 backups
size=+4096k
notifempty
missingok
create 644
copytruncate
}
Custom log processing
/medialog/*.log {
create 0644 root root
daily
rotate 30
missingok
copytruncate
notifempty
compress
delaycompress
olddir /medialog/backlog # Store archived logs in separate directories
}
People will never have enough time to do it well, but they will always have enough time to start over. However, because there is not always a chance to do it again, you must do it better. In other words, people will never have enough time to think about whether they want it, but will always have enough time to regret it