During the daily operation and maintenance, it is often necessary to manage, clean up and monitor all kinds of logs. In particular, because of the application bug, dozens of G logs can be written within one hour, resulting in full disk and system failure.

nohup.out，access.log，catalina.out

This article briefly introduces the use of Linux's own logrotate to manage various logs in the operating system.

1. Introduction to logrotate

The logrotate utility is designed to simplify the administration of log files on a system which generates a lot of log files. Logrotate allows for the automatic rotation compression, removal and mailing of log files. Logrotate can be set to handle a log file daily, weekly, monthly or when the log file gets to a certain size.

In order to use it, there are two main areas that need to be modified: one is /etc/logrotate Conf and /etc/logrotate D/ the following documents.

You can either Conf directly defines how to handle your log file. You can also use /logrotate D/ create a new file corresponding to your log to define the log processing behavior. It is recommended to use the directory /logrotate D/ create your own file below to process personalized logs.

logrotate defines how to handle logs, which itself is called periodically by crond.

A production example I used:

/usr/local/nginx/logs/*.log {
    create 0644 root root
    daily
    rotate 2
    missingok
    copytruncate
    ifempty
    compress
    noolddir
}

Save the above contents to nginxlog file and Directory: /etc/logrotate D/nginxlog
Setting permissions: owner=root group=root mode=0644

Test whether the configuration is correct:

lograte -d /etc/logrotate.d/nginxlog

2. logrotate configuration parameters

Logrotate global configuration file: /etc/logrotate Conf

Description:

Dump when the log file > = log size.
The following is the legal format: (the unit case of other formats has not been tried)
size = 5 or size 5 (> = dump after 5 bytes)
size = 100k or size 100k
size = 100M or size 100M

Example:

/home/deploy/apps/production.log {
missingok
copytruncate
rotate 10
notifempty
sharedscripts
dateext
dateformat -%Y-%m-%d-%s
size=10M
postrotate
mv /home/deploy/apps/production.log-* /data1/log/rails
gzip /data1/log/rails/production.log-*
endscript
}

Question: what is the difference between rotate and maxage?
Both of them are used to control how many log files are saved. The difference is that rotate is in number, while maxage is in days. If we rotate the logs by day, the difference between the two is not big.

4. nginx log cutting instance

vim /etc/logrotate.d/nginx   #Create nginx log cutting profile

/application/nginx/logs/*.log{
daily
rotate 10
create
dateext
}

logrotate -d /etc/logrotate.d/nginx    Commissioning test   -d debug
logrotate -d /etc/logrotate.d/nginx    Manual cutting log test

ls /application/nginx/logs/           The date indicates the cut log
access.log           bbs.log-20180228   error.log           www.log
access.log-20180228  blog.log           error.log-20180228  www.log-20180228
bbs.log              blog.log-20180228  nginx.pid

Configure the effective time of nginx cutting log

# cat /etc/anacrontab    #This document has an effective date
# /etc/anacrontab: configuration file for anacron

# See anacron(8) and anacrontab(5) for details.
 
SHELL=/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin
MAILTO=root

# the maximal random delay added to the base delay of the jobs
RANDOM_DELAY=45

# the jobs will be started during the following hours only
START_HOURS_RANGE=3-22   #The effective time range is from 3:00 to 22:00

#period in days   delay in minutes   job-identifier   command
1       5       cron.daily              nice run-parts /etc/cron.daily
7       25      cron.weekly             nice run-parts /etc/cron.weekly
@monthly 45     cron.monthly            nice run-parts /etc/cron.monthly

In other words, the configured nginx cutting log takes effect between 3:00 a.m. and 22:00 a.m., and the random delay time is 45 minutes

5. Other configuration examples

/var/log/htmlaccess.log {
 errors jim
 notifempty
 nocompress
 weekly
 prerotate
 /usr/bin/chattr -a /var/log/htmlaccess.log
 endscript
 
 postrotate
 /usr/bin/chattr +a /var/log/htmlaccess.log
 endscript
}

Continuous integration system log processing configuration

/var/log/jenkins/jenkins.log /var/log/jenkins/access_log {
    compress
    dateext
    maxage 365      #Retention maximum 365 days
    rotate 99       #Retain up to 99 backups
    size=+4096k
    notifempty
    missingok
    create 644
    copytruncate
}

Custom log processing

/medialog/*.log {
    create 0644 root root
    daily
    rotate 30
    missingok
    copytruncate
    notifempty
    compress
    delaycompress
    olddir /medialog/backlog    # Store archived logs in separate directories
}