Achieve nginx daily log segmentation under Linux (shell + scheduled task + nginx signal management)


1. Practical application: shell + scheduled task + nginx signal management, complete log storage by date, analysis idea: 00:00:01 a.m., rename yesterday's log and put it in the corresponding directory. Then the USR1 information number controls nginx to regenerate a new log file. Crontab edits scheduled tasks.

1, Custom sh script:

#LOGPATH is the log source directory to be backed up
#BASEPATH is the directory where backup logs are stored
BASEPATH=/data/nginx/logs/$(date -d yesterday +%Y%m%d)

# Create a new folder. This is the new folder created on the day of an year
mkdir -p $BASEPATH
# Build backup log file name
filename=$BASEPATH/$(date -d yesterday +%H%M).131test_access.log
#Move log files to backup directory
mv $LOGPATH $filename
touch $LOGPATH
#Use USR1 information number to control nginx to regenerate a new log file
kill -USR1 `cat /usr/local/nginx/logs/`               

2, Crontab editing scheduled tasks

Linux crontab is a command used to execute programs regularly.
After installing the operating system, the task scheduling command will be started by default.
The crond command will periodically check whether there is any work to be executed every minute. If there is any work to be executed, it will automatically execute the work.
Note: the newly created cron task will not be executed immediately. It will take at least 2 minutes. Of course, you can restart cron to execute it immediately.
linux task scheduling is mainly divided into the following two categories:
1. Work performed by the system: work to be performed periodically by the system, such as backing up system data and cleaning cache
2. Personal work: work that a user needs to do regularly, such as checking whether there are new messages on the mail server every 10 minutes. These tasks can be set by each user

The crontab command has three main parameters:
- e: edit the user's crontab.
- l: lists the contents of the user's crontab.
- r: delete the contents of the user's crontab.
Executing crontab -e will automatically open the editor. You can edit your own crontab file. The syntax is the same as that of / etc/crontab file. The difference is that you don't have to point out the executing user. You can save it after editing.

crontab -l is used to view your own crontab files. crontab -r deletes your own crontab.

crontab [ -u user ] file
crontab [ -u user ] { -l | -r | -e }
Basic format:

  • *  *  *  *  command
    Time sharing day month week order
    The first column represents 1 ~ 59 minutes, and each minute is represented by * or * / 1
    The second column indicates hour 1 ~ 23 (0 indicates 0 point)
    The third column indicates dates 1 to 31
    The fourth column indicates the month from January to December
    Column 5 identification number: 0 ~ 6 (0 indicates Sunday)
    Column 6 commands to run
30 21 * * * /usr/local/etc/rc.d/lighttpd restart
 The above example shows 21 per night:30 restart apache. 

45 4 1,10,22 * * /usr/local/etc/rc.d/lighttpd restart
 The above example shows 4 on the 1st, 10th and 22nd of each month : 45 restart apache. 

10 1 * * 6,0 /usr/local/etc/rc.d/lighttpd restart
 The above example represents 1 of every Saturday and Sunday : 10 restart apache. 

0,30 18-23 * * * /usr/local/etc/rc.d/lighttpd restart
 The above example is shown at 18 a day : 00 To 23 : 00 Restart every 30 minutes between apache. 

0 23 * * 6 /usr/local/etc/rc.d/lighttpd restart
 The example above shows 11 every Saturday : 00 pm restart apache. 

* */1 * * * /usr/local/etc/rc.d/lighttpd restart
 Restart every hour apache

* 23-7/1 * * * /usr/local/etc/rc.d/lighttpd restart
 Restart every hour between 11 p.m. and 7 a.m apache

0 11 4 * mon-wed /usr/local/etc/rc.d/lighttpd restart
 Restart on the 4th of each month and at 11:00 every Monday to Wednesday apache

0 4 1 jan * /usr/local/etc/rc.d/lighttpd restart
 Restart at 4:00 on January 1st apache

*/30 * * * * /usr/sbin/ntpdate
 Synchronize the time every half an hour

Pay attention to clearing the mail log of system users
 After each task is scheduled and executed, the system will send the task output information to the current system user in the form of e-mail. Over time, the log information will be very large, which may affect the normal operation of the system. Therefore, it is very important to redirect each task.
For example, you can crontab The following format is set in the file, and the log output is ignored:

0 */3 * * * /usr/local/apache2/apachectl restart >/dev/null 2>&1

"/dev/null 2>&1"Indicates that standard output is redirected to/dev/null,Then redirect the standard error to the standard output, because the standard output has been redirected to/dev/null,Therefore, standard errors are also redirected to/dev/null,In this way, the log output problem is solved.

crontab The log of is mainly used to view the running problems of scheduled tasks
 It is better to write the information to the specified file for easy viewing
0 6 * * *  /usr/bin/php test.php >> /home/log/crontab/mylog.log 2>&1
 Write both error and standard output to mylog.log Yes.

Tags: Linux Operation & Maintenance Nginx Back-end

Posted by FlyingIsFun1217 on Tue, 12 Oct 2021 10:01:54 +0530