Explanation and implementation of rsync service

rsync

rsync is a data image backup tool under Linux system. Use the fast incremental backup tool Remote Sync to synchronize remotely, support local replication, and synchronize data with other SSH and rsync hosts.
Note: this document is configured based on the red hat series operating system and is applicable to RHEL, CentOS and other mainstream Linux systems.

1, Usage of rsync command:

Basic format: rsync [options] original position target position

Common options:
-a archive mode, recursive and preserving object attributes
-v displays verbose information about the synchronization process
-z compress when transferring files
-H keep hard linked files
-A reserved ACL attribute

  • -delete deletes files that exist in the target location but not in the original location
    -r recursive mode, including all files in the directory and subdirectory
    -l soft link files are still copied as soft link files
    -p keep the permission mark of the file
    -t retain the time stamp of the file
    -g keep the group mark of the file (for super users only)
    -o keep the master tag of the file (for super users only)
    -D keep equipment documents and other special documents

Here we introduce two types of synchronization, one based on ssh and the other based on rsync
The basic difference between the two is that ssh needs to use a local user, while rsync can create virtual users, which will be more secure, because it is not a good thing to have too many users on a machine. However, ssh can create users, so the password can be set very complex, but the virtual users created by rsync are not able to create more complex passwords

Here, we select two Centos7 machines and install rsync. A file directory is created on the server for client synchronization. Similarly, the client also creates a repository
192.168.1.2(server)
192.168.1.10(client)

Create an authorized user on the server. Here I create mkml

[root@client client]# rsync -avz mkml@192.168.1.2:/server/ /client 
The authenticity of host '192.168.1.2 (192.168.1.2)' can't be established.
ECDSA key fingerprint is SHA256:Ibw5WhTH15faTzq4ruPkNrC7cycW9jSMd9kDfSvakk0.
ECDSA key fingerprint is MD5:b3:79:65:52:99:55:43:31:64:a9:41:ab:48:e9:40:43.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.2' (ECDSA) to the list of known hosts.
mkml@192.168.1.2's password: 
receiving incremental file list
./
file1/
file10/
file11/
file12/
file13/
file14/
file15/
file16/
file17/
file18/
file19/
file2/
file20/
file3/
file4/
file5/
file6/
file7/
file8/
file9/

sent 107 bytes  received 435 bytes  98.55 bytes/sec
total size is 0  speedup is 0.00

Start extraction, extraction succeeded

Generally speaking, many people don't need to upload, because if everyone uploads at will, downloading will also download the things uploaded by others. Generally, they are not all open. Generally, a special acl can be used to restrict

Let's demonstrate
Now there is no acl configuration or open

[root@client client]# touch mkml-fufu.txt

Error reporting

[root@client client]# rsync -avz /client/ mkml@192.168.1.2:/server
mkml@192.168.1.2's password:
sending incremental file list
rsync: failed to set times on "/server/.": Operation not permitted (1)
./
mkml-fufu.txt
rsync: mkstemp "/server/.mkml-fufu.txt.N7vc5L" failed: Permission denied (13)
sent 111 bytes  received 195 bytes  87.43 bytes/sec
total size is 0  speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1179) [sender=3.1.2]
Permission denied Once you see this, you know it's a permission problem
/server No mkml We assign a permission to the write permission of, but we can't give too much permission. We give the permission to a specific user
[root@server server]# setfacl -m user:mkml:rwx /server
[root@server server]# getfacl /server/
getfacl: Removing leading '/' from absolute path names
file: server/
owner: root
group: root
user::rwx
user:mkml:rwx
group::r-x
mask::rwx
other::r-x

Execute the command again

You can feel that the above operations are basically similar to scp, and do not reflect the benefits of rsync

Now, let's show rsync based synchronization
rsync is managed by xinetd, so we also need to install xinetd on the server
Let's start at /etc/rsync Add under conf

address = 192.168.1.2
port 873
pid file = /var/run/rsyncd.pid
log file = /var/log/rsyncd.log
[share]
comment = soft
path = /server
read only = yes
dont compress = *.gz *.bz2 *.zip
auth users = mkml-1
secrets file = /etc/rsyncd_users.db
  1. The first line is your host
  2. Port. Remember, the best port is 873. By default, if it is changed arbitrarily, the server monitoring port and the client sending port will be inconsistent during synchronization, resulting in synchronization errors
  3. pid storage
  4. Log storage
  5. This is to define a parameter. At that time, the client will not find the address according to the absolute path, but only need to know the name defined by the server to automatically obtain the path
  6. An explanation
  7. Environment, that is, your storage path
  8. Read only
  9. Compressed format
  10. Authorized virtual user
  11. Password of the virtual user

Now let's create a password repository for virtual users
Remember, generally, the plaintext stores some passwords and so on. Basically, the permission of file 600 should be given

echo 'mkml-1:123456' >> /etc/rsyncd_users.db
chmod 600 /etc/rsyncd_users.db 

All right, here we go, /etc/xinetd D the rsync service is defined below

vim /etc/xinetd.d/rsync
service rsync
{
        disable = yes
        flags = IPv6
        socket_type = stream
        wait = no
        user = root
        server = /usr/bin/rsync
        server_args = --daemon
        log_on_failure += USERID
}

Sometimes there may not be this file under xinetd. We can create it directly
Define our rsync service

{

  1. Enable or disable yes
  2. flag, you define IPv6 as supporting IPv6 and IPv4
  3. Socket types and data streams are basically fixed
  4. Whether to wait
  5. Operating user
  6. Where is the service
  7. Service parameters

}

[root@server xinetd.d]# rsync --daemon
[root@server xinetd.d]# netstat -pantu | grep "rsync"
tcp        0      0 192.168.1.2:873         0.0.0.0:*               LISTEN      5360/rsync

Start and view monitoring ports

The client can start transferring files

[root@client client]# rsync -avz mkml-1@192.168.1.2::share /client
Password:
receiving incremental file list
./
file1
file10
file11
file12
file13
file14
file15
file16
file17
file18
file19
file2
file20
file3
file4
file5
file6
file7
file8
file9
mkml-fufu.txt
sent 426 bytes  received 1,107 bytes  613.20 bytes/sec
total size is 0  speedup is 0.00

Now we can see that it is troublesome to enter a password every time

Add the parameter of rsync password to profile or export

export RSYNC_PASSWORD=123456

RSYNC_PASSWORD is a system variable, so you don't have to worry about it

[root@client client]# rsync -avz mkml@192.168.1.2::share /client
@ERROR: auth failed on module share
rsync error: error starting client-server protocol (code 5) at main.c(1649) [Receiver=3.1.2]
[root@client client]# rsync -avz mkml-1@192.168.1.2::share /client
receiving incremental file list
./
file1
file10
file11
file12
file13
file14
file15
file16
file17
file18
file19
file2
file20
file3
file4
file5
file6
file7
file8
file9
mkml-fufu.txt
sent 426 bytes  received 1,107 bytes  1,022.00 bytes/sec
total size is 0  speedup is 0.00

This is password free. If you don't want to write this directly to be seen by others' history, you can also write it in txt and extract it. It's also OK
The synchronization source of rsync has been provided here. The synchronization of rsync without password is the same as that of ssh. ssh password free login means uploading the public key to the user, that is, you can log in without password. The same is true of rsync. There is no operation here

Next, we will demonstrate the exclusion in rsync and the synchronous deletion of data owned by non servers

exclude

In many cases, when we download, we don't want to download so many things. The current one is relatively small, but if we put a few G gadgets into it, downloading is a troublesome thing
We can use an exclusion parameter included in rsync

[root@client client]# rsync -avz --exclude=mkml-fufu.txt mkml-1@192.168.1.2::share /client
receiving incremental file list
./
file1
file10
file11
file12
file13
file14
file15
file16
file17
file18
file19
file2
file20
file3
file4
file5
file6
file7
file8
file9
sent 426 bytes  received 1,039 bytes  2,930.00 bytes/sec
total size is 0  speedup is 0.00

As you can see, mkml fufu Txt excluded

If I want to exclude multiple, use {}

[root@client client]# rsync -avz --exclude={mkml-fufu.txt,file1,file2,file3} mkml-1@192.168.1.2::share /client
receiving incremental file list
./
file10
file11
file12
file13
file14
file15
file16
file17
file18
file19
file20
file4
file5
file6
file7
file8
file9
sent 402 bytes  received 892 bytes  2,588.00 bytes/sec
total size is 0  speedup is 0.00

If we write those exclusions every time, it will be a bit troublesome. We can write them in a txt. We only need to call or add them every time

exclude-from´╝îExclude some file names defined by yourself. If there are multiple file names, one on each line
echo "mkml-fufu.txt" >> exclude.txt
[root@client client]# rsync -avz --exclude-from=/root/exclude.txt mkml-1@192.168.1.2::share /client
receiving incremental file list
./
file1
file10
file11
file12
file13
file14
file15
file16
file17
file18
file19
file2
file20
file3
file4
file5
file6
file7
file8
file9
sent 426 bytes  received 1,039 bytes  2,930.00 bytes/sec
total size is 0  speedup is 0.00

It's gone

delete

OK, the exclusion has been completed. Now let's learn to delete. The deletion is not to delete the data on the server, but to delete the common data on the non server
First, we create some files on the client and add some characters to a file shared by the server

echo "123456" >> /client/file1
[root@client client]# rsync -avz --delete mkml-1@192.168.1.2::share /client
receiving incremental file list
deleting c.txt
deleting b.txt
deleting a.txt
./
file1
file10
file11
file12
file13
file14
file15
file16
file17
file18
file19
file2
file20
file3
file4
file5
file6
file7
file8
file9
mkml-fufu.txt
sent 432 bytes  received 1,107 bytes  3,078.00 bytes/sec
total size is 0  speedup is 0.00
[root@client client]# ll
total 0
-rw-r--r--. 1 root root 0 Apr 17 20:56 file1
-rw-r--r--. 1 root root 0 Apr 17 20:56 file10
-rw-r--r--. 1 root root 0 Apr 17 20:56 file11
-rw-r--r--. 1 root root 0 Apr 17 20:56 file12
-rw-r--r--. 1 root root 0 Apr 17 20:56 file13
-rw-r--r--. 1 root root 0 Apr 17 20:56 file14
-rw-r--r--. 1 root root 0 Apr 17 20:56 file15
-rw-r--r--. 1 root root 0 Apr 17 20:56 file16
-rw-r--r--. 1 root root 0 Apr 17 20:56 file17
-rw-r--r--. 1 root root 0 Apr 17 20:56 file18
-rw-r--r--. 1 root root 0 Apr 17 20:56 file19
-rw-r--r--. 1 root root 0 Apr 17 20:56 file2
-rw-r--r--. 1 root root 0 Apr 17 20:56 file20
-rw-r--r--. 1 root root 0 Apr 17 20:56 file3
-rw-r--r--. 1 root root 0 Apr 17 20:56 file4
-rw-r--r--. 1 root root 0 Apr 17 20:56 file5
-rw-r--r--. 1 root root 0 Apr 17 20:56 file6
-rw-r--r--. 1 root root 0 Apr 17 20:56 file7
-rw-r--r--. 1 root root 0 Apr 17 20:56 file8
-rw-r--r--. 1 root root 0 Apr 17 20:56 file9
-rw-r--r--. 1 1003 1003 0 Apr 17 20:59 mkml-fufu.txt
[root@client client]# cat file1
[root@client client]# 

As you can see, the original file we created is gone, and the characters added to file1 are gone

Tags: Linux server ssh inotify

Posted by nelsok on Mon, 30 May 2022 12:02:32 +0530