Article Source:
Http://xfocus.org/
Article submission:
INBURST (Inburst_AT_263.NET)
Mirror backup on the website with RSYNC
By INBURST
http://xfocus.org
For system administrators, the usual work center should focus on the maintenance system to function properly and can provide services normally. He often involves a problem with a data backup, what I know
In the case of 80% of the system administrator is not too concerned about the security of the server, but often is quite interested in the technology of the mirror image, but the hardware and software price of commercial products is quite high.
Ang, there is often a free software. The RSYNC prepared here is such software, which can meet the vast majority requirements are not particularly high backup needs.
First, profile
Rsync is a data mirror backup tool under UNIX systems, which can be seen from the software name - Remote Sync. Its characteristics are as follows:
1. Save the entire directory tree and file system image.
2, you can easily maintain the authority, time, soft and hard links of the original file.
3, there is no need to install special permissions.
4, the optimized process, the file transmission efficiency is high.
5. You can use RCP, SSH and other means to transfer files, of course, can also be connected by a direct Socket.
6, support anonymous transmission.
Second, how to use
Rsync's method is simple, I will explain the examples you use.
1, system environment
Rsync supports most UNIX systems, whether it is Linux, Solaris or BSD has passed a good test. My system environment is:
Server: FreeBSD 4.3 IP: 192.168.168.52
Client: Solaris 8 IP: 192.168.168.137
RSYNC version 2.4.6 (can
Http://rsync.samba.org/rsync/ get the latest version)
2, configure the /etc/rsyncd.conf file of the Server end
Bash-2.03 # cat /etc/rsyncd.conf
UID = Nobody
GID = NoBody
Use chroot = no # Do not use chroot
Max connections = 4 # Maximum number of connections is 4
Pid file = /VAR/Run/rsyncd.pid
Lock file = /VAR/run/rsync.lock
Log file = /VAR/LOG/Rsyncd.log # logging file
[Inburst] # This is the authenticated module name, and you need to specify at the client side.
Path = / home / inn needed to make a mirror directory
Comment = Backup Client Is Solaris 8 E250
Ignore Errors # can ignore some unrelated IO errors
Read online = yes # Read only
List = no # does not allow column files
Auth user name = Inburst # authenticated user name, if there is no line, it is anonymous
Secrets file = /etc/inburst.pas # authentication file name
[Web]
Path = / usr / local / apache / htdocs /
Comment = INBURST.ORG Web Server
3. Generate a password file /etc/inburst.pas in the Server side
Bash-2.03 # cat /etc/inburst.pas
Inburst: Hack For security purposes, the properties of the file must be readable.
4. Start rsync in the Server side to launch the daemon
Bash-2.03 # rsync --daemon
If you want to get up when you start, there are several different ways, such as:
A, add inetd.conf
Edit / etc / services, add RSYNC 873 / TCP, specifying RSYNC's service port is 873
Add /etc/inetd.conf, join RSync Stream TCP NOWAIT ROOT / BIN / RSYNC RSYNC - Daemon
B, joining rc.local
In various operating systems, the RC file is stored in the same way, and the RSync - Daemon loaded when the system starts.
5, test from the Client side
In this command line, Vzrtopg's V is Verbose, z is compressed, R is Recursive, and TOPG is the main attribute of the file, such as the owner, time parameter. --Progress refers to display
A detailed schedule, - delete means that if the server deletes this file, the client also deletes the file accordingly and remains truly. Inburst @ ip behind,
Inburst is the username in the password file. Then :: Inburst This Inburst is the module name, which is the name of the customized in /etc/rsyncd.conf. The last / TMP is backup
To the local directory name.
In this, the encrypted connection can also be established with the parameters of -e SSH. You can use -password-file = / password / path / file to specify the password file so that you can make it in the script.
It is not necessary to enter the verification password without interactively, it is necessary to note that this password file authority attribute is only readable.
Bash-2.03 # rsync -vzrtopg --Progress - Delete Inburst@192.168.168.52 :: Inburst / TMP /
PASSWORD:
Receiving File List ... DONE
./
1
785 (100%)
1PY
4086 (100%)
2.py
10680 (100%)
a
0 (100%)
IP
3956 (100%)
./
Wrote 190 bytes read 5499 bytes 758.53 bytes / sec
Total Size IS 19507 Speedup IS 3.43
6, create an update script
If there is more complex work, you can help with some common scripting languages. such as:
Bash-2.03 # cat /usr/local/bin/rsync.sh
#! / bin / sh
Date = `Date % W`
Rsync-vzrtopg --Progress - Delete Inburst@192.168.168.52 :: inburst / home / quact-file = / etc / rsync.pass>
/VAR/LOG/RSync. $ $
7, modify / etc / crontab make a good time
such as:
Bash-2.03 # echo "15 4 * * 6 root rsync.sh" >> / etc / crontab
Third, FAQ
Q: How to make RSYNC via SSH, and do not need to enter a password?
A: You can pass the following steps
1. Create SSH Keys on Server A, do not specify your password, you will see the Identity and Identity.pub files at ~ / .ssh 2. The home directory on Server B.SSH
3. Copy a Identity.pub on Server B
4. Add Identity.pub to ~ [User B] /. Ssh / authorized_keys
5. So the A user on Server A can be on User B SSH to Server B by the following command.
E.G. SSH -L Userb Serverb
This allows the user A on Server A to log in to Server B without a password without a password in the identity of User B.
Q: How to use rsync through the firewall through the firewall without harm security?
A: Answer as follows:
This usually has two situations, one is the server in the firewall, one is the server outside the firewall.
Either case, usually use SSH, it is best to create a backup user, and configure SSHD only allowing this user to enter by RSA authentication.
If the server is in the firewall, it is best to qualify the client's IP address to reject all other connections.
If the client is in the firewall, you can simply allow the firewall to open the SSH proof connection of the TCP port 22.
Q: Can I back up the files that have changed or deleted?
A: Of course you can:
You can use: rsync -other -options -backupdir = ./backup-2000-2-13 ... such commands.
This, if the source file: /path/to/some/file.c changes, then the old file will be moved to ./backup-2000-2-13/path/to/some/file.c, this directory Need yourself
Hand work
Q: What port I need to open on the firewall to adapt to rsync?
A: Depending on the situation
RSYNC can communicate directly through the 873 port of the TCP connection file, or can be passed through the SSH of the 22-port, but you can also change its port by the following command:
Rsync --port 8730 OtherHost ::
or
rsync -e 'ssh -p 2002' OtherHost:
Q: How do I copy a directory structure through rsync, ignore the file?
A: rsync -av - include '* /' --exclude '*' Source-Dir dest-Dir
Q: Why do I always have the "read-only file system" error?
A: See if I forgot to set "read only = no"
Q: Why do I appear '@Error: Invalid GID'?
A: rsync is using UID = Nobody; gid = nobody, if your system does not exist, there is such a mistake, you can try GID =
NOGROUP or other
Q: What is the failure of binding port 873?
A: If you are not running this daemon with root privilege, because 1024 ports are the privileged ports, such errors will appear. You can change it with - port parameters.
Q: Why do I have failed?
A: From your command line:
You are using:
> Bash $ RSYNC -A 144.16.251.213::Test Test> Password:
> @ERROR: Auth failed on module test
>
> I Dont Understand this. Can Somebody Explain as to how to attomplish this.
> All suggestions are well.
It should be the problem that is not landed in your username, try rsync -a max@144.16.251.213 :: test test
Fourth, some can learnable scripts
These scripts here are examples on the RSYNC website:
1, add data to the center server every seven days to increase backup
#! / bin / sh
# This Script Does Personal Backups to a rsync backup server. You will end up
# with a 7 day Rotating Incremental Backup. The incrementals Will Go
# ion in subdirectories named after the day of the week, and the current
# Full Backup Goes Into A Directory Called "Current"
# Tridge@linuxcare.com
# Directory to Backup
BDIR = / home / $ user
# Excludes File - this Contains a Wildcard Pattern Per line of files to excedlude
Excludes = $ home / cron / excluded
# The name of the backup machine
BSERVER = OWL
# Your Password on the Backup Server
Export Rsync_password = xxxxxx
######################################################################################################################################################################################################################################################################################################## #######################################
Backupdir = `Date % a`
OPTS = "- force - teleore-errors - Delete-Excluded - Exclude-from = $ EXCLUDES
--delete --backup --backup-dir = / $ backupdir -a "
Export Path = $ PATH: / BIN: / USR / BIN: / USR / local / bin
# The following line clears the last weeks incremental Directory
[-D $ homen / emptydir] || Mkdir $ Home / Emptydir
Rsync - Delete -a $ HOME / EMPTYDIR / $ BSERVER :: $ USER / $ BACKUPDIR /
RMDIR $ HOME / EMPTYDIR
# Now the actual transfer
Rsync $ OPTS $ BDIR $ BSERVER :: $ USER / CURRENT
2, back up to an idle hard disk
#! / bin / sh
Export Path = / usr / local / bin: / usr / bin: / bin
List = "rootfs usr data data2"
For D in $ list; do
Mount / Backup / $ D
rsync -ax --Exclude fstab - Delete / $ D / / BACKUP / $ D /
Umount / Backup / $ DDONE
Day = `Date" % a "`
Rsync -a - Delete / USR / Local / Apache / Data2 / Backups / $ Day
Rsync -a - Delete / Data / Solid / Data2 / Backups / $ Day
3, mirroring the CVS tree of Vger.rutgers.edu
#! / bin / bash
CD / VAR / WWW / CVS / VGER /
Path = / usr / local / bin: / usr / freeware / bin: / usr / bin: / bin
Run = `lps x | grep rsync | grep -v grep | wc -l`
IF ["$ RUN" -gt 0]; then
Echo already running
EXIT 1
Fi
Rsync -az vger.rutgers.edu::cvs/cvsroot/changelog $ homen / channelelog
Sum1 = `Sum $ homen / channelelog`
Sum2 = `sum / var / www / cvs / vger / cvsroot / channelelog`
IF [$ SUM1 "=" $ SUM2 "]; then
Echo Nothing to do
EXIT 0
Fi
rsync -az --delete --force vger.rutgers.edu::cvs/ / var / www / cvs / vger /
EXIT 0
4, use Find's clever way
Rsync -avr Remote: '`Find / Home -Name" *. [ch] "`' / TMP /
You can use this method to list a list of files that need to be backed up - this method seems to be more useful.
V. References:
1,
http://rsync.samba.org/
2, Rsync Examples
3, RSYNC FAQ
