Automatic backup on Linux
English original
content:
Simple Backup Advanced Backup Using Public Key / Private Key For Security Remote Access Using SSH-Agent Automation Using Keychain Simplified Key Access Script Backup Process Scheduling Test Your Backup Different Security Confederation Reference Information About The Author's Evaluation
related information:
OpenSSH Key Management Using Perl Automation UNIX System Management This section describes how to configure Windows to Linux Tour with Perl: Part 8. Backup and Recovery
Easy, safe, distributed network backup
Level: Intermediate
Carlos Justiniano
(Carlos.justiniano@ecuityinc.com) Software designer, ECUITY Inc. 2004 August
The loss of important data may mean fatal damage. Despite this, there is still countless professionals ignore the backup of their data. Although the reasons may vary, the most common explanation is that the implementation routine backup is indeed cumbersome. Since machines are good at completing ordinary and repetitive tasks, the automated backup process is the key to reducing the inner boring and people and people's relapse tendency.
If you use Linux, you can already use extremely powerful tools that create custom backup solutions. The solutions in this article allow you to use an open source tool that comes with almost every Linux release to perform network backups from simple to higher and secure networks. Simple Backup This article describes a step by step, as long as you follow these basic steps, this method is very intuitive. Before studying more advanced distributed backup solutions, let's take a simple and powerful archive mechanism. Let's analyze a convenient script called ARC, which allows us to create a backup snapshot in the Linux shell prompt. Listing 1. ARC shell script
#! / bin / sh
TAR CZVF $ 1. $ (DATE % Y% M% D% -h% M% s) .TGZ $ 1
EXIT $?
The ARC script receives a separate file or directory name as a parameter, creates a compressed archive file, and embed the current date into the name of the generated archive file. For example, if you have a directory called Beoserver, you can call the Arc script to deliver the Beisterver directory name to create a compressed archive file, such as: Beisterver.20040321-014844.tgz Using the date command is to embed a date And timestamp to help you organize archive files. The date of the date is the year, month, day, hour, minute, second - although there are more than the use of the spine. View the man name of the data command to find out other options. In addition, in Listing 1, we pass the -v (Verbose) option to the TAR. This makes the TAR show that it is archiving files. If you prefer to back up silently, remove this -V option. Listing 2. Archive Beoserver directory
$ ls
Arc Beisterver
$ ./ARC beoserver
Beoserver /
Beoserver / bookl.dat
Beoserver / beoServer_ab_off
Beoserver / beoServer_ab_on
$ ls
Arc beoserver beoserver.20040321-014844.tgz
Advanced Backup This simple backup is practical; however, it still contains a process of manually backed up. The best experience in the industry usually recommends backup data to multiple media and backs up to separate different geographic locations. The central idea is to avoid relying on any separate storage medium or separate location. In the next example we will respond to this challenge, we will analyze a distributed network as shown in Figure 1, which shows system management of two remote servers and an offline storage server. Figure 1. Backup files on distributed network servers # 1 and # 2 will be securely transmitted to offline storage servers, and the entire distributed backup process will be carried out regularly without artificial interference. We will use a set of standard tools (Open Safety Shell Tool Suite (OpenSSH), and tape archive (tar) and CRON task schedule service. All of our plans is to use cron to schedule, use the shell program and TAR applications to complete the backup process, using the OpenSSH Secure Shell (SSH) encryption to perform remote access, authentication, and secure shell copy (SCP) to automate file transfer. Be sure to get another manual for each tool. Safe remote access using public key / private keys In the context of digital security, key (KEY) refers to a data piece used to encrypt or decrypt other data dishes. Interesting to public key private key is, using public key encrypted data, only the corresponding private key can be decrypted. You can freely release a public key so that others can encrypt your messages sent to you. One of the reasons why public key / private key mode thoroughly changes digital security is that senders and recipients do not have to share a universal password. In addition to other contributions, public key / private key encryption uses e-commerce and other secure transmissions. In this article, we will create and use public key and private key to create a very secure distributed backup solution. Each machine to make a backup process must run the OpenSSH Secure Shell Service (SSHD), while the 22-port can be accessed through any internal firewall. If you access a remote server, it is very likely that you are using a secure shell. Our goal will be that you can safely access the machine without manual delivery. Some people think the easiest way is to set up a password access: don't do this. This is not safe. Unexpectedly, the method we will use this article may take about an hour of time, establish a system that is equally convenient to use the "None Password" account - but it is very safe. Let us first ensure that OpenSsh is already installed, then check its version number. When completing this article, the latest version of OpenSSH is released on February 24, 2004 3.8. You should consider using a newer and stable release, at least the version should be more than version 2.x. Access the OpenSSH Security web page to get details about the defects of the specific older version (see links in the references later in this article). So far, OpenSSH is very stable and has proved many of the defects reported by other SSH tools. In the shell prompt, enter SSH and give important V options to check the version number: $ ssh -v openssh_3.5p1, ssh protocols 1.5 / 2.0, openssl 0x0090701f If the version number returned by SS is greater than 2.x, the machine is in Relatively good state. In any case, it is recommended that all your software uses the latest stable version, which is especially important for security related software.
Our first step is to log in to an offline storage server machine (see Figure 1) using an account that will have privileged access servers 1 and 2. $ ssh accountname@somain.com After logging in to the offline storage server, use the SSH-KEYGEN program and give the -t DSA option to create a public key / key pair. The -t option is required to specify the type of key we have to generate. We will use Digital Signature Algorithm, DSA, which allows us to use the updated SSH2 protocol. See the SSH-KEYGEN manual to get more details. During the execution of SSH-KEYGEN, you will be prompted to enter the location of the SSH key storage before asking your password (passphrase). When you ask where to store keys, you only need to press Enter key, and then the SSH-KEYGEN program will create a hidden directory named .ssh (if it does not exist), and two files, a public key file and one Private key file. A interesting feature of SSH-KEYGEN is that when prompted to enter a password, it allows you to simply press the back key. If you have not given an export, SSH-KEYGEN will generate no encrypted key! If you think, this is not a good idea. When the password is required, make sure to enter a sufficiently long character message, preferably a mixed character, not just a simple password string. Listing 3. Always choose a good password [OFFSITE]: $ ssh-keygen -t dsa
GENERATING PUBLIC / Private DSA Key Pair.
ENTER file in which to save the key (/Home/accountname/.ssh/id_dsa):
ENTER Passphrase: (Enter Passphrase)
ENTER SAME Passphrase Again: (Enter Passphrase)
Your Identification Has Been Saved In /Home/accountname/.ssh/id_dsa.
Your public key haas been saved in /Home/accountname/.ssh/id_dsa.pub.
The key fingerprint IS:
7E: 5E: B2: F2: D4: 54: 58: 6A: FA: 6B: 52: 9C: DA: A8: 53: 1B Accountname @ Offsite
Since SSH-KEYGEN is generated. Ssh directory is a hidden "dot" directory, so you have to pass a -a option to the ls command to view the newly created directory: [Offsite] $ ls -a. .. .bash_logout .bash_profile .bashrc .emacs .gtkrc .ssh enters hidden .ssh directory and lists it: [Offsite] $ cd .ssh [offsite] $ ls -lrt id_dsa id_dsa.pub now, in hidden .ssh directory, we have It has a private key (id_dsa) and a public key (id_dsa.pub). You can analyze the contents of each key file using a text editing tool such as VI or Emacs or simply use the LESS or CAT command. You will see that the content composed of mixed characters has passed Base64 encoding. Then we need to copy the public key and install it to the servers 1 and 2. Do not use FTP. More reasonably, use a secure copy program to transfer the public to each remote machine. Listing 4. Install the public key to the remote server [OFFSite] $ scp .ssh / id_dsa.pub accountname@server1.com: offsite.pub
Accountname@server1.com's password: (Enter Password, Not New
PASSPHRASE!)
ID_DSA.PUB 100% | ************************************* | 614 00:00
[OFFSite] $ SCP.SSH / ID_DSA.PUB AccountName@server2.com: offsite.pub
Accountname@server2.com's password: (Enter Password, Not New
PASSPHRASE!)
ID_DSA.PUB 100% | ************************************* | 614 00:00
After installing the new public key, we can log in to each machine using the password specified when you create a private key and the public key. Now, log in to each machine and attach the contents of the OFFSITE.PUB file to a file called Authorized_Keys, which is stored in the .ssh directory of each remote machine. We can use a text editor or simply use the CAT command to attach the contents of the OFFSITE.PUB file to the Authorized_Keys file: Listing 5. Add Offsite.pub to the authorized key list
[OFFSITE] $ ssh accountname@server1.com
Accountname@server1.com's password: (Enter Password, Not New
PASSPHRASE!)
[Server1] $ CAT OFFSITE.PUB >> ./ssh/authorized_keys
The next step is to consider some additional security. First, we modify .SSH access rights so that only owners have permissions to read, write, and execute. Then, we ensure that the Authorized_Keys file can only be accessed by the owner. Finally, delete the previously uploaded OFFSITE.PUB key file, because no need it again. Make sure to set proper access is important because the OpenSSH server may refuse to use a key with unsecure access. Listing 6. Modify permissions using CHMOD
[Server1] $ chmod 700 .ssh
[Server1] $ chmod 600 ./ssh/authorized_keys
[Server1] $ rm offsite.pub
[Server1] $ EXIT
After the same procedure is completed on the server 2, we can return to the offline storage machine to test the access of the new password type. On Offline Servers You can enter the following: [Offsite] $ ssh -v accountname@server1.com Using -V or The Verbose tag option is displayed to display debugging information. In addition to how the authentication process is observed on a high level, you can also show important information you can not get in other ways. You may not need to specify -v tags in later connections; however it is quite useful when testing. Access the SSH-Agent program using the SSH-Agent Automation, which is as needed to secure access to the security key as needed. After SSH-Agent is started, it will run in the background and can be used by other OpenSSH applications such as SSH and SCP programs. This allows the SSH program to request a key that has been decrypted instead of asking you the security password of the private key at a time you need. Let's take a closer study of SSH -Agent. SSH-Agent is running it will output a shell command: Listing 7. SSH-Agent Application
[OFFSITE] $ ssh-agent
SSH_AUTH_SOCK = / TMP / SSH-XX1O24LS / Agent.14179; Export SSH_AUTH_SOCK;
SSH_AGENT_PID = 14180; Export SSH_AGENT_PID;
Echo Agent PID 14180;
We can use the shell's eval command to let the shell execute the output command of the SSH-Agent display: [OFFSITE] $ Eval `ssh-agent` Agent PID 14198 EVAL command tells the shell to evaluate the command generated by the SSH-Agent program. Make sure you specify the reverse quotation mark (`) instead of single quotes! After execution, the EVAL `SSH-Agent` statement will return the process identifier of the agent. After the scene, the SSH_AUTH_SOCK and SSH_AGENT_PID shell variables have been exported. You can display them in the Shell console to see their values: [OFFSITE] $ Echo $ ssh_auth_sock /tmp/ssh-xx7bhiwq/agent.14197 $ ssh_auth_sock (SSH Authentication Socket Abbreviation) is a local socket location, Applications can communicate with SSH-Agent through it. Add an Eval `SSH-Agent` statement to your ~ / .bash_profile file to make sure ssh_auth_sock and ssh_agent_pid are always registered. SSH-Agent has now become a background process that can be viewed with TOP and PS commands. Now we can already share our password using SSH-Agent. To do this, we must use a program named SSH-AddD, which adds our password (send) to the running SSH-Agent program. Listing 8. SSH-ADD for free logging in
[OFFSITE] $ ssh-add
ENTER Passphrase for /Home/accountname/.ssh/id_dsa: (Enter Passphrase)
Identity Added: /Home/accountname/.ssh/id_dsa
(/Home/accountname/.ssh/id_dsa)
Now, when we visit Server1, we will not be prompted to enter the password: [Offsite] $ ssh accountname@server1.com [server1] $ exit If you still don't believe, try removing (kill -9) ssh-agent process, Then reconnect Server1. This time, you will notice that Server1 will ask the password of the private key stored in the .ssh directory: [OFFSITE] $ KILL -9 $ ssh_agent_pid [offsite] $ ssh accountname@server1.com Enter Passphrase for Key ' /Home/accountname/.ssh/id_dsa ': Use Keychain to simplify key access to now, we have learned several OpenSSH programs (SSH, SCP, SSH-Agent, and SSH-Add), and we have created and installed Private key and public key to enable a secure and automatic login process. You may have realized that most settings work only once. For example, create a key, install the key, and perform SSH-Agent through .bash_profile, the process of executing SSH-Agent is only taken at each machine. That is a good news. Not ideal news is that each time we log in to offline machines, you must call SSH-ADD, and SSH-Agent is not directly compatible with the cron scheduled process we will use to automate backup. The Cron process cannot communicate with SSH-Agent is that the cron job is performed as a child process as a cron so that they do not inherit $ ssh_auth_sock shell variable. Fortunately, there is a solution to eliminate the limitations of SSH-Agent and SSH-ADD, but also allow us to use CRON to move a variety of processes that need to be safely password access to other machines. In the three developerWorks series of Articles in 2001, OpenSSH Key Management (see Resources for Link), Daniel Robbins introduced a shell script called Keychain, which is a front end of SSH-Add and SSH-Agent Simplify the entire password process. Over time, Keychain scripts have experienced a lot of improvements, now maintained by Aron Griffis, its latest 2.3.2-1 released release from June 17, 2004. Keychain Shell scripts are too long to be listed in this article, because many erroneous detection, rich documents and very large cross-platform code are included in the well-written scripts. However, KeyChain can easily download from the Web site (see References to get link). After downloading and installing Keychain, it is very simple. You only need to log in to each machine and add the following two lines to each .bash_profile file: Keychain ID_DSA. ~ / .Keychain / $ hostname-sh When you register in each machine, Keychain will ask you. Password. However, unless the machine is restarted, the keychain will not ask you to re-enter the password when logging in later. Best, the cron task can now use the openssh command to securely access remote machines without using passwords to interactively.
Better security and easier use, now we have already obtained. Listing 9. Initialization of Keychain 2.3.2 on each machine; http://www.gentoo.org/Projects/keychain
Copyright 2002-2004 Gentoo Technologies, Inc .; Distributed Under T
GPL
* Initializing /Home/accountname/.keychain/localhost.localdomain-sh
FILE ...
* Initializing /Home/accountname/.keychain/localhost.localdomain-csh
FILE ...
* Starting SSH-Agent
* Adding 1 Key (s) ...
ENTER Passphrase for /Home/accountname/.ssh/id_dsa: (Enter Passphrase)
Scripting Backup Procedure Our next task is to create a shell script that performs the necessary backup process. The goal is to execute the full database backup of servers 1 and 2. In our example, each server runs the MySQL database server, we use the mysqldump command line tool to export some database tables into a SQL input file. Listing 10. Server 1 DBBackup.sh shell script
#! / bin / sh
# Change INTO The Backup_Agent Directory Where Data Files Are Stored.
CD / Home / Backup_Agent
# @ use mysqldump Utility To Export The Sites Database Tables
MySQLDUMP -U SITEDB -PG0ODP @ SSWRD --ADD-DROP-TABLE SITEDB - TABLES
TBL_CCODE TBL_MACHINE TBL_SESSION TBL_STATS> Userdb.sql
# Compress and Archive
TAR CZF Userdb.tgz Userdb.sql
On Server 2, we will set a similar script to back up the unique form given in the site database. Each script is marked as executable by the following steps: [Server1]: $ chmod x dbbackup.sh After setting dbbackup.sh on the server 1 and 2, we return to offline data servers, where we will Create a shell script to call each remote DBACKUP.SH script and then transfer the compressed (.tgz) data file. Listing 11. With backup_remote_servers.sh shell scripts on offline data servers
#! / bin / sh
# uE ssh to remove the remove the dbbackup.sh Script on Server 1
/ usr / bin / ssh backup_agent@server1.com "/Home/backup_Agent/dbbackup.sh"
# uE SCP To Securely Copy The Newly Archived Userdb.tgz File
# from server 1. Note The Use of the date command to timestamp
# The File on The Offsite Data Server.
/ usr / bin / scp backup_agent@server1.com: /Home/backup_Agent/USERDB.TGZ
/ home / backups / userdb - $ (DATE % Y% M% D-% H% M% s) .tgz
# Execute dbackup.sh on server 2 / usr / bin / ssh backup_agent@server2.com "/Home/backup_agent/dbbackup.sh"
# Use scp to transfer transdb.tgz to offsite server.
/ usr / bin / scp backup_agent@server2.com: /Home/backup_Agent/Transdb.tgz
/ HOME / Backups / Transdb - $ (DATE % Y% M% D-% h% m% s) .tgz
Backup_remote_servers.sh shell scripts use the ssh command to perform scripts on the remote server. Since we have set the passwordless access, the SSH command can perform commands remotely on servers 1 and 2 through offline servers. Thanks to Keychain, the entire certification process can now be done automatically. Scheduling our next step, is the last step, is the execution of the scheduled backup_remote_servers.sh shell script on the offline data storage server. We will add two entries to the CRON schedule server to request a backup script twice a day, 3: 34 is executed once, 8: 34 is executed again. Use the Edit (-e) option on an offline server to call the crontab program. [Offsite]: $ crontab -e crontab calls the default editor specified by the Visual or Editor shell environment variable. Then, enter two entries and save and close the file. Listing 12. Crontab entry on offline servers
34 3 * * * /Home/backups/remote_db_backup.sh
34 20 * * * /Home/backups/remote_db_backup.sh
A Crontab row includes two main parts, a schedule section, and a backpoint of the back. The schedule is divided into multiple domains, used to specify when a command should be performed: Listing 13. CRONTAB format
-- minute
| ----- Hour
| | ------ DAY of the MONTH
| | | ------ Month
| | | | | ---- Day of the week
| | | | | | - Command to Execute
| | | | | | |
34 3 * * * /Home/backups/remote_db_backup.sh
Check that your backup You should check the backup to ensure that the program is correct. Automatic programs can be avoided by cumbersome work, but it will never be lazy. If your data is worth backup, it also deserves a sample check. Consider adding a CRON job to remind you to check your backup at least every month. In addition, often modifying the security key is also a good idea, and you can also schedule a cron job to remind you to do this. Additional security prevents higher security, consider installed and configure an intrusion detection system (IDS) on each machine, such as Snort. It is foreseeable that ids will notify you when the invasion is happening or in the near future. After IDS is in place, you will add other levels of security, such as digital signing and encryption for your backup. GNU Privacy Guard (GNUPG), OpenSSSL and NCRYPT and other popular open source tools can support archive files via shell scripts, but not recommended to do this without IDS (see) More information about SNORT). Conclusion The text will show you how to make your script execute on remote servers and how to perform secure automatic file transfer. I hope that you can get inspiration to consider protecting your own important data and use OpenSSH and Snort to build a new solution. Reference
In the official OpenSSH homepage and OpenSSH Security Page, you will find download, documentation, and more. Read Daniel Robbins's excellent IBM DeveloperWorks article "OpenSSH Key Management" (developerWorks, 2001), and download his Keychain app. To learn from SSH, Carlos recommends O'Reilly's SSH, The Secure Shell: The Definitive Guide (O'Reilly & Associates, 2001). The Snort Intrusion Detection System (IDS) is an open source code for detecting and reporting unauthorized access or suspicious behavior. If you are planning to automate the signature and encryption of the archive file, you must use an IDS. You can use GNU Privacy Guard (GNUPG), OpenSSL, and NCrypt to sign and encrypt the archive backup file in the shell script. If you have not used them, please refer to TCP Wrappers and XINETDs. Perl enthusiasts will also be interested in reading "Using Perl Automation Unix System Management" (developerWorks, 2001), "introducing CFENGINE" for system management "(developerWorks, 2002) and" Application configured with Perl "(developerWorks, 2000), they are all works of TED Zlatanov. DeveloperWorks Articles "Windows to Linux Tour: Part 8. Backup and Restore" (developerWorks, 2003) gives a technique for backup strategies. IBM's Tivoli Storage Manager for Linux can also automate reliable backup, archive, and centralized data management on Linux computers and servers based on custom schedules. In addition, the Tivoli Product Family provides users management, access control, and network monitoring - there are more features - products, all have a unified environment and interface. Learn more about Tivoli solutions through the IBM DeveloperWorks Tivoli area. More reference materials for Linux developers can be found in the developerWorks Linux zone. Order the release of Linux books in the Developer Bookstore Linux area. Develop and test your Linux app with the latest IBM tools and middleware through developerWorks Subscription: You can get IBM software from WebSphere, DB2, Lotus, Rational, and Tivoli, and a use of these software within 12 months. Licensing, all costs are lower than what you imagine.
Since the developerWorks Speed-start your Linux app Download no runs on Linux, the free test version to selected developerWorks Subscription products, including WebSphere Studio Site Developer, WebSphere SDK for Web services, WebSphere Application Server, DB2 Universal Database Personal Developers Edition , Tivoli Access Manager and Lotus Domino Server. To get started faster, see how-to articles and technical support for each product. About the author Carlos Justiniano is a software designer for ECUITY, INC. Aspects he interested include communication and distributed calculations. Carlos wrote an article for many technical magazines. He is still the founder and designer of Linux-based Chessbrain project, which has won 2005 Guinness World Record related to distributed computing. You can contact him through carlos.justiniano@ecuityinc.com
