Introduction
The file system is a system software used to manage and organize data saved on the disk drive, which implements the guarantee of data integrity, which is to ensure that the data written to the disk and the consistency of the content read out. In addition to saving data stored in file mode, a file system also stores and manages some important information about files and file system itself (for example: datetime, home, access, file size, storage location, etc.). This information is often referred to as metadata.
Because in order to avoid disk access bottleneck effects, the general file system is mostly working asynchronously, so if the disk operation is suddenly interrupted, the data may cause data to be lost. For example, if this happens: If you process a document on the EXT2 file system on Linux, what happens?
There are these possibilities:
When you save the file, the system crashes. This is the best situation, you won't lose any information. Just restart your computer then continue. The system crashes before you save the file. You will lose all your work, but the old version of the document will exist. The system crashes when the saved document is written to the disk. This is the worst situation: the new version of the file covers the old version of the file. There is only one part of the new part left on the disk. If the file is a binary, then the file that cannot be opened, because its file format and application are expected.
In this last case, if the system crash occurs when the drive is being written into metadata, then the situation may be worse. At this time, the file system has undergone damage, and you may lose the entire directory or the data of the entire disk partition.
The Linux Standard File System (EXT2FS) attempts to restore corrupted metadata information by calling the file scan tool fsck at restart. Since the EXT2 file system saves a backup of redundant key metadata information, it is generally not very much data loss. The location of the damaged data is calculated, or then the redundant metadata information is restored, or the file that is damaged or metadata information is directly deleted.
Obviously, the larger the file system to be detected, the longer the test process. For partitions with dozens of G, it may take a long time to test. Since Linux begins to be increasingly important applications in large servers, it is increasingly not tolerate. This requires more complex and exquisite file systems to replace EXT2.
Therefore, there will be a log style file system (Journalling FileSystems) will appear to meet this needs.
What is a log file system
Here is just a brief description of the log file system. If you need more deeper information, please refer to the article log file system, or introduce the log file system.
Most modern file systems use log technologies developed in the database system to improve crash recovery capabilities. Disk transactions are first written to the specific location of the log area (or log area) in the disk before being truly written to the magnetic disk.
According to the different log file system, the information written to the log area is not exactly the same. Some implementations only write file system metadata, while others record all write operations to the log.
Now, if the crash occurs before the log content is written, the original data is still on the disk, and the loss is only the latest update content. If the collapse occurs when the true write operation (that is, the log content has been updated), the log content of the log file system will display what operations are displayed. Therefore, when the system is restarted, it can easily restore the destroyed update according to the log content.
In either case, complete data is obtained, and the case where damaged partitions will not occur. Since the recovery process is performed according to the log, the entire process will only take a few seconds.
It should be noted that using the log file system does not mean that the file scan tool fsck is not required. The hardware and software errors of randomly occurring file systems are unable to recover according to the log, and must be used to use the FSCK tool.
Current log file system in the Linux environment
Three log file systems will be discussed in the following content: The first is EXT3, developed by Linux kernel Stephen Tweedie. EXT3 is implemented by adding log functions on an EXT2 file system, currently the default file system of RedHat7.2; Namesys developed Reiserfs log file system, can download from www.namesys.com, current Mandrake8.1 uses this log File system. SGI released an XFS log file system in March 2001. You can download in Oss.sgi.com/Projects/xfs/. The three log file systems will be used to detect and performance testing. Install EXT3
For questions about EXT3 file system technology, please refer to DR. Stephen Tweedie 's papers and interviews. The EXT3 log file is straight from the ancestral EXT2 file system. It has a key feature that is completely backward compatible, in fact, is only added to the EXT2 log files to add log functions. The biggest disadvantage is that there is no high performance that can improve file data processing speed and decompression without a modern file system.
EXT3 starts from 2.2.19 as a patched way. If you want to add support for the EXT3 file system to the kernel, you need to use a patch, you can use ftp.linux.org.uk/pub/linux/sct/fs/jfs or ftp.kernel.org/pub/linux/kernel/people / SCT / EXT3 gets a patch, a total of the following documents:
EXT3-0.0.7A.TAR.BZ2: Nuclear Patch E2FSPROGS-1.21-WIP-0601.tar.bz2 Support Ext3 E2FSProgs Program Suite
Copy linux-2.2.19.tar.bz2 and ext3-0.0.7a.tar.bz2 to / usr / src directory:
MV Linux Linux-Old
Tar -ixvf linux-2.2.19.tar.bz2
Tar -ixvf ext3-0.0.7a.tar.bz2
CD Linux
Cat ../ext3-0.0.7A/Linux-2.2.19.kdb.diff | patch -sp1
Cat ../ext3-0.0.7A/Linux-2.2.19.ext3.diff | patch -sp1
First add the KDB core debugger patch of SGI, the second is the EXT3 file system patch. Just configure the kernel, answer Yes on the "Enable SecondEded Fs Development Code" of the file system part. Then compile.
After the kernel compilation installation, you need to install the E2FSProgs software suite:
Tar -ixvf E2FSPROGS-1.21-WIP-0601.tar.bz2
CD E2FSPROGS-1.21
./configure
Make
Make Check
Make Install
The job to do now is to create an EXT3 file system on the partition, restart using the new kernel, this time you have two options to create a new log file system or upgrade to an existing EXT2 file system to the EXT3 log vectors.
For the case where you need to create a new EXT3 file system, you only need to use the MKE2FS command in the installed E2FSProgs package to add -f parameters to create a new EXT3 file system: mke2fs -j / dev / xxx
Here / DEV / XXX is a new partition that wants to create an EXT3 file system. -J parameter represents the creation of EXT3 instead of an EXT2 file system. You can use parameter "-jsize =
You can upgrade your file system and unloaded file system. If the current file system is being loaded, the file .journal will be created in the directory where the file system loading point is located. If it is upgraded a file system that has not been loaded at the time, use an implicit system inode to log the log, this time all the contents of the file system will be reserved not destroyed. You can load EXT3 file system using the following command: mount -t ext3 / dev / xxx / mount_dir
Since EXT3 is actually an EXT2 file system with logging, an EXT3 file system can be loaded in EXT2.
Install XFS file system
If you need to understand the XFS file system from the technical aspect, refer to the SGI's XFS file system and the SGI information page. You can also refer to FAQ.
XFS is a log file system in the Linux environment developed by SGI. It is a mature technology that is originally used in the IRIX system. XFS follows the GPL copyright declaration. The latest version of the XFS file system is 1.02. You can http://linux-xfs.sgi.com/projects/xfs/102_release.html From the download to get the kernel XFS file system support patch or download the RPM package mode, below We will explain how to 2.4. 14 kernel use XFS. First download the following
PATCH-2.4.14-XFS-1.0.2.BZ2 PATCH-2.4.14-XFS-1.0.2-kdb.bz2
Copy the Linux kernel Linux-2.4.2.tar.bz2 to / usr / src directory, modify the old kernel directory name, then decompress the new kernel:
MV Linux Linux-Old
Tar -ixf inux-2.4.2.tar.bz2
Copy each patch into the kernel source directory (for example: / usr / src / linux), and patch:
Zcat Patch-2.4.14-xfs-1.0.2.bz2 | patch -p1
ZCAT PATCH-2.4.14-XFS-1.0.2-kdb.bz2 | Patch -p1
The kernel is then configured, open the kernel options of the file system section: "XFS FileSystem Support" (config_xfs_fs) and "Config_page_buf). At the same time, you need to upgrade these system tools to the following or higher version:
MODUTILS-2.4.0 AutoConf-2.13 E2FSProgs-Devel-1.18
Install the new kernel and restart the server.
Then download the XFS tool. This package includes the following command to handle the file system, use the following command to install the package ::
Tar -zxf xfsprogs-1.2.0.src.tar.gz
CD xfsprogs-1.2.0
Make configure
Make
Make Install
After installing these commands, you can create a new XFS file system:
MKFS -T XFS / DEV / XXX
If XXX is an existing file system, you need to use the "-f" parameter to create a new partition, but remember that this will undermine all the data of the partition.
MKFS -T XFS -F / DEV / XXX
After creating, you can use the following command to load the new file system:
Mount -t xfs / dev / xxx / mount_dir
Installing the Reiserfs file system If you want to know more from the technical aspect of the Reiserfs file system, please refer to
Namesys and
FAQ.
The Reiserfs file system starts from 2.4.1-Pre4 is a formal support for the Linux kernel. In order to use the Reiserfs file system, you first need to install the file system support tool on the system (such as the Mkreiserfs tool for creating the Reiserfs file system). The latest REISERFS file system version can be added to a 2.2.x or 2.4.x kernel in a patch. Here we take 2.2.19 as an example: first step, first under the kernel source, at 2.2.19 patch at the Reiserfs file system, the latest version of the patch is Linux-2.2.19-Reiserfs-3.5.34-patch. BZ2. At the same time, you should download the tool package: Reiserfsprogs-3.x.0j.tar.gz.
Then decompress the kernel source code and packets to / usr / src:
Tar -ixf linux-2.2.19.tar.bz2
BZCAT Linux-2.2.19-Reiserfs-3.5.34-Patch.bz2 | PATCH -P0
Compile the kernel to support Reiserfs, install the kernel. Then install the file system tool software:
CD / USR / SRC / Linux / FS / Reiserfs / Utils
Make
Make Install
Install the new kernel and restart. You can now create a new Reiserfs file system and load:
Mkreiserfs / dev / xxxx
mount -t reiserfs / dev / xxx / mount_dir
File system performance test
The computer environment used in the test environment is as follows: Pentium III - 16 MB RAM - 2 GB HD, the operating system is redhat6.2. All file systems work normally, so the Benchmark analysis is performed to perform performance comparisons. First, I directly unplug the system power supply to simulate the system to power out to test the log file system recovery process. All file systems successfully passed the file scan detection phase, and the system was scanned after several seconds and then launched the system.
Next, the Bonnie Performance Test Program (www.coker.com.au/bonnie ) is used, this program is accessible to a file, created, read, and delete small files, these operations for Squid, Inn Or mail server programs (QMAIL) in Maildir format are the most common operations. Performance test commands are:
Bonnie -d / Work1 -S10 -R4 -U0
It performs 10MB (-S10) testing on the file system loaded under the / WORK1 directory. Therefore, you must create an appropriate type of file system before performing the test and load it under directory / WORK1. Other parameters specify the number of Memory Size (-R4), and run the test program as root, the test results are as follows:
Sequential OutputSequential InputRandom SeeksSize: Chunk SizePer CharBlockRewritePer CharBlockK / sec% CPUK / sec% CPUK / sec% CPUK / sec% CPUK / sec% CPU / sec% CPUext210M1471971481367130914150694488915309.810ext310M136698236138182422148294493514317.810xfs10M120694951277135133129998477980229.111reiserfs10M145599425331234026147793559326174.35
Sequential CreateRandom CreateNum FilesCreateReadDeleteCreateReadDelete / sec% CPU / sec% CPU / sec% CPU / sec% CPU / sec% CPU / sec% CPUext2169499278994929795992841009341ext3168998274100458969399288999745xfs16929925196436989199311999041reiserfs161307100896310019149912459993161001725100 Each test has two sets of data: the file system rate (K / sec) and the CPU occupancy rate ( % CPU). The higher the speed, the better the file system. For the CPU rate, the smaller the number, the better. It can be seen that the Reiserfs file system is best in the file operation (Sequential Create and Random Create section), which is more than 10 times more than other file systems. In other aspects (Sequential Output and Sequential Input) and other file system performance is incapable. There is no particularly obvious difference for other file systems. XFS performance is close to the EXT2 file system, and the EXT3 file system is slightly slower than EXT2 (because the log log requires some additional time). Finally, the performance test program MONGO obtained from www.namesys.com is modified to test three log file systems. Here, you add a command to add the load XFS and EXT3 file system to the Mongo.pl and format it, and then start performance test analysis. This script format partitions / dev / xxxx, loads it and runs specified number of processes at each phase: creation, copy, symbolic connection processing, read, display file status information, rename, and delete files. At the same time, the program calculates the fragmentation after the creation and copy phase.
FRAGM = Number_of_fragments / number_of_files
You can get the same test comparison result in the result file:
LOG - Original results log.tbl - The output result of the comparison process log_table - the result of the table format
The following command is tested:
Mongo.pl ext3 / dev / hda3 / work1 Logext3 1
If you want to test another file system, you need to modify EXT3 in the parameters of the above command to REISERFS or XFS. Other parameters are the partitions, load paths, and save the file names of the test results and the number of processes that are started.
The following table is the test result. The data unit is second. The lower the value, the better the performance. The data block used by the first table test is 100 bytes, the second table is 1000 bytes, the last one is 10,000 bytes.
ext3 files = 68952 size = 100 bytes dirs = 242XFS files = 68952 size = 100 bytes dirs = 241reiserFS files = 68952 size = 100 bytes dirs = 241Create90.07267.8653.05Fragm.1.321.021.00Copy239.02744.51126.97Fragm.1.321.031.80Slinks0203. 54105.71Read782.751543.93562.53stats108.65262.25225.32162.25225.3270.72nd6205.1870.72delete23.80389.7985.51
ext3 files = 11248 size = 1000 bytes dirs = 44XFS files = 11616 size = 1000 bytes dirs = 43ReiserFS files = 11616 size = 1000 bytes dirs = 43Create30.6857.9436.38Fragm.1.381.011.03Copy75.21149.4984.02Fragm.1.381.011.43Slinks16. 6829.5919.29Read225.74348.99409.45Stats25.6046.4189.23Rename16.1133.5720.69Delete6.0464.9018.21ext3 files = 2274 size = 10000 bytes dirs = 32XFS files = 2292 size = 10000 bytes dirs = 31reiserFS files = 2292 size = 10000 bytes dirs = 31Create27. 1325.9922.27Fragm.1.441.021.05Copy55.2755.7343.24Fragm.1.441.021.12Slinks1.332.511.43Read40.5150.2056.34Stats2.341.993.52Rename0.991.101.25Delete3.408.991.84
From the above table, you can see that EXT3 is better in terms of state deletion and rename, and the Reiserfs file system is better in file creation and copy performance. It can also see that REISERFS is equivalent to its technical documentation.
in conclusion
At least there are at least two robust log file systems (XFS and Reiserfs) for selection (XFS and REISERFS), which are widely used. For example, Mandrake8.1 supports the Reiserfs file system by default.
As can be seen from the results of performance test, Reiserfs is the best choice.
