You can find more information at http://www-900.ibm.com/developerworks/cn/
Logs and Reiserfs
Daniel Robbins President / CEO, Gentoo Technologies, Inc2001 June
With the release of Linux 2.4, a large number of file system probability, including REISERFS, XFS, GFS, and other file systems. These file systems have sounded very cool, but what do they really do, what are they doing, and how can they use them safely in the Linux product environment? In the Advanced File System Implementator Guide, Daniel Robbins answered the above questions by setting up these new advanced file systems in the Linux 2.4 environment. Compliance with this method, he provides valuable advice, performance information, and important technical attention in the actual implementation process to facilitate pleasant experience in the new file system. Here, in the first article of this series, he explained the advantages of logs and reiserfs.
The purpose of this series of articles is to introduce you to all new file systems for Linux, including Reiserfs, XFS, JFS, GFS, EXT3, and other file systems. I want you to know some of the necessary practical knowledge, with these knowledge you can start using these file systems. My goal is to help you avoid potential hidden dangers as much as possible; this is to say, we will carefully understand the stability, performance issues (or bad) of the file system, and any negative application interaction you should know. , The best match for kernel and patches and more. You can regard this series of articles as the "Insider Guide" of these next-generation file systems.
This is what is ready. But to start this series of work, I have an article to get out of this theme, used to prepare for the next itinerary. I will involve two topics for Linux development communities - the design concept after log and Reiserfs. The log is very important because it is the technology we have always expected for a long time, but now I have finally appeared. It is used in Reiserfs, XFS, JFS, EXT3, and GFS. Exactly understand what the log is to do and why Linux needs it to be very important. Even if you have some mastery of the log, I still hope that the introduction of the log can be a good model to explain this technology to others, or as a convention, to benefit the whole world to start the world. Transforms these new log file systems. This process is usually started by "Linux Guy / Gal", just like you will convince others to do this.
In the second half of this article, we will look at the design concept after Reiserfs. Through this, we can master a fact that these new file systems are not just a little more than the old system than the old system. They also allow us to take things with a very impossible method. Developers should keep this in mind when reading this series of articles. The ability of these new file systems will probably have an impact on your future Linux software development project.
Understand Log: Metadata is as you understand, the presence of a file system allows you to store, retrieve, and operational data. To achieve this, the file system needs to maintain an intrinsic data structure that makes your data organized and easy to access. This internal data structure (exact, "data about data") is called metadata. That is the structure of this metadata provides its specific identity and performance characteristics for file systems.
Typically, we don't directly deal with the metadata of the file system. It is a special Linux file system driver to work for us. The Linux file system driver is specifically used to operate complex metadata. However, in order to make the file system driver work normally, there is an important necessary condition; it needs to find metadata in a certain reasonable, consistent, and no interference. Otherwise, the file system driver cannot understand and operate metadata, then you cannot access files. Understanding Log: fsck This will lead to FSCK. When the Linux system starts, FSCK starts and scans all local file systems listed in the system / etc / fstab file. The FSCK's work is to ensure that the metadata of the file system to be loaded is in the usable state. Most of them are available. When Linux is closed, it carefully transfers all buffer data to disk and make sure the file system is completely uninstalled to ensure that the system can be used again. Typically, FSCK scans those file systems that will be loaded, and they have been completely unloaded and make reasonable assumptions - all metadata is no problem.
However, we all know that there will be some unexpected occurrences from time to time, such as unexpected power failures or systems hang. When these misfortunes appear, Linux does not have the opportunity to completely uninstall the file system. When the system restarts, FSCK starts scanning, it detects these file systems that have not been completely uninstalled, and make reasonable hypothesis - file systems may not be ready for the Linux file system driver. This is very likely that the metadata is in some case in some cases.
So, in order to make up for this situation, FSck will begin thorough scanning and fully check the metadata and correct any errors found in this process. Once fsck completes this work, the file system can be used. Although unexpected power failures or system suspends may cause recently modified data loss, the file system can be loaded and put into use because the metadata is now consistent.
FSCK's question so far, in order to ensure the consistency of the file system, this method may sound is not a bad idea, but it is not the best solution. The problem comes from the fact - FSCK must scan all metadata of the file system to ensure the consistency of the file system. Thorough consistency checking for all metadata is an extremely time time. It is usually at least for a few minutes to complete. Worse, the larger the file system, the longer the time spent on this thorough scan. This is a big problem, because when fsck is doing it yourself, your Linux system is actually cut off, and if you have a huge file system storage, your system may spend half an hour. Or longer time to perform FSCK. Of course, in the environment of the task is desirable, the standard FSCK work may cause destructive results in an environment where the system is operating in an extremely important system. Fortunately, there is a better solution.
The log log file system solves this FSCK issue by adding a new data structure called a log. This log is the structure on the disk. Before making any changes to metadata, the file system driver writes an entry to the log. This entry describes what it will do. Then, it continues and modifies the metadata. In this way, the log file system has a history of recent metadata modified. When checking the consistency problem of a file system that is not completely uninstalled, this record is at hand.
You can look at the log file system - except for storage data (your material) and metadata (about material data), they have a log. You can call them metadata (data about material data).
How do fsck handle log file systems? In fact, usually don't do anything. It just ignores the file system and allows it to be loaded. Behind the fast recovery of the file system reaches the consistency state, it is true that in the Linux file system driver. When the file system is loaded, the Linux file system driver views whether the file system is intact. If you have problems due to some reasons, you need to fix metadata, but don't perform a thorough scan of metadata (just like FSCK), but check the log. Since the log contains a recent metadata modification record in chronological order, it simply views the metadata that is recently modified. Thus, it can restore the file system to the consistent state within a few seconds. And the traditional methods used in FSCK are different, and this log playback process does not take more time on large file systems. Thanks to the log, hundreds of file system metadata can revert to the consistency in an instant. Reiserfs now, let's talk about Reiserfs, which is the first in several log file systems we will study. Reiserfs 3.6.x (Version part of Linux 2.4) is developed by Hans Reiser and his development group in Namesys. Hans and his team members believe that the best file system is those file systems that help to create independent shared environments or namespaces, which can interact more directly, effectively and powerful. In order to achieve this goal, the file system should meet the needs of our users on performance and functionality. In that way, the user can continue to use the file system directly without having to build a special purpose layer running on the file system (such as the database).
The performance of small files is then how can the file system adapt to the environment? Namesys has decided to focus on one aspect of the file system, at least - the performance of small files. Typically, file systems such as EXT2 and UFS do not very well, often forcing developers to turn to databases or special organizations to acquire certain properties they need. Over time, this method of "encoding the problem" is not a good thing. This is not a good thing.
There is an example of how EXT2 encourages this programming. EXT2 is very good at storing a large number of files above 20k, but for 2,000 50-bytes of files, it is not an ideal technology. When EXT2 must handle very small files, it is not just a significant decline, and storage efficiency also decreases, because EXT2 is a block in 1k or 4K to allocate space (set when it is created in file system).
Now, General Wise Make the Fa will prompt you should not store so many files on the file system. Instead, it should be stored in a database running on the file system. As a response to this statement, Hans Reiser pointed out wheneted when you need to build a layer on the top of the file system, it means that the file system does not satisfy your needs. If the file system meets your needs, you first avoid the use of special purpose solutions. This saves development time and eliminates code expansion. These codes may be when you manually handle your own personal memory or buffer mechanism, or when you interaction with a library of a database.
In theory, this is theoretical. But in practical use, how will Reiserfs's small file performance? It's so surprised. In fact, when the file is processed less than 1K, Reiserfs is about 8 to 15 times faster than EXT2! Sometimes, these performance improvements are not at the expense of performance loss of other file types. Typically, Reiserfs is almost superior to EXT2, but it really reflects its flash point when processing small files. How does REISERFS technology provide such excellent small file performance? Reiserfs uses special optimization B * balanced trees (one per file system) to organize all file system data. This provides very good performance improvements to itself, and can also reduce the artificial constraints in the design of the file system. For example, now a directory can accommodate 100,000 subdirectors. Another benefit of using the B * tree is that Reiserfs can dynamically assign index festival as needed, without having to establish a fixed index feature when the file system is created. This helps the file system more flexibly adapted to its various storage needs while providing additional space efficient.
REISERFS has many features that are particularly directed to the performance of improving small files. Unlike EXT2, Reiserfs does not securely allocate storage space with 1k or 4K block, but the exact size required to assign. And Reiserfs also includes a special optimization-centricular-centered-centered - tail file refers to the end of files and files smaller than file system blocks. To improve performance, Reiserfs can store files in the leaf node of the B * tree instead of storing the data at other places where the disk is pointing again.
This made two things. First, it significantly increases the performance of small files. Since the file data and the STAT_DATA information is close to the store, they are usually read by the same disk IO operation. Second, Reiserfs can compress the tail file to save a lot of disk space. In fact, REISERFS file system with tail file compression (default) can store more 6 percentage points more than the equivalent EXT2 file system, which is amazing.
However, since the document compression forces Reiserfs to reload the data when the file is modified, this results in a slight discount in performance. In view of this reason, the Reiserfs tail file compression can be turned off, allowing system administrators to choose at speeds and space efficiently, or sacrificed some storage capabilities to exchange higher speeds.
Reiserfs is indeed a very good file system. In my next article, I will guide you to complete the full process of reiserfs installation in Linux 2.4. We will also take care of the performance adjustment, application interaction (and how to work around them) and the best kernel used.
With the release of Linux 2.4, there is a possibility of using many new file systems, including Reiserfs, XFS, GFS, and other file systems. These file systems sound "cool", but what can they do, what they are good at, and how can you use them safely in a product Linux environment? In this series of articles, Daniel Robbins answered these questions by setting up these new high-end file systems in Linux 2.4 in Linux 2.4. in his
In the previous article in this series, Daniel introduces the benefits of logs and Reiserfs. In this article, Daniel will teach you to install a very stable Linux 2.4-based REISERFS system.
In this article, I will show you how to let Reiserfs run under the 2.4 series kernel. I will also share a lot of technical information about different topics, including the best 2.4 kernel, performance precautions, and so on with Reiserfs. Because I will talk about it first, I suggest you read this article first, then follow the installation instruction. In this way, when you start running Reiserfs on your own system, there will be all technical annotations in your mind so you can adjust the necessary adjustments in each step. Looking for a good kernel To use Reiserfs on your system, you first need to find a suitable kernel. If you are already familiar with the development of the 2.4 core, you will know that this process is complicated than it sounds. When this is completed, the latest kernel is 2.4.6-pre2; but I suggest you use 2.4.4 (standard Linus kernel) or 2.4.4-AC9 (slightly improved Alan Cox core) on its own Reiserfs system or 2.4.4-AC9 . From my test, 2.4.5 seems to be very unstable, so I don't recommend using this kernel as a product; let us hope 2.4.6 will be much better than it.
If you want to use other kernels other than 2.4.4 or 2.4.4-AC9 in its own product Reiserfs system, you must make the necessary checks to ensure that Reiserfs (and kernels) are stable. Of course, if you are installing Reiserfs on a test server, you can use any kernel whenever you do not lose important data.
In general, pay attention to the problem of kernel stability, especially the stability of Reiserfs, there are two reasons. Because Reiserfs is a "experiment" kernel feature, you can't assume that a Reiserfs that uses the new kernel has just been able to run well from Tarball. The second reason (perhaps the problem is currently a more important issue) is that most of the 2.4 kernel issuance and patch are unstable, so you still need to be cautious when you act. In theory, all 2.4 issuances should be quasi-productized, because the 2.4 version should be a stable series; however, in fact they (still) is not, so strongly encourage you to carefully use new, no tested kernels .
This information is not to scare you dare not use Reiserfs or Linux 2.4, but to give those people who are more dare to venture. Don't always use a variety of kernels that are still in an important system; if so, you will suffer. When you use an unreliable kernel, you are not only in danger of system locking; you are also in danger of losing data and file system crash, which is absolutely undesirable. Even if the Reiserfs implementation itself is stable, the main error of other parts of the kernel is also likely to cause the production of file system crash.
If you don't have the latest source of kernel stability information, I recommend that you regularly access Linux weekly news (see Refer to this article), timely understand the latest possible kernel issues (information updated every Thursday). I hope that I have already convince more readers who like to adventure to use 2.4.4 or 2.4.4-AC9 kernel as the product Reiserfs configuration, let us continue.
Standard Kernel OK, we can now talk about three options for installing and running a quasi-productized Reiserfs system. The first choice is to use only standard 2.4.4 Linux kernel. The second option is to use 2.4.4 kernels while using the Reiserfs big patch, which includes some specialized patches to make Reiserfs to reach quota compatible and more compatible with NFS servers running. The third option is that we can use 2.4.4 kernel and AC9 patches (ie 2.4.4-AC9), plus or increasing patchs. I usually recommend using 2.4.4-AC9 and big patches because there is no negative effect on large patches, and you may need it, and AC9 is much better than the standard kernel. However, if you don't want to use the AC kernel, the standard 2.4.4 is not bad. I will simply introduce you to the steps of setting 2.4.4-AC9 and big patch, but if you don't want to install these two patches or one, you can skip this step. Now let's get started. First, download 2.4.4 kernel source code from Kernel.org and enter your / usr / src directory. Remove any Linux directories or symbolic connections to the site, if it is a directory, it will be renamed. If it is a symbol connection, simply remove it. then:
# CD / USR / SRC
# cat /path/to/linx-2.4.4.tar.bz2 | BZIP2 -D | TAR XVF -
AC9 patch and big patch If you plan to use only standard 2.4.4 kernels, you already have all the resources you need, you can skip the rest of the patch. However, I recommend you to continue using the AC patch and large patch below.
To use the AC9 patch, download the Alan Cox AC9 patch from Kernel.org. Then type:
# CD / USR / SRC / Linux
# bzip2 -dc /path/to/patch-2.4.4-ac9.bz2 | Patch -p1
Once the standard kernel is installed, go to DICE to download the Dice's Reiserfs big patch. This step is still optional, but it is also recommended, especially if you will run the NFS server on this system or you need quotas (if not, this patch will not have any disadvantages). To use a big patch, please follow these steps:
# CD / USR / SRC / Linux
# bzip2 -dc /path/to/bigpatch-2.4.4.diff.bz2 | Patch -p1
Once all optional corrections are installed, you can configure kernel for Reiserfs.
Note: If you need more guidance on how to compile Linux kernels, please refer to Compile Linux kernel free tutorial on developerWorks. Below is a simple summary.
The configuration kernel is very simple. First, type "make menuconfig". In the Code Maturity Level Options section, make sure the "Prompt for Development and / or Incomplete Code / Drivers" option is enabled. Then go to the "File Systems" section to enable the "Reiserfs Support" option. You set REISERFS to compile it to the kernel (not as a module); and usually "Have Reiserfs Do Extra InterNal Checking" is not a good idea. Now, save your settings, compile your kernel ("Make Dep; Make Bzimage; Make Make Make "_INSTALL), then configure your boot loader to load new kernels configured with Reiserfs. Key: Save the current kernel and configure the boot loader (boot loader) is always a good idea, so that your new kernel cannot work, you can start with the original kernel.
Use Reiserfs
The installation tool we need to install the "Reiserfsprogs" tool before restarting, which includes "mkreiserfs", "resize_reiserfs" (useful to the LVM user), and "fsck.reiserfs". You can download the latest version of "Reiserfsprogs" from the Namesys.com download page (now "3.x.0j"). Once these tools are downloaded, you can compile and install "Reiserfsprogs" as follows:
# CD / TMP
# TAR XzVF Reiserfsprogs-3.x.0j.tar.gz
# cd reiserfsprogs-3.x.0j
# ./configure
./configure Output Will Appear Here
# Make
Make Output Will Appear Here
# make install
Make Install Output Will Appear Here
Since the tool has been installed, you can now create any new partitions as needed (using "fdisk" or "cfdisk") or LVM logical volume (using "lvcreate" and restart your system. If you are creating a standard partition, you can simply mark this partition as a "Linux Native File System" (83).
After creating and installing the file system, you can create a Reiserfs file system on an empty partition, as shown below:
# mkreiserfs / dev /
HDXY
In the example above, / dev / hdxy should be a device node corresponding to an idle partition. Install it like installing other file systems:
# mount / dev /
HDXY
/ MNT / Reiser
Moreover, if you want to add a Reiserfs file system in your own / etc / fstab file, simply set "FREQ" and "Passno" fields to "0", as shown below:
/ dev / hdc1 / home reiserfs defaults 0 0
From this, your Reiserfs file system should be inferior to its opponent EXT2, and you don't have to worry about "FSCK" for a long time, and overall performance will be much better - especially for small documents. Reiserfs Technical Note
File System Stability I have used the REISERFS of the 2.4.4 version of the kernel in a product environment (on the cvs.gentoo.org development server), there is no crash problem at all. 2.4.4 and 2.4.4-AC9 have been very stable. Our server performs a large number of disk IO tasks because it is our CVS profile, "dev-wiki", gentoo.org mail server, a mailing list based on postman, and there are many other things.
REISERFS Limitations Although Reiserfs is better than the EXT2 file system in almost every type of application, some places Reiserfs are still a bit lack. What is gratifying is that these problems are not the limitations that Reiserfs cannot overcome, but the Namesys developers have not yet have to be coded or optimized.
Without Dump / Restore, this is true; Reiserfs does not have "dump" and "restore" implementation. If you want to use Reiserfs, it happens to be a person who likes to use "dump", you must find other ways to back up data. In fact, it is true that this is not a problem, because the 2.4 series kernel is not compatible from the "Dump" and "Restore" commands from the beginning. To learn more about the incompatibility of Dump and 2.4 kernel, read Linus Torvalds Posts (see Resources), he said in the post "DUMP is a stupid program from the beginning. Don't touch it. "
Performance Issues Although Reiserfs usually rushed to EXT2's head, it does have performance deficiency in a particular situation. The first is the ability to handle sparse files. Reiserfs' ability to handle sparse files is much better than EXT2. This will change when Namesys developers have been trying to optimize this part of Reiserfs. Prior to this, EXT2 is a preferred solution for applications that have a high demand for sparse files.
There may also encounter problems in the code that performs multiple STAT () calls for a large number of files. A application that might triggering this performance defect (it only shows the REIERFS implementation of the 2.4 series kernel, and there is no existence in the 2.2 core) is "MUTT" mail user agent (see Resources), which is used to read Performance defects are triggered when a large MAILDIR form is email. Obviously, each mail file MUTT is STAT twice, which will be more influential than usual. The Reiserfs Development Team has realized this special problem and has identified it. If there is no solution in the near future, you should be able to see the corresponding solution in Reiserfs 4.
Performance Adjustment is that there are several simple and universal performance adjustment methods to ease these issues. The first is to install your Reiserfs file system with the "NoAtime" option (a installation option for Reiserfs and other file systems). You may know that the UNIX system records an activity for each object on the file system, or is called access time, and attemes will be updated each time you read the file. For most people, the ATIME postmark function is not very useful, and there is almost no application (I don't think of) relying on what important tasks for ATIME. For this reason, it is usually safely turned off, so it can bring a good comprehensive performance. Typically, unless you do know you need Atime support, you should also use the NoAtime option to install the file system. Use the / etc / fstab entry as shown below: / dev / hdc1 / home reiserfs noatime 0 0
In the first article about Reiserfs, I mentioned that Reiserfs has a special feature called "Tail Packing". In Reiserfs terms, "tail" is a file that is less than a system file block (4K), or cannot fill the end of the file of a complete file system block. Reiserfs has a small file processing performance because it can merge these TAIL into its B * Tree (its main data organization structure) so that they can really approach STAT-DATA (Reiserfs medium with indexes Node's unit). However, because these TAIL can't fill a complete block, they waste a lot of disk space (of course, said relatively). To solve this problem, Reiserfs uses its "Tail Packing" feature to compress these TAIL to a space occupied as much as possible. Usually, doing the Reiserfs file system more than about 5% of the EXT2 file system than the size.
More about NOTAIL, however, Tail Packing also has its shortcomings. First, it does bring a small but not ignored impact to the performance. Fortunately, Reiserfs developers have expected that some people prefer to sacrifice about 5% of disk space for a little extra performance, so they create "NOTAIL" installation options. When the file system is installed with this option, Tail Packing will be closed so that your storage capacity is reduced, but there is a faster speed. Typically, attach importance to the performance of the file system performance, while enabling "NOTAIL" and "NoAtime" options to install their file system, bringing significant performance improvement:
/ dev / hdc1 / home reiserfs noatime, NOTAIL 0 0
Even if you want to save a little disk space, sometimes it is a good thing to use the "NOTAIL" option to install the file system. In particular, most boot loaders load a problem when the kernel created on the Reiserfs file system enabled TAIL PACKING is enabled. If you are using a Lilo that is lower than version 21.6, you will encounter this problem. When you use the latest version of GRUB, you will also encounter your Stage1 and Stage1_5 files, although there is no problem when loading the actual kernel. If you are already experiencing this problem, you can fix this problem - install the file system using the "NOTAIL" option to move the file to another file system, then move them back. When the file is recreated, there will be no TAIL. In addition, remember that you can easily reinstall the file system (with new options) without uninstalling it. This special example uses the "NOTAIL" option to reinstall the root file system. Typically, if you want to use Tail Packing, you also need to boot loader load auxiliary file from the root file system (such as kernel), this command is useful: # mount / -o remount, notail
QMAIL Note If you are using qmail in Reiserfs, you should learn about some important references. First, you should use the QMAIL 1.03 source of patch. It fixes QMAIL asynchronous calls "LINK ()" and "unlink ()" problems that are not only Reiserfs, but also the problem of EXT2. Next, you should look at JEDI's QMAIL tuning page, which contains a lot of good suggestions about how to use qmail performance as much as possible. Finally, be sure to view JEDI's Reisersmtp package. Reisersmtp contains a GPL plugin instead of QMAIL's SMTP section; this replacement of JEDI is especially true for Reiserfs, depending on the new queue processing routine, which can give you double mail charge performance.
Conclusion I found that Reiserfs is an incredible file system that provides a good small file processing performance and a very good (usually better than EXT2) regular file processing performance. Because with Reiserfs, my developers can complete the Gentoo Linux "CVS" update in just 15 seconds, and use EXT2 usually spend about two minutes. Reiserfs makes our developers' life more enjoyable, but also makes our CVS server to handle a large number of concurrent IOs without causing hard drives and negative impact on interaction performance.
However, in addition to all of this, the most exciting place for Reiserfs is its future development direction. Hans Reiser has a very radical and innovative plan for Reiserfs, including planning extension file systems to use it as a high-performance database that develops mature, and finally includes transaction support and advanced query. This means that Reiserfs will not only "only another high performance file system"; the opposite, it will open new possibilities and roads, so that we can solve traditional storage problems in innovation. With the cooperation of Namesys, Linux will be very exciting in the future - this is definitely a good thing.
With the release of Linux 2.4, a large number of file system probability, including REISERFS, XFS, GFS, and other file systems. These file systems have sounded very cool, but what do they really do, what are they doing, and how can they use them safely in the Linux product environment? Daniel Robbins answered the above questions by setting up these new advanced file systems in a Linux 2.4 environment. At this section, Daniel briefly introduces TMPFS, a VM-based file system, and introduces you to the new possibility of "binding" installation function in version 2.4. In this series I have previously, I introduced the benefits of creating logs and using Reiserfs, and showing how to install a solid Linux 2.4-based Reiserfs system. In this article, we have to talk about several relative topics. First, we will briefly introduce TMPFS, which is the virtual memory (VM) file system we know. TMPFS may be the best system that can use the RAM disk now, and is a new function of the 2.4 core. Then, we will simply introduce new features of another 2.4 kernel called "Binding Installation", which brought great flexibility when installing (and reinstall) file systems. In the next article, we will focus on DEVFS, and we will take some time to further familiarize yourself with the new EXT3 file system.
Introduction TMPFS If I have to say clear TMPFS, I will say that TMPFS is like a virtual disk (RAMDISK), but not the same. Like virtual disks, TMPFS can use your RAM, but it can also be stored using your swap partition. And the traditional virtual disk is a block device and requires a command such as MKFS to truly use it. TMPFS is a file system, not block devices; you just install it, it can be used. All in all, this allows TMPFS to become the best RAM-based file system I have the opportunity to encounter.
TMPFS and VM Let's take a look at TMPFS more interesting features. As I mentioned earlier, TMPFS can use RAM or the exchange partition. I just started to look at it, but I remember that TMPFS is also the "virtual memory file system" we know. Moreover, you may also know that the virtual memory resource of the Linux kernel comes from your RAM and swap partition. The VM subsystem in the kernel allocates these resources to other parts of the system, and is responsible for managing these resources in the background, typically transparently moving the RAM page to a switched partition or from a swap partition to the RAM page.
The TMPFS file system requires a page of the VM subsystem to store files. TMPFS does not know that these pages are in the swap partition or in RAM; doing this decision is the work of the VM subsystem. The TMPFS file system is known that it is using some form of virtual memory.
Not a block device here is another interesting feature of the TMPFS file system. Unlike most "standard" file systems, such as EXT3, EXT2, XFS, JFS, Reiserfs, and other systems, TMPFS is not existing on a bottom block device. Because TMPFS is built directly above the VM, you can create a TMPFS file system with a simple mount command.
# mount TMPFS / MNT / TMPFS -T TMPFS
After performing this command, a new TMPFS file system is installed in / mnt / tmpfs, ready to use. Note that there is no need to run mkfs.tmpfs; in fact, it is impossible because there is no such command exists. After the mount command is executed, the file system is immediately installed and can be used, and the type is TMPFS. This and the Linux virtual disk uses a large phase mirror; the standard Linux virtual disk is block device, so you must format it with the file system you choose before using them. Instead, TMPFS is a file system. So you can use it simply to install it. Advantages of TMPFS
Dynamic file system size You might want to know how much the TMPFS file system installed in front of / mnt / tmpfs is in front. The answer to this question is a bit unexpected, especially when the disk-based file system is compared. / MNT / TMPFS will only have a small space, but with the copy and creation of the file, the TMPFS file system driver allocates more VM and dynamically increases the space of the file system as required. Moreover, when the file in / mnt / tmp is deleted, the TMPFS file system driver dynamically reduces the file system and releases the VM resource, which can return the VM to the loop in the system to use in the system. Because the VM is a valuable resource, you must not want anything to waste more than the actual VM, TMPFS benefits is that these are automatically handled. See Resources.
Another major advantage of speed TMPFS is its lightning speed. Because typical TMPFS file systems will remain completely in RAM, read and write can almost instantly. Even with some exchange partitions, performance is still excellent, this part of the TMPFS file system is moved to the RAM when more idle VM resources can be used. Allows the VM subsystem to move some TMPFS file systems to switch partitions actually good performance, because doing this allows the VM subsystem to release space of the process that requires RAM. This point along with its dynamic adjustment of size, which can make the operating system more overall performance and flexibility than choosing a traditional RAM disk.
No persistence This seems to be a positive factor, TMPFS data will not be retained after restarting, because virtual memory is essentially volatile. I think you may guess the reason for TMPFS known as "TMPFS", isn't it? However, this can actually be a good thing. It allows TMPFS to save data that you don't need to keep (such as temporary files, you can find some parts of / TMP, and some parts of the / VAR file system tree).
Using TMPFS In order to use TMPFS, you need to enable the 2.4 series core of "Virtual Memory FiLe System Support" option; this option is in the "File Systems" section of the kernel configuration option. Once you have a kernel that enabled TMPFS, you can start installing the TMPFS file system. In fact, open the TMPFS option in all the 2.4 kernels is a good idea, whether you plan to use TMPFS. This is because you need kernel TMPFS support to use POSIX shared memory. However, the memory shared by System V does not require TMPFS in the kernel. Note that you don't need to install the TMPFS file system in order to make POSIX shares; you only need to support TMPFS in the kernel. The memory shared by POSIX is now not much used, but this situation may change over time.
Avoid low VM situations that TMPFS is doubtful as needed: If your TMPFS file system is increased to it exhausted all virtual memory, you don't have the remaining RAM or exchange partition, then what happens? In general, this situation is a bit annoying. If it is 2.4.4 core, the core will lock immediately. If it is 2.4.6 kernel, the VM subsystem has been corrected in many ways, although it is not a good experience, things will not completely fail. If the 2.4.6 core has not been assigned more VM, you obviously don't want to write any new data to the TMPFS file system. In addition, some things may happen. First, other processes of the system will not be able to allocate more memory; usually, this means that the system will become extremely slow and there is almost no response. In this way, the super user should take the necessary steps to alleviate this low VM, or when it is very often consumed. In addition, the kernel has a built-in final line system that releases memory when there is no memory available, which will find the process of the VM resource and terminate the process. Unfortunately, this "Termination Process" solution usually causes adverse consequences in the case of TMPFS. The following is a reason. TMPFS itself cannot (nor) is terminated because it is part of the kernel rather than a user process, and there is no easy way to let the kernel find that that process is full of TMPFS file systems. Therefore, the kernel will incorrectly attack the maximum occupational VM that it can find, usually the X Server, if you happen to use it. So, your X server will be terminated, and the root cause of low VM is not resolved. ICK.
Low VM: Solution is fortunate, TMPFS allows you to specify the maximum upper limit of the file system capacity when installing or reinstalling the file system. In fact, from the 2.4.6 core to the 2.11G core, these parameters can only be set at installation instead of reinstall, but we can expect to set these parameters when reinstalling in the near future. The best setting of the maximum value of TMPFS relies on the use of resources and your specific Linux host; this idea is to prevent a TMPFS file system that completely uses resources from exhausting all virtual memory results, which leads us a bad low VM mentioned earlier. Happening. A good way to find a good TMPFS upper limit is to use TOP to monitor the use of the exchange partitions of your system in the peak use phase. Then, make sure the specified TMPFS upper limit is slightly smaller than all of these peak use times, the sum of the space exchange partitions and idle RAMs.
It is easy to create a TMPFS file system with the maximum capacity. To create a new maximum 32 MB of TMPFS file system, type:
# mount TMPFS / DEV / SHM -T TMPFS -O Size = 32M
This time, we didn't install the TMPFS file system in / mnt / TMPFS, but found in / dev / shm, which is exactly the "formal" installation point of the TMPFS file system. If you are using DEVFS, you will find this directory has been created for you.
Also, if we want to limit the capacity of the file system within 512 kb or 1 GB, we can specify size = 512k and size = 1g, respectively. In addition to restricting capacity, we can also limit index nodes (file system objects) by specifying NR_INODES = X parameters. When using NR_INODES, X can be a simple integer, and there is also a k, m or g to designate thousands, millions or billion (!) Index nodes. Moreover, if you want to add the equivalent function of the above mount tmpfs command to / etc / fstab, it should be like this:
TMPFS / DEV / SHM TMPFS SIZE = 32M 0 0
When installing 2.2 on an existing installation point, trying to install anything else installed in the installed mounting point that has been installed again. However, the rewritten kernel installation code makes multiple usage installation points no longer a problem. Here is an example: assuming that we have an existing file system installed in / tmp. However, we decided to start using TMPFS to make / TMP storage. In the past, your unique choice is to uninstall / TMP and reinstall your new TMPFS / TMP file system in its location, as shown below:
# umount / tmp
# mount TMPFS / TMP -T TMPFS -O Size = 64M
However, this solution may be for you. There may be a lot of running processes in / tmp; if so, when trying to uninstall / TMP, you will encounter the following error:
Umount: / TMP: Device is busy
However, using the most recent 2.4 kernel, you can install your new / TMP file system without encountering the "Device IS busy" error:
# mount TMPFS / TMP -T TMPFS -O Size = 64M
With an order, your new TMPFS / TMP file system is installed on / tmp and is installed on a partition that has been installed without being directly accessed. However, although you cannot access the original / TMP, any process on which you have opened a file on the original file system can continue to access them. Moreover, if you unmount Based on TMPFS / TMP, the original installed / TMP file system will reappear. In fact, you can install any number of file systems at the same installation point, the installation point is like a stack; uninstall the current file system, the last recently installed file system will reappear.
Binding Installation Use Binding Installation, we can install all or even partially installed file systems to another, while two mount points can access the file system at the same time. For example, you can use binding installation to install your existing root file system to / home / drobbins / nifty as follows:
# mount - Bind / / Home / Drobbins / Nifty
Now, if you observe the inside of / home / drobbins / nifty, you will see your root file system (/ home / drobbins / nifty / etc, / home / drobbins / nifty / opt, etc.). Moreover, if you modify the file in the root file system, you can also see the changes in / home / drobbins / nifty. This is because they are the same file system; the kernel is just simply maping the file system to two different installation points. Note that when you install a file system on another, any file system installed inside the binding installation file system will move. In other words, if you have / usr on a separate file system, the binding installation we perform before we will make the / home / drobbins / nifty / usr is empty. You will need additional binding installation commands to enable you to browse / usr / usr / usr: # mount --bind / usr / us / drobbins / nifty / usr
Binding Installation Some file system binding installation makes more wonderful things. Suppose you have a TMPFS file system installed in its traditional location / dev / shm, you decide to start using TMPFS using TMPFS at the current root file system. Although a new TMPFS file system can be installed in / tmp (this is possible), you can also decide to let the new / TMP share the currently installed / dev / shm file system. However, although you can complete in / TMP binding installation / dev / shm, your / dev / shm also contains some directory you don't want to appear on / TMP. So, what do you do? In this way:
# MKDIR / DEV / SHM / TMP
# chmod 1777 / dev / shm / tmp
# mount --bind / dev / shm / tmp / tmp
In this example, we first created a / dev / shm / tmp directory, then give it 1777 permissions, appropriate license for / TMP. Since our directory is ready, we can install, and can only install / dev / shm / tmp to / tmp. So, although / TMP / FOO will map to / dev / shm / tmp / foo, you have no way to access the / dev / shm / bar file from / TMP.
As you can see, the binding installation is very powerful, so you can easily modify the file system design, don't have to be busy. Next article, we talk about Devfs. As for now, you may want to see the following reference.
With the release of Linux 2.4, a large number of file system probability, including REISERFS, XFS, GFS, and other file systems. These file systems have sounded very cool, but what do they really do, what are they doing, and how can they use them safely in the Linux product environment? Daniel Robbins answered the above questions by setting up these new advanced file systems in a Linux 2.4 environment. Compliance with this method, he provides valuable suggestions, performance information, and important technical attention in the actual implementation process to facilitate as possible to experience as much as possible in new file systems. In this article: Daniel explains the meaning and benefits of using the device management file system DEVFS, allowing you to know how to display you in the next article to install DEVFS.
Introduction DEVFS
The equipment, everywhere, is also a device file system (Device FileSystem), which is designed to provide a new (more rational) managing all block devices and character devices typically located / dev. You may know that typical / dev trees contain hundreds of special files and character special files, which are all on the root file system. Each special document allows the user space process to easily interact with the kernel device. For example, by performing operations on these special files, your X server can access video hardware, FSCK can execute a file system check, and the LPD can send data to the printer via the parallel port. In fact, usually Linux and UNIX more "cool" are not simply hidden in the obscured API, but is truly as ordinary files, directory, and symbolic links on the file system. Because characters and block devices are mapped to ordinary file system namespace, we can usually use a meaningful way to interact with hardware, and you can use only standard UNIX commands such as CAT and DD. In addition to interesting, this also makes us more power and improve productivity.
Equipment management problem However, although the equipment special document itself is a good thing, typical Linux systems manage these special documents in a less ideal and troublesome manner. Today, Linux supports many different kinds of hardware. This means that in strict sense, there are hundreds of special documents in / dev to represent all of these devices. Not allowed, most of these special documents will not be mapped to devices in the system (but they need them exist, just considering that we will eventually add new hardware / drives in the system), which makes things more Confusing.
From this point of view, we know that / dev needs to be completely repaired, and the clear purpose of creating DEVFs is to let / dev back back. In order to understand how DEVFS solves the vast majority of / dev management issues, we will see DEVFS from the perspective of the device driver.
The equipment management insider is better understood by DEVFS, it is best to understand how DEVFS changes from the perspective of the device driver. Traditionally (not using DEVFS), according to whether or not to register, the kernel-based device driver is registered with the rest of the system_blkdev () or register_chrdev () or register_chrdev ().
You must provide a master number (a unsigned 8-bit integer) as a parameter of register_blkdev () or register_chrdev (); then, after the device is registered, the kernel will know that this particular primary device number corresponds to the execution of Register_ ??? DEV () Call the specific device driver.
Then, what should the device driver developers should call the main device number provided by the register _ ??? dev ()? If the developer is not intended to share the device driver with the outside world, what is the number, as long as it does not conflict with other main device numbers used in the current kernel. Developers can also choose to dynamically allocate the main device number of the device called the Register _ ??? dev (). However, such a solution is usually only possible when the driver is not used by other people.
Get the number, however, if the developer wants to share the driver with the outside world (most Linux developers often use this method), then just draw a primary device number in "Vacuum" or use the dynamic main device number assignment. . Instead, developers must contact Linux kernel developers so that he (her) specific devices can allocate a "official" master number. Then, in the world's world, this particular device (only this device) will be associated with that specific primary device number. There is a "formal" master number to be important because the administrator must create a special file in / dev with a specific device. When the device node (special file) is created, the main device number it uses must be exactly the same as the internal core. In this way, the process will know what device driver should be referenced when performing operations on the device. Maps that make special documents to the kernel driver becomes the main device number, not the real device name (it is independent of the non-DEVFS system).
Once the device driver has a formal master number, the device can be published, the device node can start incorporated into different distribution / dev trees, and their formal / dev / makedev scripts (used to help superusers Use the correct master from device number, permissions, and ownership to create device nodes.
The traditional problem is unfortunately, this method has many scalability issues. Not only the equipment driver developers contact the kernel developers to get the official main device number is an annoying thing, but the kernel developers find out how they allocate all of these primary device numbers even more annoying. This task is very interesting in many ways very like system administrators tracking company LAN static IP addresses - this is not very interesting. As system administrators can use DHCP to alleviate this management burden, if the device registers some similar approach is good.
Not just this, Linux is still exhausted by the main device number and the secondary number. Although this problem can be used to simply expand the number of bits used by the primary device number and the secondary code, the first maintenance of these primary device numbers is very annoying, so we are considering that there is no better way to handle these things. Fortunately, there is such a method; enter DEVFS.
Enter DEVFS
DEVFS_REGISTER () This is a simple and clear quick outline for DEVFS how to process things and solve these problems. Once DEVFS is configured correctly (including the Nuclear Add DEVFS support and some slightly complex changes to the startup script), the super user restarts the system. The kernel then starts to start, and the device driver starts to register the device to the remainder of the system. You will remember that in a non-DEVFS system, register_blkdev () and register_chrdev () calls (along with the main device number provided) are for this purpose. However, DEVFS is now enabled, and the device driver is used in a new, improved kernel call to register devices, called devfs_register ().
Here is DEVFS_REGISTER () call interesting place. Although the main device number and the secondary code specified for compatibility purposes are possible, it is no longer necessary. Instead, devfs_register () calls the acceptance device path (that is, it may appear in the form of / dev) as a parameter. For example, it is assumed that the Foo device driver wants to use the DEVFS registration device. It provides a FOO0 parameter to devfs_register (), telling the kernel to create a new FOO0 device at the root directory of the DEVFS namespace. Correspondingly, devfs_register () adds the FOO0 device node to the root directory of the DEVFS namespace and records this new FOO0 node to map to the Foo device driver in the kernel. Run DEVFS Once all device drivers start and register the appropriate device to the kernel, the kernel starts to start / sbin / init and system initialization script. At the beginning of the startup process (before the file system check), the RC script installs the DEVFS file system in / dev, / DEV contains the expression of the DEVFS namespace. This means that after installing / dev, all registered devices (such as / dev / foO0 above) can be accessed, just as in non-DEVFS. When they are accessed, the kernel maps to the appropriate device driver through the DEVFS device name, not through the primary device number.
The advantage of this system is that all required equipment nodes (no other) are automatically created by the kernel. This is not only a no longer need MakeDev (because all registered devices only "appear" in / dev), it also means that / dev is no longer flooded by hundreds of "useless" device nodes. In fact, using devfs, you can know what device on the system as long as you want to view / dev. So, if you have a laptop that supports hot-swappable, this means you can even make the equipment appear and disappear from / dev when you can insert and pull out the PC card. This makes DEVFS a very thorough and practical solution for the previous clumsy situation.
DEVFS's Advantage DEVFS makes many things to be much easier. Consider the issue of creating a Linux bootable CD, which includes a boot loader located on the CD, an initrd, a kernel, and a return file system. When the CD boot, the boot loader loads the kernel and initrd, then the kernel performs the / LinuxRC script on initrd. / LinuxRC's main task is to install the CD, so that the return file system itself can also be installed and accessed.
Without DEVFS, LinuxRC requires many special files in "View" / dev, which may have no real hardware that is connected to the system. For example, LinuxRC requires detection / dev / hdc, / dev / SCD0, / DEV / HDB, and other devices to detect "active" optical drive devices. In the detection process, you are likely to hit a few "useless" device nodes.
However, using DEVFS, LinuxRC is only available in / dev / cdroms, which contains special files associated with all and active disc drives in the system, whether it is IDE or SCSI. Due to this convenient new DEVFS, no need to guess; only active devices will be listed, and the device detects that the code does not have to worry about the details of the bottom of the optical drive, such as what IDE channel or what SCSI ID is used. In fact, this is another main benefit of DEVFS; in my next article, we will see that the equipment in DEVFS / DEV has a new default location.
In fact, if you want to access a specific block device (such as disk, partition, optical drive, etc.), in fact there are several different special files can be referenced. For example, my server has only one SCSI optical drive; if DEVFS is enabled, I can access it by installing / dev / cdroms / cdrom0 or / dev / scsi / host / bus0 / target4 / lun0 / cd. Both are referenced by the same device, I can quote the most convenient special document I think. If you prefer, I can also use an old-fashioned device name (/ dev / sr0) to access the disc drive, because there is a very convenient applet called devfsd. DEVFSD is a multi-functional program that is responsible for creating old-fashioned "compatibility" special files, allowing you to customize / dev in many ways. In my next article, we will discuss DEVFSD in detail, and I will always guide you to start DEVFS and run it on your own system. Before that, please refer to the reference information below for more information on DEVFS. Reference
The NameSys page is more about Reiserfs. To understand the current more in-depth Reiserfs information, the Reiserfs mailing list is an excellent resource. Also, you must see the Reiserfs Mail List Archive. You can discuss in-depth discussion of the difference between UFS, EXT2, Reiserfs, and more file systems in Juan I. Santos Florido's Linux Gazette log file system. JEDI's Reiserfs / Qmail Tuning Page contains many information about QMAIL users. Also view Reisersmtp - JEDI's Drop-in QMAIL component set, which provides a striking QMAIL performance. Linux Weekly News is a very good reference to the latest kernel development. Please read the JFS overview of Steve Best on developerWorks. Please read the JFS base tutorial on developerWorks.
The Namesys web page is wherever understand the REISERFS. Note Access DICE download pages, pay attention to the latest Reiserfs big patch. The REISERFS mailing list is an excellent source of information that can get current, more in-depth REISERFS information. Be sure to view the Reiserfs Mail list profile. Linux weekly news is a good reference to our timely understanding of the latest kernel development. You can find some differences in the metadata between UFS, EXT2, REISERFS in Linux Gazette, on Linux Gazette, and a very detailed information on the metadata between ufs, ext2, reiserfs, and other content. JEDI's REISERFS / QMAIL adjustment page contains many information useful for qmail users. Be sure to view the numerous QMAIL components collected by Reisersmtp - JEDI, which significantly enhances the performance of QMAIL. Take a look at the MUTT email customer, it can be used to read a large mailbox in the large maildir form. Read Linus Torvalds Recently About Dump and Restore reviews. Participate in the free tutorial to compile the Linux kernel on developerWorks. Browse more Linux reference information on developerWorks. Browse more open source reference information on developerWorks.
Read the Daniel this series, he introduced the benefits of creating logs and using Reiserfs, and showing how to install a solid Linux 2.4-based Reiserfs system. Linux Weekly News is a good reference to maintain synchronization with the latest kernel development. Util-Linux (Links) collects a variety of important Linux applications, including Mount and Unmount. You may want to upgrade to the latest available version so you can use the mount --bind syntax (instead of using mount -o bind). Because TMPFS and binding installations are relatively new, most of them are no new features with document description, learning their best ways is to learn the relevant part of Linux kernel source code. The Namesys page is to learn more about Reiserfs. Reiserfs Mail List is a well-informed of the current REISERFS information. You must also look at the Reiserfs mail list archive. In Juan I. Santos Florido's Linux Gazette Journal File Systems review, you can find a deeper explanation of metadata differences between UFS, EXT2, and Reiserfs and other content. JEDI's Reiserfs / QMAIL Tuning page contains many information on QMAIL users. You must also look at REISERSMTP, JEDI collected a lot of QMAIL components that provide powerful QMAIL performance. Read the JFS overview of Steve Best on developerWorks. Participate in the Free JFS basic tutorial for Daniel on developerWorks. Browse more Linux reference information on developerWorks. Browse more open source resources on DeveloperWorks. O'Reilly's Linux Device Drivers, 2nd Edition usually for those who want to learn more about equipment registration and Linux device driver, is indeed good books and reference materials. Be sure to read the Linux devfs FAQ of Richard Gooch (founders of Linux devfs). It is complete, detailed and latest. What can you still ask? There is a DEVFS mailing list to be used. If you want to subscribe, email to Majordomo@oss.sgi.com and indicate Subscribe in the message body section. The DEVFS list file can be obtained at http://oss.sgi.com/projects/devfs/archive/. You may also want to access the home page of Richard Goo; it contains devfs and some other good things. Linux Weekly News is a good reference to maintain synchronization with the latest kernel development.
