This article starts from the perspective of the basic principles of the database, through the analysis of the Exchange Server Store module, reveals the working principle and maintenance techniques of the Exchange Server mail storage system. Article is suitable for professional IT staff with a certain Exchange Server management experience to make readers to know how to know when maintaining the Exchange Server mail system. The hierarchical relationship between the Information Store and Extensible Storage Engine is well known that in Exchange Server, the Information Store (IS) Service is critical to the Information SERVER. This service controls the operational request for the mailbox and public folder database. In more step further, in fact, Exchange Server's database system is managed by a database engine named Extensible Storage Engine (ESE). This ESE engine is Microsoft developed to save non-relational data. There are applications in Many systems in Microsoft: For example, AD database (NTDS.DIT file), Windows DHCP, Windows Wins, SRS, etc., background is Supported by the ESE database. We know that Exchange Server's database consists of an EDB file, a STM file, and a large number of log files. Inside these files, Microsoft uses one of the internal data structures named "B Tree". One of the tasks of the ESE engine is to translate these requests to read and write access to internal data structures when the INFORMATION Store service requests access to the database. . The B tree is characterized by providing fast access capabilities to data stored on the disk. Microsoft uses B Tree as a reason for the ESE background structure, which is to improve the I / O performance when accessing data as possible. The structure of these B trees is transparent to the Exchange Server Store service, and the Store only needs to send the request to the ESE, and ESE will operate the data structure. In addition, as a database system, ESE is responsible for providing support for transaction-level operation, and maintains the integrity and consistency of the entire database. For modern database systems, when we refer to transactions, generally use the abbreviations such as ACID to describe the characteristics of the transaction: We will discuss how Exchange Server and ESE implement the above requirements in the later spaces. For Information Store Service, the ESE encapsulates all details of the database operation, and IS can be called as long as the interface provided by the ESE can be called. In Exchange Server 2000, the process corresponding to the IS service is Store.exe, each Storage Group generates an instance of an ESE engine in the Store.exe process. The new feature of the Exchange Server 2000/2003 storage system is updated and improved when Microsoft publishes Exchange Server 2000.
From the perspective of the ESE engine, the ESE version is upgraded to ESE98 from the ESE97 in 5.5, and has been improved in the following aspects: 1.I / O performance is further optimized and improved 2. Add the calculation check for the log file. And the operation, further reducing the possibility of the database error 3. Improve the speed of the maintenance tool such as Eseutil, more attractive, for example: 1. Provide more than each server STORAGE Group and Store support, this is one of the biggest features of 5.5. 2. Introduction of STM stream file format in the database, improves the performance of the Internet mail 3.Web Storage System, users can use multiple protocols Accessing Database EDB files and STM files In Exchange Server 5.5, the database only extension is an edb file. When Exchange Server 5.5 is released, Microsoft's focus is a mail transmission system within an enterprise. At that time, the protocol of the main push is the MAPI protocol, which is Microsoft's private mail protocol, and the Database for EDB format makes specialized optimizations for this protocol. Therefore, Exchange Server 5.5 must support the Idb-enabled format that converts it to EDB each time you process the Internet message, which makes it possible to recognize the huge performance loss. In Exchange Server 2000, Microsoft has increased support for the Internet standard protocol SMTP. Therefore, storage for the Internet format mail is born: this is the STM file. The MAPI format is based on Microsoft's RPC and binary standards, and the Internet format is made of plain text messages and MIME-encoded character streams. The characteristics of the two determine that they cannot share a file of a database structure. Therefore, in Exchange Server 2000, Microsoft uses the EDB files and STM files, respectively, and establishes associations and references between EDB and STM files. For the user, his mailbox content is actually constructed from the contents of the EDB and STM files. It is worth mentioning that in addition to the actual letters information, the EDB file also saves the mailbox structure of each user, the content list and view of each folder. This is where only the STM is only saved in the word stream. We discuss the use of EDB and STM files in several situations: 1. Use Outlook uses Outlook Use the MAPI protocol and send and access mail 2. Users use the SMTP / POP3 and other Internet protocols to access Exchange Server. Scenario 1: When the message is submitted from the client of the MAPI protocol (usually Outlook in Microsoft Office), the message content is saved in the EDB file. When the user reads access to the message in the mailbox via the MAPI protocol, if the requested message is saved in the EDB file, then the letter is turned on directly to the user. If the requested letters are saved in the STM file (this letter is SMTP format), then the Exchange Server database engine will first make a conversion, convert the data format in the STM file into the format that can be identified, and then send it to Client. This process is called "On-Demand Conversion". Second: Users use SMTP / POP3 clients (such as Outlook Express, Foxmail, etc.) to connect with the mailbox. When the SMTP protocol submits the message to the Exchange Server, the contents of the message are saved in the STM file.
As mentioned earlier, the EDB file contains a list of folders and letters contents of the user's mailbox. Therefore, after the message is saved to the STM file, the database engine is of some important information (usually in the email header). The location of the letter in the STM file is extracted, saved to the EDB file, this process is called "Property PROMOTION". There is this process, and the user can get a complete list of mailbox content, and the MAPI client needs to access the message located in the STM file, thereby obtaining the correct storage location of the letter in the STM file. When the user uses the POP3 protocol to read the message, if the visited message is in the EDB file, a transformation from the MAPI to the Internet will also occur quietly in the background. Through the above description, we know that these two files are closely associated in the actual Exchange Server environment. Don't operate these two files at all times, you must always treat them as a whole. The EDB file contains the content list of each mailbox. When the client needs to get the content of the folder, it must be issued to the EDB file. Two format files provide support for both types of protocols, which effectively reduces unnecessary format conversion. The role of the log file We discuss the email storage of Exchange Server, you have to talk about its log files. I have listened to the administrator who heard Exchange Server complained: Days to documents are mad every day, too hard to consume the hard disk space. Let's take a look at what these log files do. For each Storage Group, Exchange Server produces a series of log files with it. The size of these log files is 5m, the extension is log, their prefix is E0X, where x is the number of Storage Group corresponding to the log file [Footnotes: Although there is "log file prefix" in the Storage Group attribute Box, but actually this is not changed. ]. So the first Storage Group's log file prefix is E00, the second is E01, and push it accordingly. The purpose of this is to avoid the administrator "Zhang Guan Li Dai" when the administrator is maintained when there is a number of Storage Group. In addition, in addition to the continuous log file, we can also see files such as E0x.chk, res1.log, res2.log. Many administrators have a headache for log files, then what is Microsoft introduced to the Log file in Exchange Server's database system? We will see from the following aspects: 1. As an enterprise-class mail database system, we must make data security and integrity. Must be able to face the crashes and downtime that may occur at any time, what happens if we crash? The loss of data is reduced to the latest level. 2. Must provide high-performance mail throughput, which must be recorded on the storage medium immediately after the transaction of the mail in the database is completed (persistence of the transaction). 3. When the disaster occurs, the backup recovery of the database must return to the database status before the disaster occurred. Now let's take a further look at that when I want to modify the content in the mailbox, the modified content is first read out in memory. Actual modifications have occurred in memory. When the modification is complete, these content must be written back to the storage medium to indicate that a modification is successfully completed. For such modifications, at the database level, we are called a "transaction". We know that in order to ensure the integrity and consistency of the database, the operation of the transaction is "atomic level".
If a transaction is successful, it marks that the change he has made is preserved; if a transaction fails, the system must return to the state before the transaction begins. When the system is completed in memory, the transaction is not completed. If this time is downtime, saved in the database still has no changes. So how do you make sure the modifications completed in memory be written to the database in the first time (to reach the durability of database transaction)? Note that if it is the first time, the sooner the better. If we write directly to the EDB file, we cannot do the fastest, because the EDB file is usually very large, the I / O system will take a lot of time when the big file is randomly written, and the time takes a lot of time waiting to find it. Suitable tracks and sectors, this will be a bottleneck when the system is busy. Therefore, the database system uses a log file. When the changes in the memory are complete, first write to the log file. The log file is small, and the write performance is far better than the huge EDB file. After the write is completed, the transaction also saves successfully on the storage medium. Exchange Server's database engine writes the contents of the log file into the database in the background, because the transaction operation has been completed, even if it is powered down or downtime, the completed transaction is lost. This is the first role of log files: ensuring that transactions can be saved to non-volatile storage media in the first time. (Provide Persistent Durable Support) According to the above description, we know that the Exchange Server database in the run is composed of three parts - memory has been modified, but there is no content written to the log file (Dirt Page). - There is no log file content written to the database file. --EDB and STM files. For data (Dirt Page), this data is lost when the system is powered down or crashed. Exchange Server uses a file called E0x.chk (Check Point) to record those log files have been written to the database file. This is a record of similar pointers. We can use the command eSeutil / MK to view the contents of this file.
C: /.../ Exchsrvr / bin> Eseutil / mk "c: /" EXCHSRVR / MDBDATA / E00.CHK "Microsoft (R) Exchange Server (TM) Database Utilities Version 6.0 CopyRight (c) Microsoft Corporation 1991 . -2000 All Rights Reserved Initiating FILE DUMP mode ... Checkpoint file:. C: / program files / exchsrvr / mdbdata / e00.chk LastFullBackupCheckpoint: (0x0,0,0) Checkpoint: (0x8,26DA, 30) FullBackup: (0x0, 0) Fullbackup Time: 00/00/1900 00:00:00 IncBackup: (0x0, 0) IncBackup Time: 00/00/1900 00:00:00 Signature: Create Time: 03/28 / 2004 20:26:10 Rand: 6519986 Computer: ENV (Circlog, Session, OpenTBL, VERPAGE, CURSORS, LOGBUFS, LOGFILE, BUFFERS) (Off, 202, 10100, 1365, 10100, 128, 10240, 40828) Operation Completed SuccessFully In 1.47 Seconds. In the output of the command, CheckPoint: <0x8, 26da, 30> indicates the LOG full position currently submitted to the database file. Where 0x8 is the serial number of the log file, generally correspond to E0x00008.log, and the remaining two parameters are the number of the Log file internal page (Page).
Let's take a look at the role of the log file on the system backup and recovery.
As mentioned earlier, Exchange Server requires recovery to a state before the disaster occurs after the disaster. For a general system, we always backed up every week, then how to protect the data before the backup and the disaster occurred? The answer is a log file. We know that for any changes to the database, we will first write to the log file, and then update by the log file to the database. We now assume that there is such a system, back up 3:00 AM daily, after the backup is complete, the system is working properly. If the system has failed at 12:00 at noon, the administrator recovered the system with a tape of 3:00 AM, then data from 3:00 AM to 12:00 AM, will be filled by the log file. The specific case is that when the backup recovery of 3: 00 AM is complete, Exchange Server automatically scans the log folder associated with this Store. If you find that there is a new log, Exchange Server will automatically put these logs. Write in order to the database. Therefore, the changes made by the database from 3:00 AM to 12:00 AM can be recovered back. This is the second important role of log files. (The premise is not to turn on the loop log function)
The second role of the log: guarantee the integrity of the system backup and recovery. Of course, the premise is that there is no cycle log! ! (See it, the harm of using the loop log is quite big, compared to your data, do you do more backups don't make meaning?
Some people may ask, what should I do if the database file and log file are damaged? The answer is this: Avoid this happening. First, the probability of damaged database file is far greater than log files, and Microsoft recommended is to place database files and log files on different disks, respectively. We will focus on discussing this problem in the next article. The complaints of the administrator for log files are constantly growing every day, and consume hard disk space. For this problem, the only reasonable solution is: Regularly do full backup or incremental backup for Storage Group. Because Exchange Server is removed after the full backup or incremental backup is complete, the Log file generated before this backup is removed. Many administrators manually delete log files, or launch the "loop log" to reduce the consumption of the hard disk space, which is incorrect. The incomplete log file will not restore the recent state when the system is restored. If your system is a full backup, you happen to delete some log files after backup, then you may lose data after your backup when you need to recover. Remember, data is always more precious than disk space.
The ESE database engine and the startup and shutdown of the INFORMATION Store service
When the ESE engine loads a database file, it checks a special flag of the database file. This logo saves whether the database file is last properly closed. This status is represented by "consistent" or "inconsistent". For a normal shutdown database file, all content in the log file and memory should have been submitted to the database file, only at this time, the database is marked as "consistent". One thing to pay attention to, in the running database, its status must be "inconsistent", because there is definitely available to the database file content in the log file. For a database that has been closed and the state is labeled "Inconstent" does not mean that this database library file is corrupted, "inconsistent" is only indicated, and there is no content that has not been written to the database file in the log file.
Use the command ESEUTIL / MH to look at the shutdown state of the database.
C: /.../ Exchsrvr / bin> Eseutil / MH "C: /" EXCHSRVR / MDBDATA / Priv1.edb "Microsoft (R) Exchange Server (TM) Database Utilities Version 6.0 Copyright (c) Microsoft Corporation 1991 . -2000 All Rights Reserved Initiating FILE DUMP mode ... Database:. C: / program files / exchsrvr / mdbdata / priv1.edbFile Type: DatabaseFormat ulMagic: 0x89abcdefEngine ulMagic: 0x89abcdefFormat ulVersion: 0x620,9Engine ulVersion: 0x620,9Created ulVersion: 0x620, 9db Signature: Create Time: 03/28/2004 20:26:24 Rand: 6536656 Computer: CBDBPAGE: 4096DBTIME: 63139 (0-63139) State: Clean shutdown <----- Indicates the status log when the database is closed Required: 0-0Streaming File: Yeshadowed: Yeslast Objid: 574 ... <> ... Operation Completed Success In 1.391 Seconds.
"Clean Shutdown" of the State field indicates that the database is at the consistent state. When the ESE loads the database file, for the "consistent" database file, it directly mount's Store; for the "in constent" database file, ESE will perform the process called "Soft Recovery", in which no The log content submitted to the database file in time will be written to the database. When all logs are written, the database is marked as "consistent" status, and then load it normally.
When Soft Recovery starts, the ESE performs writing of the log file according to the location pointed to by the check point file (if the check point file is damaged or does not exist, the database starts from the oldest log file). When eSe writes data from the log file to the Store, it decides whether to write the log file to the database according to the timestamp of the DBTIME.
In this process, there will be the following records in the Event log.
Event Type: Information Event Source: Ese98 Event Category: Logging and Recovery Event ID: 301 Date: 10/17/2001 Time: 5:52:11 AM User: N / A Computer: Information Store (xxxx) The Database Engine Has Begun replaming logfile ..log.
We can also perform "Soft Recovery" for "Dis-Mount" and is in "Inconsistent".
The specific command is "eseutil / r", followed by the path of the database file. (Recommended this command after power-down reboot, you can run Eseutil / MH to determine the database status, if it is "inconsistent", then execute this command)
From this we can find that Exchange Server has the ability to "self-fix" on a database that is not closed. Therefore, the ESE ensures that the database can still be restored even if it is suddenly powered down, and the status detection and recovery are automatically completed after the restart service.
M disk of the dragon
When Exchange Server 2000 is released, Microsoft proposes the concept of "Web Storage System", whose core is to provide a variety of ways to access the Exchange Server database. These routes include
File system / IFS
--HTTP WebDAV
--Exoledb / ado
--CDO
Among them, IFS technology to provide file system services is a module that causes more disputes. After installing Exchange Server 2000, a M disk will appear. This M disk is a mapping of a database to a file system implemented by Microsoft through the IFS (Installable File System) technology. Developers can access the Exchange Server email and email through standard file operations API (such as CREATEFILE, OpenFile, etc.).
Open your M disk, you can see a folder named after you currently domain. Add to this file, you will see a folder containing all mailboxes, named MBX. MBX is under the mailbox folder named after the user's name. You can see the contents of Inbox, Outbox, etc. below each folder. Each letter is expressed by a file named EML. Exifs uses a special shared name called //./backoffICESTORGE to point to the database file. You can run "Dir //BackofficeStorage/Domain.con/mbx" in the command line, the implementation results of this command are the same as the disk.
We can change the drive letter mapped by Exchange Server by modifying the registry.
HLKM / SYSTEM / CURRENTCONTROLSET / SERVICES / EXIFS / ParametersName: DriveletterData Type: REG_SZVALUE: Drive Letter for IFS (Drive, no need to follow the colon)
After changing the registry, you need to restart the Information Store Service to make the changes take effect.
We can also use the following command line tool to change the mapping of the M disk:
Subst X: //./back oFICESTORAGE Note: Map the Exchange Store to X disk Subst / D m: Note: Delete mapping to the M disk
If we remove the M disk, we can also access the Exchange Server database via //./backofficestorage this shared name.
EXIFS is running as a hidden service in Windows. The following registry key value defines the parameters of this service:
HLKM / SYSTEM / CURRENTCONTROLSET / SERVICES / EXIFS / PARAMETERS
Since this is a hidden service, we have no way to control this service through the Service Control Panel. But we can do it through the command line:
Net Start EXIFS Note: Start service NET STOP EXIFS Note: Stop service
The following picture shows the architecture of Exifs.
EXIFS is implemented using an Exifs.sys driver running in a Windows kernel mode.
We know that the file system and Exchange Server's Store is two completely different architectures. The files in the file system contain only relatively few properties, and the messages saved in the Store have their specific properties, and in the Store, there is a very complex relationship between the mail (relationship with the mailbox, mailbox file) Clip view, etc.). Therefore, the file (mail) existing in the form of EML is reflected in a subset of all the personal properties and relationships of the message. Some of the inappropriate operation of the M disk often disrupt the relationship within the database, causing database damage. A more typical example is that anti-virus software scans the M disk and discovers "suspect virus" and clear it. According to the statistics of the Microsoft Technical Support, this is one of the main reasons for the damage of the Exchange Server Store database. Because anti-virus software is cleared by the virus file (EML file), take "barbaric construction" means, often destroy the association and mail structure inside the database, thereby causing damage to the internal structure of the database file.
