Microsoft software test method
Jeff Wjoined: 19 Dec 2002 Total Posts: 91
Microsoft's software test method pos: 19 Nov 2004 02:36 PMPOSTED: 19 Nov 2004 02:36 PM Domestic recent issues on software testing and discussion is increasingly active. However, from the general speaking, the exchange software test technology is generally discussed less. The "technology" here refers to specific tactical problems, such as how to use some kind of tool to solve a particular test problem, or what test methods for a type of software, etc. And the "method" here refers to a macro strategic issue, or methodology, which includes the concept or concept of software testing, to the enterprise software quality control system; from the process of software testing, the setting of the test team and its responsibilities Definition, etc. As a tester, it is passionate about "technology" discussion and communication is a grateful thing. It can feel the beginning and potential of software testing in China. But as the management decision maker of the company, is it necessary to think about the "method" problem with the same enthusiasm? Especially when a software company's software test is not available, or when the company has certain software testing, it is found that it is not significant, even due to the introduction of the test, it has brought new management confusion. This time the methodology thinks is more conducive to the foundation of the problem. Even a base layer tester, after accumulating a certain technical experience, it should also come out from the daily specific work. Review summary and reference at a higher level, and try to propose some optimization and improvement. Measures, this is very meaningful for this regardless of professional growth. Microsoft has a lot of experience in software testing, here I want to take some discussions with my personal experience and thinking. Here, there is a special statement, although Microsoft's approach has been successful in the practice of Microsoft, it is very effective, but this does not mean that these methods have a wide range of feasibility in China's software companies. Whether a method is feasible to be affected by many other factors, such as enterprise types (Microsoft is a production platform software and universal software products), corporate management system, corporate culture, etc. So my purpose is just some ideas and reference. Two types of classic software test methods before describing Microsoft's software testing methods, I would like to understand what is software testing from the concept, or the idea level, and the purpose is to export the theoretical root cause of Microsoft test methods. Traditionally, the method of software testing is considered to be two categories. The first type of test method is to try to verify the software "work", so-called "work" means that the software function is implemented in advance; the second type of test method is to try to prove software is "not working" . Representatives of the first type of approach are the pioneer Dr. Bill Hetzel in the field of software testing ("The Complete Guide to Software Testing), he organized the first in the United States in Northern Carolina University in the United States in June 1972. Subject to the forum for software testing. He first tests a such definition in 1973: "It is to establish a confidence that the program can operate on the expected idea." That IT IS supposed to do. "Later in 1983 he will Definition revision is: "Evaluate the characteristics or capabilities of a program and system and determine if it reaches the expected result. Software test is any behavior of this. Any Activities AIMED AT EVALUATING An Attribute or Capability of a Program OR System. "In his definition" ideas "and" expected results "are actually the user needs or functional design we are now mentioned. He also defined the quality of the software as "conformity".
The first type of test can simply describe such a process: running the software in the design specified environment, comparing its results with user needs or design results, if the test is passed, if not, it is considered to be bugs. . The ultimate goal of this process is to run all of the functions of the software in all design specified, and pass. The first type of method is generally mainstream and industry standards in the software industry. The IEEE / ANSI standard in 1990 made such a definition of software testing: "Under the established condition conditions, run a system or formation, observe the results, and evaluate some aspects .the process of operation a system or component under specified conditions, observing or recording the results, and making an evaluation of some aspect of the system or component (IEEE / ANSI, 1990 [Std 610.12-1990] "here, the term" predetermined condition "is also understood to Demand or design. Despite this, this method is still questionable and challenged by many autonomous authority. Representatives are Glenford J. MYERS (representative "The Art of Software Testing"). He believes that test should not focus on verification software is working In the opposite should first determine that the software is wrong, then discovers as much error as possible. He also argued as possible from the perspective of human psychology, "verifying software is working" as the purpose of testing, very bad for testers Find the software's error. So he proposed his definition of software testing in 1979: "It is the process of running the program with the purpose of discovering the error .The process of executing a program or system with the intent of findinger." This is The second type of software test is simply that verification software is "not working", or there is a mistake. He even extremely believed that a successful test must be a test of Bug, otherwise it is worthless. This is like a patient (assuming that this person is not sick), go to the hospital to do a medical examination, and the various indicators are normal, that the medical examination is not worth the patient's condition, it is failed. I don't fully agree with this view. The second type of software test method is also very popular in the industry, received by many academic experts. Everyone is familiar with Ron Patton in "Software Test" (Chinese version is published by Machinery Industry Press, It is said that this book is currently a classic textbook for the newcomer. The book is a book, there is a clear and simple definition: "Software tester is to find software defects, as early as possible, and ensure that it is fixed. "Some software companies use bugs as an indicator for assessment test staff, in fact, this is the way. The advantages and disadvantages of the two types of methods Although the total purpose of software test is for the quality of software products, it is obvious These two types of test methods are truly opposite to specific goals or guiding ideas. This also determined that they had a lot of differences in ideas, process and priorities, and their aphasic. The first type of test method is based on demand and design, so it is conducive to defining the category of testing, and facilitates the side focus of deployment tests, and strengthens targeted. This is especially important for large software testing, especially in limited time and human resources. The second type of test method is not necessarily related to demand and design. If the planned management is not properly managed, the test activity is easy to lose its focus, and go to astray.
The first type of test method can cooperate with the planning of the software architecture and software development, so that software test activities will be launched one by one, so that software function and quality have planned gradually improve and improve (on the level of testing, I It will be specifically introduced in the future discussion). The second type of test method does not have the graduality of this process. The disadvantage of the first type of test method is the lack of flexibility, which is not conducive to the subjective initiative of the test personnel. It is like Mr. MYERS, which is not easy to find the error (bug). And this is the strength of the second type of test method. Microsoft's strategy is because of the various types of test methods, Microsoft combines two types of methods in software testing activities, based on the first type of test method and the main clues, and use second types of test methods. . Microsoft's first type of test Microsoft's first type of test is generally divided into three steps: audit requirements and design -> Design test -> Implement operation test. The foregoing test is the correctness of the software to verify the software with demand and design. Everyone is naturally, the demand and design itself also has the right problem. The test will also be futile based on incorrect demand and design. Therefore, verification requirements and design are the first step in Microsoft for the first type of test. It is necessary to point out that the needs and design described herein generally include: (1) demand text written by the project manager according to user requirements (information from market sectors, user support, etc.) (Requirement Specification ); (2) Functional Design Specification, which is written by the project manager; (3) Implementation Design Specification, which is written by the developer according to the function text. Microsoft's testers should participate in all of these text audits. As a tester, the audit focus is the detection of the integrity, rigid and functional design of the text on user needs. At the same time, this review is a warm-up activity for testers to enable them to enter technology and business. In the second step, the tester should design test cases based on the review and design of the reviewed demand and design. In the three text mentioned earlier, the functional design text is the main basis. The reason is very simple, this type of test is concerned about whether the software can implement the function correctly, not how these functions are implemented. From here you can see this is a typical "black box test." It is true that Microsoft's test is mainly black boxes from the perspective of the user. The completion of this step means the completion of the "test plan" and "test case design". The "Test Plan" text mainly describes the category, field, method, tool, resource and plan schedule, etc. of the test. "Test Case Design" text To list test cases, the settings, execution steps, and expected results of each use case. These two texts tested should also be reviewed by project managers and developers. In this way, through various mutual audits, everyone has formed a basic consensus on the project. The implementation of the operation of the third step is a longest and most complicated phase throughout the development process. Overall, the process of testing the previous design is implemented in accordance with the process of implementation. This includes writing an automated test program, repeatedly running the automated test programs, and also includes a phased execution manual test case. This phase of the test must be carried out under the careful plan, I have mentioned in front, this is the characteristics and strengths of the first type of test. This plan is first reflected in the development and testing of mutual coordination, according to the relying on the structure of the product, according to the overall plan of the project, according to the overall plan of the project. From the process of testing, you always run or perform simple use cases, then repeat the use case; first verify a single basic function, then the end-to-end function; first find the solution to the surface, the influential BUG, and then deep It is not easy to reproduce the bug.
Therefore, with the process of project development and testing, the function of the product is constantly improving, and the quality is constantly increasing. There is a little more specifically that there are many test cases to be repeated, especially basic automation tests, each build is running. Although most of these tests are passed, rarely discover new bugs, but their value is obvious, in order to prevent quality regression. It can be seen that MYERS is not applicable here. At this stage, the tester has a cumbersome but very important job, which is the maintenance of existing test cases. For example, in the following two cases, some test use cases are added. First, for the field of test design, the second is that the external bug (such as report from the Beta customer) is not covered by the existing test example. When the function design of the product changes (in Microsoft this is the common thing), the test case involved is of course modified accordingly. Microsoft's second type of test Microsoft's second type of test is phased, often with randomness and assault as needed. For such tests, there is a special name in Microsoft: "BUG Bash (BSH Sweeping)". BUG Bash usually happens at the end of the project development (Microsoft Milestone), such as the beta release, draw a special time period (usually 1-3 days), all participating in the project during this period, concentrated all energy To use all aspects of knowledge, do full wisdom to search the project's bug. This is a very interesting event, but it is not easy to organize such an activity. Generally there are the following points: (1) Although this is a test activity, participants are not limited to testers. Project managers, developers even managers should participate, as they mobilize them. The purpose is to brainstorm; (2) to encourage departments, field cross search, because new ideas and perspectives usually help find more bugs; (3) To mobilize enthusiasm, enhance fun, can introduce competition mechanisms, For example, when the activity is over, the discovery bug is discovered, and the most serious bug is discovered, gives substances and spiritual rewards. (4) You can expand, such as security, user interface availability, internationalization, and localization. Microsoft's second type of test In addition to BUG Bash, there are often some professional tests, most typical is for security attacks. Generally, the company is invited to search for the company to search for the safety vulnerability of the product. The above, from the perspective of traditional software test concept, I introduced Microsoft's strategy and the specific practices of two types of traditional test methods, and their focus. This is actually just a foundation, a very original basis. Software Tests The role of Microsoft software product development, is far from these original methods, nor is it a letter of traditional software test concepts. Microsoft has a lot of unique practices in software testing, and conceptual breakthroughs, such as "software test information service function", "user-centered macro quality system", "grading test", "quality management system" , "BUG trial", "test automation" and "software test hardware - sectors, teams, people and infrastructure", etc. These I will introduce them in the next discussion. Jeff Wang [MSFT] This post is provided with "status quo" and there is no guarantee, and there is no right to grant any rights.
BitterJoined: 06 May 2004 TOTAL POSTS: 30
Re: Microsoft's software test method posted: 29 nov 2004 04:37 PMPOSTED: 29 Nov 2004 04:37 PM long experience, thanks to Jeff W. I want to ask an old topic, how do you decide when to end? For example, what you said, this method I have not practiced, it can also be confident that it is worth it. But how can I not pay too much? The project test resources I have experienced are limited (time, month), so Generally, it is naturally ended when the resource is exhausted, and of course, the coverage rate, risk, discovery of the number of bugs (less and less), but mainly in the main reason to stop testing. I don't know what experiences and methods do Microsoft doing on this issue? Http://groups.yahoo.com/group/china-testing
TL_GEONGJOINED: 20 AUG 2003 TOTAL POSTS: 251
Re: Microsoft software test method pos: 30 Nov 2004 01:38 Pmposted: 30 Nov 2004 01:38 PM First, I think this is a strategic problem. In general, most companies are the first type of test strategy. The purpose is very simple, it is to meet the expectations of our customers. But on the other hand, customers' expectations may exceed the definition of demand, and the expectations of end users are often higher than customers (customer pay, end users are used) imagined high. The final user's question tends to bring pressure to customers, which affect customer satisfaction. For the angle of the quality of the product itself, we generally add a random test. Similar to the nature of bug cleaning. This random test sometimes is very good, sometimes there is no results, and sometimes it will find a serious divergence of defective trends that have converged. Since this random test may bring a lot of pressures to the project group (for example, there will be ideas after the customer see so many problems), so management and coupling pressure is a more difficult thing.
Shen Wei MSN: T628Shen@hotmail.com
BitterJoined: 06 May 2004 TOTAL POSTS: 30
Re: Microsoft's software test method pos: 30 Nov 2004 01:49 Pmposted: 30 Nov 2004 01:49 PM I think the difference between the random test and bug Bash is quite big.
http://groups.yahoo.com/group/china-testing
Jeff Wjoined: 19 Dec 2002 Total Posts: 91
Re: Microsoft's software test method Posted: 30 nov 2004 04:42 PMPOSTED: 30 NOV 2004 04:42 When is PM test? At the end of the day ended by the plan! I have listened to this answer. But this answer tells you the most basic principles that Microsoft basis, this is the plan. When I introduced Microsoft's first type of test, I mentioned "Test Plan", this "test plan" is actually an investment problem to answer the test, including human resources, time limits, and processes. Determine the test plan has such a few dependent: 1) The function of the product. The amount and complexity of the function directly affect the test workload; 2) Quality standards, the standards, the standards of the industry, market feedback standards and customer requirements, etc .; 3) experience, experience in the past products And there are also personal experience. This "test plan" is also approved by the parties (development, project management) of the project, thereby forming a consensus throughout the product department, which is finally included in the project overall plan. For the second type of test, it is also part of the overall plan of the total project, and it is also known. Generally speaking, there will be a few "bug Bash" in each milestone, and each time you lasted 1-3 days. There is always one day in Microsoft's project plan, called "test completion day Test Complete Day". It marks all the test activities of all programs have all been completed, all discovered bugs are all resolved and verified by test (there are some reasons to be decided to delay). For the above analysis, you may still be unsatisfactory: Is Microsoft's plan always completed on schedule? Of course, it will often be. Almost certainly, the actual implementation of the project will have more or less gaps with the pre-plan. Microsoft will take some methods in the project process to perceive this gap, such as the code "coverage" analysis of Bitter and the change trend of BUG number. The goal is to find gaps, reassess and revise the plan as soon as possible. This plan can be changed, but the test is always over the day of the plan. Jeff Wang [MSFT] This post is provided with "status quo" and there is no guarantee, and there is no right to grant any rights.
BitterJoined: 06 May 2004 TOTAL POSTS: 30
Re: Microsoft software test method pos: 01 dec 2004 11:35 amposted: 01 dec 2004 11:35 amjeff, thank you very much, you explain very clear. I hope to see your subsequent content soon. Your post is the most valuable content I have seen in the domestic forum. Your expression is also very appreciated.
http://groups.yahoo.com/group/china-testing
Jeff Wjoined: 19 Dec 2002 Total Posts: 91
Re: Microsoft's software test method pos: 01 dec 2004 04:18 pmposted: 01 DEC 2004 04:18 PM Thanks to Bitter's praise. The defective trend of convergence of the randomized test mentioned in TL_Geong has a serious divergence, and there is also in Microsoft. Usually bug Bash will generate an excellent quantity of bugs. Generally, we believe that the larger the amount of bugs is, the better. Because if the number of bugs is generated, it is difficult to judge because the quality of the product is really high, or bug Bash is doing it. And the fact is often the latter. So what should I do if a large number of bugs generated by BUG BASH? In Microsoft, we have the system of "Bug Triage (Test, Development and Project Management, Triple Association)". For each bug, after review, there is no more than the following three in the following three (generally): (1) confirmed as "defective" bug, such bugs must be resolved by developers, and then verified by the original discovery. (2) Adjusting to "defective" bug, do not use developers to make any changes, but must be incorporated into product user documents, clearly explain to users, and tell users how to avoid and respond. For example, the product will be temporarily stopped in the case of serious system memory, and there is a lot of warning messages that are not easy to be understood by the user. This is obviously a bug (according to Microsoft standard), the correct thing should be, first, the software should not completely stop working, followed by a number of warnings, third, warning information should be concise and easy to understand, and give users measures and suggestions. However, in consideration, this situation occurs very low when the user actually uses the product, and on the other hand, from the development angle, there is a large technical difficulty in solving this problem, and the impact is too large. In this case, this BUG will change this bug to "Text" BUG, that is, the text is required to make this situation to make this situation, and it is recommended that users do not use this product with other software that consumes a lot of memory. This kind of bug is very common in bug Bash, because everyone is more often super in this test activity. (3) It is completely negative, immediately shut down, no longer entanglement. This kind of situation is also common in bug bash. Because people involved in Bash don't know the accurate usage of product functions, it is inevitable. Although there is no direct follow-up measures for such issues, this information is still a certain value, because the newcomers in the future users can make the same problem, and if the product support department can be in time, it will be accurate in time Provide help. So this information is saved in the BUG's management library for future product support department queries. After such a review, screening, if (1) (2) class bug, especially (1) class bug still, the test department is likely to reject the original test plan and test case design, see if it is necessary to add test cases. . The overall plan and release date will be made as soon as possible. The appearance of a large number of bugs may not be a pleasant thing, but compared to the user, the cost is too much. In short, the BUG of the product is to be like the body's disease. Jeff Wang [MSFT] This post is provided with "status quo" and there is no guarantee, and there is no right to grant any rights.
TL_GEONGJOINED: 20 AUG 2003 TOTAL POSTS: 251
Re: Microsoft software test method pos: 01 dec 2004 05:26 PMPOSTED: 01 DEC 2004 05:26 PM I used product class test for wireless products, now do J2EE project class test. In the product type test, almost all the scheduled time after the completion of the schedule can be used to find a defect, so there is no statement of bug cleaning. But the test of the project class is not the same. The problems of the progress and cost of each person in the project test should exceed the product test. Therefore, I temporarily called it "random test." The difference between these two will definitely, and sometimes it is very big. On the other hand, when doing projects, the software level of testers and customers is uneven, and differ. To make them understand that the test is needed to use different languages. Relatively, "random test" is more likely to understand. It is often encountered that the progress of the project development delay, bug is not changed, so there is a risk of insufficient testing time, which is basically unable to organize "random test". When the development and correction efficiency is less efficient, the cost of testing will often increase the cost, and there will be cost overhead, and the "random test" is not organized. In project class test, the pressure of defect divergence is far more than the larger. Many customers can see the situation of defects, and many customers will directly ask the grassroots employees: "Why is this simple question now? Why don't you fix it? Why do you want to fix this?". In this way, the project group that itself and the quality pressures have been greatly affected, which is likely to "explode". This is what the company is not willing to see. Our current approach is: Whether it is a random test problem, or other tests, as long as it is not correct, it needs to be tested and developing Manager for communication, related tests and developers statements and suggestions. If the Manager can't resolve, then the problem is to rise to an organization similar to the CCB for ruling. Generally, this time is before the last round of tests, it is on the previous time. No one is correct, no one will be entangled. Shen Wei MSN: T628Shen@hotmail.com
Jeff Wjoined: 19 Dec 2002 Total Posts: 91
Re: Microsoft's software test method Posted: 02 dec 2004 04:25 PMPOSTED: 02 DEC 2004 04:25 PMTL_GEONG, thank you for providing some experience in project development, it is indeed different from product development. Here I have some questions to ask you: First, why do you put the "random test" in the end, waiting for almost all established planning completed? Is it that "random test" is not as important as "established plan test"? According to my experience, there are very few projects will have a so-called short time in the later stage to do some willing work. Placing in the final time, the result is often canceled or grass. We believe that one type, two types of tests are equally important, second types of tests have great effects on verifying the coverage of first types of tests (no tools can be replaced), and the second type of test finds a lot of real bugs. The original design and plan of the first class test must be adjusted accordingly. So Microsoft will have "bug bash" in each of the scapement, not to put it in the end. I feel that "random test" is finally like a gamble, gambling your test plan is very close. If this is not the case, in the end, a successful "random test" will expose a lot of problems to you, and just then the date of delivery. How to clean up this offer? You know that this is not the worst ending. The worst case is that you don't have the opportunity to do "random test", or do but unsuccessful, as a lot of bugs slipped into your customers. This will be a disaster for you or your users. The second question is, why do you add pressure diverged on the grassroots employee? Does the original test plan not reviewed? If you have been reviewed, even if you have no circumstances, even the low-level mistakes should not be borne. The third question is about the CCB organization you mentioned, is this an upper right part? In Microsoft, BUG fate is made by the test department. If the development and management is dissatisfied, it is generally requested to provide further information because they are closer to customers. Jeff Wang [MSFT] This post is provided with "status quo" and there is no guarantee, and there is no right to grant any rights.
MSLGNJOINED: 13 DEC 2004 TOTAL POSTS: 44
Re: Microsoft software test method pos: 13 dec 2004 11:19 pmposted: 13 dec 2004 11:19 PM ideas open, thank you Mr. W
Maxwell Liu [hwtc]
Jeff Wjoined: 19 Dec 2002 Total Posts: 91
Re: Microsoft software test method Posted: 16 Jan 2005 01:42 PMPOSTED: 16 Jan 2005 01:42 PM Microsoft Software Test Method (2) I medided with Microsoft's two in the "Microsoft Software Test Method" Basic test methods, the basic idea should be more familiar because they are only a comprehensive integrated software test method. So single from the form, it does not reflect the breakthrough to the traditional framework. But from another level to examine Microsoft software test, you will be surprised by some basic facts. For example, "Microsoft's tester and developer number is substantially equal or slightly", "Microsoft's product cost is about 40%", etc. People will have questions, is it only as a functional verification and search for Bug? Can I consume such a large amount of resources? Is it necessary to pay such a big price? There should be reason to believe that Microsoft uses a software company, and each of its investment is meaningful, so it can also determine that Microsoft's efforts in software test must exceed the conventional test method. Historical review In order to better understand microsystem testing in methods and ideas, I want to review the development history of software development and software testing, and reveal their inevitability. Edward Kit In his bestseller "Software Testing in The Real World: Improving The Process (1995, ISBN: 0201877562)" The entire software development history is divided into three phases: the first stage is the 1960s and its previous, then When the size of the software is small, the complexity is low, the process of software development is free. The developer's Debug process is considered a unique test activity. In fact, this is not a software test in the modern sense. Of course, there is no special tester in a stage. The second stage is the 1970s, and the software developed at this stage is still not complicated, but people have begun to think about the development process, and propose the concept of "software engineering software engineering". However, this phase of people's understanding of software tests is limited to basic functional verification and bug search, and the test activity only appears in the later stage of the entire software development process, although testing is undertaken by specialized testers, but testers are industries and Software professional entrance newcomers. The third stage is in the 1980s, and the software and IT industry have entered the great development. The software tends to be large. In contrast, people have designed a variety of complex and precise processes and management methods for software development, and integrate the concept of "quality" into it. Software Test has an industry standard (IEEE / ANSI), it is not a one-time, but only the post-development activities, but integrates with the entire development process. Software tests have become a professional, need to use specialized methods and means, special talents and experts need to be borne. The integration of testing and development is most worth noting in this historical development process. The trend of testing and development process fusion. People may not be unfamiliar with this integration. For example, early development of test activities, allows testers to participate in the verification of user needs, participate in functional design and implementation review. For example, testers work closely with developers, with gradual implementation of unit testing, module function testing and system integration test with development progress. It is indeed the form of expression and development fusion, and the initial fusion is only reflected in this level. After the 1990s, the size and complexity of software increased rapidly, and this form of integration also quickly went to a deeper level, more practical. Specifically, this fusion is the dependence of the entire software development activity. Traditionally, only software quality control depends on testing, but the practice of modern software development proves that not only software quality control is dependent on testing, and the development itself will not be able to advance. Project management has left the test. in accordance with.
In Microsoft, the test does have such status and role. That's why Microsoft has such a large investment in the software test. Developing modern software development for testing, especially large software development, usually encounters two issues: (1) In the initial stage of development, how to launch a large-scale team, the group is in turn, while maintaining the order of development. Thus efficiently utilize resources and shorten the development cycle. (2) In the late development, how to solve deep BUGs, how to face design changes, and ensure that the quality of the product does not appear or falls. For small and simple software, these two problems also exist, but do not protrude, and easy to solve. But for complex large software development, these two problems often become difficult to overcome. Usually the function of large projects is rich, but the architecture, and the level will be quite complicated. Sustained development method is that a small number of people have been investing in one time, and it is stabilized by layer. However, this way is clear that the resource utilization is low, the development cycle is long, and the modern software and the high-speed development of the IT industry are unable to develop. So large projects require large teams. In Microsoft, product development teams (mainly including development, testing and project management) generally have more than 100 people, some products, even thousands of people (more than 3,000 of Windows2000 development departments). Such a large-scale human resource acts in a dynamic, internal interconnected system, if there is no effective synergy, its confusion is inevitable. Imagine that there are two developers, which are developed two different functional modules, which are dependent on each other. In order to coordinate each other, they can discuss all at any time. If this relationship occurs between five developers and five functional modules, this coordination can only be done through a regular meeting. And a large project will have a lot of such relationships, and many times this relationship has uncertainty and unpredictability. When a developer writes a new code or modifies the existing code, he often does not determine, or cannot fully determine which related modules will be affected, and what should be What results will be brought about. Because the complexity of the system is far exceeding the ability of people's logical thinking, skills, and experience. Therefore, this traditional coordination means is far from needing. In Microsoft, this coordination is achieved by testing. Specifically: daily construction automation test. With regard to the daily compilation and automation test, I will introduce it in the future. It is simple to build a new version every day. Each version is running through a certain automatic test case to test the quality of work on the day. The quality mentioned here is of course a concept of quality in general, but it also reflects the overall coordination of the project during the development process. The maximum advantage of automatic test is its height repeatability. An ideal automatic test system can operate a lot of test cases at any time, convenient and quickly run. Therefore, a developer can analyze the quality of the code before the day of automatic test results (afterward check), or before the time is deposited in the day, run automatic testing to further ensure the quality of the code (prior inspection). In Microsoft, every day construction is starting at midnight, and then the completion is a comprehensive automatic test, it will automatically send the results automatically through E-mail before morning. The first thing to develop after work is often checking the test results. If there is no problem, you will start a new job. If there is a test use case, the developer must cooperate with the test personnel to find out the cause and solve the new code. Sometimes a small mistake will cause a large-scale test case failure, a large part of the development team will be affected.
To avoid this, the developer is required to run a certain amount of automatic test on his personal build version before depositing the code, all of which are passed after depositing. If the developer does not follow such a request, the unauthorized test fails, this irresponsible behavior is to be severely criticized. As can be seen from this process, developers depend on tests to ensure the quality of development work, so that the development is integrally coordinated forward. When the development enters the later stage, although the project has been overall, developers will encounter some technical challenges from time to time. For example, some bug addresses the adjustment of the deep-hierarchical structure of the project; for example, the design is modified due to the opinion of customer feedback. Every such modification and adjustment is actually destroying a stable system. If the improper handling is often a bug modification, there will be a lot of new bugs, just like a series of interlocking vicious circles. Many project duration delays are caused. To avoid or at least reduce this damage to a minimum, the developer first needs to know the influence of this damage. The logic thinking, skills and experiences here, skills and experience are far less than, and automatic testing has once again become an effective tool. Often developers will make more than one solution, running all over the same set of automatic test cases, then compare results, elect the best solution. The effects of automatic testing in this regard not only during the development of the product, but it continues until the product is released. The product support department also relies on automated testing when providing emergency solutions for our customers. Management's dependence on testing in Microsoft, software project management is the management of BUG, where the most direct specific management activity is "Bug Triage". The meeting generally hosts project management Program Manager (PM), developers and testers to participate (so called Tri-party meeting). Each newly generated bug is discussed and decided to accept this bug; (2) BUG's severity level and priority; (3) BUG is responsible for providing further details by testing, Or is also a developer to solve, and the general solution, etc. The meeting also resolved the progress on the old BUG. This seminar often occurs, requiring testers to have sufficient technical foundation and user experience to defend the quality of the product. It can be said that the project is developed to a certain stage is driven by this BUG management. This is from the test. A very important job in project management is a very important job is to measure the progress of the project, including the status of judging the project, determine if the project can be completed. In this regard, the test provides two very important parameters, one is the trend of bugs and the other is the trend of test results. The BUG trend is a trend chart to a new bug number and the BUG number that is resolved daily. Generally, the new BUG count curve will rise in the beginning of the project, and the BUG count curve will be increased in the middle of the project, and the number of new BUG should decline, and the two curves tend to zero. PM will continue to observe this chart to ensure the healthy development of the project, while analyzing the predictive project bug tend to tend to tend to tend to tend to zero. Based on certain historical experiences, the analysis uses this chart will get a lot of valuable information. For example, it is possible to analyze whether the development and testing is appropriate on the comparison of human resources, and can analyze a serious bug. The fluctuation of the project quality. The daily automatic test results can also form a similar chart. It is also very helpful to understand the quality status of the current project and develop test progress. These data generated by tests not only provide effective basis for project management not only in project development, but also the necessary conditions of the product through the release. In Microsoft, each product has to be reviewed.
The few charts described earlier are important contents of the publishing review. If there is a large quality fluctuation before the review is found, the reviewers will question this. Therefore, software project management depends on software testing provides its basic management material. It can be said that the development and management of the modern large software development is a fundamental factor for testing and development process. From another perspective, testing and development process fusion is not just a simple time synchronization, not approaching between the two spaces, but an external performance of this inherent dependency. The development of this dependence on testing has put forward higher requirements for testing and testing. At the idea, software testing is not just the authentication of software functions and the search of bugs; in specific methods, the use of automatic testing and testing tools has become a basic requirement. In Microsoft, testing not only uses some universal tools, each product is also specially developed special tool library, and the test amount is often exceeded by the code of the project itself. A software businesses should improve their software development, especially for large-scale rapid development capabilities for large software, is necessary to break through the traditional concepts and methods in terms of testing. Microsoft's practice is a good confirmation. Jeff Wang [MSFT] This post is provided with "status quo" and there is no guarantee, and there is no right to grant any rights.
Liuyang630_630joined: 12 Nov 2004 Total Posts: 14
Re: Microsoft's software test method Posted: 17 Jan 2005 10:32 Pmposted: 17 Jan 2005 10:32 PM read your article indeed made me more than the earnings of this kind of beginner, I hope to have the opportunity to learn from you later.
DFGJOINED: 12 DEC 2002 TOUR POSTS: 5
Re: Microsoft software test method pos: 18 Jan 2005 09:56 amposted: 18 Jan 2005 09:56 AM thanks to Jeff Wang's detailed explanation, give me the guidance of theory and methods!
Carol Mojoined: 19 Jan 2005 Total Posts: 1
Re: Microsoft's software test method pos: 19 Jan 2005 07:57 Pmposted: 19 Jan 2005 07:57 PM Mourning Open! Especially the relationship between development and testing! Looking forward to Jeff Master's next wonderful topic
Herahjoined: 30 Jul 2004 Total Posts: 41
Re: Microsoft software test method pos: 24 Jan 2005 12:46 Pmposted: 24 Jan 2005 12:46 PM Track
TL_GEONGJOINED: 20 AUG 2003 TOTAL POSTS: 251
Re: Microsoft's software test method pos: 26 Jan 2005 03:00 Pmposted: 26 Jan 2005 03:00 PM top
Shen Wei MSN: T628Shen@hotmail.com
Xaleejoined: 31 Jan 2005 Total Posts: 2
Re: Microsoft's software test method Posted: 31 Jan 2005 02:56 PMPOSTED: 31 Jan 2005 02:56 PM Indeed The success of the work. I think Microsoft still pays more attention to the second theory. The description of the test work on Microsoft's HR website is "We Break It to Build IT". :) Thank you Jeff sharing. If there is time, can jeff please talk about Microsoft's test team? How to improve the quality of test work from the perspective of management? THANKS. Xalee
Deltadavijoined: 01 Feb 2005 Total Posts: 4
Re: Microsoft's software test method pos: 02 feb 2005 10:28 amposted: 02 Feb 2005 10:28 AM specially to apply for users to reply! The best discussion is seen. I look forward to your sharing with you. YUMEN1JOINED: 16 Feb 2004 TOTAL POSTS: 15
