NPERF, A Performance Benchmark Framework for .NET
Introduction
This article present NPerf? A flexible performance benchmark framework. The framework provides custom attributes that the user uses the tag benchmark? Classes and methods. If you are familiar with NUnit [1], this is similar to the custom attributes they provide. ??
........................................ ...CRIPLILE, TELILS.
At The end of the article, Illustrate NPERF with SOME METAPHYSIC .NET Question: Interface Vs Delegates, String ConcaTanation Race, Fastest Dictionary.
QuickStart: Benchmarking iDictionary
Let's start with a small introductory example:. Benchmarking the [] assignement for the different implementation of IDictionary To do so, we would like to test the assignment on a growing number of assignment calls.
All the customer attributes are locate name, nperf.framework.dll assembly.
Perftester Attribute: Defining Tester
First, You NEED TO CREATE A Tester Class That Will Contains Method to Do The Benchmark. This Tester Method Has To Be Decorated with The Perftester Attribute.
Using nperf.framework; [gerftester (iDictionary), 10)] public class Dictionarytester {...}
The Perftesterattribute Construction Takes Two Argument:
The Type of The Test Class, Interface OR Struct, The Number of Test Runs.?the?framework Will Use this value to call test methods? multiple times? (Explained Below).
Perftest Attribute: Adding Benchmark Tests
The Perftest Attribute Marks a Specific Method Inside a class That Has Already Been Marked with The Perftester Attribute
, as a performance test method.
The Method Should Take The TESTED TYPE AS Parameter,? IDictionary? Here,? And the return type shop be?
Void:
[PerfTester (typeof (IDictionary), 10)] public class DictionaryTester {// explained below private int count; private Random rnd = new Random (); [PerfTest] public void ItemAssign (IDictionary dic) {for (int i = 0; I Perfsetup and Perfteardown Attributes Often, you will need to set up you tester and tested class before actually starting the benchmark test. In our example, we want to update the number of insertion depending the test repetition number. The PerfSetUp attribute can be used to tag a method that will BE CALLED BEFORE Each Test Repetition. IN Our test case, we use this method to update the dictionarytester.count member: [PerfTester (typeof (IDictionary), 10)] public class DictionaryTester {private int count; private Random rnd = new Random (); [PerfSetUp] public void SetUp (int index, IDictionary dic) {this.count = index * 1000; } The Set-up Method Must Return Void and Take Two Arguments: Index, Current Test Repetition Index. this value can be used to modify the number of elements Tested, Collection Size, ETC ... DIC, TESTED CLASS INSTANCE If you need to clean up resource the Tests Are Run, you can use the perfteardown attribute to tag a cleaning method: [Perftester (iDictionary), 10)] Public Class Dictionarytester {... [Perfteardown] Public Void TEARDOWN (iDictionary DIC) {...}} Perfrundescriptor Attribute: Giving Some Information To The Framework In our example, we test the IDictionary object with an increasing number of elements It would be nice to store this number in the results, and not store just the test index:. We would like to store 1000,2000, .... and NOT 1, 2, ... The Perfrundescriptor Attribute Can Be Used to Tag a Method That Returns A Double From The Test Index. This Double Is Typically Used for Charting The Results, as x coordinate. [Perftester (iDictionary), 10)] Public class Dictionarytester {[Perfrundescriptor] public double count (int index) {return index * 1000;}} Full Example Source. The Full Source of the Example Is as Follows: using System; using System.Collections; using NPerf.Framework; [PerfTester (typeof (IDictionary), 10)] public class DictionaryTester {private int count = 0; private Random rnd = new Random (); [PerfRunDescriptor] public double Count ( INDEX) {Return INDEX * 1000;} [perfsetup] public void setup (int index, idictionary DIC) {this.count = (int) Math.floor (count (index));} [perftest] public void itemassign (iDictionary) DIC) {for (int i = 0; i Compiableing and rainning Compile this class to an assembly and copy the nperf binaries in the output folder: (Nperf.cons.exe, nperf.core.dll, nperf.framework.dll, nperf.report.dll, scpl.dll). NPerf.Cons.exe is a console application that dynamically loads the tester assemblies (that you need to specify), the assemblies that contains the tested types (you need to specify), runs the test and output charts using ScPl [2] (ScPl IS A Chart Library Under GPL). The call to nperf.cons.exe loospook NPERF.CONS -TA = myperf.dll -tdfap = system -tdfap = mscorlibwhere ta defines an assembly that contains terster classes (DictionaryTester), tdfap defines an assembly that contains tested type. Moreover, the assembly names are given as partial name and will be loaded by AssemblyLoadWithPartialName. There Are A Number of Other Options That You Can Get by Typing NPERF.CONS - H. RUNNING THE COMMAND LINE Above Will Produce THE FOLLOWING? CHART: In the graph, you can see That Some Type Failedorcollection. It is possible to specify to nperf to avoid those type by passing the in the commit line: Nperf.cons -ta = myperf.dll -tdfap = system -tdfap = mscorlib -it = protydescriptorCollection Saving to XML You can also output the results to xml by adding the result, .net xml serialization is buy to render the results to xml. A few remarks You can add as ma you test? As you want in the perftester classes, You Can Want, you can load tester / tsted type? From multiple assembly Overview of the core The NPERF.CORE NAMESPACE Contains The Methods That Do The Job in The Background. I Do Not Plan To Explain The in Details But I'll Discuss Some Problem I Ran Into While Writing The Framework. Getting the Machine Properties . Getting the physical properties of the machine was a suprisingly difficult task It took me a bunch of google tries to get on the right pages Anyway, here's the self-explaining code that get the machine properties.: ManagementObjectSearcher query = new ManagementObjectSearcher ( "SELECT * From Win32_ComputerSystem"); foreach (ManagementObject obj in query.Get ()) {long ram = long.Parse (obj [ "TotalPhysicalMemory"] ToString ().); Break;} query = new ManagementObjectSearcher ( "SELECT * From Win32_Processor"); foreach (ManagementObject obj in query.Get ()) {string cpu = (string) obj [ "Name"]; long cpuFrequency = long.Parse (obj [ "CurrentClockSpeed"]. Tostring ()); Break;} TypeHelper, Easier CustomAttribute Support A Type Helper Static Class Was Added to Automate Tedious Tasks Like Check for Custom Attribute, Get A Custom Attribute, Etc ... The TypeHelper Class Declaration is as follows: public sealed class TypeHelper {public static bool HasCustomAttribute (Type t, Type customAttributeType); public static bool HasCustomAttribute (MethodInfo t, Type customAttributeType) public static Object GetFirstCustomAttribute (Type t, Type customAttributeType) public static Object GetFirstCustomAttribute (MethodInfo mi, Type customAttributeType) public static MethodInfo GetAttributedMethod (Type t, Type customAttributeType) public static AttributedMethodCollection GetAttributedMethods (Type t, Type customAttributeType); public static void CheckSignature (MethodInfo mi, Type returnType, params Type [] argumentTypes); public static void CheckArguments (MethodInfo mi, params Type [] argumenttypes; Benchmark Bonus In Order to Illustrate The Framework, I Have Written A Few Benchmark Testers for Classic Performance Questions About .NET. All these Benchmarks Are Provided in The System.Perf Project.idictionary Benchmark String ConcateNation Benchmark Interface vs delegate
