Evaluation report: .NET performance is still far behind Java
Everyone has seen a variety of different Benchmark, which proves that .NET is fast than Java, and also proves that Java is fast. In some people's hands, Benchmark is a magic mirror that can see what they want to see. So, when this is a Benchmark called Cameron, he thinks he is a waste of time, because it will definitely prove that .NET is fast than Java.
Incidentally, Mr. Cameron proposes 10 "Do not use the choice between .NET and JAVA", including:
Ø In a set of specific Benchmark, one is more faster than another;
Ø The report of Microsoft or Sun (or Oracle, IBM ...) is much better than another;
Ø Do you like (or don't like) Bill Gates or Scott McNeally (or Larry Ellison);
Ø You think Microsoft or Sun (or IBM or Oracle) is very evil (or great);
Ø You have read a "no prejudice" article in THSERVERSIDE.COM or GOTDOTNET.COM (or Microsoft.com);
In fact, this Benchmark stems from Cameron and Trolf in the THSERVERSIDE.COM website for a quarrel of Web service performance (http://www.theserverside.com/reviews/thread.jsp?thread_id=19226). In this Benchmark, Cameron tested the following versions:
.NET 1.0SP2:
Microsoft (R) Visual C # .NET Compiler Version 7.00.9466
For Microsoft (R) .NET Framework Version 1.0.3705
CopyRight (c) Microsoft Corporation 2001. All Rights Reserved.
.NET 1.1:
Microsoft (R) Visual C # .NET Compiler Version 7.10.3052.4
For Microsoft (R) .NET Framework Version 1.1.4322
Copyright (c) Microsoft Corporation 2001-2002. All Rights Reserved.
Sun Hotspot Server JVM from JDK 1.3.1_04:
Java Version "1.3.1_04"
Java (TM) 2 Runtime Environment, Standard Edition (Build 1.3.1_04-B02)
Java Hotspot (TM) Server VM (Build 1.3.1_04-B02, MIXED MODE)
Sun Hotspot Server JVM from JDK 1.4.0_02:
Java Version "1.4.0_02"
Java (TM) 2 Runtime Environment, Standard Edition (Build 1.4.0_02-B02)
Java Hotspot (TM) Server VM (Build 1.4.0_02-B02, MIXED MODE)
Sun Hotspot Server JVM from JDK 1.4.1_02: Java Version "1.4.1_02"
Java (TM) 2 Runtime Environment, Standard Edition (build 1.4.1_02-b06)
Java Hotspot (TM) Client VM (Build 1.4.1_02-B06, MIXED MODE)
In terms of code, Cameron mainly chose a large number of ArrayList to test. The following is his test code:
C #
// c # crete 100,000 people in an arraylist and access them
Using system;
Using system.collections;
Public Class Manypeople
{
Public static void main ()
{
For (int i = 0; i <100; i )
Test (i);
}
Public Static Void Test (Int Iiter)
{
DateTime start = datetime.now;
For (int i = 0; i <20; i )
NEW MANYPEOPLE (). AVERAGE ();
Datetime finish = datetime.now;
Console.writeline ("Total Time for Iteration Iiter ": (Finish - Start);
}
Private long average ()
{
ArrayList List = New ArrayList (100000);
For (int i = 0; i <100000; i )
List.add (New Person (i, "john" i));
Long silly = 0;
Foreach (Person P in List)
Silly = p.id;
Return Silly / 100000;
}
}
// Person.cs: a Very Simple Guy
Public Class Person
{
Int ID;
String name;
Public Person (int anid, string aname)
{
THIS.ID = anid;
THIS.NAME = aname;
}
Public Int ID
{
Get {returnim.
}
}
Java:
// java create 100,000 people in an ArrayList and Access Them
Import java.util. *;
Public Class Manypeople
{
Public static void main (string [] args)
{
For (int i = 0; i <100; i )
Test (i);
}
Public Static Void Test (Int Iiter)
{
Long Start = system.currenttimemillis ();
For (int i = 0; i <20; i )
NEW MANYPEOPLE (). AVERAGE ();
Long finish = system.currenttimemillis ();
System.out.println ("Total Time for Iteration" IITER ": (Finish - start));
Private long average ()
{
ArrayList List = New ArrayList (100000);
For (int i = 0; i <100000; i )
List.add (New Person (i, "john" i));
Long silly = 0;
For (int i = 0; i <100000; i )
Silly = ((PERSON) List.get (i)). getId ();
Return silly;
}
}
// Person.java: a Very Simple Guy
Class Person
{
Private int ID;
PRIVATE STRING NAME;
Public Person (int anid, string aname)
{
THIS.ID = anid;
THIS.NAME = aname;
}
Public int getID ()
{
Return this.id;
}
}
The test machine used by CAMERON is a notebook of P3 1GHz, 1G memory, and the operating system is Windows 2000 SP2. The following is the result of the .NET test (take the best 6 times):
.NET 1.0SP2:
Manypeople
Total Time for ity 0: 00: 07.3004976
Total Time for ity 1: 00: 07.0501376
Total Time for ity 2: 00: 07.2504256
Total Time for ity 3: 00: 07.7311168
Total Time for Iterations 4: 00: 07.5007856
Total Time for ity 5: 00:07.5007856
.NET 1.1:
Manypeople
Total Time for ity 0: 00: 08.0916352
Total Time for ity 1: 00: 08.5222544
Total Time for ity 2: 00: 08.3520096
Total Time for ity 3: 00: 08.6324128
Total Time for Iterations 4: 00: 08.3419952
Total Time for ity 5: 00: 08.1617360
It can be seen that the same code is more than 15% slower than .NET 1.0 SP2. This is a very strange phenomenon. Also, Cameron observed that the running speed of the same code on .NET did not increase with the increase of the number of runs, indicating that Net CLR simply compiled JIT compilation. And on HotSpot Server, not only the performance of the performance is advantageous, but the speed will continue to improve:
Sun Hotspot Server JVM from JDK 1.4.1_02:
Java-Server -XMS128M -XMX128M MANYPEOPLE
Total Time for Iteration 0: 6370
Total Time for Iteration 1: 5788
Total Time for Iteration 2: 5868 Total Time for Iteration 3: 6029
Total Time for Iteration 4: 5748
Total Time for ITERATION 5: 5738
Total Time for Iteration 6: 5729
Total Time for Iteration 7: 5948
Total Time for Iteration 8: 5688
Total Time for Iteration 9: 5679
Total Time for Iteration 10: 5658
Total Time for Iteration 11: 6028
Total Time for Iteration 12: 5699
Total Time for Iteration 13: 5708
Total Time for Iteration 14: 5678
Total Time for Iteration 15: 5969
Total Time for ITERATION 16: 5628
Total Time for Iteration 17: 5538
Total Time for Iteration 18: 5608
Total Time for Iteration 19: 5498
Total Time for Iteration 20: 5768
Total Time for Iteration 21: 5518
Total Time for Iteration 22: 5307
Total Time for Iteration 23: 4247
Total Time for Iteration 24: 4696
Total Time for Iteration 25: 4617
Total Time for Iteration 26: 4777
Total Time for Iteration 27: 4286
Total Time for Iteration 28: 4677
Total Time for Iteration 29: 4626
Total Time for Iteration 30: 4697
Total Time for Iteration 31: 4286
Total Time for Iteration 32: 4697
Total Time for Iteration 33: 4617
Total Time for Iteration 34: 4696
Total Time for ITERATION 35: 4307
Total Time for Iteration 36: 4686
Total Time for Iteration 37: 4807
Total Time for Iteration 38: 4517
Total Time for ITERATION 39: 4306
Total Time for Iteration 40: 4657
Total Time for ITERATION 41: 4807
Total Time for Iteration 42: 4596
Total Time for ITERATION 43: 4206
Total Time for Iteration 44: 4777
Total Time for Iteration 45: 4717
Total Time for ITERATION 46: 4607
Total Time for Iteration 47: 4196
Total Time for Iteration 48: 4796
Total Time for ITERATION 49: 4707
Total Time for ITERATION 50: 4777
Total Time for ITERATION 51: 4196
Total Time for Iteration 52: 4627 Total Time for Iteration 53: 4687
Total Time for Iteration 54: 4806
Total Time for Iteration 55: 4186
Total Time for ITERATION 56: 4627
Total Time for Iteration 57: 4697
Total Time for Iteration 58: 4807
Total Time for Iteration 59: 4166
Total Time for Iteration 60: 4616
Total Time for Iteration 61: 4697
Total Time for Iteration 62: 4717
Total Time for ITERATION 63: 4346
Total Time for Iteration 64: 4717
Total Time for Iteration 65: 4617
Total Time for Iteration 66: 4626
Total Time for Iteration 67: 4367
Total Time for Iteration 68: 4706
Total Time for Iteration 69: 4617
Total Time for Iteration 70: 4617
Total Time for ITERATION 71: 4366
Total Time for Iteration 72: 4687
Total Time for ITERATION 73: 4616
Total Time for Iteration 74: 4196
Total Time for Iteration 75: 4787
Total Time for ITERATION 76: 4687
Total Time for Iteration 77: 4807
Total Time for Iteration 78: 4436
Total Time for Iteration 79: 4387
Total Time for Iteration 80: 4676
Total Time for Iteration 81: 4807
Total Time for Iteration 82: 4417
Total Time for Iteration 83: 4296
Total Time for Iteration 84: 4686
Total Time for Iteration 85: 4797
Total Time for Iteration 86: 4266
Total Time for Iteration 87: 4697
Total Time for Iteration 88: 4617
Total Time for Iteration 89: 4717
Total Time for Iteration 90: 4276
Total Time for ITERATION 91: 4707
Total Time for ITERATION 92: 4616
Total Time for ITERATION 93: 4697
Total Time for ITERATION 94: 4296
Total Time for ITERATION 95: 4677
Total Time for ITERATION 96: 4546
Total Time for ITERATION 97: 4697
Total Time for ITERATION 98: 4296
Total Time for ITERATION 99: 4677
It can be seen that as the number of runs increases, Sun Hotspot Server will continue to be large, and finally more than .NET 1.1 is twice. The .NET CLR will not be able to reduce the runtime to 8 seconds, while Sun HotSpot Server can be kept below 5 seconds after multiple run. Therefore, CAMERON believes that for the application of the Universiade, HOTSPOT JVM is much faster than Microsoft CLR. Original and complete Benchmark data, see Cameron's Blog:
http://www.freeroller.net/page/cpurdy/20030516
