tag:blogger.com,1999:blog-6877629428951398731.post5029913752413656776..comments2024-01-09T09:02:40.935-08:00Comments on Java Persistence Performance: What is faster? JVM PerformanceJameshttp://www.blogger.com/profile/07275512393744882781noreply@blogger.comBlogger23125tag:blogger.com,1999:blog-6877629428951398731.post-85175533782078958432013-06-15T05:55:06.428-07:002013-06-15T05:55:06.428-07:00Useless synthetic microbenchmark. Data sets of 10,...Useless synthetic microbenchmark. Data sets of 10, 100 or 1000 objects aren't relevant for big data, and queries of 10000000s per sec aren't relevant for small data. Besides, what was said already - change the testing conditions, you'll get different results. Only thing that IS sure is that sync'ed classes are slower than not sync'ed ones. Period.Vaxquishttps://www.blogger.com/profile/08016926050037840747noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-83911969367497078402011-07-20T07:40:47.953-07:002011-07-20T07:40:47.953-07:00Good job mate for putting this data. By the way ha...Good job mate for putting this data. By the way have you done this test using -server class of JVM <br /><br />Javin<br /><a href="http://javarevisited.blogspot.com/2011/07/java-debugging-tutorial-example-tips.html" rel="nofollow">10 tips on debugging Java Program in eclipse </a>javin paulhttps://www.blogger.com/profile/15028902221295732276noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-3335775628175448522011-01-11T08:10:35.734-08:002011-01-11T08:10:35.734-08:00I corrected the results for volatile
Thanks to @s...I corrected the results for volatile<br /><br />Thanks to @sp8472 for finding the bug in the test.<br /><br />>><br /><b>CORRECTION</b><br />Originally volatile was showing an improvement in performance because of a bug in the volatile test. After correcting the bug the test now shows a huge overhead in performance. The usage of volatile on the int field that is being incremented in the test method is causing a 15x performance overhead. This is surprising, and much larger than the synchronized overhead, but still smaller than the reflection overhead. This is especially odd as the field is an int which one would think is atomic anyway. I imaging the affect the volatile has on the JVM memory usage is somehow causing the overhead. In spite of the overhead, I would still use volatile when required, in concurrent pieces of code where the same variable is concurrently modified. I have done multi-threaded tests comparing its concurrency with synchronizing the method, and using volatile is better than using synchronization for concurrency.Jameshttps://www.blogger.com/profile/07275512393744882781noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-54622394160908855882011-01-04T06:50:24.141-08:002011-01-04T06:50:24.141-08:00@sp8472 good catch on the volatile test, not sure ...@sp8472 good catch on the volatile test, not sure how I missed that. I will fix the test and update the volatile results.<br /><br />@williamlouth, @Marcus - the tests are run 5 times in the same JVM in intermixed order, the min/max results are rejected and middle 3 averaged, so if the JVM was hot/cold for the first run, the result would be rejected. If the JVM got faster as it warmed up, this would affects all tests the same. I have changed the order of the tests and re-run the tests, the results are the same, and very consistent.<br /><br />@Nikolay Tsankov - The difference between these tests and yours is that the creation of the Map is included in the test time, where are you are not measuring the creation. If your results are correct, then it would appear the creation cost is why HashMap is performing so poorly. I will look into confirming this. All these are run 5 times in mixed order and averaged (and standard deviation computed), where as you only run your test once, in a defined order, so may be seeing bad results.Jameshttps://www.blogger.com/profile/07275512393744882781noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-58648204351399844952010-12-28T18:43:07.210-08:002010-12-28T18:43:07.210-08:00thankthankG.Rajeshhttps://www.blogger.com/profile/05035043699578591010noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-90480019991232234922010-12-27T07:47:11.572-08:002010-12-27T07:47:11.572-08:00@blog
Well the JavaDoc seems to think it is. On th...@blog<br />Well the JavaDoc seems to think it is. On the Hashtable one it says:<br />"Unlike the new collection implementations, Hashtable is synchronized."<br /><br />And on the HashMap one it says:<br />"The HashMap class is roughly equivalent to Hashtable, except that it is unsynchronized and permits nulls"Tiran Kenjahttps://www.blogger.com/profile/00149892231563027817noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-68988685106172881902010-12-27T02:12:01.395-08:002010-12-27T02:12:01.395-08:00One point: "Hashtable, and does not suffer fr...One point: "Hashtable, and does not suffer from its limitation of using synchronized methods, which are suppose to be slow."<br /><br />That's not correct; Hashtable isn't synchronized! It just doesn't support null values or keys.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-25088548495745176102010-12-21T12:49:33.787-08:002010-12-21T12:49:33.787-08:00How does the reflection performance change if you ...How does the reflection performance change if you call setAccessible(true) on the method first?David Phillipshttps://www.blogger.com/profile/02563947024752844788noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-45938538249410541622010-12-21T10:41:00.869-08:002010-12-21T10:41:00.869-08:00Wrt IdentityMap being slower -- that would not be ...Wrt IdentityMap being slower -- that would not be surprising to me. When I last profiled it (granted it has been a while, so things may have changed), cost came from System.identityHashCode() which is a native method, and bit costly as such. It also uses linear probing which is not necessarily faster than buckets, although this really depends.<br /><br />I would agree with other suggestions -- micro-benchmarks must use proper warm-up and runtimes; as well as JVM isolation. Japex, for example, would provide these, as do most other suitable benchmarks.Unknownhttps://www.blogger.com/profile/02343617472112278520noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-67861564747490281442010-12-21T08:59:36.533-08:002010-12-21T08:59:36.533-08:00I think these tests are flawed because they don...I think these tests are flawed because they don't attempt to deoptimize megamorphic method calls. Notice that in both tests, the test that ran first is the fastest. As the jvm encounters me implementations of Map and List, the inlined calls to get, put, etc have to be un-inlined.Craighttps://www.blogger.com/profile/15806804843602653454noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-73892902587859848222010-12-20T11:00:09.852-08:002010-12-20T11:00:09.852-08:00I did a very simple test on maps and my results ar...I did a very simple test on maps and my results are quite different. As I expected, IdentityHashMap is the fastest, followed closely by HashMap. Hashtable, ConcurrentHashMap were twice as slow. I expect ConcurrentHashMap to be faster than Hashtable, if writes are less frequent.<br /><br />Here is the code:<br /><br />int size = 100; <br />Map[] maps = new Map[] { <br /> new HashMap(size), <br /> new ConcurrentHashMap(size), <br /> new IdentityHashMap(size),<br /> new Hashtable(size) };<br /><br />for (Map m : maps) {<br /> int loops = 100000;<br /> long now = System.currentTimeMillis();<br /> while (loops-- > 0) {<br /> for (int i = 0; i < size; i++) {<br /> m.put(i, i);<br /> }<br /> for (int i = 0; i < size; i++) {<br /> m.get(i);<br /> }<br /> for (int i = 0; i < size; i++) {<br /> m.remove(i);<br /> }<br /> }<br /> long time = System.currentTimeMillis() - now;<br /> System.out.println(m.getClass().getSimpleName() + " " + time);<br />}Nikolay Tsankovhttps://www.blogger.com/profile/11757734655917263440noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-63887034590945183162010-12-20T01:43:15.965-08:002010-12-20T01:43:15.965-08:00I find it a little strange that you wouldn't t...I find it a little strange that you wouldn't try an iterator with the LinkedList in your Iteration Performance Comparison. It would be one of the obvious use cases for it (and one of the main reasons for me to choose it)Tiran Kenjahttps://www.blogger.com/profile/00149892231563027817noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-7071947660741604272010-12-20T01:19:25.835-08:002010-12-20T01:19:25.835-08:00Also, is sufficient time spent warming up the JVM ...Also, is sufficient time spent warming up the JVM before measuring? If not, you may get code generation overhead included in the results, and initial measurements will not reflect optimally generated code.Unknownhttps://www.blogger.com/profile/12267690665246153250noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-5951336773816316362010-12-20T01:18:27.705-08:002010-12-20T01:18:27.705-08:00This comment has been removed by the author.Unknownhttps://www.blogger.com/profile/12267690665246153250noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-64229920581979613292010-12-19T13:03:23.470-08:002010-12-19T13:03:23.470-08:00Before I look at the benchmarks please tell me tha...Before I look at the benchmarks please tell me that all tests are run completely isolated from each other (i.e. separate process execution) so that the ordering of the individuals tests within a group has no bearing (positive/negative) on the performance of subsequent tests (hint: gc, heap sizing, native compilation,....).Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-28036922092320487552010-12-19T03:20:06.785-08:002010-12-19T03:20:06.785-08:00>I used 1.6.0_7 for no particular reason, it wa...>I used 1.6.0_7 for no particular reason, it was just what a had installed.<br /><br />I'm sorry mate, but this doesn't sound really professional, does it?Robert Whanehttps://www.blogger.com/profile/17891302243856789222noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-58093969789050322082010-12-18T11:23:34.298-08:002010-12-18T11:23:34.298-08:00Hi James, really cool that you've done this an...Hi James, really cool that you've done this analysis!<br /><br />I'll echo the request to see that test under the 1.6.0_23 JVM. One of the hugely underrated parts of Java 6 are the non-trivial enhancements to the JVM with their point releases.<br /><br />Let me know if you want to run the tests against the latest Open JDK (1.7) build as well (I might be able to help). Will be interesting to if there are any differences.Martijn Verburghttps://www.blogger.com/profile/01458162075331573781noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-80033424134058021172010-12-18T06:22:33.598-08:002010-12-18T06:22:33.598-08:00I really like this idea of getting some sense of t...I really like this idea of getting some sense of the performance of the JVM internals, even if you really shouldn't do anything with this knowledge in day to day programming ;) However, here it sounds like you're really not comparing apples to apples with the different-sized HashMap and HashTable. I was also a bit surprised by the volatile field result, but a quick glance at the test explains a lot: it's the only test that only does 1 series of fields operations instead of the 100 in a tight loop of the others. Big grains of salt are needed here...Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-66587781675050239832010-12-17T12:42:41.733-08:002010-12-17T12:42:41.733-08:00@Ismael - Thanks for the info, I will try to check...@Ismael - Thanks for the info, I will try to check if the latest JDK has any different behavior.<br /><br />I double checked the HashMap test code, and it seems valid. I have not profiled the test, so cannot say for sure why it has worse performance. One difference is that Hashtable uses the size directly, where as HashMap uses the closest power of 2, so would use 16 instead of 10, which may be impacting its performance. But its poor results are consistent for the 100 and 1000 size as well. It would be interesting to test it with a size of 16. Anyway all of the source for the tests is available, you are welcome to look at, or run the tests yourself if you think HashMap should be faster.Jameshttps://www.blogger.com/profile/07275512393744882781noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-61814030370269637462010-12-17T11:47:12.731-08:002010-12-17T11:47:12.731-08:00Hi James,
Actually JDK6u23 is a big change at the...Hi James,<br /><br />Actually JDK6u23 is a big change at the VM level when compared to JDK6u7. Sun (and now Oracle) have decoupled HotSpot's release cycle from the JDK and new versions have been added throughout the cycle (with major new features like compressed references and scalar replacement in some of them).<br /><br />A new HotSpot was integrated in JDK6u4, JDK6u10, JDK6u14, JDK6u18, JDK6u21 and JDK6u23 (if I recall correctly).<br /><br />One other comment, HashMap is generally used for JDK code, so it's quite strange that HashTable does better. I'd check to make sure that the test doesn't have issues.<br /><br />Best,<br />IsmaelIsmael Jumahttps://www.blogger.com/profile/17398483226873559286noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-75260223106690694282010-12-17T11:02:39.567-08:002010-12-17T11:02:39.567-08:00@dave - Yes, I thought this was odd as well. I wo...@dave - Yes, I thought this was odd as well. I would guess it is just variance, notice the % standard deviation was 1.4% and the cost of calling the test method, versus it calling the set method is pretty marginal. It would have been better to put the set call in a 100 loop like I did for the method execution test, but here I was more interested in comparing field vs method reflection, not in-line versus set method, as the previous test already did that.<br /><br />I used 1.6.0_7 for no particular reason, it was just what a had installed. That latest JDK version (today) is 1.6.0_23, I could re-run using it, I would not expect a big difference between patch releases, but would expect I big difference in different JVMs or difference JVM versions. I will try to get some results on different JVMs.Jameshttps://www.blogger.com/profile/07275512393744882781noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-82630200121338629652010-12-17T10:01:37.305-08:002010-12-17T10:01:37.305-08:00These seem to be not very interesting micro benchm...These seem to be not very interesting micro benchmarks. Sort of like comparing the performance of different kind of hammers at hammering in framing nails. I expect a framing hammer would do a better job than a drywall hammer. Each kind of hammer is more useful for different kinds of nails and jobs.BJ Hargravehttps://www.blogger.com/profile/02289107040084648957noreply@blogger.comtag:blogger.com,1999:blog-6877629428951398731.post-32476265059811785932010-12-17T09:12:22.881-08:002010-12-17T09:12:22.881-08:00Interesting results... In the Reflection Performan...Interesting results... In the Reflection Performance Comparison, the Set method is slightly faster than In-lined -- this seems odd to me, can you comment on why this is?<br /><br />Also, the JVM you tested with (1.6.0_7) is relatively old now -- was there a reason you didn't use the latest (1.6.0_22 I think)?davehttps://www.blogger.com/profile/08872088478657058117noreply@blogger.com