6

I have a project with a unit test that works under Java 7, but not under Java 8. Is there a good way to investigate such things? (I'm sure that the test is correct; this suggests that there's a subtle bug in the implementation.)

Really I suppose what I would like is a quick way to identify where the code paths diverge. But this is hard, because there might be all sorts of differences in the code paths at a very low level through the JDK, and I don't want to get bogged down in irrelevant differences that are down to tiny optimisations.

So the nice thing would maybe be to ask at what top level trace the paths diverge; and then, starting from just before that point, to ask at what second level trace the paths diverge; and so on.

But I've no idea whether there's a way to do this. I fear I could waste a lot of time if I don't have a systematic approach.

The code, by the way, is the Apache Phoenix repository, where under Java 8, I get the following failure:

Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.006 sec <<< FAILURE! - in org.apache.phoenix.schema.PMetaDataImplTest
testEviction(org.apache.phoenix.schema.PMetaDataImplTest)  Time elapsed: 0.006 sec  <<< FAILURE!
java.lang.AssertionError: expected:<3> but was:<2>
    at org.junit.Assert.fail(Assert.java:88)
    at org.junit.Assert.failNotEquals(Assert.java:834)
    at org.junit.Assert.assertEquals(Assert.java:645)
    at org.junit.Assert.assertEquals(Assert.java:631)
    at org.apache.phoenix.schema.PMetaDataImplTest.testEviction(PMetaDataImplTest.java:98)
chiastic-security
  • 19,689
  • 3
  • 33
  • 62
  • 6
    How about you just debug the code, stepping through it? – Andreas Sep 30 '15 at 16:09
  • @Andreas I can do that, yes. But it's going to come down to the specifics of a JDK collection implementation, I think. And I would also need two machines in order to step through it twice simultaneously with different JDKs. – chiastic-security Sep 30 '15 at 16:12
  • Why? You can have multiple JDK's and IDE's installed on a single machine, if that's what you need. – Andreas Sep 30 '15 at 16:37
  • 6
    Don’t waste time trying to compare the two JREs. Do a step debug of the failed test, and compare what the code does with what the code is supposed to do. If that’s too complicated, you should rethink the granularity of your unit tests, i.e. separate the test into smaller tests until you find the broken operation. – Holger Sep 30 '15 at 16:40
  • @Holger I suspect you're right that this is the best approach to take. I was just hoping there was something that would help analyse this sort of case. You can see that it might occur in other contexts: you update a dependency to a new version, and things break, and you want to know exactly where. The point about granularity of unit tests is spot on, but in this case they're not my tests, so I don't really have control of that. – chiastic-security Sep 30 '15 at 19:22
  • If I were you I would add "phoenix" tag instead of say javac. Guys from that project might want to look at your question and they probably do it by tag. – rsutormin Sep 30 '15 at 19:58
  • @rsutormin Good call. Done. The Phoenix guys are aware of the issue, though. – chiastic-security Sep 30 '15 at 20:42
  • If you are updating one dependency and it has a single significant change, awareness regarding that change might help to spot the problematic code. However, there are too many changes between Java 7 and Java 8. – Holger Oct 01 '15 at 08:16

1 Answers1

4

One of the most visible changes between Java 7 and Java 8 is a change to how hash data structures handle collisions - previously linked lists were used, now the entries are stored in balanced trees. In some cases this can affect the iteration order of these data structures. This isn't a language regression since these data structures explicitly do not specify an iteration order, but careless code and tests can easily be written that assume a certain order.

In rare situations, this change could introduce a change to the iteration order of HashMap and HashSet. A particular iteration order is not specified for HashMap objects - any code that depends on iteration order should be fixed.

~ Collections Framework Enhancements in Java SE 8

So the first thing I would do when debugging Java 7 -> 8 failures is look for miss-uses of HashMap, HashSet, and ConcurrentHashMap. Once you've found the mistake you have a few choices:

  • Replace the data structure with one that explicitly provides an iteration order, e.g. LinkedHashMap, TreeMap, or ImmutableMap.
  • Remove iteration-order assumptions from your production code (for instance assuming that a certain key/value will be the first element in a map).
  • Improve your tests to actually check for specified behavior (which elements) rather than unspecified behavior (what order). Often test data will be defined in an easy-to-write but incorrect way, like using the inline array syntax int[] expected = {1,2,3} and then looping over the elements. You can make this easier using Guava's static constructors (ImmutableSet.of(1,2,3)) or Truth's fluent assertion syntax (assertThat(someSet).containsExactly(1, 2, 3)).

If hash ordering proves to not be the root cause your issue is likely much more subtle and your best option will likely be to step through with a debugger, as suggested. Be sure to at least skim What's New in JDK 8 as well for clues.

Community
  • 1
  • 1
dimo414
  • 42,340
  • 17
  • 131
  • 218