Java HashMap/Hashtable does not retrieve its own keys

Question

For a school project, I am currently coding a program which learns how to play TicTacToe thanks to a Q-Learning algorithm, based on the following sources : https://github.com/aimacode/aima-java/blob/AIMA3e/aima-core/src/main/java/aima/core/learning/reinforcement/agent/QLearningAgent.java

But I identified a more specific problem, and in spite of all my researches and trials to debug it, I can't solve it.

The algorithm uses a HashMap to store Q-Values, here is its definition :

Map<Pair<S, A>, Double> Q = new Hashtable<Pair<S, A>, Double>();

Then, I implemented my classes such that S is a Grid, and A is a DynamicAction, and I concretely use instances of TicTacToeAction and NoOpAction, which both extends DynamicAction.

Here are hashCode and equals for Grid :

@Override
public int hashCode() {
    final int prime = 31;
    int result = 1;
    result = prime * result + ((matrix == null) ? 0 : matrix.hashCode());
    return result;
}

@Override
public boolean equals(Object obj) {
    if (this == obj)
        return true;
    if (obj == null)
        return false;
    if (getClass() != obj.getClass())
        return false;
    Grid other = (Grid) obj;
    if (matrix == null) {
        if (other.matrix != null)
            return false;
    } else {
        System.out.println("this : " + this);
        System.out.println("other : " + other);
        if (!matrix.equals(other.matrix)) {
            return false;
        }
    }
    System.out.println("this and other are equals");
    return true;
}

With matrix being defined like this :

private ArrayList<ArrayList<String>> matrix = new ArrayList<ArrayList<String>>();

For TicTacToeAction it is a bit more complicated, because it extends DynamicAction, which itself extends a class ObjectWithDynamicAttributes, which implements hashCode and equals, but those part of the code come from the GitHub at the previous link.

For Pair, hashCode and equals are simply defined like this :

@Override
public boolean equals(Object o) {
    if (o instanceof Pair<?, ?>) {
        Pair<?, ?> p = (Pair<?, ?>) o;
        return a.equals(p.a) && b.equals(p.b);
    }
    return false;
}

@Override
public int hashCode() {
    return a.hashCode() + 31 * b.hashCode();
}

But I have some huge problems when the program try to use put and get methods from the QLearningAgent class. It appends that it seldom retrieves previously entered values, and worse, the keySet() seems to contain duplicate elements! Moreover, thanks to prints in equals methods, I saw that it is a bit like if the put and get methods did not check the entire KeySet to see if the entry exists, so the same entries are put again and again... However, when I try to put values by hand in a similar HashMap, defined with the same classes, such problems don't seem to happend.

Maybe the most ununderstandable example is this one :

This part of code :

    for(Pair<S, A> pair : Q.keySet()) {
        System.out.println(pair);
        System.out.println("Contains? " + Q.keySet().contains(pair));
        System.out.println("value : " + Q.get(pair));
    }

Returns this (among other things...) :

< -------------
| 1 | O | X |
-------------
| 4 | X | O |
-------------
| X | O | X |
-------------
 , Action[name=Play Cell 6] > 
Contains? false
value : null
< -------------
| 1 | O | X |
-------------
| 4 | X | O |
-------------
| X | O | X |
-------------
 , Action[name=Play Cell 6] > 
Contains? false
value : null

So, firsly, the two pairs look equals but can't be by definition of a Set, but how is it possible for the HashMap's KeySet (and same result replacing it with an HashTable, that can't admit null values so the "value : null" means that no such entry exists in the map) to not retrieve the pair which directly comes from Q.keySet()? Because even if there is a problem in an equals method that I don't see, using contains in this case would be like comparing an instance with itself using equals at some point, no? So I guess the first "this == obj" condition would be checked.

I must be missing something, but I tried a lot of things and I have no more idea, so I hope someone will be able to help me...

Thank you for your precious time,

Paul

Edit 1 : as suggested by @tevemadar, I add prints in order to show hashCodes.

    for(Pair<S, A> pair : Q.keySet()) {
        System.out.println(pair);
        System.out.println("Contain? " + Q.keySet().contains(pair));
        System.out.println("value : " + Q.get(pair));
        System.out.println("Pair hashCode : " + pair.hashCode());
        System.out.println("Grid hashCode : " + pair.getFirst().hashCode());
        System.out.println("Action hashCode : " + pair.getSecond().hashCode());
    } 

And here is a returned example :

< -------------
| 1 | X | O |
-------------
| 4 | O | X |
-------------
| O | X | O |
-------------
 , Action[name=Play Cell 3] > 
Contain? false
value : null
Pair hashCode : 1710044996
Grid hashCode : 79268846
Action hashCode : -224488982
< -------------
| 1 | X | O |
-------------
| 4 | O | X |
-------------
| O | X | O |
-------------
 , Action[name=Play Cell 3] > 
Contain? false
value : null
Pair hashCode : 1710044996
Grid hashCode : 79268846
Action hashCode : -224488982

So both elements that both belong to the keySet() seem to have the same hashCodes too.

Answer 1

Actually, this is because the key Pair in this case is mutable, and you can change it internal attributes.

Consider the below example, we have two keys, with different 'names', and then .put will not see them same, since they have different hashCode, however, similar to your case, you seem to change the attributes inside that pair, and in this case, the map will not remove the duplicate, since it does this only on put.

class MyAction extends DynamicAction {
    public MyAction(String name) {
        super(name);
    }
}

@Test
public void test() {
    Map<Pair<String, MyAction>, Double> Q = new Hashtable<Pair<String, MyAction>, Double>();
    Pair<String, MyAction> pair1 = new Pair<String, MyAction>(
            "Play Cell", new MyAction("something1"));
    Pair<String, MyAction> pair2 = new Pair<String, MyAction>(
            "Play Cell", new MyAction("something2"));

    // different hashcodes
    System.out.println(pair1.hashCode());
    System.out.println(pair2.hashCode());
    Q.put(pair1, 1d);
    Q.put(pair2, 1d);
    // so the size is 2
    System.out.println(Q.size());

    // however, we can change the hashcode of the key afterwards
    System.out.println("Setting attributes");
    pair2.getSecond().setAttribute(DynamicAction.ATTRIBUTE_NAME, "something1");
    for (Object o : Q.keySet()) {
        Pair p = (Pair) o;
        System.out.println(p.hashCode());
    }
    // and the size is still 2
    System.out.println(Q.size());
}