No side-effect nested dictionary implementation

Question

There are at least two decent implementations of nested dictionary I've come across on SO, one is to use defaultdict, and the other is to subclass dict.

Both methods work fine for most functionalities, except that they both have a side-effect when accessing a non-existent key-value pair: it creates an empty dictionary for that key, stores that and returns it.

Ideally, I would like an implementation which would return None without creating an entry (e.g. an empty dictionary) upon an attempted access to a non-existent key. Is this feasible?

p.s. I am aware that we can avoid nested dictionary by using tuples as keys, but this implementation does not work me as I need to access the collection of entries at each level of the nested dictionary.

Would the `get` method suit your need? d.get(key, None) for dictionary `d`? — beardc, May 28 '12 at 18:11
@skyork That's the same as `d=mydict[key]; d[sub_key]=value`. By your definition, `d` would be `None`. — Janne Karila, May 28 '12 at 18:23
@BirdJaguarIV, that could be a workaround, but I'd still like to see a more proper solution — skyork, May 28 '12 at 18:23

Martijn Pieters · Answer 1 · 2012-05-28T18:24:47.493

The only thing the two implementations you point to do over normal dicts is to return a dict when accessing a non-existing key. You want to revert that again, thus leaving you again with the default dict type:

>>> example = {}
>>> example['foo']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
KeyError: 'foo'
>>> example['foo'] = {}
>>> example['foo']['bar'] = 1

If you want to return None instead of a dict, then just use defaultdict(lambda: None) instead:

>>> from collections import defaultdict
>>> example = defaultdict(lambda: None)
>>> example['foo'] is None
True

Note that you cannot have it both ways; Python first must find the first key and resolve that to a dict before it can look up the second key:

>>> example['foo']['bar']
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is unsubscriptable

Janne Karila · Accepted Answer · 2012-05-29T06:51:32.663

You must give up your requirement of returning None if you wish d[key][subkey] = value to work with missing keys. If d[key] is None, d[key][subkey] = value is equivalent to None[subkey] = value, which cannot work.

What you can do with missing values is to return an empty dict-like object without assigning it to a key just yet. If that object holds a reference to the parent, the assignment can be delayed until there is an explicit assignment down the hierachy.

An example implementation (this is incomplete, you must do more than override setitem to have a fully functional dict subclass):

class NestedDict(dict):
    def __init__(self, parent=None, parentkey=None):
        self.parent = parent
        self.parentkey = parentkey

    def __missing__(self, key):
        return NestedDict(self, key)

    def __setitem__(self, key, value):
        if self.parent is not None:
            self.parent[self.parentkey] = self
            self.parent = None
        super(NestedDict, self).__setitem__(key, value)

>>> d = NestedDict()
>>> d[1][2][3] = 4
>>> d[2]
{}
>>> d.keys()
[1]
>>> d[1][2][3] 
4

An alternative approach would be to override __getitem__ and __setitem__ to do a nested lookup when the key is a tuple. This version gives a KeyError from __getitem__ for missing keys in order to be consistent with regular dict. You can change it easily to return None instead if you wish.

class NestedDict(dict):
    def __getitem__(self, key):
        if isinstance(key, tuple):
            try:
                x = self
                for k in key:
                    x = x[k]
                return x
            except (KeyError, TypeError):
                raise KeyError(key)
        else:
            return super(NestedDict, self).__getitem__(key)

    def __setitem__(self, key, value):
        if isinstance(key, tuple):
            d = self
            for k in key[:-1]:
                d = d.setdefault(k, NestedDict())
            d[key[-1]] = value
        else:
            super(NestedDict, self).__setitem__(key, value)

>>> d = NestedDict()
>>> d[1,2,3] = 4
>>> d[1,2,3]
4
>>> d[1,2,4]
KeyError: (1, 2, 4)
>>> d
{1: {2: {3: 4}}}

the wolf · Answer 3 · 2012-05-28T23:56:03.603

1

Python support duck typing for such situations:

>>> d={}
>>> d[1]='Some'
>>> try:
...    att=d[1]
... except KeyError:
...    att=None
... 
>>> print att
Some
>>> try:
...    att=d[1][2][3]
... except KeyError:
...    att=None
... 
>>> print att
None

Roll that into a class or a function and it should easily support what I think you are looking for.

edited May 28 '12 at 23:56

answered May 28 '12 at 23:27

the wolf

29,808
12
50
71

No side-effect nested dictionary implementation

3 Answers3