python3 additional problem with nested default dict

Question

In question Nested defaultdict of defaultdict is described how to create a nested dictionary. This works so far until I want to increment a non-existing key. See the following example:

#!/usr/bin/python3

from collections import defaultdict

mydict = defaultdict(lambda: 0)
print("mydict['a'] =", mydict['a'])
mydict['b'] = 2
print("mydict['b'] =", mydict['b'])
mydict['c'] += 4
print("mydict['c'] =", mydict['c'])

def rec_dd():
    return defaultdict(rec_dd)

x = rec_dd()
x['a']['b']['c']['d'] = 2
print("x['a']['b']['c']['d'] =", x['a']['b']['c']['d'])

x['a']['b']['c']['e'] += 2
print("x['a']['b']['c']['e'] =", x['a']['b']['c']['e'])

which creates the following output

mydict['a'] = 0
mydict['b'] = 2
mydict['c'] = 4
x['a']['b']['c']['d'] = 2
Traceback (most recent call last):
  File "./nested.py", line 19, in <module>
    x['a']['b']['c']['e'] += 2
TypeError: unsupported operand type(s) for +=: 'collections.defaultdict' and 'int'

For the defaultdict with just one level it is all right to add a value to a non-existing key (mydict['c'] += 4), for the nested dictionary this does not work, the line x['a']['b']['c']['e'] += 2 fails. How can I achieve this?

Your nested defaultdict is **infinitely** nested. If you are only ever going to access 4 layers deep, then make a function that defaults to `0` after 4 layers. — Aplet123, Jan 01 '21 at 14:03
@Aplet123 Got sidetracked by a [similar, recent question](https://stackoverflow.com/questions/65529456/python3-building-nested-dictionary). — MisterMiyagi, Jan 01 '21 at 14:07
Is this increment an occasional one? In which case, the `try`, `except` can be used. — sotmot, Jan 01 '21 at 14:08
Can you clarify what are the rules for a layer to be a dict-to-be-filled or a number-to-be-incremented? Is the fourth layer always a dict of numbers? Is there any reason why you need this to be created dynamically on access, instead of explicitly defining the data structure? — MisterMiyagi, Jan 01 '21 at 14:09
@MisterMiyagi The link you have provided does not solve my problem (it is in fact similar to the question that I have mentioned in my post). My question is an extension to the problem that is described there. I don't want to assign a value, I want to increment it, and therefore I haven't found any solution yet. — drhuhdd, Jan 02 '21 at 10:59

score 1 · Accepted Answer · answered Jan 01 '21 at 15:58

First of all, we need to understand why this is happening and what are we going to sacrifice by fixing this error:

TypeError: unsupported operand type(s) for +=: 'collections.defaultdict' and 'int'

This error tells you that x['a']['b']['c']['e'] += 2 is adding two to a default dictionary. This is obviously happening because the defaultdict sets all new elements to defaultdict. This would allow you to do something like:

x['a']['b']['c']['e']['another_one']

That wouldn't be a problem because just like you've seen from your error, the element at ['e'] is a default dictionary allowing you to further index it with something like ['another_one']. Now we can change your code to make the dictionary have integers at level four, but you would be sacrificing the ability to further index the dictionary to levels beyond 4:

Solution 1 (?)

from functools import partial

def rec_dd(depth=0):
    if depth == 4:
        return 0

    return defaultdict(partial(rec_dd, depth + 1))

The function above returns a default dictionary until you reach level 4 at which you want to have integers instead.

Now let's take a look at the modified function above, it stops at depth 4 resulting in the following behavior:

x['a']['b']['g'] = 2

Will work without any problems, because you're setting the dictionary at depth 3 to an int.

x['a']['b']['h'] + 3

Will not work because everything up to depth 4 is dictionary and you're trying to add an int to a dictionary.

x['a']['b']['c']['d'] = 2
x['a']['b']['c']['e'] += 2

Both of those options will work because the level 4 now contains ints and not dictionaries. HOWEVER:

x['a']['b']['c']['e']['f'] = 2

You sacrifice the ability to index past level 4 because level 4 now has ints and ints are not subscriptable.

This solution would totally fix the example that you have provided but might not be the optimal solution if you want customizable depth. Luckily for you, there are already a lot of very high quality posts showing how to implement a nested dictionary where you can add any behavior to your nested container:

What is the best way to implement nested dictionaries?

I recommend reading through that first if you want further functionality.

Thank you, this answer solved my problem! – drhuhdd Jan 02 '21 at 10:42 — drhuhdd, Jan 02 '21 at 10:42

python3 additional problem with nested default dict

1 Answers1