PERL-like autovivification with default value in Python, and returns a default value from non-existing arbitrary nesting?

Question

Suppose I want PERL-like autovivication in Python, i.e.:

>>> d = Autovivifier()
>>> d = ['nested']['key']['value']=10
>>> d
{'nested': {'key': {'value': 10}}}

There are a couple of dominant ways to do that:

1. Use a recursive default dict
2. Use a __missing__ hook to return the nested structure

OK -- easy.

Now suppose I want to return a default value from a dict with a missing key. Once again, few way to do that:

1. For a non-nested path, you can use a __missing__ hook
2. try/except block wrapping the access to potentially missing key path
3. Use {}.get(key, default) (does not easily work with a nested dict) i.e., There is no version of autoviv.get(['nested']['key']['no key of this value'], default)

The two goals seem in irreconcilable conflict (based on me trying to work this out the last couple hours.)

Here is the question:

Suppose I want to have an Autovivifying dict that 1) creates the nested structure for d['arbitrary']['nested']['path']; AND 2) returns a default value from a non-existing arbitrary nesting without wrapping that in try/except?

Here are the issues:

1. The call of d['nested']['key']['no key of this value'] is equivalent to (d['nested'])['key']['no key of this value']. Overiding __getitem__ does not work without returning an object that ALSO overrides __getitem__.
2. Both the methods for creating an Autovivifier will create a dict entry if you test that path for existence. i.e., I do not want if d['p1']['sp2']['etc.'] to create that whole path if you just test it with the if.

How can I provide a dict in Python that will:

1. Create an access path of the type d['p1']['p2'][etc]=val (Autovivication);
2. NOT create that same path if you test for existence;
3. Return a default value (like {}.get(key, default)) without wrapping in try/except
4. I do not need the FULL set of dict operations. Really only d=['nested']['key']['value']=val and d['nested']['key']['no key of this value'] is equal to a default value. I would prefer that testing d['nested']['key']['no key of this value'] does not create it, but would accept that.

?

Answer 1

To create a recursive tree of dictionaries, use defaultdict with a trick:

from collections import defaultdict

tree = lambda: defaultdict(tree)

Then you can create your x with x = tree().

above from @BrenBarn -- defaultdict of defaultdict, nested

Answer 2

Don't do this. It could be solved much more easily by just writing a class that has the operations you want, and even in Perl it's not a universally-appraised feature.

But, well, it is possible, with a custom autoviv class. You'd need a __getitem__ that returns an empty autoviv dict but doesn't store it. The new autoviv dict would remember the autoviv dict and key that created it, then insert itself into its parent only when a "real" value is stored in it.

Since an empty dict tests as falsey, you could then test for existence Perl-style, without ever actually creating the intermediate dicts.

But I'm not going to write the code out, because I'm pretty sure this is a terrible idea.

Answer 3

While it does not precisely match the dictionary protocol in Python, you could achieve reasonable results by implementing your own auto-vivification dictionary that uses variable getitem arguments. Something like (2.x):

class ExampleVivifier(object):
    """ Small example class to show how to use varargs in __getitem__. """

    def __getitem__(self, *args):
        print args

Example usage would be:

>>> v = ExampleVivifier()
>>> v["nested", "dictionary", "path"]
(('nested', 'dictionary', 'path'),)

You can fill in the blanks to see how you can achieve your desired behaviour here.