How do I Implement a Multi-Key Hash in Perl as a Nested Dictionary in Python?

Question

I have some basic code to deal with a Perl hash where I can address the elements like: $data{"WV2"}{789}{PP1} (or use that actual text in an assignment) ...but I would like to do something like that using Python dictionaries.

A couple of simple programs in both Perl and Python that illustrate what I've been trying to replicate follow:-

So, the Perl code:-

# hash.pl

use strict;
use warnings;

use Data::Dumper;

my %data = ();

my @reg_list = ( "MC1", "CA2", "WV2" );
my @site_list = ( 123, 456, 391, 287 );

$data{MC1}{4564}{PP}{1} = "-15,-15C";
$data{MC1}{4564}{PP}{2} = "5,5C";
$data{MC1}{4564}{PP}{3} = "-19,-19C";
$data{MC1}{4564}{PP}{4} = "-12,-12C";

printf("---- One:\n");
print Dumper(%data);                 # Ok, shows the full strucure

printf("---- Two:\n");
print Dumper($data{"MC2"});          # Shows as undef (sensible)

printf("---- Three:\n");
print Dumper($data{"MC1"});          # Ok, showing the key:values for each "site" key

printf("---- Four:\n");
print Dumper($data{"MC1"}{"4564"});  # Ok, shows the actual equality value above


# ---- This works Ok

my %xdata = ();
$xdata{"MC1"}{123}{"PP"} = "-15,-15C";
$xdata{"MC1"}{456}{"PP"} = "5,5C";
$xdata{"MC1"}{391}{"PP"} = "-19,-19C";
$xdata{"MC1"}{287}{"PP"} = "-12,-12C";

printf("---- One:\n");
print Dumper(%xdata);                # Ok, shows the full strucure


#pprint.pprint(data["MC2"]) 
#pprint.pprint(data["MC1"}{391]) 


# [eof]

...and the Python code:-

# dict.py

import pprint
import collections

reg_list  = [ "MC1", "CA2", "WV2" ]
site_list = [ 123, 456, 391, 287 ]

#data = {}
data = collections.defaultdict(dict) # {}
data["MC1"][123] = "-15,-15C"
data["MC1"][456] = "5,5C"
data["MC1"][391] = "-19,-19C"
data["MC1"][287] = "-12,-12C"

print("---- One:")
pprint.pprint(data)              # Ok, shows the full strucure

print("---- Two:")
pprint.pprint(data["MC2"])       # Shows: {} [...Ok, undefined...]

print("---- Three:")
pprint.pprint(data["MC1"])       # Ok, showing the key:values for each "site" key

print("---- Four:")
pprint.pprint(data["MC1"][391])  # Ok, shows the actual equality value above

# ---- Cannot get the following to work

xdata = collections.defaultdict(dict) # {}
xdata["MC1"][123]["PP"] = "-15,-15C"  # ERROR: Key error 123
xdata["MC1"][456]["PP"] = "5,5C"
xdata["MC1"][391]["PP"] = "-19,-19C"
xdata["MC1"][287]["PP"] = "-12,-12C"

#pprint.pprint(data["MC2"]) 
#pprint.pprint(data["MC1"][391]) 

# [eof]

Outputs from each of the programs follow:-

# Perl Output:

---- One:
$VAR1 = 'MC1';
$VAR2 = {
          '4564' => {
                      'PP' => {
                                '4' => '-12,-12C',
                                '1' => '-15,-15C',
                                '3' => '-19,-19C',
                                '2' => '5,5C'
                              }
                    }
        };
---- Two:
$VAR1 = undef;
---- Three:
$VAR1 = {
          '4564' => {
                      'PP' => {
                                '4' => '-12,-12C',
                                '1' => '-15,-15C',
                                '3' => '-19,-19C',
                                '2' => '5,5C'
                              }
                    }
        };
---- Four:
$VAR1 = {
          'PP' => {
                    '4' => '-12,-12C',
                    '1' => '-15,-15C',
                    '3' => '-19,-19C',
                    '2' => '5,5C'
                  }
        };
---- One:
$VAR1 = 'MC1';
$VAR2 = {
          '391' => {
                     'PP' => '-19,-19C'
                   },
          '456' => {
                     'PP' => '5,5C'
                   },
          '123' => {
                     'PP' => '-15,-15C'
                   },
          '287' => {
                     'PP' => '-12,-12C'
                   }
        };

...and from the Python:-

# Python Output:-

---- One:
defaultdict(<class 'dict'>,
            {'MC1': {123: '-15,-15C',
                     287: '-12,-12C',
                     391: '-19,-19C',
                     456: '5,5C'}})
---- Two:
{}
---- Three:
{123: '-15,-15C', 287: '-12,-12C', 391: '-19,-19C', 456: '5,5C'}
---- Four:
'-19,-19C'
Traceback (most recent call last):
  File "C:\Projects\00-Development\LXQuery\CDB-Review\dict.py", line 30, in <module>
    xdata["MC1"][123]["PP"] = "-15,-15C"  # ERROR: Key error 123
KeyError: 123

I've tried to look-up info about Nesting Dictionaries... but everything I've looked at doesn't clearly explain how the concept is supposed to work (to my mind, anyway).... particularly when there are 'deeper' levels of the dictionaries in use.

I've been writing Perl code for ~25 years but am only starting with Python.

Running ActiveState Perl v5.16.3, Build 1603 and Anaconda Python 3.6.5 under Windows 10 x64.

Thanks a lot for any thoughts or suggestions.

The problem is you are initialising your `defaultdict` to return (normal) `dict` instances when a key is not found and that's why the second dimension fails. Not marking as a duplicate because it is not exactly the same question but have a look at [this](https://stackoverflow.com/questions/2600790/multiple-levels-of-collection-defaultdict-in-python) — Selcuk, Feb 18 '19 at 03:17
Thanks for pointer, Selcuk.. I only used the defaultdict as I thought it would remove the need to initialize through the entire structure (as @mob identified in my thinking)... but it seems like I will still need to do that, so maybe defaultdict wasn't actually the best choice. More study to do, obviously... — Skeeve, Feb 18 '19 at 04:23

score 1 · Accepted Answer · answered Feb 18 '19 at 03:27

Python does not autovivify multilevel dictionaries the way that Perl does with its hashes. At the second and deeper levels, you have to assign an empty dict to the higher level dicts before you add more keys to them:

xdata = collections.defaultdict(dict)
xdata["MC1"] = collections.defaultdict(dict)
xdata["MC1"][123]["PP"] = "-15,-15C"  # ERROR: Key error 123
xdata["MC1"][456]["PP"] = "5,5C"
xdata["MC1"][391]["PP"] = "-19,-19C"
xdata["MC1"][287]["PP"] = "-12,-12C"

Skeeve · Answer 2 · 2019-02-20T02:36:47.980

The simple way around the problem seems to be something like:-

xdata = collections.defaultdict(dict) # {}
xdata["MC1"][123] = {}                # Define the dict before using it
xdata["MC1"][123]["PP"] = "-15,-15C"  # Works Ok

...but that still means I have to 'manually' define a dict every time I 'discover' a new 'value'... blah

Notwithstanding the inherent Gotcha!'s with mis-typing and (possible) corruption of the dict content, What is the best way to implement nested dictionaries? seems to be a good way to deal with the problem... particularly as the values (in my current application, anyway) "never see the light of day" (they are machine generated and validated before coming into my application)... So, some code that would likely work might be:-

class Vividict(dict):
    def __missing__(self, key):
        value = self[key] = type(self)() # retain local pointer to value
        return value                     # faster to return than 
                                         # ...'dict lookup'

ydata = Vividict()

ydata["MC1"][123]["PP"] = "-15,-15C"
ydata["MC1"][456]["PP"] = "5,5C"
ydata["MC1"][391]["PP"] = "-19,-19C"
ydata["MC1"][287]["PP"] = "-12,-12C"

pprint.pprint(ydata)              # Ok, shows the full strucure

Thanks a heap for the suggestions and pointers.

How do I Implement a Multi-Key Hash in Perl as a Nested Dictionary in Python?

2 Answers2