Python: Dictionary merge by updating but not overwriting if value exists

Question

If I have 2 dicts as follows:

d1 = {('unit1','test1'):2,('unit1','test2'):4}
d2 = {('unit1','test1'):2,('unit1','test2'):''}

In order to 'merge' them:

z = dict(d1.items() + d2.items())
z = {('unit1','test1'):2,('unit1','test2'):''}

Works fine. Additionally what to be done, if i would like to compare each value of two dictionaries and only update d2 into d1 if values in d1 are empty/None/''?

[EDIT] Question: When updating d2 into d1, when the same key exists, I would like to only maintain the numerical value (either from d1 or d2) instead of empty value. If both values are empty, then no problems maintaining empty value. If both have values, then d1-value should stay. :) (lotsa if-else .. i'd try myself in the meantime)

i.e.

d1 = {('unit1','test1'):2,('unit1','test2'):8,('unit1','test3'):''}
d2 = {('unit1','test1'):2,('unit1','test2'):'',('unit1','test3'):''}

#compare & update codes

z = {('unit1','test1'):2,('unit1','test2'):8, ('unit1','test2'):''} # 8 not overwritten by empty.

please help to suggest.

Thanks.

Answer 1

Just switch the order:

z = dict(d2.items() + d1.items())

By the way, you may also be interested in the potentially faster update method.

In Python 3, you have to cast the view objects to lists first:

z = dict(list(d2.items()) + list(d1.items())) 

If you want to special-case empty strings, you can do the following:

def mergeDictsOverwriteEmpty(d1, d2):
    res = d2.copy()
    for k,v in d2.items():
        if k not in d1 or d1[k] == '':
            res[k] = v
    return res

Answer 2

Python 2.7. Updates d2 with d1 key/value pairs, but only if d1 value is not None,'' (False):

>>> d1 = dict(a=1,b=None,c=2)
>>> d2 = dict(a=None,b=2,c=1)
>>> d2.update({k:v for k,v in d1.iteritems() if v})
>>> d2
{'a': 1, 'c': 2, 'b': 2}

Answer 3

To add to d2 keys/values from d1 which do not exist in d2 without overwriting any existing keys/values in d2:

temp = d2.copy()
d2.update(d1)
d2.update(temp)

Answer 4

Here's an in-place solution (it modifies d2):

# assumptions: d2 is a temporary dict that can be discarded
# d1 is a dict that must be modified in place
# the modification is adding keys from d2 into d1 that do not exist in d1.

def update_non_existing_inplace(original_dict, to_add):
    to_add.update(original_dict) # to_add now holds the "final result" (O(n))
    original_dict.clear() # erase original_dict in-place (O(1))
    original_dict.update(to_add) # original_dict now holds the "final result" (O(n))
    return

Here's another in-place solution, which is less elegant but potentially more efficient, as well as leaving d2 unmodified:

# assumptions: d2 is can not be modified
# d1 is a dict that must be modified in place
# the modification is adding keys from d2 into d1 that do not exist in d1.

def update_non_existing_inplace(original_dict, to_add):
    for key in to_add.iterkeys():
        if key not in original_dict:
            original_dict[key] = to_add[key]

Answer 5

d2.update(d1) instead of dict(d2.items() + d1.items())

Answer 6

Merging Only Non-zero values

to do this we can just create a dict without the empty values and then merge them together this way:

d1 = {'a':1, 'b':1, 'c': '', 'd': ''}
d2 = {'a':2, 'c':2, 'd': ''}
merged_non_zero = {
    k: (d1.get(k) or d2.get(k))
    for k in set(d1) | set(d2)
}
print(merged_non_zero)

outputs:

{'a': 1, 'b': 1, 'c': 2, 'd': ''}

• a -> prefer first value from d1 as 'a' exists on both d1 and d2
• b -> only exists on d1
• c -> non-zero on d2
• d -> empty string on both

Explanation

The above code will create a dictionary using dict comprehension.

if d1 has the value and its non-zero value (i.e. bool(val) is True), it'll use d1[k] value, otherwise it'll take d2[k].

notice that we also merge all keys of the two dicts as they may not have the exact same keys using set union - set(d1) | set(d2).

Python 3.5+ Literal Dict

unless using obsolete version of python you better off using this.

Pythonic & faster way for dict unpacking:

d1 = {'a':1, 'b':1}
d2 = {'a':2, 'c':2}
merged = {**d1, **d2}  # priority from right to left
print(merged)

{'a': 2, 'b': 1, 'c': 2}

its simpler and also faster than the dict(list(d2.items()) + list(d1.items())) alternative:

d1 = {i: 1 for i in range(1000000)}
d2 = {i: 2 for i in range(2000000)}

%timeit dict(list(d1.items()) + list(d2.items())) 
402 ms ± 33.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

%timeit {**d1, **d2}
144 ms ± 1.12 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

more on this from PEP448:

The keys in a dictionary remain in a right-to-left priority order, so {**{'a': 1}, 'a': 2, **{'a': 3}} evaluates to {'a': 3}. There is no restriction on the number or position of unpackings.

Answer 7

In case when you have dictionaries with the same size and keys you can use the following code:

dict((k,v if k in d2 and d2[k] in [None, ''] else d2[k]) for k,v in d1.iteritems())

Answer 8

If you want to ignore empty spaces so that for example merging:

a = {"a": 1, "b": 2, "c": ""}
b = {"a": "", "b": 4, "c": 5}
c = {"a": "aaa", "b": ""}
d = {"a": "", "w": ""}

results in:{'a': 'aaa', 'b': 4, 'c': 5, 'w': ''}

You can use these 2 functions:

def merge_two_dicts(a, b, path=None):
    "merges b into a"
    if path is None:
        path = []
    for key in b:
        if key in a:
            if isinstance(a[key], dict) and isinstance(b[key], dict):
                merge_two_dicts(a[key], b[key], path + [str(key)])
            elif a[key] == b[key]:
                pass  # same leaf value
            else:
                if a[key] and not b[key]:
                    a[key] = a[key]
                else:
                    a[key] = b[key]
        else:
            a[key] = b[key]
    return a


def merge_multiple_dicts(*a):
    output = a[0]
    if len(a) >= 2:
        for n in range(len(a) - 1):
            output = merge_two_dicts(output, a[n + 1])

    return output

So you can just use merge_multiple_dicts(a,b,c,d)