Why is Python 3.3+ dict ordering not only undefined, but variable?

Question

I'm asking this question out of curiosity (specifically, insight into how python is working under the hood).

I am fully aware that the python dict object is unordered - you can put in a bunch of items, and print out some representation of the dict, and the ordering of the items in the representation will have no correspondence to the order that the items were entered.

What I'm curious about, however, is why the ordering changes from one execution of the code to the next?

I have a really simple python script that is printing a dict to the console. The dict looks like this (content is pretty irrelevant):

{
    'hello': 'hi',
    'goodbye': 'bye',
    'hahaha': 'lol',
}

Printing it to the console causes the items to not only be printed in a random order, but to be printed in a different order each time I run the program. Here's my question: Why is this the case?

NOTE: The python code is inside a django project (but I'm not using any django functionality in this case - just mentioning this in case it's somehow relevant).

Python 2, or Python 3? You're probably running into hash randomization, in which case [this](http://stackoverflow.com/questions/14956313/dictionary-ordering-non-deterministic-in-python3) would be an appropriate dupe target. — user2357112 supports Monica, Dec 10 '15 at 19:02
Python 3? And did you mean 'variable between OSes (Windows, Linux/MacOS, Android, iOS...) and/or between implementations (CPython/PyPi)'? This is a good question but please add detail! — smci, Dec 10 '15 at 19:07
@sudomakeinstall2 - It can be enabled in python 2.7 if you want by setting an environment variable `export PYTHONHASHSEED=random`. — tdelaney, Dec 10 '15 at 19:08
@tdelaney: is there any more Pythonic way/package to get that backward-compatibility in Python 2.x? — smci, Dec 10 '15 at 19:10
@smci how would he know? Do you normally test all of these platforms before asking a question? — tdelaney, Dec 10 '15 at 19:11
@smci I'm not sure of other ways to do it. I suspect that the decision needs to be made early so by the time you could do it in python, its too late. — tdelaney, Dec 10 '15 at 19:12
@smci: It's probably Python 3, but that shouldn't be edited into the question by someone other than the questioner. It'd be more appropriate to say in an answer that the behavior strongly implies Python 3, or dupe-vote for the Python 3 specific dupe. — user2357112 supports Monica, Dec 10 '15 at 19:16
Not an answer to your question, but you can get an ordered dict object in Python 3.3 [as seen here](https://docs.python.org/2/library/collections.html#collections.OrderedDict) — tom, Dec 10 '15 at 19:21
@user2357112: this was already getting downvoted (undeservedly), and needed urgent rescuing before it disappeared. In the unlikely event this is not a dupe of the Python 3.3+ hashseed issue, the OP is perfectly free to edit the tags and title — smci, Dec 10 '15 at 21:35

score 7 · Accepted Answer · edited Dec 10 '15 at 19:13

There is a note about this here: https://docs.python.org/3/reference/datamodel.html#object.__hash__

Here is the note:

Note: By default, the __hash__() values of str, bytes and datetime objects are “salted” with an unpredictable random value. Although they remain constant within an individual Python process, they are not predictable between repeated invocations of Python. This is intended to provide protection against a denial-of-service caused by carefully-chosen inputs that exploit the worst case performance of a dict insertion, O(n^2) complexity. See http://www.ocert.org/advisories/ocert-2011-003.html for details. Changing hash values affects the iteration order of dicts, sets and other mappings. Python has never made guarantees about this ordering (and it typically varies between 32-bit and 64-bit builds).

See also PYTHONHASHSEED.

Changed in version 3.3: Hash randomization is enabled by default.

Why is Python 3.3+ dict ordering not only undefined, but variable?

1 Answers1