Sort a list of tuples with None values in Python3

Question

In Python 3 the ordering comparison operators (<, <=, >=, >) raise a TypeError exception when the operands don’t have a meaningful natural ordering

This change in comparison creates difficulties to order a list of tuples, if there exist None values.

In Python 2:

>>> unordered_list = [('3', '1', None), ('3','1', '4'), ('3', '1', None)]
>>> sorted(unordered_list, reverse=True)

[('3', '1', '4'), ('3', '1', None), ('3', '1', None)]

In Python 3:

>>> unordered_list = [('3', '1', None), ('3','1', '4'), ('3', '1', None)]
>>> sorted(unordered_list, reverse=True)

TypeError: unorderable types: str() < NoneType()

Do you have any idea how to have the same behavior with Python 2 an elegant way?

NOTE: In the above examples I have integers, but it's only an example. The elements of the tuples will have the same type and they could be any type.

ex.2 [('test3','test1', 'test4'), ('test3', 'test1', None)]
ex.3 [( 3, 1, 4), (3, 1, None)]
ex.4 [( 3.1, 1.1, 4.1), (3.1, 1.1, None)]

Have fun re-implementing the old [comparison rules](https://docs.python.org/2/reference/expressions.html#value-comparisons) :-P — Kelly Bundy, Mar 15 '20 at 17:03
*"In the above examples I have integers"* - No you don't. Those are strings. — Kelly Bundy, Mar 15 '20 at 17:05
Does this answer your question? [Sorting list by an attribute that can be None](https://stackoverflow.com/questions/12971631/sorting-list-by-an-attribute-that-can-be-none) — Querenker, Mar 15 '20 at 17:06
The elegant way is not to have any ``None`` values in your list. Is there a reason why you are required to have this mixed data in the first place? — MisterMiyagi, Mar 15 '20 at 17:18
If `None` is the only problem, I guess you could sort with `key=lambda t: [(v is None, v) for v in t]`. — Kelly Bundy, Mar 15 '20 at 17:21
Yes @MisterMiyagi I know that this is the root of the problem but I can avoid them in my application because are meaningful values in my domain and I can't replace them with zeros or empty strings. — ggeop, Mar 15 '20 at 17:22
A way could be writing a function that compares tuples (or other objects) of disparate types, then using `functools.cmp_to_key(custom_cmp)` wherever you need to in functions that require a key. @ggeop — Todd, Mar 16 '20 at 01:33

score 1 · Answer 1 · answered Mar 15 '20 at 16:56

1

In this case you can use a custom key function to translate all Nones to an empty string, eg:

sorted(unordered_list, key=lambda L: tuple(el or '' for el in L), reverse=True)

Which'll give you:

[('3', '1', '4'), ('3', '1', None), ('3', '1', None)]

answered Mar 15 '20 at 16:56

Jon Clements

124,071
31
219
256

Hi, thanks for your answer!But this answer covers only the case which we have strings. I want a generic solution, for other types :-) – ggeop Mar 15 '20 at 16:58
@ggeop You can incorporate a `if-else` in your tuple comprehension if the values is `None` store `float('-inf')` – Ch3steR Mar 15 '20 at 17:00
1

@ggeop there isn't really a generic case... you can do what Ch3steR has suggested and use negative/positive inf. floats if you know you've got numbers to compare against, but then that'll break your example with strings... – Jon Clements Mar 15 '20 at 17:01
I know that is difficult, thats the reason for the question :-) I have add more examples in the question in order to be more clear. – ggeop Mar 15 '20 at 17:05
@ggeop difficult is one way of describing it... what about `[('bob', '1', 'x'), (None, 1, 3), (3.0, '1', None)]` ? – Jon Clements Mar 15 '20 at 17:05
I would like to have the same behavior with Python2. Ok your example its not meaningful, and I don't have this case with mixed types. I will re-edit my question in order to be more clear – ggeop Mar 15 '20 at 17:09
3

@ggeop thanks... can I ask why you want the same behaviour as Py2 though? It's a kind of mistake it ever had that behaviour (hence it's gone now) and the ordering is pretty meaningless anyway – Jon Clements Mar 15 '20 at 17:11
Agreed. Doesn't make sense to order None in any way. If anything leaving it to a custom key to handle it in whichever way is wanted seems like the only sensible option – yatu Mar 15 '20 at 17:23

yatu · Answer 2 · 2020-03-15T17:08:20.510

One approach could be to define a custom function as a key prior to sorting, catching the TypeError. Also note that you want to cast the numerical values to int or float, otherwise the ordering process will be lexicographic.

def handle_none(t):
    out = tuple()
    for i in t:
        try:
            out += (float(i),)
        except TypeError:
            out += (-float('inf'),) 
    return out

For the first case:

unordered_list = [('3', '1', '4'), ('3', '1', None), ('3', '1', None)]

sorted(unordered_list, key=handle_none, reverse=True)
# [('3', '1', '4'), ('3', '1', None), ('3', '1', None)]

Another example:

unordered_list = [( 3.1, 1.1, 4.1), ( 3.1, 1.1, 7.1), (3.1, 1.1, None), ( 3.1, 1.1, 8.1)]

sorted(unordered_list, key=handle_none, reverse=True)
# [(3.1, 1.1, 8.1), (3.1, 1.1, 7.1), (3.1, 1.1, 4.1), (3.1, 1.1, None)]

Sort a list of tuples with None values in Python3

2 Answers2