31

I'm trying to sort a list of objects using

my_list.sort(key=operator.attrgetter(attr_name))

but if any of the list items has attr = None instead of attr = 'whatever',

then I get a TypeError: unorderable types: NoneType() < str()

In Py2 it wasn't a problem. How do I handle this in Py3?

jsalonen
  • 26,663
  • 14
  • 82
  • 104
AlexVhr
  • 1,726
  • 1
  • 16
  • 28

4 Answers4

31

The ordering comparison operators are stricter about types in Python 3, as described here:

The ordering comparison operators (<, <=, >=, >) raise a TypeError exception when the operands don’t have a meaningful natural ordering.

Python 2 sorts None before any string (even empty string):

>>> None < None
False

>>> None < "abc"
True

>>> None < ""
True

In Python 3 any attempts at ordering NoneType instances result in an exception:

>>> None < "abc"
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: NoneType() < str()

The quickest fix I can think of is to explicitly map None instances into something orderable like "":

my_list_sortable = [(x or "") for x in my_list]

If you want to sort your data while keeping it intact, just give sort a customized key method:

def nonesorter(a):
    if not a:
        return ""
    return a

my_list.sort(key=nonesorter)
jsalonen
  • 26,663
  • 14
  • 82
  • 104
  • Yes, I've allready noticed that. What I'm asking for is a solution. If `attr` was of some `custom_type`, I whould just override `custom_type.__lt__()`, but `None` is a built in. – AlexVhr Oct 19 '12 at 09:59
  • 1
    It wasn't me - but one improvement would be to use `""` instead of `0` as the `None` substitute since the OP appears to be comparing strings. – Tim Pietzcker Oct 19 '12 at 10:08
  • But that will change the actual data inside the objects being sorted, will it not? – AlexVhr Oct 19 '12 at 10:09
  • @AlexVhr: check the edit again for a solution that doesn't change your list. – jsalonen Oct 19 '12 at 10:24
  • 1
    It works, but requires objects to have comparison methods, like `__lt__()` Otherwise it couses `TypeError: unorderable types: MyObj() < MyObj()` – AlexVhr Oct 19 '12 at 10:59
30

For a general solution, you can define an object that compares less than any other object:

from functools import total_ordering

@total_ordering
class MinType(object):
    def __le__(self, other):
        return True

    def __eq__(self, other):
        return (self is other)

Min = MinType()

Then use a sort key that substitutes Min for any None values in the list

mylist.sort(key=lambda x: Min if x is None else x)
augurar
  • 9,426
  • 4
  • 40
  • 57
  • 1
    Not really that general: `TypeError: unorderable types: type() < tuple()` If your list contains tuples. – letmaik Feb 10 '15 at 22:16
  • 1
    @neo You made a mistake in your test. Obviously using this class won't prevent other, unrelated type errors. Notice that neither of the arguments in the type error were of type `MinType`. – augurar Feb 10 '15 at 23:31
  • Ah, sorry, I had used `MinType` (which is a `type`) instead of `Min`. – letmaik Feb 11 '15 at 07:45
  • 1
    Much better than the other solution, IMHO, and very elegant. The kind of thing I'd never have found alone. Thanks a lot ! – JulienD Sep 29 '15 at 10:00
  • 1
    Be aware that this might lead to inconsistent ordering if you have more than one of these objects competing to be the smallest. A safe way would be to have `__le__` compare `id(self)` and `id(other)` when the two objects are both instances of `MinType`. – T Tse May 31 '18 at 01:08
  • 1
    @ShioT The intent is to use the `Min` object as a singleton. – augurar Jun 18 '18 at 20:07
  • @augurar Hmmm.. Sorry sometimes my Java instinct still kicks in and tell me to defensively make the `__init__` method throw an exception and have a `getInstance` class method / static method instead – T Tse Jun 21 '18 at 01:41
  • Yeah, this is more of a "sketch" of the approach, but you could certainly make it more robust in various ways. – augurar Jun 21 '18 at 20:33
  • How do set an attribute this solution compares objects by? Like, I want to sort instances of a class `Person` by age. The standard way would be `my_list.sort(key=operator.attrgetter('age'))`. How do I do that with this solution? – AlexVhr Jul 04 '20 at 14:28
10

The solutions proposed here work, but this could be shortened further:

mylist.sort(key=lambda x: x or 0)

In essence, we can treat None as if it had value 0.

E.g.:

>>> mylist = [3, 1, None, None, 2, 0]
>>> mylist.sort(key=lambda x: x or 0)
>>> mylist
[None, None, 0, 1, 2, 3]
fralau
  • 2,040
  • 18
  • 33
  • doesn't work if you're sorting numerical lists with possible None, eg `[3,1,None,2]` – alancalvitti Nov 01 '19 at 15:52
  • That puts `None` arbitrarily in the middle of the list if there's negatives and nonnegatives, eg `[3, 1, None, None, -2, 0]` – alancalvitti Nov 01 '19 at 18:52
  • Yes. The question was about removing the error, and that removed the error. Since it did not specify what one was supposed to do with the None, I left the Nones there. – fralau Nov 02 '19 at 06:18
3

Since there are other things besides None that are not comparable to a string (ints and lists, for starters), here is a more robust solution to the general problem:

my_list.sort(key=lambda x: x if isinstance(x, str) else "")

This will let strings and any type derived from str to compare as themselves, and bin everything else with the empty string. Or substitute a different default default key if you prefer, e.g. "ZZZZ" or chr(sys.maxunicode) to make such elements sort at the end.

alexis
  • 43,587
  • 14
  • 86
  • 141