36

All the docs I've seen imply that you might be able to do that, but there isn't anything official w/r/t ulong64/uint64 fields. There are a few off-the-shelf options that look quite promising in this arena:

  • BigIntegerField ... almost, but signed;
  • PositiveIntegerField ... suspiciously 32-bit-looking; and
  • DecimalField ... a fixed-pointer represented with a python decimal type, according to the docs -- which presumably turns into an analogously pedantic and slow database field when socked away, á la the DECIMAL or NUMERIC PostgreSQL types.

... all of which look like they might store a number like that. Except NONE OF THEM WILL COMMIT, much like every single rom-com character portrayed by Hugh Grant.

My primary criterion is that it works with Django's supported backends, without any if postgresql (...) elif mysql (...) type of special-case nonsense. After that, there is the need for speed -- this is for a model field in an visual-database application that will index image-derived data (e.g. perceptual hashes and extracted keypoint features), allowing ordering and grouping by the content of those images.

So: is there a good Django extension or app that furnishes some kind of PositiveBigIntegerField that will suit my purposes?

And, barring that: If there is a simple and reliable way to use Django's stock ORM to store unsigned 64-bit ints, I'd like to know it. Look, I'm no binary whiz; I have to do two's complement on paper -- so if this method of yours involves some bit-shifting trickery, don't hesitate to explain what it is, even if it strikes you as obvious. Thanks in advance.

fish2000
  • 3,985
  • 2
  • 37
  • 67
  • 8
    +1 for `much like every single rom-com character portrayed by Hugh Grant` made me chuckle on a dreary and humid morning. – Burhan Khalid May 21 '12 at 04:29

1 Answers1

26

Although I did not test it, but you may wish to just subclass BigIntegerField. The original BigIntegerField looks like that (source here):

class BigIntegerField(IntegerField):
    empty_strings_allowed = False
    description = _("Big (8 byte) integer")
    MAX_BIGINT = 9223372036854775807

    def get_internal_type(self):
        return "BigIntegerField"

    def formfield(self, **kwargs):
        defaults = {'min_value': -BigIntegerField.MAX_BIGINT - 1,
                    'max_value': BigIntegerField.MAX_BIGINT}
        defaults.update(kwargs)
        return super(BigIntegerField, self).formfield(**defaults)

Derived PositiveBigIntegerField may looks like this:

class PositiveBigIntegerField(BigIntegerField):
    empty_strings_allowed = False
    description = _("Big (8 byte) positive integer")

    def db_type(self, connection):
        """
        Returns MySQL-specific column data type. Make additional checks
        to support other backends.
        """
        return 'bigint UNSIGNED'

    def formfield(self, **kwargs):
        defaults = {'min_value': 0,
                    'max_value': BigIntegerField.MAX_BIGINT * 2 - 1}
        defaults.update(kwargs)
        return super(PositiveBigIntegerField, self).formfield(**defaults)

Although you should test it thoroughly, before using it. If you do, please share the results :)

EDIT:

I missed one thing - internal database representation. This is based on value returned by get_internal_type() and the definition of the column type is stored eg. here in case of MySQL backend and determined here. It looks like overwriting db_type() will give you control over how the field is represented in the database. However, you will need to find a way to return DBMS-specific value in db_type() by checking connection argument.

Brad Solomon
  • 29,156
  • 20
  • 104
  • 175
Tadeck
  • 117,059
  • 25
  • 140
  • 191
  • 1
    Wow, that looks too good to be true, so to speak -- I'm not too concerned about validating human input via `FormField` (as this datum will come from an image analysis function) and if you take that stuff out, we're left with the `description` text, the `MAX_BIGINT` attribute, and the `get_internal_type` specification... which none of those additions let the subclass tell the ORM how to treat it like a uint64. I could very well be mistaken -- if I'm missing something here with any of these, do let me know. – fish2000 May 21 '12 at 00:02
  • 1
    @fish2000: You were right, see my edit. The part I was missing is `db_field()` that was meant to return DBMS-specific type based on internal type returned by `get_internal_type()` and looked up in DBMS-specific `data_types` dictionary (seen [here](https://github.com/django/django/blob/master/django/db/backends/mysql/creation.py#L8)). Does it solve your problem now? – Tadeck May 21 '12 at 05:19
  • Nice! It's good to know where the ORM stashes its necessary heap of proprietary edge-case SQL fragments -- I hadn't run across that before and it's what one needs to implement `get_internal_type()` methods with the same vendor-specific logic as Django itself. That's a great tip, thank you! – fish2000 May 21 '12 at 22:04
  • 1
    As far as I can see, the `db_field` method should be `db_type` (submitted an edit which is waiting on peer review). – Tikitu Mar 01 '13 at 14:13
  • 2
    This also works for SQLite, however the actual stored value will be a **signed** 64bit INTEGER, so the max allowed value will be 2 ^ 63 - 1 – Tzach Jul 20 '15 at 13:51
  • Small fix. In 'max_value' expression, there should be +1 instead of -1. The true max value is 2^64-1 = 18446744073709551615, while the expression above calculates 18446744073709551613. The exact value could be hard-coded, by the way, similar to BigIntegerField implementation. – Marcin Wojnarski Mar 05 '20 at 16:19