Django Tastypie Advanced Filtering: How to do complex lookups with Q objects

Question

I have a basic Django model like:

class Business(models.Model):
    name = models.CharField(max_length=200, unique=True)
    email = models.EmailField()
    phone = models.CharField(max_length=40, blank=True, null=True)
    description = models.TextField(max_length=500)

I need to execute a complex query on the above model like:

qset = (
    Q(name__icontains=query) |
    Q(description__icontains=query) |
    Q(email__icontains=query)
    )
results = Business.objects.filter(qset).distinct()

I have tried the following using tastypie with no luck:

def build_filters(self, filters=None):
    if filters is None:
        filters = {}
    orm_filters = super(BusinessResource, self).build_filters(filters)

    if('query' in filters):
        query = filters['query']
        print query
        qset = (
                Q(name__icontains=query) |
                Q(description__icontains=query) |
                Q(email__icontains=query)
                )
        results = Business.objects.filter(qset).distinct()
        orm_filters = {'query__icontains': results}

    return orm_filters

and in class Meta for tastypie I have filtering set as:

filtering = {
        'name: ALL,
        'description': ALL,
        'email': ALL,
        'query': ['icontains',],
    }

Any ideas to how I can tackle this?

Thanks - Newton

score 42 · Accepted Answer · edited Feb 07 '13 at 22:40

You are on the right track. However, build_filters is supposed to transition resource lookup to an ORM lookup.

The default implementation splits the query keyword based on __ into key_bits, value pairs and then tries to find a mapping between the resource looked up and its ORM equivalent.

Your code is not supposed to apply the filter there only build it. Here is an improved and fixed version:

def build_filters(self, filters=None):
    if filters is None:
        filters = {}
    orm_filters = super(BusinessResource, self).build_filters(filters)

    if('query' in filters):
        query = filters['query']
        qset = (
                Q(name__icontains=query) |
                Q(description__icontains=query) |
                Q(email__icontains=query)
                )
        orm_filters.update({'custom': qset})

    return orm_filters

def apply_filters(self, request, applicable_filters):
    if 'custom' in applicable_filters:
        custom = applicable_filters.pop('custom')
    else:
        custom = None

    semi_filtered = super(BusinessResource, self).apply_filters(request, applicable_filters)

    return semi_filtered.filter(custom) if custom else semi_filtered

Because you are using Q objects, the standard apply_filters method is not smart enough to apply your custom filter key (since there is none), however you can quickly override it and add a special filter called "custom". In doing so your build_filters can find an appropriate filter, construct what it means and pass it as custom to apply_filters which will simply apply it directly rather than trying to unpack its value from a dictionary as an item.

Dictionary has no method 'extend'. Should be: orm_filters.update({'custom': qset}) — Karmo Rosental, Dec 05 '12 at 22:06
This solution causes calling the DB twice (for semi_filtered and then for custom filter). A slightly different code works for me: if 'custom' in applicable_filters: custom = applicable_filters.pop('custom') return Outreaches.objects.filter(custom) else: return super(OutreachResource, self).apply_filters(request, applicable_filters) — Israel Zalmanov, Sep 23 '13 at 22:16

Ted · Answer 2 · 2012-08-20T21:30:53.863

I solved this problem like so:

Class MyResource(ModelResource):

  def __init__(self, *args, **kwargs):
    super(MyResource, self).__init__(*args, **kwargs)
    self.q_filters = []

  def build_filters(self, filters=None):
    orm_filters = super(MyResource, self).build_filters(filters)

    q_filter_needed_1 = []
    if "what_im_sending_from_client" in filters:
      if filters["what_im_sending_from_client"] == "my-constraint":
        q_filter_needed_1.append("something to filter")

    if q_filter_needed_1:
      a_new_q_object = Q()
      for item in q_filter_needed:
        a_new_q_object = a_new_q_object & Q(filtering_DB_field__icontains=item)
      self.q_filters.append(a_new_q_object)

  def apply_filters(self, request, applicable_filters):
    filtered = super(MyResource, self).apply_filters(request, applicable_filters)

    if self.q_filters:
      for qf in self.q_filters:
        filtered = filtered.filter(qf)
      self.q_filters = []

    return filtered

This method feels like a cleaner separation of concerns than the others that I've seen.

It's a really bad idea to put request-specific information on a resource instance. So `self.q_filters.append(a_new_q_object)`. This is because in a deployed environment with multiple threads, you might end up with one request's state influencing another's. So for example, all the filters built up in one request could actually be applied to a completely different one, depending on the timing. See the docs here: http://django-tastypie.readthedocs.io/en/latest/resources.html#why-class-based This is the problem that passing a `bundle` object around everywhere solves. — Cameron Lee, Dec 04 '17 at 22:28

Cameron Lee · Answer 3 · 2017-12-05T18:20:45.637

Taking the idea in astevanovic's answer and cleaning it up a bit, the following should work and is more succinct.

The main difference is that apply_filters is made more robust by using None as the key instead of custom (which could conflict with a column name).

def build_filters(self, filters=None):
    if filters is None:
        filters = {}
    orm_filters = super(BusinessResource, self).build_filters(filters)

    if 'query' in filters:
        query = filters['query']
        qset = (
                Q(name__icontains=query) |
                Q(description__icontains=query) |
                Q(email__icontains=query)
                )
        orm_filters.update({None: qset}) # None is used as the key to specify that these are non-keyword filters

    return orm_filters

def apply_filters(self, request, applicable_filters):
    return self.get_object_list(request).filter(*applicable_filters.pop(None, []), **applicable_filters)
    # Taking the non-keyword filters out of applicable_filters (if any) and applying them as positional arguments to filter()

Django Tastypie Advanced Filtering: How to do complex lookups with Q objects

3 Answers3

Linked

Related