32

I use Postgres for production and development, but I'd like to use sqlite to run some tests. I don't see an easy way to configure one engine for tests and another for dev / production. Am I missing something?

jMyles
  • 10,434
  • 6
  • 39
  • 54
  • For now, I'm using this solution, although it seems dreadfully hacky: http://seanhayes.name/2010/01/09/test-database-django/ – jMyles Jun 15 '11 at 04:56

3 Answers3

48

Append the following lines in your settings:

import sys
if 'test' in sys.argv or 'test_coverage' in sys.argv: #Covers regular testing and django-coverage
    DATABASES['default']['ENGINE'] = 'django.db.backends.sqlite3'

Make sure your actual database setting comes before them.

Torsten Bronger
  • 6,770
  • 7
  • 28
  • 39
shanyu
  • 8,990
  • 5
  • 55
  • 68
  • 2
    As I said in the comment, this is essentially what I'm doing now. But don't you think this solution has a bit of the suck? – jMyles Jun 16 '11 at 13:19
  • 1
    I saw your comment just after adding this post :) Nevertheless, this solution is perfectly fine for testing purposes: You run manage.py with a 'test' argument, Python passes it to the code via sys.argv, and the code behaves accordingly. I see nothing unpythonic here, do you? – shanyu Jun 16 '11 at 15:44
  • Well, what if I'm not running manage.py? I might be running django-admin.py some other way or using PyUnit or who knows what else? My point is that django needs to be *aware* of its test database. – jMyles Jun 17 '11 at 17:27
  • But no, I can't say it's unpythonic per se. – jMyles Jun 17 '11 at 17:27
  • 2
    This solution will behave well as long as it receives 'test' as an argument from the command line. OTOH you're right, it would be nice if django provided a way to differentiate test settings. There we have issues more troubling than database settings, for example if your tests manipulate dirs or files under MEDIA_ROOT that's where you'll need hacky stuff. – shanyu Jun 18 '11 at 04:22
  • 8
    Set ENGINE to 'django.db.backends.sqlite3' to avoid a deprecation warning with recent versions of Django. – tobych Sep 22 '11 at 04:11
  • 5
    ^^ And make sure engine is in capitals: 'ENGINE' – funkotron Jul 16 '13 at 15:49
  • Caution with this `"test" in sys.argv`; it may trigger when you don't want it to, e.g. `manage.py collectstatic -i test`. `sys.argv[1] == "test"` is a more precise condition that should not have that problem. – keturn Jan 08 '15 at 23:28
  • Word of caution: You can't use a statement like `Thing.related_object.distinct('field_of_related_object')`—it will work in Postgres, but SQLite doesn't support calling distinct on fields (so if anything you're testing does this, it'll fail when using an SQLite DB) – AmagicalFishy Apr 26 '19 at 19:12
  • Also adding DATABASES['default']['NAME'] = ':memory:' speeds up tests – quux Mar 06 '21 at 08:12
8

This is not a direct answer, but yes, you are missing one big problem - testing a Postgres app on SQLite is tricky - they are so different. I suggest you rather create a ram-disk (e.g. using tmpfs) and create your Postgres test database there. It won't be as fast as SQLite, but possibly an order of magnitude faster than regular Postgres database stored on HDD.

Tomasz Zieliński
  • 15,300
  • 7
  • 55
  • 74
  • Well, most tests don't do anything that differs from one engine to another. In my experience, 9/10 (or more) tests use only the most straightforward manager methods. So, to address your concern, I'd be fine with specifying for particular tests that I want to use postgres. But if I just want to run the test suite in the field (and not be connected to the DB), I really need to be able to do that. That's pretty basic. – jMyles Jun 15 '11 at 20:04
  • 11
    I don't get this answer. In what ways are Postgres and Sqlite so different that a typical Django application (I assume it is using ORM) can't be reliably tested on Sqlite, or another database? Aren't we talking about a database-agnostic application here? – shanyu Jun 16 '11 at 12:06
  • @shanyu: Noone forces you to belive me, if you have different experience then I'm perfectly fine with this. – Tomasz Zieliński Jun 16 '11 at 14:46
  • @Tomasz: Why not share your experience? All in all I may be missing something. – shanyu Jun 16 '11 at 15:48
  • 1
    Ok, let me reconsider and change the word "worthless" - which fits projects I've been working on - to word "tricky". – Tomasz Zieliński Jun 16 '11 at 16:11
  • Ok guys, I've tried sqlite after a long time and either it improved meanwhile or I had some tricky cases last time I used it. Anyway, it looks good enough to perform quick tests on the fly, and switch to MySQL/PostgreSQL only once in a while to confirm that tests work with them as well. – Tomasz Zieliński Jun 25 '11 at 21:41
  • 2
    @shanyu and what if the app has a JsonField in one of the models? This is an excellent answer as it highlights a flaw in the OP's plan, and provides a solution the problem the OP was presumably trying to solve through their plan (i.e. performance) – andyhasit Apr 07 '19 at 17:13
  • @shanyu One problem I just ran into: SQLite doesn't allow you to call `distinct` on specific fields while Postgresql does – AmagicalFishy Apr 26 '19 at 19:13
4

You could try a setup similar to what is suggested here by Zachary Voase: http://blog.zacharyvoase.com/2010/02/03/django-project-conventions/

(The entire post is useful, but scroll down to the section on "Settings" for the part most relevant here.)

Zach's strategy is to create a settings folder and marks it as a python package using a __init__.py file. You can then have a separate sub-module for each of your deployment types, structured as follows:

settings/
|-- __init__.py     # Empty; makes this a Python package
|-- common.py       # All the common settings are defined here
|-- development.py  # Settings for development
|-- production.py   # Settings for production
|-- staging.py      # Settings for staging

Following this concept, you could set up a deployment for postgres and a separate deployment for sqlite, and separate the configurations for each as needed.

Aman
  • 38,643
  • 7
  • 32
  • 37