67

In development mode, I have the following directory tree :

| my_project/
| setup.py
| my_project/
    | __init__.py
    | main.py
    | conf/
        | myproject.conf

I use ConfigParser to parse the myproject.conf file.

In my code, it's easy to load the file with a good path : my_project/conf/myproject.conf

The problem is : When I install my project using the setup.py, the configuration file are situated (thanks to setup.py) in the /etc/my_project/myproject.conf and my application in the /usr/lib/python<version>/site-packages/my_project/.

How can I refer my my_project/conf/myproject.conf file in my project in "production" mode, and refer to the local file (my_project/conf/myproject.conf) in "devel" mode.

In addition, I would like to be portable if possible (Work on windows for example).

What's the good practice to do that ?

Sandro Munda
  • 36,427
  • 21
  • 94
  • 117

7 Answers7

39

Have you seen how configuration files work? Read up on "rc" files, as they're sometimes called. "bashrc", "vimrc", etc.

There's usually a multi-step search for the configuration file.

  1. Local directory. ./myproject.conf.

  2. User's home directory (~user/myproject.conf)

  3. A standard system-wide directory (/etc/myproject/myproject.conf)

  4. A place named by an environment variable (MYPROJECT_CONF)

The Python installation would be the last place to look.

config= None
for loc in os.curdir, os.path.expanduser("~"), "/etc/myproject", os.environ.get("MYPROJECT_CONF"):
    try: 
        with open(os.path.join(loc,"myproject.conf")) as source:
            config.readfp( source )
    except IOError:
        pass
S.Lott
  • 359,791
  • 75
  • 487
  • 757
  • 4
    This isn't how it works in Windows, though, which the question asked for. – endolith Mar 18 '12 at 23:48
  • 1
    This will "Work on windows". It may not be "typical" for Windows, but it certainly works. – S.Lott Mar 20 '12 at 09:39
  • 2
    Uh - why do you have a try catch around a with statement? – David Jul 18 '13 at 14:07
  • 3
    The referenced files may not exist. Also note, that for Linux there is the XDG spec, so you should store config in ~/.config// – dom0 Jul 26 '13 at 02:11
  • 2
    `os.path.expanduser("~")` has problems if your windows account is associated with windows directory services. – Akshay Jun 14 '16 at 06:24
21

The appdirs package does a nice job on finding the standard place for installed apps on various platforms. I wonder if extending it to discover or allow some sort of "uninstalled" status for developers would make sense.

julienc
  • 15,239
  • 15
  • 74
  • 77
nealmcb
  • 9,525
  • 6
  • 56
  • 80
17

If you're using setuptools, see the chapter on using non-package data files. Don't try to look for the files yourself.

patrys
  • 2,621
  • 15
  • 27
  • The above link no longer works. I guess the new link is [Non-Package Data Files](https://setuptools.readthedocs.io/en/latest/setuptools.html#non-package-data-files) but things have changed. The current docs point to [the old docs](https://github.com/pypa/setuptools/blob/52aacd5b276fedd6849c3a648a0014f5da563e93/docs/setuptools.txt#L970-L1001), which is what I guess this comment intended to point to. – sylvain Jan 02 '20 at 20:04
  • 1
    As of now, it seems that using the `data_files` option as documented in [Installing Additional Files](https://docs.python.org/3/distutils/setupscript.html#installing-additional-files) may work. – sylvain Jan 02 '20 at 20:05
4

Update for S.Lott's answer.

Now the method configparser.ConfigParser.readfp() is deprecated。

You can use configparser.ConfigParser.read() method directly for multi files.

For example:

import configparser

config = configparser.ConfigParser()

all_config_files = ['/path/to/file1', '/path/to/file2', '/path/to/file3']

config.read(all_config_files)

Note:

  • The config options in later config file will overwrite the previous.
  • If the config file not exists, read() will ignore it.
  • If you have required options need read from a file, use read_file() first.

configparser.ConfigParser.read()

alex li
  • 181
  • 1
  • 4
2

I don't think there is a clean way to deal with that. You could simply choose to test for the existence of the 'local' file, which would work in dev mode. Otherwise, fall back to the production path:

import os.path

main_base = os.path.dirname(__file__)
config_file = os.path.join(main_base, "conf", "myproject.conf")

if not os.path.exists(config_file):
    config_file = PROD_CONFIG_FILE   # this could also be different based on the OS
Rodrigue
  • 3,465
  • 2
  • 36
  • 49
  • 1
    Hey, could you please remove the brackets from os.path.join command or put * before the list, it will take variable arguments rather than a list. – Rahul Gupta Feb 05 '19 at 13:13
1

Another option is to keep all the .cfg and .ini files in the home directory, like 'boto' does.

import os.path
config_file = os.path.join(os.path.expanduser("~"), '.myproject')
tashuhka
  • 4,524
  • 3
  • 40
  • 62
  • 3
    Using `~/.myapp` is discouraged nowadays. You should follow the [XDG Base Directory spec](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html) instead, and put it at `~/$XDG_CONFIG_HOME/myapp`, which will likely be `~/.config/myapp`. – DeusXMachina Jun 28 '19 at 23:47
-1

The docs for Writing the Setup Script, under section 2.7. Installing Additional Files mention:

The data_files option can be used to specify additional files needed by the module distribution: configuration files, message catalogs, data files, anything which doesn’t fit in the previous categories.

sylvain
  • 189
  • 1
  • 7