Can Pip install dependencies not specified in setup.py at install time?

Question

I'd like pip to install a dependency that I have on GitHub when the user issues the command to install the original software, also from source on GitHub. Neither of these packages are on PyPi (and never will be).

The user issues the command:

pip -e git+https://github.com/Lewisham/cvsanaly@develop#egg=cvsanaly

This repo has a requirements.txt file, with another dependency on GitHub:

-e git+https://github.com/Lewisham/repositoryhandler#egg=repositoryhandler

What I'd like is a single command that a user can issue to install the original package, have pip find the requirements file, then install the dependency too.

score 36 · Accepted Answer · edited May 23 '17 at 12:00

This answer helped me solve the same problem you're talking about.

There doesn't seem to be an easy way for setup.py to use the requirements file directly to define its dependencies, but the same information can be put into the setup.py itself.

I have this requirements.txt:

PIL
-e git://github.com/gabrielgrant/django-ckeditor.git#egg=django-ckeditor

But when installing that requirements.txt's containing package, the requirements are ignored by pip.

This setup.py seems to coerce pip into installing the dependencies (including my github version of django-ckeditor):

from setuptools import setup

setup(
    name='django-articles',
    ...,
    install_requires=[
        'PIL',
        'django-ckeditor>=0.9.3',
    ],
    dependency_links = [
        'http://github.com/gabrielgrant/django-ckeditor/tarball/master#egg=django-ckeditor-0.9.3',
    ]
)

Edit:

This answer also contains some useful information.

Specifying the version as part of the "#egg=..." is required to identify which version of the package is available at the link. ~~Note, however, that if you always want to depend on your latest version, you can set the version to dev in install_requires, dependency_links and the other package's setup.py~~

Edit: using dev as the version isn't a good idea, as per comments below.

the trick with the "dev" does only work for the very first time and not the subsequent times. The setup.py does only check the "dev" string as a version for itself — DanEEStar, Jul 05 '12 at 15:39
@DanEEStar That's right. Once the _dev_ version of the package has been installed `setuptools` will consider the requirement satisfied. As demonstrated [in the linked answer](http://stackoverflow.com/a/2163919/396967) you would need to synchronously update the package version in *all 3 places*: the dependency's `setup.py` and `install_requires` and `dependency_links` - not really practical. — kynan, Dec 17 '12 at 02:11
yup, @DanEEStar you're right. I've edited out the recommendation to use 'dev' version. Thanks you two! — Gabriel Grant, Feb 21 '13 at 18:28

score 13 · Answer 2 · answered Feb 03 '12 at 07:55

Here's a small script I used to generate install_requires and dependency_links from a requirements file.

import os
import re

def which(program):
    """
    Detect whether or not a program is installed.
    Thanks to http://stackoverflow.com/a/377028/70191
    """
    def is_exe(fpath):
        return os.path.exists(fpath) and os.access(fpath, os.X_OK)

    fpath, _ = os.path.split(program)
    if fpath:
        if is_exe(program):
            return program
    else:
        for path in os.environ['PATH'].split(os.pathsep):
            exe_file = os.path.join(path, program)
            if is_exe(exe_file):
                return exe_file

    return None

EDITABLE_REQUIREMENT = re.compile(r'^-e (?P<link>(?P<vcs>git|svn|hg|bzr).+#egg=(?P<package>.+)-(?P<version>\d(?:\.\d)*))$')

install_requires = []
dependency_links = []

for requirement in (l.strip() for l in open('requirements')):
    match = EDITABLE_REQUIREMENT.match(requirement)
    if match:
        assert which(match.group('vcs')) is not None, \
            "VCS '%(vcs)s' must be installed in order to install %(link)s" % match.groupdict()
        install_requires.append("%(package)s==%(version)s" % match.groupdict())
        dependency_links.append(match.group('link'))
    else:
        install_requires.append(requirement)

score 1 · Answer 3 · answered Jun 18 '12 at 19:35

1

does this answer your question?

setup(name='application-xpto',
  version='1.0',
  author='me,me,me',
  author_email='xpto@mail.com',
  packages=find_packages(),
  include_package_data=True,
  description='web app',
  install_requires=open('app/requirements.txt').readlines(),
  )

answered Jun 18 '12 at 19:35

daigorocub

667
7
15

3

`open('app/requirements.txt').readlines()` only works if `requirements.txt` only contains "plain" version specifications, not URLs. These will need to be split out in the `dependency_links` as done by Simon Charette's answer. – kynan Dec 17 '12 at 02:13

Can Pip install dependencies not specified in setup.py at install time?

3 Answers3

Linked