Cannot collect patch dependency on a local Artifactory Pypi repository

Question

While testing out conan, I had to "pip install" it.

As I am working in a fully offline environment, my expectation was that I could simply

Manually deploy all dependencies listed in https://github.com/conan-io/conan/blob/master/conans/requirements.txt to a local Pypi repository called myrepo-python
Install conan with

pip3 install --index http://myserver/artifactory/api/pypi/myrepo-python/simple conan

This works fine for some packages and then fails for the dependency on patch == 1.16

[...]
Collecting patch==1.16 (from conan)
  Could not find a version that satisfies the requirement patch==1.16 (from conan) (from versions: )
No matching distribution found for patch==1.16 (from conan)

Looking into the Artifactory logs, this shows that even though I manually deployed patch-1.16.zip (from https://pypi.org/project/patch/1.16/#files) into the repository, it is not present in the index...

The .pypi/simple.html file doesn't contain an entry for 'patch' (checked from the Artifactory UI)
The server logs ($ARTIFACTORY_HOME/logs/artifactory.log) show the file being added to the indexing queue but don't contain a line saying that it got indexed

Does anyone know why patch-1.16.zip is not indexed by Artifactory?

This is on Artifactory 5.8.4.

For now, my only workaround is to gather all the dependencies into a local path and point pip3 at it

scp conan-1.4.4.tar.gz installserver:/tmp/pip_cache
[...]
scp patch-1.16.zip installserver:/tmp/pip_cache
[...]
scp pyparsing-2.2.0-py2.py3-none-any.whl installserver:/tmp/pip_cache
ssh installserver
installserver$ pip3 install --no-index --find-links="/tmp/pip_cache" conan

score 4 · Accepted Answer · answered Jun 25 '18 at 19:01

The reason you are unable to install the "patch" Pypi package via Artifactory is that it does not comply with the Python spec.

Based on Python spec (https://www.python.org/dev/peps/pep-0314/ and https://packaging.python.org/tutorials/packaging-projects/), the structure of a Python package should be, for example:

└── patch-1.16.zip
    └── patch-1.16
        ├── PKG-INFO
        ├── __main__.py
        ├── patch.py
        └── setup.py

However, the zip archive (can be found here https://pypi.org/project/patch/1.16/#files) is structured like that:

└── patch-1.16.zip
    ├── PKG-INFO
    ├── __main__.py
    ├── patch.py
    └── setup.py

Artifactory is searching for the metadata file (PKG-INFO in this case) in a certain pattern (inside any folder). Since the PKG-INFO is in the root of the zip (and not in a folder), it cannot find it, therefore, this package's metadata will not be calculated and it will not appear in the "simple" index file (see the error in artifactory.log). As a result, you are unable to install it with pip.

Workaround:

What you can do is manually changing the structure to the correct one.

Create a folder named patch-1.16 and extract the zip to it. Then, zip the whole folder, so you will get the structure like the example above. Finally, deploy this zip to Artifactory. This time, the PKG-INFO file will be found, the metadata will be calculated and pip will be able to install it.

Thanks, so if I understand correctly, you are saying that Artifactory implements its indexing with a strict reading of PEP-314, whereas pypi.org accepts non-compliant packages. That would explain the indexing problem, I will check your workaround next week. — Arnaud Jeansen, Jun 26 '18 at 10:40
Yes this solves it, massaging the zip to add the top level folder was enough to get it indexed and working correctly. I now get a failure to collect deprecation-2.0.3-py3-none-any.whl but that's another problem ;-) — Arnaud Jeansen, Jul 03 '18 at 08:39
good to hear! Let me know if I can help with the other error ;) — avivblo, Jul 04 '18 at 09:32

Cannot collect patch dependency on a local Artifactory Pypi repository

1 Answers1