38

Does somebody know the logic behind Python modules names vs the name of the actual package used in easy_install?

A few (amongst others) example that seem a bit unlogical to me:

  • We do easy_install mysql-python, but the import is in fact import MySQLdb
  • We do easy_install python-memcached, but the import is in fact import memcache (without the trailing d)

I didn't find a consistent way of finding the equivalence. For some modules, it took me a lot of browsing to find it. What am I doing wrong?

Colonel Panic
  • 119,181
  • 74
  • 363
  • 435
Gerard Yin
  • 1,173
  • 11
  • 23

2 Answers2

20

Regrettably, there's no method to the madness. The name in the package index is independent of the module name you import. Disastrously some packages share module names. If you install both, your application will break with even odds. (Ruby has this problem too)

Packaging in Python is generally dire. The root cause is that the language ships without a package manager. Ruby and Nodejs ship with full-featured package managers Gem and Npm, and have nurtured sharing communities centred around GitHub. Npm makes publishing packages as easy as installing them. Nodejs arrived 2009 and already has 14k packages. The venerable Python package index lists 24k. Ruby Gems lists 44k packages.

Fortunately, there is one decent package manager for Python, called Pip. Pip is inspired by Ruby's Gem, but lacks some vital features (eg. listing packages, and upgrading en masse). Ironically, Pip itself is complicated to install. Installation on the popular 64-bit Windows demands building and installing two packages from source. This is a big ask for anyone new to programming.

Python's devs are ignorant of all this frustration because they are seasoned programmers comfortable building from source, and they use Linux distributions with packaged Python modules.

Until Python ships with a package manager, thousands of developers will needlessly waste time reinventing the wheel.

Python 3 solves many problems with packaging. There aren't any packages for Python 3.

Yuri Feldman
  • 1,648
  • 1
  • 16
  • 20
Colonel Panic
  • 119,181
  • 74
  • 363
  • 435
  • 1
    I sure hope this answer is partially tongue-in-cheek. I keep telling myself parts of this *can't* be serious, but other parts are spot on... –  Jul 12 '12 at 17:37
  • Is installing pip on win64 really that bad? I usually just download the setuptools installer then run `easy_install pip`. Though I am using 32bit python, so maybe that's where the difference lies. – obmarg Jul 13 '12 at 09:58
  • That's right. There's no [setuptools](http://pypi.python.org/pypi/setuptools/) installer for 64 bit Python. "Currently, the provided .exe installer does not support 64-bit versions of Python for Windows, due to a distutils installer compatibility issue". I discourage friends from installing 64 bit Python. – Colonel Panic Jul 23 '12 at 08:55
  • 2
    ``pip`` can list packages: installed packages with ``pip freeze`` and online packages (stored on PyPI) with ``pip search``. – Balthazar Rouberol Sep 17 '13 at 07:03
  • 1
    This answer is outdated. But python dependency is still a mess. – Fabich Dec 06 '19 at 13:16
  • “reinventing the wheel” Haha, I see what you did there! – LeopardShark Jan 24 '21 at 15:32
10

Unlike the accepted answer suggests, it is actually possible, although indeed quite cumbersome.

The module name(s) can be found top_level.txt file in the metadata directory of a package installation.

To access it with python (replace python-memcached with your package name):

>>> import pkg_resources as pkg # included in setuptools package
>>> metadata_dir = pkg.get_distribution('python-memcached').egg_info
>>> open('%s/%s' % (metadata_dir, 'top_level.txt')).read().rstrip()
'memcache'

Or with an equivalent bash "one liner":

cat $(python -c "import pkg_resources; \
print(pkg_resources.get_distribution('python-memcached').egg_info)")/top_level.txt

Keep in mind that some packages install multiple modules, so the method I'm presenting can return multiple module names.

Jakub Kukul
  • 6,869
  • 1
  • 35
  • 39