3

I have three different Python 2.7s at:

/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7

I use a number of packages that come from different sources. I am currently installing packages from port (MacPorts), easy_install, pip (installed by easy_install), and Mercurial. There are also some that I have to install from image or build from source. I have more control over those.

The problem is that easy_install and pip seem to be installing to one location (/Library/Frameworks/...) and MacPorts installs to another (/opt/local/Library/Frameworks/...).

What's my best action now? Delete /Library/Frameworks/.../python2.7 and move easy_install and pip to the MacPorts one at /opt/local/...? Link the two directories? Move the MacPorts installation to /Library/Frameworks/...?

How can I consolidate these Pythons? I have tried putting both site-packages locations in my path, but only certain packages are available only for one Python and not the other and others vice versa, and I need them all available at once.

wxs
  • 288
  • 3
  • 18
Dannid
  • 1,173
  • 14
  • 17
  • 2
    Whatever you do, *do* *not* attempt to move or delete anything in `/System/Library/Frameworks/`. Those are Apple-supplied files as part of OS X, including the system Pythons. They are needed for proper operation of your system. – Ned Deily Mar 12 '13 at 18:46
  • 2
    Also, in general, you cannot move a Python framework from one directory to another. So don't try to move the MacPorts Python from `/opt/local/`. – Ned Deily Mar 12 '13 at 18:47

4 Answers4

4

It seems that you have control over the stuff you're building yourself. This is how I consolidate macports with pip:

I like using Macports for all my stuff, so I just make sure that pip and easy_install build into macports' installation of python (the one in /opt/local/...).

You can tell where pip and easy_install will install things by using:

readlink `which pip`

(those are backticks)

If you want pip to install to the macports direcectories, use macports to install pip:

sudo port install py-pip

Then, be sure that which pip points to something like:

askewchan@rock:~$ which pip
/opt/local/bin/pip
askewchan@rock:~$ readlink `which pip`
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin/pip-2.7

From the comments below (thanks @Jonathan and @Ned) you can do the same with easy_install but its port is called py-distribute:

sudo port install py-distribute

But as far as I know, you never need to use easy_install because anything that can be easy_installed can be piped better.

Note the port descriptions:

askewchan@rock:Tracking {master *}$ port search *easy*install*
py-pip @1.2.1 (python, www)
    An easy_install replacement

askewchan@rock:Tracking {master *}$ port search py*distribute
py-distribute @0.6.35 (python, devel)
    Replacement for setuptools
Community
  • 1
  • 1
askewchan
  • 39,694
  • 13
  • 105
  • 128
  • 1
    I just about never use `pip` and recommend against it, because of a longstanding bug where `pip install` tries to download and install every package, even if a dependency is already met. https://github.com/pypa/pip/issues/304 – Jonathan Vanasco Mar 12 '13 at 18:36
  • @Jonathan Good to know. Anyway, the question asks for help in where pip should install to, not whether to use pip. – askewchan Mar 12 '13 at 18:40
  • 1
    The same approach applies to `easy_install`. The key is understanding that each Python you want to use needs its own copy of `easy_install` (from `Distribute` or `setuptools`) or `pip` and then ensure you use that copy. The standard way to do that is to manage your shell PATH environment variable. – Ned Deily Mar 12 '13 at 19:00
  • True, @Ned, but macports has no `easy_install` port, so that python installation can't (easily) have its "own" `easy_install`. – askewchan Mar 12 '13 at 19:01
  • 1
    Not true! `port install py-distribute` – Ned Deily Mar 12 '13 at 19:11
  • @Ned, aha! `port search *easy*install*` doesn't return that. LOL: py-pip An easy_install replacement; py-distribute Replacement for setuptools – askewchan Mar 12 '13 at 19:15
  • @askewchan i'm just commenting on your 'you can pip better'. – Jonathan Vanasco Mar 12 '13 at 19:39
  • @Jonathan I just meant `pip > easy_install`. Do you use something instead of `pip` and `easy_install`, or do you just build things yourself? – askewchan Mar 12 '13 at 19:46
  • I use `easy_install` for everything by default. on the off chance that it fails, I'll try `pip` before having to build it myself. The pip 'always install dependencies' behavior makes it a complete pain for development work or production deployments -- I've had it try to install/upgrade already-met-dependencies for 20+ minutes. – Jonathan Vanasco Mar 12 '13 at 19:51
  • I appreciate the advice about pip. I especially appreciate the how-to for getting both pip (py-pip) and easy_install (py-distribute) to work through macports. – Dannid Mar 12 '13 at 22:50
1

I suggest deciding on one and only one Python for your development work ( personally, I use distribution from Python.org )

  • You can't get rid of /Library/Frameworks - that's the default OSX one, and you could break things
  • of the two remaining Pythons, I'm assuming one is Macports and the other is Python.org -- you need to choose which one you want to be your development env and to stick with that.

I would strongly recommend against using pip or easy_install from one Python to install modules for another. The reason is that there can be differences in the compile options. It can be hard enough as-is to get certain packages to compile on OSX properly -- if you start compiling against different binaries ( which might support different architectures ) you're just going to increase your headaches.

I personally chose the following path:

  1. I use the Python.org package for all development.
  2. On a terminal login, I run shell scripts to prioritize my Python choice
  3. All of my projects have their own virtualenv , and I disable system packages
  4. When starting to work on any project, I tend to have an environment setup script. I just type in go_myproject.source ; that cds me to the right directory and runs the source /path/to/virtualenv/bin/activate to get me set up for that project.

There's a tiny bit of overhead on getting things setup, but I have been in complete heaven ever since. Managing projects and not needing to worry about dependencies/upgrades for one thing killing something else is... blissful.

Jonathan Vanasco
  • 13,535
  • 8
  • 42
  • 67
  • This seems to highlight some of the advantages of virtualenv. I'll think about this, but it seems like a lot of work to convert to this solution if most things are already set up. One thing that led me into this headache in the first place is that I inherited this machine from the previous developer and had to continue working with little knowledge about how things had been installed before me. (Plus, this machine had gone through several OSX upgrades before my arrival, too.) – Dannid Mar 12 '13 at 23:00
  • Having each library you need installed across any one of 3 Python installations isn't having `most things already set up`. Anything with a C-extension ( simplejson, most sql apis, PIL, reportlab, to name a few ) are likely to cause a lot of errors down the road if you try and condense everything into one library. Think of it like this -- imagine it's not Python packages, but some apps like iTunes, Word, Photoshop. If you copy them from one machine to the next, they *might* work, but most often they'll crash unless you ran the actual installer. If your packages are compiled, same problem. – Jonathan Vanasco Mar 12 '13 at 23:14
1

While not a general solution, I install Mercurial and other Python-based applications using virtualenv. In particular, pip and easy_install will install to the respective virtual environment only and not clutter any system folder. The downside is, of course, that I will have duplicates of some packages; the advantage is that I have a clean, self-contained environment with a known version of Python (which for things such as Mercurial and other mission-critical applications is more important for me).

Another downside is that I need to link individual applications to my personal bin directory or add the bin directories of the virtual environments to my path. (Personally, I manage this with some simple scripts that do the symlinking for me.)

Reimer Behrends
  • 7,837
  • 9
  • 13
  • I haven't used virtualenv (yet?) and wonder what its advantages are. Can it handle multiple users on the same machine (such that we can all access the installed packages)? Do I want yet another install manager on top of port, easy_install, pip and mercurial? – Dannid Mar 12 '13 at 22:55
  • Well, `virtualenv` is not an install manager. It is basically a sandbox that contains copies/links for an existing Python installation. Within that sandbox, you have the Python interpreter and binaries for `pip` and `easy_install` that install packages and programs within that sandbox. Note that "sandbox" doesn't mean that this is a restricted version of Python, just that the installation is contained within its own directory. – Reimer Behrends Mar 13 '13 at 17:20
-2

I sugest to move all python installations to the one place and create symlinks. After that configure python environment to avoid problems with imports and "visibility" of the modules. Try to use commands:

# easy_install
env PYTHONPATH=/custom/path easy_install –install-dir /custom/path

#pip
pip install --install-option="--prefix=$PREFIX_PATH" package_name
Kamil
  • 2,424
  • 27
  • 37
  • I thought about this, but it seems dangerous to me (moving python installations). Tricky, too, since some things were already installed twice (let alone the two different python builds). Still, the commands for specifying the install locations from easy_install and pip seem helpful. – Dannid Mar 12 '13 at 23:02