Error: pg_config executable not found when installing psycopg2 on Alpine in Docker

Question

I'm trying to build a Flask app using Postgres with Docker. I'd like to connect to an AWS RDS instance of Postgres, but use Docker for my Flask app. However, when trying to set up psycopg2 it runs into an error because it can't find pg_config. Here's the error:

Building api
Step 1/5 : FROM python:3.6.3-alpine3.6
 ---> 84c98ca3b5c5
Step 2/5 : WORKDIR /usr/src/app
 ---> Using cache
 ---> 407c158f5ee4
Step 3/5 : COPY . .
 ---> 966df18d329e
Step 4/5 : RUN pip install -r requirements.txt
 ---> Running in 284cc97aeb63
Collecting aniso8601==1.3.0 (from -r requirements.txt (line 1))
  Downloading aniso8601-1.3.0.tar.gz (57kB)
Collecting click==6.7 (from -r requirements.txt (line 2))
  Downloading click-6.7-py2.py3-none-any.whl (71kB)
Collecting Flask==0.12.2 (from -r requirements.txt (line 3))
  Downloading Flask-0.12.2-py2.py3-none-any.whl (83kB)
Collecting Flask-RESTful==0.3.6 (from -r requirements.txt (line 4))
  Downloading Flask_RESTful-0.3.6-py2.py3-none-any.whl
Collecting Flask-SQLAlchemy==2.3.2 (from -r requirements.txt (line 5))
  Downloading Flask_SQLAlchemy-2.3.2-py2.py3-none-any.whl
Collecting itsdangerous==0.24 (from -r requirements.txt (line 6))
  Downloading itsdangerous-0.24.tar.gz (46kB)
Collecting Jinja2==2.9.6 (from -r requirements.txt (line 7))
  Downloading Jinja2-2.9.6-py2.py3-none-any.whl (340kB)
Collecting MarkupSafe==1.0 (from -r requirements.txt (line 8))
  Downloading MarkupSafe-1.0.tar.gz
Collecting psycopg2==2.7.3.1 (from -r requirements.txt (line 9))
  Downloading psycopg2-2.7.3.1.tar.gz (425kB)
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/psycopg2.egg-info
    writing pip-egg-info/psycopg2.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
    Error: pg_config executable not found.

    Please add the directory containing pg_config to the PATH
    or specify the full executable path with the option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-01lf5grh/psycopg2/
ERROR: Service 'api' failed to build: The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

Here's my Dockerfile:

FROM python:3.6.3-alpine3.6

WORKDIR /usr/src/app

COPY . .

RUN pip install -r requirements.txt

CMD ["python", "app.py"]

Many others seem to have a similar issue locally, but none of them involve using Docker. This seems like a Docker issue because I can set up a local virtual environment and the setup works just fine since I have Postgres installed locally and it's able to find my local pg_config.

It appears that during the container build/setup, Docker is trying to find pg_config within the container. Is there a way to install a pg_config in the container, even though I won't be using a containerized instance of Postgres, but rather the instance on RDS?

Any and all suggestions on how to get around this are welcomed.

Possible duplicate of [pg\_config executable not found](https://stackoverflow.com/questions/11618898/pg-config-executable-not-found) — Klaus D., Oct 12 '17 at 14:29
These examples all seem to involve setting up locally, but not specifically within a Docker container that doesn't have Postgres installed. — hidace, Oct 12 '17 at 14:34
There's one answer for Alpine (see the first line of your Dockerfile): https://stackoverflow.com/a/44938314/3929826 You should be able to abstract the answer for Docker. — Klaus D., Oct 12 '17 at 14:38
Thanks @KlausD for pointing that out. Appreciate it. Will give it a try. — hidace, Oct 12 '17 at 14:51

Norrius · Accepted Answer · 2018-12-13T09:56:14.220

143

Tested with Python 3.4.8, 3.5.5, 3.6.5 and 2.7.14 (just replace 3 with 2):

# You can use a specific version too, like python:3.6.5-alpine3.7
FROM python:3-alpine

WORKDIR /usr/src/app

COPY requirements.txt .

RUN \
 apk add --no-cache postgresql-libs && \
 apk add --no-cache --virtual .build-deps gcc musl-dev postgresql-dev && \
 python3 -m pip install -r requirements.txt --no-cache-dir && \
 apk --purge del .build-deps

COPY . .

CMD ["python3", "app.py"]

Explanation: to build Psycopg you need the packages gcc musl-dev postgresql-dev. Then you also need that pg_config executable: while simply installing postgresql-dev will work, postgresql-libs does fine too and takes up some 12 MB less space.

Here's the original version of the answer (based on this Dockerfile) where I manually install Python onto a pure Alpine image because at that time Python did not provide the Docker image with Python 3.6 and Alpine 3.7. If you want to install Python 2.7 like that, also do apk add py2-pip (called py-pip in older Alpine repos).

FROM alpine:3.7

WORKDIR /usr/src/app

COPY requirements.txt .

RUN \
 apk add --no-cache python3 postgresql-libs && \
 apk add --no-cache --virtual .build-deps gcc python3-dev musl-dev postgresql-dev && \
 python3 -m pip install -r requirements.txt --no-cache-dir && \
 apk --purge del .build-deps

COPY . .

CMD ["python3", "app.py"]

edited Dec 13 '18 at 09:56

answered Dec 18 '17 at 15:01

Norrius

6,118
4
34
47

Thanks. It helped me with the same problem for python:3.6.4-alpine3.7 – Wojteks Mar 22 '18 at 19:57
I used this Dockerfile but changed python3 to python2 (we still use 2.7.x) and I get an error: /usr/bin/python2: No module named pip. – Mike Stoddart Apr 05 '18 at 13:50
1

@MikeStoddart I updated the answer, please see if it helps. – Norrius Apr 06 '18 at 11:09
1

For Ubuntu/Debian, install `libpq-dev`. See this [SO answer](https://stackoverflow.com/a/12037133/1398841). – phoenix Oct 11 '19 at 11:56
2

`postgresql-libs` depends on `libpq`, so you can use `libpq` instead https://pkgs.alpinelinux.org/package/edge/main/x86_64/postgresql-libs – Devy Oct 17 '19 at 02:47
1

Alpine 3.10.3: installing postgresql-libs did not solve the problem, but using postgresql-dev did. – rschwieb Jan 20 '20 at 18:41
`build-deps` is not a valid package for recent versions (e.g. 3.11) of Alpine. – Jason McVetta Feb 26 '20 at 06:40
I am using "tiangolo/uvicorn-gunicorn:python3.8-slim" as my base image. I get "apk command not found". what do i do? – Naveen Kumar Oct 22 '20 at 08:36
At least it got to install `psycopg2` but there is an issue with `purge`. I get this error when I run the script above `Usage: pip3 install [options] [package-index-options] ... pip3 install [options] -r [package-index-options] ... pip3 install [options] [-e] ... pip3 install [options] [-e] ... pip3 install [options] ... no such option: --purge`. The commands return a non-zero code: 2 – Gitau Harrison Oct 30 '20 at 07:05
To solve it, I did `RUN \ apk update && \ apk add postgresql-dev gcc python3-dev musl-dev && \ /bin/pip3 install -r requirements.txt`. This successfully got `pyscopg2` installed. – Gitau Harrison Oct 30 '20 at 07:25
@GitauHarrison It sounds like the command got mangled. --purge is a flag to apk, not pip. – Norrius Oct 30 '20 at 13:48
This failed for me today with an error about `libffi` while installing requirements.txt in a docker container. I was able to get it to work by adding `apk add --no-cache libffi-dev && \` to the beginning of the run command – Nathan Clement Nov 04 '20 at 17:52
better to separately install `psycopg2` and the rest of deps. to make the best out of docker cache. – Ali Kazemkhanloo Feb 24 '21 at 09:20

score 22 · Answer 2 · answered Feb 04 '20 at 07:25

22

Simply add these commands (psycopg2 dependencies) before installing dependencies from requirements.txt in Dockerfile

# install psycopg2 dependencies
RUN apk update
RUN apk add postgresql-dev gcc python3-dev musl-dev

Source: https://testdriven.io/blog/dockerizing-django-with-postgres-gunicorn-and-nginx/

answered Feb 04 '20 at 07:25

Tasawar Hussain

482
3
16

I was having an issue with `purge` from the solution example above. This was able to work for me – Gitau Harrison Oct 30 '20 at 07:20

score 6 · Answer 3 · edited Mar 01 '19 at 22:43

6

You could try:

pip install psycopg2-binary

edited Mar 01 '19 at 22:43

Paul Roub

35,100
27
72
83

answered Mar 01 '19 at 22:31

Vivin Veerali

181
2
1

1

You still need to install postgres binaries, independently. – merqurio Mar 26 '19 at 11:27
3

not at all. psycopg2-binary is not compatible with Alpine. – piro Feb 13 '20 at 20:39
The `psycopg2-binary` package is meant for beginners to start playing with `Python` and `PostgreSQL` without the need to meet the build requirements. **For production use you are advised to use the source distribution.** Hence the need to use `psycopg` (without the binary) – Gitau Harrison Oct 30 '20 at 07:28

Error: pg_config executable not found when installing psycopg2 on Alpine in Docker

3 Answers3

Linked

Related