How to install Python packages over SSH Port Forwarding?

Question

I am controlling a remote unit over SSH and OPENVPN.

On the remote unit I want to install some Python packages using pip but:

the remote company firewall allows only traffic on port 22 (and not 443, needed by pip);
DNS is not installed on the remote unit;
I cannot modify any OPENVPN settings (or I would like to avoid this option as it means to access some remote sysadmin and try to convince him that the configuration must be changed);
all systems are Linux (Ubuntu + Debian). Non Windows involved.

Stripping down hours of attempts (I am not a system admin and my knowledge on this subject is very limited), the idea was to open an obvious SSH port forwarding:

ssh -R 9999:pypi.python.org:443 xxxx@XX.XX.XX.XX

and then, on the remote unit play with pip install:

pip install pymodbus==1.3.2 --proxy localhost:9999

But this command returns:

Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pymodbus==1.3.2

/root/.pip/pip.log is:

  Getting page https://pypi.python.org/simple/pymodbus/
  Could not fetch URL https://pypi.python.org/simple/pymodbus/: connection error: ('Connection aborted.', BadStatusLine("''",))
  Will skip URL https://pypi.python.org/simple/pymodbus/ when looking for download links for pymodbus==1.3.2
  Getting page https://pypi.python.org/simple/
  Could not fetch URL https://pypi.python.org/simple/: connection error: ('Connection aborted.', BadStatusLine("''",))
  Will skip URL https://pypi.python.org/simple/ when looking for download links for pymodbus==1.3.2
  Cannot fetch index base URL https://pypi.python.org/simple/
  URLs to search for versions for pymodbus==1.3.2:
  * https://pypi.python.org/simple/pymodbus/1.3.2
  * https://pypi.python.org/simple/pymodbus/
  Getting page https://pypi.python.org/simple/pymodbus/1.3.2
  Could not fetch URL https://pypi.python.org/simple/pymodbus/1.3.2: connection error: ('Connection aborted.', BadStatusLine("''",))
  Will skip URL https://pypi.python.org/simple/pymodbus/1.3.2 when looking for download links for pymodbus==1.3.2
  Getting page https://pypi.python.org/simple/pymodbus/

It is obvious the remote unit cannot read the index page on pypi.pthon.org because the connection is refused.

What is the correct syntax for what I am trying to achieve?

According to answer https://stackoverflow.com/a/33611028/95735 you should specify it as `--proxy https://localhost:9999` — Piotr Dobrogost, Jan 19 '18 at 15:06
Piotr, yes, I forgot to tell that this was an attempt I already tried: `--proxy https://localhost:9999 --no-check-certificate` (the `--no-check-certificate` because otherwise it would complain that it is not present on `localhost`) but got the same `Cannot fetch index base URL https://pypi.python.org/simple/`. — Alex Poca, Jan 20 '18 at 12:44

score 2 · Accepted Answer · answered Nov 03 '18 at 22:08

2

Proxy is going to be tricky. I suggest that you scp the pip module source file and install it locally from source. Use
pip install package —download="/pth/to/downloaded/file” to get the package, scp it to the dest server and use pip install “/pth/to/scp/file”

answered Nov 03 '18 at 22:08

David

36
2

I believe --download is deprecated. `pip download ` works though. – Jeppe Dec 04 '19 at 11:34
1

Not maintainable for multiple packages with multiple dependencies. – ibarrond Aug 06 '20 at 10:20

score 2 · Answer 2 · edited Apr 01 '19 at 09:51

It's look like my problem. after exploration, I have found a solution. And because in my region, pypi.python.org is slow, so I change my pip.conf and use pypi.douban.com/simple, as my index-url. this website use http protocol. so in my solution. I use 80 port as my target port.

Problem: I have two host. host1 could connect Pypi.douban.com. and host2 couldn't. but I can connect host2 in host1 through ssh.

so in host2, I open a tmux session and open a ssh tunnel by local port forwarding(not remote port forwarding):

ssh -L 9999:pypi.douban.com:80 username@host1

after this redirect, I can use

pip install scikit-learn --proxy localhost:9999

to install package in host2.

How to install Python packages over SSH Port Forwarding?

2 Answers2