Trying to run python code on jenkins in ubuntu

Question

all.

I recently started working with Jenkins, in an attempt to replace cronjob with Jenkins pipeline. I have really a bit knowledge of programming jargon. I learned what I learned from questions on stackoverflow. So, if you guys need any more info, I would really appreciate if you use plain English.

So, I installed the lastest version of Jenkins and suggested plugins plus all the plugins that I could find useful to python running.

Afterwards, I searched stackoverflow and other websites to make this work, but all I could do was

#!/usr/bin/env python

from __future__ import print_function

print('Hello World')

And it succeeded.

Currently, Jenkins is running on Ubuntu 16.04, and I am using anaconda3's python (~/anaconda3/bin/python).

When I tried to run a bit more complicated python code (by that I mean import pandas), it gives me import error.

What I have tried so far is

1. execute python script build: import pandas - import error
2. execute shell build: import pandas (import pandas added to the code that worked above)
3. python builder build: import pandas - invalid interpreter error
4. pipeline job: sh python /path_to_python_file/*.py - import error

All gave errors. Since 'hello world' works, I believe that using anaconda3's python is not an issue. Also, it imported print_function just fine, so I want to know what I should do from here. Change workspace setting? workdirectory setting? code changes?

Thanks.

Answer 1

Since 'hello world' works, I believe that using anaconda3's python is not an issue.

Your assumption is wrong.

There are multiple ways of solving the issue but they all come down to using the correct python interpreter with installed pandas. Usually in ubuntu you'll have at least two interpreters. One for python 2 and one for python 3 and you'll use them in shell by calling either python pth/to/myScript.py or python3 pth/to/myScript.py. python and python3 are in this case just a sort of labels which point to the correct executables, using environmental variable PATH.

By installing anaconda3 you are adding one more interpreter with pandas and plenty of other preinstalled packages. If you want to use it, you need to tell somehow your shell or Jenkins about it. If import pandas gives you an error then you're probably using a different interpreter or a different python environment (but this is out of scope here).

Coming back to your script

Following this stack overflow answer, you'll see that all the line #!/usr/bin/env python does, is to make sure that you're using the first python interpreter on your Ubuntu's environment path. Which almost for sure isn't the one you installed with anaconda3. Most likely it will be the default python 2 distributed with ubuntu. If you want to make sure which interpreter exactly is running your script, instead of 'Hello World' put inside:

#!/usr/bin/env python

import sys

print(sys.executable) # this line will give you the exact path to the interpreter
print(sys.version)    # this one will give you the version

Ok, so what to do?

Well, run your script using the correct interpreter. Remove #!/usr/bin/env python from your file and if you have a pipeline, add there:

sh "/home/yourname/anaconda3/bin/python /path_to_python_file/myFile.py"

It will most likely solve the issue. It's also quite flexible in the sense that if you ever want to use this python file on a different machine, you won't have your username hardcoded inside.