Compiling python into a shared library

Question

I have a bunch of python code that I would like to "compile" into a shared library with a C interface that can be linked with other C/c++ programs and work without depending on too many other libs (perhaps python and some other dlls but they should all be included into a directory with the final lib).

I don't really want to rewrite the python code into C++ for this. I can of course, but it would be best to have a standalone lib that can be used like a dll/so lib.

I have tried cython and wanted to compile python to C and then just compile C code into a dll but that doesn't seem to work quite yet (I haven't been able to make it work flawlessly yet). And then I also tried bbfreeze - but does bbfreeze support creating an .so file? Wasn't able to find out how to do it. Does anyone know?

Do you know of any other options that are more straightforward? the python code only needs to be compiled once. And best of all would be if it creates a single .so file no matter how big it is that just works without too many dependencies.

http://stackoverflow.com/questions/138521/is-it-feasible-to-compile-python-to-machine-code — Jonathan Vanasco, Apr 05 '13 at 22:10
http://stackoverflow.com/questions/4421832/compiling-python-to-c-using-cython — Jonathan Vanasco, Apr 05 '13 at 22:11
@JonathanVanasco: +1 for those great links. They're pretty old (`PyPy` has come a long, long way since 2008!), and they're focused on compiling for performance than for convenient distribution, so I think this is reasonable as a separate question, but they're still useful info. — abarnert, Apr 05 '13 at 22:14

score 3 · Answer 1 · answered Oct 25 '15 at 21:03

You can use Python as a library from inside your C++ application: it's called the Python/C API.

The idea is that you initialize your own interpreter, then you load a script in it. You can expose your C++ objects through global variables or modules inside the Python interpreter for your code to interact with, your you can just run Python functions directly from your C++ code.

Now, I understand that you might want to have your Python scripts embedded inside the shared library: this is not necessarily an easy task, with the traditional GNU toolchain. There are multiple techniques to achieve this, but none is official, and all seem way too complicated as opposed to just having the script in an external file.

If your goal is to hide the scripts from the end-user, you could sign them or encrypt them with a private key and embed the public key inside your library, but keep in mind that the key can easily be retrieved by anyone with enough motivation.

score -2 · Answer 2 · edited Feb 05 '14 at 21:21

-2

Python is more set up to work the other way around: Your code is called from Python, the overall logic is in Python while problem-specific (and performance critical code) is written in C/C++.

edited Feb 05 '14 at 21:21

Kara

5,650
15
48
55

answered Apr 05 '13 at 23:20

vonbrand

10,026
8
27
47

2

That is not what I asked. The necessity is to make a c++ shared library from already written python code. I need the c++ linkage option because I need the functionality implemented in python accessible from c++. – mkschreder Apr 05 '13 at 23:26
@user1953157, it's really not possible due to python's dynamic typing and C++'s static typing. What you're asking for would be the equivalent of a Python to C++ JIT engine, and this doesn't exist. – Nathan Ernst Apr 06 '13 at 01:26
@NathanErnst, it _is_ possible, just totally pointless. It would combine the worst traits of Python (rather inefficient) and C++ (long edit-compile-link-run cycle). – vonbrand Apr 06 '13 at 01:35
2

@vonbrand: nonsense! There is definitely a point in embedding Python inside a C++ (or any other language) app; your app can then be scriptable through Python. [Blender](https://www.blender.org/) does it extensively. Most demanding operations are written in C++, for performance reasons, but almost all the UI is implemented in Python, and nothing stops you from performing demanding operations in Python. – sleblanc Oct 27 '15 at 22:59
@vonbrand, also if you carelessly assume that you will gain performance by calling compiled functions from Python, you might spend more time just releasing and acquiring the GIL. Imagine you need to do lots of square roots, and you wrote a hand-optimized algorithm in assembler; calling it repeatedly in a Python loop will waste lots of resources, as opposed to writing a compiled function that receives a Python iterable as parameter and returns a list. – sleblanc Oct 27 '15 at 23:02

Compiling python into a shared library

2 Answers2

Linked