I wrote a python c extension that uses routine from Intel's math kernel library (mkl). This is the first time that I write a c extension. I just learned that today.
The c extension compiled. But when I import it in python, it says undefined symbol, and can't find a function that is defined in the mkl.
How to include any external c library in a python c extension?
Thank you for your help.
mkl_helper.c:
#include "Python.h"
#include "numpy/arrayobject.h"
#include "mkl.h"
static PyObject* test4 (PyObject *self, PyObject *args)
{
// test4 (m, n,
// a, ja, ia,
// c, jc, ic)
PyArrayObject *shape_array;
PyArrayObject *a_array; // csr_matrix.data
PyArrayObject *ja_array; // csr_matrix.indices
PyArrayObject *ia_array; // csr_matrix.indptr
PyArrayObject *c_array;
PyArrayObject *jc_array;
PyArrayObject *ic_array;
if (!PyArg_ParseTuple(args, "O!O!O!O!O!O!O!",
&PyArray_Type, &shape_array,
&PyArray_Type, &a_array,
&PyArray_Type, &ja_array,
&PyArray_Type, &ia_array,
&PyArray_Type, &c_array,
&PyArray_Type, &jc_array,
&PyArray_Type, &ic_array))
{
return NULL;
}
long * ptr_int = shape_array->data;
int m = ptr_int[0];
int n = ptr_int[1];
int k = n;
float * a_data_ptr = a_array->data;
float * ja_data_ptr = ja_array->data;
float * ia_data_ptr = ia_array->data;
float * c_data_ptr = c_array->data;
float * jc_data_ptr = jc_array->data;
float * ic_data_ptr = ic_array->data;
char trans = 'T';
int sort = 0;
int nzmax = n*n;
int info = -3;
int request = 0;
mkl_scsrmultcsr(&trans, &request, &sort,
&m, &n, &k,
a_data_ptr, ja_data_ptr, ia_data_ptr,
a_data_ptr, ja_data_ptr, ia_data_ptr,
c_data_ptr, jc_data_ptr, ic_data_ptr,
&nzmax, &info);
return PyInt_FromLong(info);
}
static struct PyMethodDef methods[] = {
{"test4", test4, METH_VARARGS, "test2(arr1)\n take a numpy array and return its shape as a tuple"},
{NULL, NULL, 0, NULL}
};
PyMODINIT_FUNC
initmkl_helper (void)
{
(void)Py_InitModule("mkl_helper", methods);
import_array();
}
setup.py:
from distutils.core import setup, Extension
import numpy as np
ext_modules = [ Extension('mkl_helper', sources = ['mkl_helper.c']) ]
setup(
name = 'mkl_helper',
version = '1.0',
include_dirs = [np.get_include()], #Add Include path of numpy
ext_modules = ext_modules
)
test.py:
import mkl_helper
result of running test.py:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: /home/rxu/local/lib/python2.7/site-packages/mkl_helper.so: undefined symbol: mkl_scsrmultcsr
Update 6/16/2016:
this seems to be useful:
1.12. Providing a C API for an Extension Module in https://docs.python.org/2/extending/extending.html says even including one c extension in another c extension can have problems if linked as share library. so, i guess I have to link the mkl as a static library? or add inlcude mkl.h to the python.h?
But then, in python (without c), I can use ctypes.cdll.LoadLibrary("./mkl_rt.so") to load the mkl's shared library and then use c function from the shared library without problem (as in here). Yet Python/C api cannot do the same thing in c?
For linking external c library staticly, the setup.py might need: extra objects in class distutils.core.Extensions at https://docs.python.org/2/distutils/apiref.html?highlight=include#distutils.ccompiler.CCompiler.add_include_dir
Related question about cython with no answer: Combining Cython with MKL
This one seems more helpful: Python, ImportError: undefined symbol: g_utf8_skip
This one use dlopen which is deprecated: Undefined Symbol in C++ When Loading a Python Shared Library