Call python code from c via cython

Question

So I'd like to call some python code from c via cython. I've managed to call cython code from c. And I can also call python code from cython. But when I add it all together, some things are missing.

Here is my python code (quacker.pyx):

def quack():
    print "Quack!"

Here is my cython "bridge" (caller.pyx):

from quacker import quack

cdef public void call_quack():
    quack()

And here is the c code (main.c):

#include <Python.h>
#include "caller.h"

int main() {
  Py_Initialize();
  initcaller();
  call_quack();
  Py_Finalize();
  return 0;
}

When I run this I get this exception:

Exception NameError: "name 'quack' is not defined" in 'caller.call_quack' ignored

The missing pieces I'm suspecting:

I haven't called initquacker()
I haven't included quacker.h
Cython didn't produce any quacker.h - only quacker.c
caller.c doesn't import quacker.h or call initquacker()

I'm not really sure that it's even possible to do what I'm trying to do, but it seems to me that it ought to be. I'd love to hear any input you might have.

Edit:

This is how I cythonize / compile / link / run:

$ cython *.pyx
$ cc -c *.c -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
$ cc -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython2.7 -ldl *.o -o main
$ ./main

Can the quacker file just be a python .py file, or does it have to be a .pyx one? — brm
– brm, Commented Mar 23, 2014 at 13:29
@brm Yes, in fact I would prefer that, since all the python code I want to access is already in .py files. — pius
– pius, Commented Mar 23, 2014 at 15:51

brm · Accepted Answer · 2014-03-23 17:22:41Z

11

If you rename the quacker.pyx to quacker.py, everything is actually correct. The only problem is that your program won't search for python modules in the current directory, resulting in the output:

Exception NameError: "name 'quack' is not defined" in 'caller.call_quack' ignored

If you add the current directory to the PYTHONPATH environment variable however, the output becomes the one you'd expect:

$ PYTHONPATH=".:$PYTHONPATH" ./main 
Quack!

When running the python shell, according to the documentation the current directory (or the directory containing the script) is added to the sys.path variable automatically, but when creating a simple program using Py_Initialize and Py_Finalize this does not seem to happen. Since the PYTHONPATH variable is also used to populate the sys.path python variable, the workaround above produces the correct result.

Alternatively, below the Py_Intialize line, you could add an empty string to sys.path as follows by just executing some python code, specified as a string:

PyRun_SimpleString("import sys\nsys.path.insert(0,'')");

After recompiling, just running ./main should then work.

Edit

It's actually interesting to see what's going on if you run the code as specified in the question, so without renaming the quacker.pyx file. In that case, the initcaller() function tries to import the quacker module, but since no quacker.py or quacker.pyc exists, the module cannot be found, and the initcaller() function produces an error.

Now, this error is reported the python way, by raising an exception. But the code in the main.c file doesn't check for this. I'm no expert in this, but in my tests adding the following code below initcaller() seemed to work:

if (PyErr_Occurred())
{
    PyErr_Print();
    return -1;
}

The output of the program then becomes the following:

Traceback (most recent call last):
  File "caller.pyx", line 1, in init caller (caller.c:836)
    from quacker import quack
ImportError: No module named quacker

By calling the initquacker() function before initcaller(), the module name quacker already gets registered so the import call that's done inside initcaller() will detect that it's already loaded and the call will succeed.

edited Mar 23, 2014 at 17:22

answered Mar 23, 2014 at 16:52

brm

3,8761 gold badge17 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

hivert Over a year ago

Alternatively you can set the PYTHONPATH with PySys_SetPath("..."); where ... is what you want.

brm Over a year ago

@hivert Just doing that would set sys.path to only '.', so I'm guessing that would make it difficult to find other standard modules

pius Over a year ago

Great answer :-). I think I prefer to put the sys.path.insert call at the top of caller.pyx. And that PyErr_Occurred trick works great.

AlexF · Accepted Answer · 2023-06-02 11:06:40Z

9

In case there's anyone wondering how would it work in Python 3, here's my solution after struggling a bit as a Cython newbie.

main.c

#include <Python.h>
#include "caller.h"

int
main() 
{
    PyImport_AppendInittab("caller", PyInit_caller);
    Py_Initialize();
    PyImport_ImportModule("caller");
    call_quack();
    Py_Finalize();
    return 0;
}

caller.pyx

# cython: language_level=3
import sys
sys.path.insert(0, '')

from quacker import quack

cdef public void call_quack():
    quack()

quacker.py

def quack():
    print("Quack!")

Finally, here's the Makefile that compiles everything:

target=main
cybridge=caller

CC=gcc
CFLAGS= `python3-config --cflags`
LDFLAGS=`python3-config --ldflags`

all:
        cython $(cybridge).pyx
        $(CC) $(CFLAGS) -c *.c
        $(CC) *.o -o $(target) $(LDFLAGS)

clean:
        rm -f $(cybridge).{c,h,o} $(target).o $(target)
        rm -rf __pycache__

edited Jun 2, 2023 at 11:06

AlexF

4375 silver badges9 bronze badges

answered Nov 13, 2019 at 23:10

cassota

1322 silver badges5 bronze badges

1 Comment

AlexF Over a year ago

On Windows with MSYS2, PYTHONHOME and PYTHONPATH had to be set to run the executable without errors or warnings. In my setup it is set to c:\msys64\mingw64\lib\python3.10

hivert · Accepted Answer · 2014-03-23 15:05:49Z

3

Maybe this is not what you want but I got it working by the following changes:

in quacker.pyx I added

cdef public int i

To force Cython to generate the .h file.

An then in the main:

#include <Python.h>
#include "caller.h"
#include "quacker.h"

int main() {
  Py_Initialize();
  initquacker();
  initcaller();
  call_quack();
  Py_Finalize();
  return 0;
}

answered Mar 23, 2014 at 15:05

hivert

10.7k3 gold badges34 silver badges58 bronze badges

3 Comments

pius Over a year ago

Nice, that works :-). It is a bit hacky though. To use this method I have to put bogus attributes in all my python files. It also means that my c code has to know about all my python modules. I'd prefer that the c code only knows about the bridge (caller.pyx).

brm Over a year ago

You actually don't need that header file. In C functions can be declared implicitly, so the code also works without the header file, just calling initquacker()

pius Over a year ago

That's right! I actually just gave up after the compiler gave me this warning: main.c:7:3: warning: implicit declaration of function 'initquacker' is invalid in C99 [-Wimplicit-function-declaration]

Ani · Accepted Answer · 2021-01-13 02:44:42Z

1

I needed to do this using CMake and ended up recreating this sample. You can find the repository with complete working example here.

You can build and run the example using either Docker on the CLI or a Visual Studio devcontainer.

answered Jan 13, 2021 at 2:44

Ani

11k3 gold badges30 silver badges49 bronze badges

Collectives™ on Stack Overflow

Call python code from c via cython

4 Answers 4

Edit

3 Comments

1 Comment

3 Comments

Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

4 Answers 4

Edit

3 Comments

1 Comment

3 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related