# Try to Integrate Python and C++
###### tags: `Python`

Make a extension with c++ code for python3.

Not included cPython, pybind11, boost.python, ctype, SWIG.
This only use native and official method.

Table:
[TOC]

## Official Document
* [Intro](https://docs.python.org/3.7/extending/extending.html#a-simple-example)
* [python c-api](https://docs.python.org/3.7/c-api/)

## Concept
All of python instances are object(`PyObject*`), so the c++ function get pyobject as input and send pyobject as output.

Then, you need to wrap your function as a module for your python code to import, just like the module you install.

## Minimal example
`test1.cpp`

#define PY_SSIZE_T_CLEAN
#include <Python.h>

PyObject * spam_six(PyObject*, PyObject*);

static PyMethodDef SpamMethods[] = {
    {"six",  spam_six, METH_VARARGS, "return 6"},
    {NULL},
};

static struct PyModuleDef spammodule = {
    PyModuleDef_HEAD_INIT, "spam", NULL,  -1, SpamMethods
};

PyMODINIT_FUNC PyInit_spam(void)
{
    return PyModule_Create(&spammodule);
}

PyObject* spam_six(PyObject *self, PyObject *args)
{
    return PyLong_FromLong(6);
}

`setup.py`

from distutils.core import setup, Extension

module = Extension('spam',
                    sources = ['test1.cpp'])

setup (name = 'spam',
       version = '1.0',
       description = 'This is a demo package',
       ext_modules = [module])

`test.py`

import spam
print(spam.six())
print(type(spam.six()))

and then run

python3 setup.py install
python3 test.py

## Explain for minimal example
Let me explain it from top to down.

### python code
In `test.py`, me import a module called `spam` and it has a method `six` which always return a constant interger 6.

### setup code
In `setup.py`, the `spam` module can be built by `python3 setup.py install`, where you can see the extension built by gcc from `test1.cpp`.

module = Extension('spam',
                    sources = ['test1.cpp'])

### Module Init
When calling `import spam` in Python, the interpreter will find `PyInit_{name}` to get the module instance, in this case is `PyInit_spam`.

PyMODINIT_FUNC PyInit_spam(void)
{
    return PyModule_Create(&spammodule);
}

The spammodule define module name, documents
and its methods.

static struct PyModuleDef spammodule = {
    PyModuleDef_HEAD_INIT,
    "spam", /* name of module */
    NULL,   /* module documentation, may be NULL */
    -1,     /* size of per-interpreter state of the module,
               or -1 if the module keeps state in global variables. */
    SpamMethods
};

And the methods defined in here

static PyMethodDef SpamMethods[] = {
    {"six",  spam_six, METH_VARARGS, "return 6"},
    {NULL},
};

`"six"` is the name of method,
`spam_six` is the self-defined function you want to call.
`METH_VARARGS` is about to deal with the parameters in your function, I will talk it later.
`"return 6"` is the documents of this function.

Practically, you can add as many entries as you want in here, note that you should add `{NULL}` in the last.

### Main function

PyObject* spam_six(PyObject *self, PyObject *args)
{
    return PyLong_FromLong(6);
}

This function only return 6, however it wrapped into pyobject by `PyLong_FromLong`. The wrapper can find in [c-api doc](https://docs.python.org/3.7/c-api/concrete.html).

## Second, try to read parameters
We take `arange` for example.

In Python, we pass a number to this method

print(spam.arange(4))
print(spam.arange(14))

Add

{"arange",  spam_arange, METH_VARARGS, "arange"},

in `SpamMethods`.

The function:

PyObject* spam_arange(PyObject *self, PyObject *args)
{
    int n;
    if (!PyArg_ParseTuple(args, "i", &n))
        return NULL;
    PyObject* obj = PyList_New(n);
    for(int i=0; i<n; ++i)
        PyList_SetItem(obj, i, PyLong_FromLong(i));
    return obj;
}

The parameters can be parsed by `Pyarg_ParseTuple`, and the whole function return a list wrapped by pyobject.

The detail usage of parsing, you can see [tutorial](https://docs.python.org/3.7/extending/extending.html#extracting-parameters-in-extension-functions) or the [api](https://docs.python.org/3.7/c-api/arg.html).

## Next, read list

spam.array([1, 3 ,2])

PyObject* spam_array(PyObject *self, PyObject *args)
{
    // parse into list
    PyObject *listObj;
    if (!PyArg_ParseTuple(args, "O!", &PyList_Type, &listObj))
        return NULL;

    // get length
    Py_ssize_t numLines = PyList_Size(listObj);
    if (numLines < 0)
        return NULL; /* Not a list */

    // foreach
    PyObject* obj = PyList_New(numLines);
    for (int i=0; i<numLines; i++){
        long x = PyLong_AsLong(PyList_GetItem(listObj, i));
        printf("%d -> %ld\n", i, x);
        PyList_SetItem(obj, i, PyLong_FromLong(x));
    }
    return obj;
}

reference https://stackoverflow.com/questions/3253563/pass-list-as-argument-to-python-c-module

## Same thing for python dictionary

print(spam.get({'a': 123, 'b': 345}, 'a'))

PyObject* spam_get(PyObject *self, PyObject *args)
{
    PyObject* dictObj;
    char* s;
    // parse into list
    if (!PyArg_ParseTuple(args, "O!s", &PyDict_Type, &dictObj, &s))
        return NULL;

    long v = PyLong_AsLong(PyDict_GetItem(dictObj, PyUnicode_FromString(s)));
    printf("%s -> %ld\n", s, v);
    return PyLong_FromLong(v);
}

## Finally, raising error

static PyObject* SpamError = PyErr_NewException("spam.error", NULL, NULL);
PyObject* spam_arange(PyObject *self, PyObject *args)
{
    ...
    if (n < 0) {
        PyErr_SetString(SpamError, "Number should be nonzero");
        // PyErr_SetString(PyExc_IndexError, "Number should be nonzero");
        return NULL;
    }
    ...

and the object should be carefully removed if you don't want. **No auto garbage collection in here.**

    PyObject* tmpobj = PyList_New(10000000);
    // Py_XDECREF(tmpobj);  // obj can be NULL
    Py_DECREF(tmpobj);  // obj NOT NULL

The default exceptions are listed [here](https://docs.python.org/3.7/c-api/exceptions.html#standard-exceptions).

## Extra
Bulid with c++17

module = Extension('spam',
                   extra_compile_args=['-std=c++17'],
                   sources = ['test1.cpp'])

## Overall
test.py

import spam
print(spam.six())
print(spam.array([1, 2, 3]))
print(spam.get({'a': 123, 'b': 345}, 'a'))
print(spam.get({'a': 123, 'b': 345}, 'b'))
print(spam.arange(3))
print(spam.arange(-3))

test1.cpp

#define PY_SSIZE_T_CLEAN
#include <Python.h>

PyObject * spam_six(PyObject*, PyObject*);
PyObject * spam_arange(PyObject*, PyObject*);
PyObject * spam_array(PyObject*, PyObject*);
PyObject * spam_get(PyObject*, PyObject*);

static PyMethodDef SpamMethods[] = {
    {"six",  spam_six, METH_VARARGS, "return 6"},
    {"arange",  spam_arange, METH_VARARGS, "arange"},
    {"array",  spam_array, METH_VARARGS, "array"},
    {"get",  spam_get, METH_VARARGS, "get from dictionary"},
    {NULL},
};

static struct PyModuleDef spammodule = {
    PyModuleDef_HEAD_INIT, "spam", NULL,  -1, SpamMethods
};

PyMODINIT_FUNC PyInit_spam(void)
{
    return PyModule_Create(&spammodule);
}

PyObject* spam_six(PyObject *self, PyObject *args)
{
    return PyLong_FromLong(6);
}

static PyObject* SpamError = PyErr_NewException("spam.error", NULL, NULL);
PyObject* spam_arange(PyObject *self, PyObject *args)
{
    int n;
    PyObject* tmpobj = PyList_New(10000000);
    // Py_XDECREF(tmpobj);  // obj can be NULL
    Py_DECREF(tmpobj);  // obj NOT NULL
    if (!PyArg_ParseTuple(args, "i", &n))
        return NULL;
    if (n < 0) {
        PyErr_SetString(SpamError, "Number should be nonzero");
        // PyErr_SetString(PyExc_IndexError, "Number should be nonzero");
        return NULL;
    }
    PyObject* obj = PyList_New(n);
    for(int i=0; i<n; ++i)
        PyList_SetItem(obj, i, PyLong_FromLong(i));
    return obj;
}

PyObject* spam_array(PyObject *self, PyObject *args)
{
    PyObject *listObj;
    // parse into list
    if (!PyArg_ParseTuple(args, "O!", &PyList_Type, &listObj))
        return NULL;

    // get length
    Py_ssize_t numLines = PyList_Size(listObj);
    if (numLines < 0)
        return NULL; /* Not a list */

    // foreach
    PyObject* obj = PyList_New(numLines);
    for (int i=0; i<numLines; i++){
        long x = PyLong_AsLong(PyList_GetItem(listObj, i));
        printf("%d -> %ld\n", i, x);
        PyList_SetItem(obj, i, PyLong_FromLong(x));
    }
    return obj;
}

PyObject* spam_get(PyObject *self, PyObject *args)
{
    PyObject *dictObj;
    char* s;
    // parse into dict
    if (!PyArg_ParseTuple(args, "O!s", &PyDict_Type, &dictObj, &s))
        return NULL;

    printf("%s\n", s);
    long v = PyLong_AsLong(PyDict_GetItem(dictObj, PyUnicode_FromString(s)));
    printf("%s -> %ld\n", s, v);
    return PyLong_FromLong(v);
}

## On Windows
Buld cpp in windows is not such easy thing.

First, make sure Python and your g++ are 64bits.

The g++ (MinGW32) in 64 bits can be download here
https://sourceforge.net/projects/mingw-w64/

To check:
* `g++ -v` the target should be x86_64-w64-mingw32
* `python -c 'import sys;print("%x" % sys.maxsize)'` The output of 64bits python should be `7fffffffffffffff`

If there is not PATH to g++, you can setup manually.
`set PATH=/your/path/MinGW/bin:$PATH`

Add below compiler setting in `.../Python37/Lib/distutils/distutils.cfg`

[build]
compiler = mingw32

Also, patch `.../Python37/Lib/distutils/cygwinccompiler.py`

def get_msvcr():
    ...
    if msc_ver == '1300':
        ...
    else:
        return []

Reference:
https://stackoverflow.com/questions/1405913/how-do-i-determine-if-my-python-shell-is-executing-in-32bit-or-64bit-mode-on-os
https://stackoverflow.com/questions/34135280/valueerror-unknown-ms-compiler-version-1900