How to call C++ functions in Python

About

pybind11 is a light-weigth header-only type library in C++ which helps using native C++ types for a function called by python.

To call a C++ function you need to include this library, define a PYBIND11_MODULE and build the C++ source code as a shared library. Then you can call it via python code.

In this tutorial, I will show you step by step, how to call a C++ via python in Ubuntu Linux environment. If you are using any other operating system, you will have some minor variation in your steps.

This library, receive a list of floating point price numbers and calculate the average, median, minimum and maximum of them. There are three functions: average, median and examine. The first two function will return a single number out of the array. The examine function will return

Prerequisites

To use pybind11, you the first step is to install python3-dev as follows:

sudo apt install python3-dev

The next step is to install pybind11 as follows:

pip install pybind11

Make sure you can run the following command successfully in your command line:

python3 -m pybind11 --includes

Define a C++ library

Here, I will define a none-trivial library to demonstrate how easily the C++ types are recognized in the python environment. Here is the C++ code:

price_analyzer.cpp

#include <algorithm>
#include <cmath>
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <string>
#include <vector>

namespace py = pybind11;
using std::vector;
using std::string;
using std::unordered_map;

typedef unordered_map<string, double> ResultType;



double average(const vector<double> &prices)
{
   if(!prices.size())
      return std::numeric_limits<double>::quiet_NaN();
   double sum = 0.0;
   for(double x : prices)
      sum += x;
   return sum / (double) prices.size();
}


double median(const vector<double> &prices)
{
   size_t len = prices.size();
   if(!len)
      return 0.0;
   vector<double> sorted_copy = prices;
   std::sort(sorted_copy.begin(), sorted_copy.end());
   if(len % 2 == 0)
      return (sorted_copy[len/2 - 1] + sorted_copy[len/2]) / 2.0;
   else
      return sorted_copy[len/2];
}


ResultType examine(const vector<double> &prices)
{
   ResultType result;
   if(prices.size())
   {
      result["average"] = average(prices);
      result["median"] = median(prices);
      result["max"] = *std::max_element(prices.begin(), prices.end());
      result["min"] = *std::min_element(prices.begin(), prices.end());
   }
   return result;
}


PYBIND11_MODULE(price_analyzer, m)
{
   m.def("examine", &examine, "Analyze a list of prices");
   m.def("average", &average, "Find the average of prices");
   m.def("median", &median, "Find the median of prices");
}

As you have noticed, I have defined 3 functions: average, median and examine.

I would like a python script to be able to call any of them. Therefore, I need to define a PYBIND11_MODULE section at the end of the C++ library and introduce each of these entry functions there. Thus, a python script can call them.

Building the C++ library

To build the C++ script, you will need a g++ or another available compiler. If you have no compiler on your computer yet, you can simply install it via

sudo apt install g++

The command to build the library is as follows

g++ -O3 -Wall -shared -std=c++11 -fPIC $(python3 -m pybind11 --includes) price_analyzer.cpp -o price_analyzer.so

If you are using windows, you will need to replace .so extension with .dll. Please not that python interpreter is very sensitive to the extension of these binary libraries.

The command python3 -m pybind11 --includes will provide you a list of -I switches for the compiler to include required paths for pybind11.

Call the built module features in python

The last step is to call your library module features in your python script as follows:

price_test.py

import price_analyzer



prices = [9.11, 7.4, 3.2, 8.9, 22]
result = price_analyzer.examine(prices)

print('List of prices:', prices)

# call the analyzer function
print('Price analyzer result:', result)


# call the features separately
print('Average:', price_analyzer.average(prices))
print('Median:', price_analyzer.median(prices))

The price_analyzer.so file can be simply be located in the same directory as your python script.

Then call your python script:

python3 price_test.py

And observe the output result:


List of prices: [9.11, 7.4, 3.2, 8.9, 22]
Price analyzer result: {'min': 3.2, 'max': 22.0, 'median': 8.9, 'average': 10.122}
Average: 10.122
Median: 8.9

It is that simple!

Impediments

If you do not perform the steps correctly or if you have a wrong setup, you may encounter some errors.

The one of the common errors is as follows:


ImportError: dynamic module does not define module export function (PyInit_price_analyzer)

This error is generated when in the module definition

PYBIND11_MODULE(price_analyzer, m)

, the name of the claimed module name does not match the real module name.

I hope this tutorial helps you enjoying the fast speed of C++ functions in python without any headache.