wrapped C++ code poor performance

[SchroedingersLion]

Greetings, I am new to Nanobind, so apologies for the potential beginners' mistakes.

I have a performance-critical C++ simulation code that I want to wrap in a Python package such that the user simply needs to do

import MySim

# specify some args

simu = MySim.simulation(args)

simu.run()

All the heavy lifting happens within the run function, and there is no explicit interface crossing within.

My issue is that this approach leads to run becoming slower by 30% to 50% compared to running it from a compiled C++ main file.

I post my module creation file that also holds the simulation wrapper class that will be exposed to Python. I also post my CMakeList.txt file below.
The code uses OpenMP multithreading under the hood, but I benchmarked by using only one thread.

Is there anything wrong with my setup?

Module creation:

#include <nanobind/nanobind.h>
#include <nanobind/stl/string.h>
#include "simulation.h"
#include "model.h"
#include <string>
#include <memory>
#include <variant>

namespace nb = nanobind;


// using variant because my model comes in 1D, 2D, and 3D versions.
using ModelVariant = std::variant<
    std::unique_ptr<model<1>>, 
    std::unique_ptr<model<2>>, 
    std::unique_ptr<model<3>>
>;

// same for simulation
using SimuVariant = std::variant<
    std::unique_ptr<simulation<1>>, 
    std::unique_ptr<simulation<2>>, 
    std::unique_ptr<simulation<3>>
>;



// Wrapper class exposed to Python.
// For each dimension, a model is created which is then passed by reference to a simulation class.

class PythonWrapper {

    public:
        
        PythonWrapper(int dim, /* other args*/) 
        {
            if (dim == 1){
                model_variant = std::make_unique<model<1>>(/*args*/);
                simulation_variant=  std::make_unique<simulation<1>>( *std::get<std::unique_ptr<model<1>>>(model_variant), /* other args */);
            }
            else if (dim == 2) {
                model_variant = std::make_unique<model<2>>(/*args*/);
                simulation_variant =  std::make_unique<simulation<2>>( *std::get<std::unique_ptr<model<2>>>(model_variant), /* other args */);
            }
            else if (dim == 3) {
                model_variant = std::make_unique<model<3>>(/*args*/);
                simulation_variant =  std::make_unique<simulation<3>>( *std::get<std::unique_ptr<model<3>>>(model_variant), /* other args */);
            }
            else throw std::runtime_error("Unsupported dimension. Only 1D, 2D, and 3D are allowed.");         
        }

        void run() {
            std::visit([](auto& simulation) { simulation->run(); }, simulation_variant);    // This will be much slower when run from Python for some reason.
        }

    private:

        ModelVariant model_variant;   
        SimuVariant simulation_variant;

};


// make wrapper class known to Python

NB_MODULE(MySim, m) {
    
    // Bind wrapper class.
    nb::class_<PythonWrapper>(m, "MySim")
        .def(nb::init</*args*/>())
        .def("run", &PythonWrapper::run)
        
}

CMakeList.txt

cmake_minimum_required(VERSION 3.15...3.27)
project(MySim) 

if (CMAKE_VERSION VERSION_LESS 3.18)
  set(DEV_MODULE Development)
else()
  set(DEV_MODULE Development.Module)
endif()

find_package(Python 3.8 COMPONENTS Interpreter ${DEV_MODULE} REQUIRED)

if (NOT CMAKE_BUILD_TYPE AND NOT CMAKE_CONFIGURATION_TYPES)
  set(CMAKE_BUILD_TYPE Release CACHE STRING "Choose the type of build." FORCE)
  set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS "Debug" "Release" "MinSizeRel" "RelWithDebInfo")
endif()

# Detect the installed nanobind package and import it into CMake
execute_process(
  COMMAND "${Python_EXECUTABLE}" -m nanobind --cmake_dir
  OUTPUT_STRIP_TRAILING_WHITESPACE OUTPUT_VARIABLE nanobind_ROOT)
find_package(nanobind CONFIG REQUIRED)

# Find OpenMP
find_package(OpenMP REQUIRED)

# Add nanobind module
nanobind_add_module(MySim nanobind.cpp)  // nanobind.cpp holds the python wrapper class.

# Enable OpenMP flags
if(OpenMP_CXX_FOUND)
    target_link_libraries(MySim PRIVATE OpenMP::OpenMP_CXX)
    target_compile_options(MySim PRIVATE -fopenmp)
endif()

# Enable -O3 Optimization
target_compile_options(MySim PRIVATE -O3)

target_compile_definitions(MySim PRIVATE -DNDEBUG)

[wjakob]

Dear @SchroedingersLion,

I believe that passing NOMINSIZE to nanobind_add_module should fix your performance issue, please see the documentation for details.

[SchroedingersLion]

Hi @wjakob

thanks for the response!

I just tried it, but it did not help. Somehow, the function being launched with nanobind is slower than launching it from a main.cpp. I had the same experience with pybind11, which is why I ended up here.

[SchroedingersLion]

To compile the main.cpp, I don't even use Make, merely g++ -fopenmp -O3 -o mysim.exe main.cpp.

For the nanobind version I use the Cmake file above (with added NOMINSIZE).

[wjakob]

Can you post the compilation flags used to compile the binding .cpp file?

[wjakob]

Also, are you sure that CMAKE_BUILD_TYPE is set to Release?

[SchroedingersLion]

When running CMake in verbose mode, the output to build nanobind.cpp looks like this:

[ 92%] Building CXX object CMakeFiles/mysim.dir/nanobind.cpp.o
/usr/bin/c++ -DNDEBUG -Dmysim_EXPORTS -I/path_to_conda_env/include/python3.12 -I/path_to_conda_env/lib/python3.12/site-packages/nanobind/include -O3 -DNDEBUG -fPIC -fvisibility=hidden -fno-stack-protector -fopenmp -O3 -ffunction-sections -fdata-sections -MD -MT CMakeFiles/mysim.dir/nanobind.cpp.o -MF CMakeFiles/mysim.dir/nanobind.cpp.o.d -o CMakeFiles/mysim.dir/nanobind.cpp.o -c /path_to_project/src/nanobind.cpp

[100%] Linking CXX shared module mysim.cpython-312-x86_64-linux-gnu.so
/path_to_conda_env/bin/cmake -E cmake_link_script CMakeFiles/mysim.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC -O3 -DNDEBUG -Wl,-s -Wl,--gc-sections -shared  -o mysim.cpython-312-x86_64-linux-gnu.so CMakeFiles/mysim.dir/nanobind.cpp.o  libnanobind-static.a /usr/lib/gcc/x86_64-linux-gnu/11/libgomp.so /usr/lib/x86_64-linux-gnu/libpthread.a 

So it adds more flags, but I thought they are necessary to create the .so object

[SchroedingersLion]

Also had it print the selected build type under the build type logic in the CMakeFile. It does print Release.