|
1 | | -# InterOp |
2 | | -A Clang-based C++ Interoperability library |
| 1 | +## CppInterOp Introduction |
| 2 | +The CppInterOp library provides a minimalist approachfor other languages to |
| 3 | +identify C++ entities (variables, classes, etc.). This |
| 4 | +enables interoperability with C++ code, bringing the speed and |
| 5 | +efficiency of C++ to simpler, more interactive languages like Python. |
| 6 | + |
| 7 | +### Incremental Adoption |
| 8 | +CppInterOp can be adopted incrementally. While the rest of the framework is |
| 9 | +the same, a small part of CppInterOp can be utilized. More components may be |
| 10 | +adopted over time. |
| 11 | + |
| 12 | +### Minimalist by design |
| 13 | +While the library includes some tricky code, it is designed to be simple and |
| 14 | +robust (simple function calls, no inheritance, etc.). The goal is to make it |
| 15 | +as close to the compiler API as possible, and each routine to do just one |
| 16 | +thing that it was designed for. |
| 17 | + |
| 18 | +### Further Enhancing the Dynamic/Automatic bindings in CPPYY |
| 19 | +The main use case for CppInterOp is with the CPPYY service. CPPYY is an |
| 20 | +automatic run-time bindings generator for Python & C++, and supports a wide |
| 21 | +range of C++ features (e.g., template instantiation). It operates on demand |
| 22 | +and generates only what is necessary. It requires a compiler (Cling[^1] |
| 23 | +/Clang-REPL[^2]) that can be available during program runtime. |
| 24 | + |
| 25 | +Once CppInterOp is integrated with LLVM's[^3] Clang-REPL component (that can |
| 26 | +then be used as a runtime compiler for CPPYY), it will further enhance |
| 27 | +CPPYY's performance in the following ways: |
| 28 | + |
| 29 | +- **Simpler codebase**: Removal of string parsing logic will lead to a |
| 30 | +simpler code base. |
| 31 | +- **LLVM Integration**: The CppInterOp interfaces will be a part of the LLVM |
| 32 | +toolchain (as part of Clang-REPL). |
| 33 | +- **Better C++ Support**: C++ features such as Partial Template |
| 34 | +Specialization will be available through CppInterOp. |
| 35 | +- **Fewer Lines of Code**: A lot of dependencies and workarounds will be |
| 36 | +removed, reducing the lines of code required to execute CPPYY. |
| 37 | +- **Well tested interoperability Layer**: The CppInterOp interfaces have full |
| 38 | + unit test coverage. |
| 39 | + |
| 40 | +### 'Roots' in High Energy Physics research |
| 41 | +Besides being developed as a general-purpose library, one of the long-term |
| 42 | +goals of CppInterOp is to stay backward compatible and be adopted in the High |
| 43 | + Energy Physics (HEP) field, as it will become an essential part of the Root |
| 44 | +framework. Over time, parts of the Root framework can be swapped by this API, |
| 45 | + adding speed and resilience with it. |
| 46 | + |
| 47 | +### Build Instructions for Linux (Debian) |
| 48 | +Build instructions for CppInterOp and its dependencies are as follows. |
| 49 | + |
| 50 | +#### Setup Clang-REPL |
| 51 | +Clone and checkout the LLVM project repository. |
| 52 | +``` |
| 53 | +git clone https://github.com/llvm/llvm-project.git |
| 54 | +git checkout release/16.x |
| 55 | +``` |
| 56 | +Get the following patches required for development work. |
| 57 | +``` |
| 58 | +compgen -G "../patches/llvm/clang16-*.patch" > /dev/null && find ../patches/llvm/clang16-*.patch -printf "%f\n" && git apply ../patches/llvm/clang16-*.patch |
| 59 | +``` |
| 60 | +##### Build Clang |
| 61 | +Clang-REPL is an interpreter that CppInterOp works alongside. Build Clang (and |
| 62 | +Clang-REPL along with it). |
| 63 | +``` |
| 64 | +mkdir build && cd build |
| 65 | +cmake -DLLVM_ENABLE_PROJECTS=clang \ |
| 66 | + -DLLVM_TARGETS_TO_BUILD="host;NVPTX" \ |
| 67 | + -DCMAKE_BUILD_TYPE=Release \ |
| 68 | + -DLLVM_ENABLE_ASSERTIONS=ON \ |
| 69 | + -DLLVM_USE_LINKER=lld \ |
| 70 | + -DCLANG_ENABLE_STATIC_ANALYZER=OFF \ |
| 71 | + -DCLANG_ENABLE_ARCMT=OFF \ |
| 72 | + -DCLANG_ENABLE_FORMAT=OFF \ |
| 73 | + -DCLANG_ENABLE_BOOTSTRAP=OFF \ |
| 74 | + ../llvm |
| 75 | +cmake --build . --target clang clang-repl --parallel $(nproc --all) |
| 76 | +``` |
| 77 | +Note the 'llvm-project' directory location. |
| 78 | +``` |
| 79 | +cd ../ |
| 80 | +export LLVM_DIR=$PWD |
| 81 | +cd ../ |
| 82 | +``` |
| 83 | +Next, use the following export commands. |
| 84 | +``` |
| 85 | +export CB_PYTHON_DIR="$PWD/cppyy-backend/python" |
| 86 | +export INTEROP_DIR="$CB_PYTHON_DIR/cppyy_backend" |
| 87 | +``` |
| 88 | +#### Build 'LLVM-Project' related dependencies |
| 89 | +Following steps are required to help build CppInterOp alongside the |
| 90 | +LLVM-Project. |
| 91 | +``` |
| 92 | +mkdir build && cd build |
| 93 | +export INTEROP_BUILD_DIR=$PWD |
| 94 | +cmake -DCMAKE_BUILD_TYPE=Release \ |
| 95 | + -DUSE_CLING=OFF \ |
| 96 | + -DUSE_REPL=ON \ |
| 97 | + -DLLVM_DIR=$LLVM_BUILD_DIR \ |
| 98 | + -DLLVM_USE_LINKER=lld \ |
| 99 | + -DBUILD_SHARED_LIBS=ON \ |
| 100 | + -DCMAKE_INSTALL_PREFIX=$INTEROP_DIR \ |
| 101 | + ../ |
| 102 | +cmake --build . --target install --parallel $(nproc --all) |
| 103 | +``` |
| 104 | +#### Build Cling related dependencies |
| 105 | +Besides the Clang-REPL interpreter, CppInterOp also works alongside the Cling |
| 106 | +interpreter. Cling depends on its own customised version of `llvm-project`, |
| 107 | +hosted under the `root-project` (see the git path below). |
| 108 | +Use the following build instructions. |
| 109 | +``` |
| 110 | +git clone --depth=1 https://github.com/root-project/cling.git |
| 111 | +git clone --depth=1 -b cling-llvm13 https://github.com/root-project/llvm-project.git |
| 112 | +cd llvm-project |
| 113 | +mkdir build && cd build |
| 114 | +cmake -DLLVM_ENABLE_PROJECTS=clang \ |
| 115 | + -DLLVM_EXTERNAL_PROJECTS=cling \ |
| 116 | + -DLLVM_EXTERNAL_CLING_SOURCE_DIR=../../cling \ |
| 117 | + -DLLVM_TARGETS_TO_BUILD="host;NVPTX" \ |
| 118 | + -DCMAKE_BUILD_TYPE=Release \ |
| 119 | + -DLLVM_ENABLE_ASSERTIONS=ON \ |
| 120 | + -DLLVM_USE_LINKER=lld \ |
| 121 | + -DCLANG_ENABLE_STATIC_ANALYZER=OFF \ |
| 122 | + -DCLANG_ENABLE_ARCMT=OFF \ |
| 123 | + -DCLANG_ENABLE_FORMAT=OFF \ |
| 124 | + -DCLANG_ENABLE_BOOTSTRAP=OFF \ |
| 125 | + ../llvm |
| 126 | +cmake --build . --target clang --parallel $(nproc --all) |
| 127 | +cmake --build . --target cling --parallel $(nproc --all) |
| 128 | +cmake --build . --target gtest_main --parallel $(nproc --all) |
| 129 | +``` |
| 130 | +Note the 'llvm-project' directory location. |
| 131 | +``` |
| 132 | +cd ../ |
| 133 | +export LLVM_DIR=$PWD |
| 134 | +cd ../ |
| 135 | +``` |
| 136 | +Next, export the following directory. |
| 137 | +``` |
| 138 | +export INTEROP_DIR=$PWD/cppyy-backend/python/cppyy_backend |
| 139 | +``` |
| 140 | +#### Build CppInterOp |
| 141 | +Finally, clone the CppInterOp repository. |
| 142 | +``` |
| 143 | +git clone https://github.com/compiler-research/CppInterOp.git |
| 144 | +cd InterOp |
| 145 | +mkdir build && cd build |
| 146 | +INTEROP_BUILD_DIR=$(PWD) |
| 147 | +Execute the following. |
| 148 | +cmake -DBUILD_SHARED_LIBS=ON -DUSE_CLING=ON -DUSE_REPL=Off -DCling_DIR=$LLVM_DIR/build -DCMAKE_INSTALL_PREFIX=$INTEROP_DIR .. |
| 149 | +cmake --build . --target install |
| 150 | +``` |
| 151 | + |
| 152 | +--- |
| 153 | +Further Reading: [C++ Language Interoperability Layer](https://compiler-research.org/libinterop/) |
| 154 | + |
| 155 | +[^1]: Cling is an interpretive Compiler for C++. |
| 156 | +[^2]: Clang-REPL is an interactive C++ interpreter that enables incremental |
| 157 | +compilation. |
| 158 | +[^3]: LLVM is a Compiler Framework. It is a collection of modular compiler |
| 159 | +and toolchain technologies. |
0 commit comments