Skip to content

README.md

Latest commit

 

History

History
183 lines (123 loc) · 5.98 KB

README.md

File metadata and controls

183 lines (123 loc) · 5.98 KB

TsFile C++ Document

TsFile Logo

Introduction

This directory contains the C++ implementation of TsFile. The C++ version currently supports the query and write functions of TsFile, including time filtering queries.

The source code can be found in the ./src directory. C/C++ examples are located in the ./examples directory, and a benchmark for TsFile_cpp can be found in the ./bench_mark directory. Additionally, a C function wrapper is available in the ./src/cwrapper directory, which the Python tool relies on.

How to make contributions

We use clang-format to ensure that our C++ code adheres to a consistent set of rules defined in .clang-format. This is similar to the Google style.

mvn spotless:apply uses clang-format v17.0.6 for C++ code formatting. Please make sure the clang-format in your PATH matches this version before submitting code.

How to install clang-format v17.0.6:

  • macOS
brew install llvm@17
ln -sf /opt/homebrew/opt/llvm@17/bin/clang-format /opt/homebrew/bin/clang-format
  • Windows
choco install llvm --version 17.0.6 --force

You can verify the installed version with:

clang-format --version

To format the C++ code, run:

mvn spotless:apply -P with-cpp

If you need to skip code formatting temporarily, you can add -Dspotless.skip=true, for example:

mvn clean verify -P with-cpp -Dspotless.skip=true

Platform Support

TsFile C++ now supports:

  • Linux: GCC/Clang
  • macOS: Clang
  • Windows: MSVC 2017+ and MinGW

All code must compile without errors on all supported platforms before submission.

We welcome any bug reports. You can open an issue with a title starting with [CPP] to describe the bug, like: #94

Build

Requirements

TsFile C++ supports three toolchains:

Linux (GCC/Clang):

sudo apt-get update
sudo apt-get install -y cmake make g++ clang-format libuuid-dev

Windows (MSVC):

  • Visual Studio 2017 or later
  • CMake 3.11+

Windows (MinGW): If you compile using MinGW on windows and encounter an error, you can try replacing MinGW with the following version that we have tried without problems:

  • GCC 14.2.0 (with POSIX threads) + LLVM/Clang/LLD/LLDB 18.1.8 + MinGW-w64 12.0.0 UCRT - release 1
  • GCC 12.2.0 + LLVM/Clang/LLD/LLDB 16.0.0 + MinGW-w64 10.0.0 (UCRT) - release 5
  • GCC 12.2.0 + LLVM/Clang/LLD/LLDB 16.0.0 + MinGW-w64 10.0.0 (MSVCRT) - release 5
  • GCC 11.2.0 + MinGW-w64 10.0.0 (MSVCRT) - release 1

Build Instructions

To build tsfile, use Maven which automatically detects and uses the appropriate toolchain:

mvn clean verify -P with-cpp

Toolchain Selection:

Maven will automatically select the compiler based on your platform:

  • Linux: GCC/Clang
  • macOS: Clang
  • Windows: MinGW (default) or MSVC

To explicitly specify a toolchain on Windows:

# Use MinGW (default on Windows)
mvn clean verify -P with-cpp -Dcpp.toolchain=mingw

# Use MSVC
mvn clean verify -P with-cpp -Dcpp.toolchain=msvc

Then you can find the shared library at ./cpp/target/build/lib.

Before you submit your code to GitHub, please ensure that the compilation is correct.

configure the cross-compilation toolchain

Modify the Toolchain File cmake/ToolChain.cmake, define the following variables:

  • CMAKE_C_COMPILER: Specify the path to the C compiler.
  • CMAKE_CXX_COMPILER: Specify the path to the C++ compiler.
  • CMAKE_FIND_ROOT_PATH: Set the root path for the cross-compilation environment (e.g., the directory of the cross-compilation toolchain).

In the cpp/ directory, run the following commands to create the build directory and start the compilation:

mkdir build && cd build
cmake .. -DToolChain=ON
make

Parallel Write

TsFile C++ supports thread pool-based parallel column encoding for the table write path (write_table). When enabled, each column (time and value columns) is written in parallel using precomputed page boundaries, while maintaining aligned page sealing across columns.

Build Options

Parallel write is controlled by the ENABLE_THREADS CMake option (ON by default):

cmake .. -DENABLE_THREADS=ON   # enable (default)
cmake .. -DENABLE_THREADS=OFF  # disable — all thread code is stripped at compile time

Runtime Configuration

#include "common/global.h"

// Enable or disable parallel write at runtime (auto-disabled on single-core machines)
storage::set_parallel_write_enabled(true);

// Set the number of worker threads (must be called before creating TsFileWriter)
storage::set_write_thread_count(4);

By default, parallel write is enabled when the machine has more than one CPU core, and the thread count is set to the number of hardware cores (capped at 64).

Use TsFile

You can find examples on how to read and write data in demo_read.cpp and demo_write.cpp located under ./examples/cpp_examples. There are also examples under ./examples/c_examples on how to use a C-style API to read and write data in a C environment. The examples will be built automatically when you run the main build command.