Enable ODBC build on MSVC CI

alinaliBQ · alinaliBQ · commit eb5292071f8c · 2025-11-25T12:06:55.000-08:00
Code Clean up and enable ODBC tests in CI Still need to modify ODBCUtilEnvironment if we decide to use it. Still need to add ODBC V2 support so a different env and conn is used for ODBC 2 tests. Draft enable ODBC global setup/teardown Run ODBC test once outside of test script If ODBC test is run using MinGW Shell, segfault occurs. I am not getting any seg fault on my local MSVC Windows when I run the tests without the bash script. But if Windows CI breaks from running the standalone exe then I will look into this Prepend vcpkg to search vcpkg before `<prefix>/lib/cmake` Since we are installing dependencies on vcpkg, if `lib/cmake` is searched first, then cmake will look into that directory and use the wrong paths. Convert VCPKG Windows path to MSYS path Set `VCPKG_ROOT` in test phase Fix ODBC dll name Use `arrow_flight_sql_odbc.dll` instead of `libarrow_flight_sql_odbc.dll` which is the naming convention used on MinGW Windows Link ODBC library on all platforms On my local MSVC Windows machine, Visual Studio is used to build without needing to link ` ${ODBCINST}` explicitly, its behavior might be different from Ninja which is what CI uses. `${ODBCINST}` is likely needed by Linux as well, so adding it for all platforms. Attempt to resolve `arrow-compute-grouper-benchmark` build issue I think `if(ARROW_FLIGHT_TEST_LINKAGE STREQUAL "static")` might be more appropriate here, as I am seeing build issues due to both dynamic and static linking occurring. I see in apache@59903d0, this check is used for `arrow-filesystem-s3fs-benchmark` Disable `UNITY_BUILD` for ODBC test due to conflict on `sqlite_sql_info.cc` On some workflows, unity build is set to ON to make the build faster. This is an attempt to resolve the build issue. Remove debug messages Fix lint Add `sql_info_undef.h` to sqlite_sql_info.cc Undefine duplicates in SQLGetInfo Add `#pragma once` to odbc_test_suite.h To avoid redefinition error Attempt to resolve conflict with sql/types.h Attempt to include Arrow headers before ODBC headers to avoid conflict with arrow/flight/sql/types.h [apacheGH-48084] Replace boost::optional with std::optional Addresses comment apache#40939 (comment) Replace boost::optional with std::optional in ODBC codebase apache#48084 Fix lint Remove `BOOST_SOURCE=BUNDLED` to use vcpkg's dynamic link to boost Since we need to use link to boost dynamically, we need to remove the bundled boost library flag. Add debug messages. Remove `ARROW_BOOST_USE_SHARED = OFF` Since we have a release build, can enable boost as shared. With `ARROW_BOOST_USE_SHARED = OFF`, I am getting error ``` D:\a\arrow\arrow\build\cpp\vcpkg_installed\x64-windows\include\boost/filesystem/config.hpp(96): fatal error C1189: #error: Must not define both BOOST_FILESYSTEM_DYN_LINK and BOOST_FILESYSTEM_STATIC_LINK ``` Also increased time limit, since 2 hours don't seem to be enough to finish the vcpkg build Change to release build Getting ``` orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in unity_2_cxx.cxx.obj orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in unity_2_cxx.cxx.obj ``` when building with debug and presumably ARROW_ORC Set ARROW_BOOST_USE_SHARED to OFF `ARROW_BOOST_USE_SHARED` is restored to `OFF`, which according to Windows doc should help with the debug build now. Retore value of `ARROW_BUILD_BENCHMARKS`, though noting it is set to `OFF` in GLib MSVC workflow. Add VCPKG set up Borrowed from the Glib workflow Add `/EHsc` flag * Add back `CMAKE_CXX_STANDARD: "17"` Remove `CMAKE_CXX_STANDARD` 17 * reason: gtest can't be used with C++17. Arrow project doesn't support C++ 17 yet. Extend run-time to 2hr windows-mingw also has timeout of 2hr Empty commit to trigger CI Specify `VCPKG_BINARY_SOURCES` and `VCPKG_DEFAULT_TRIPLET` Specify VCPKG_TARGET_TRIPLET as x64 windows Set ARROW_DEPENDENCY_SOURCE to VCPKG To make it easier to manage dependencies Enable static build on MSVC `ARROW_BUILD_BENCHMARKS` prevents Flight from being built Remove `ARROW_BOOST_USE_SHARED` Having static vs. shared issue Enable Flight & Flight SQL for ODBC Enable ODBC build on MSVC CI Enable regular ctest tests Remove undef items Add concurrency and permissions to odbc yml Create cpp_odbc.yml
diff --git a/.github/workflows/cpp_odbc.yml b/.github/workflows/cpp_odbc.yml
@@ -0,0 +1,160 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: C++ ODBC
+
+#-AL- TODO change into ODBC workflow
+# path should be odbc only.
+
+on:
+  push:
+    branches:
+      - '**'
+      - '!dependabot/**'
+    tags:
+      - '**'
+    paths:
+      - '.github/workflows/cpp_odbc.yml'
+      - 'ci/scripts/cpp_*'
+      - 'cpp/src/arrow/flight/sql/odbc/*'
+  pull_request:
+    paths:
+      - '.github/workflows/cpp_odbc.yml'
+      - 'ci/scripts/cpp_*'
+      - 'cpp/src/arrow/flight/sql/odbc/*'
+
+concurrency:
+  group: ${{ github.repository }}-${{ github.head_ref || github.sha }}-${{ github.workflow }}
+  cancel-in-progress: true
+
+permissions:
+  contents: read
+
+jobs:
+  windows:
+    runs-on: windows-2022
+    timeout-minutes: 240
+    env:
+      ARROW_BUILD_SHARED: ON
+      ARROW_BUILD_STATIC: ON
+      # -AL- todo have Github issue for this if it fails
+      ARROW_BUILD_TESTS: ON
+      ARROW_BUILD_TYPE: release
+      ARROW_DATASET: ON
+      ARROW_DEPENDENCY_SOURCE: VCPKG
+      ARROW_FLIGHT: ON
+      ARROW_FLIGHT_SQL: ON
+      ARROW_FLIGHT_SQL_ODBC: ON
+      ARROW_SIMD_LEVEL: AVX2
+      CMAKE_CXX_STANDARD: "17"
+      CMAKE_GENERATOR: Ninja
+      CMAKE_INSTALL_PREFIX: /usr
+      VCPKG_BINARY_SOURCES: 'clear;nuget,GitHub,readwrite'
+      VCPKG_DEFAULT_TRIPLET: x64-windows
+    steps:
+      - name: Disable Crash Dialogs
+        run: |
+          reg add `
+            "HKCU\SOFTWARE\Microsoft\Windows\Windows Error Reporting" `
+            /v DontShowUI `
+            /t REG_DWORD `
+            /d 1 `
+            /f
+      - name: Checkout Arrow
+        uses: actions/checkout@v5
+        with:
+          fetch-depth: 0
+          submodules: recursive
+      - name: Download Timezone Database
+        shell: bash
+        run: ci/scripts/download_tz_database.sh
+      - name: Install msys2 (for tzdata for ORC tests)
+        uses: msys2/setup-msys2@v2
+        id: setup-msys2
+      - name: Install cmake
+        shell: bash
+        run: |
+          ci/scripts/install_cmake.sh 4.1.2 /usr
+      - name: Install ccache
+        shell: bash
+        run: |
+          ci/scripts/install_ccache.sh 4.12.1 /usr
+      - name: Setup ccache
+        shell: bash
+        run: |
+          ci/scripts/ccache_setup.sh
+      - name: ccache info
+        id: ccache-info
+        shell: bash
+        run: |
+          echo "cache-dir=$(ccache --get-config cache_dir)" >> $GITHUB_OUTPUT
+      - name: Cache ccache
+        uses: actions/cache@v4
+        with:
+          path: ${{ steps.ccache-info.outputs.cache-dir }}
+          key: cpp-ccache-windows-${{ inputs.arch }}-${{ hashFiles('cpp/**') }}
+          restore-keys: cpp-ccache-windows-${{ inputs.arch }}-
+      - name: Checkout vcpkg
+        uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+        with:
+          fetch-depth: 0
+          path: vcpkg
+          repository: microsoft/vcpkg
+      - name: Bootstrap vcpkg
+        run: |
+          vcpkg\bootstrap-vcpkg.bat
+          $VCPKG_ROOT = $(Resolve-Path -LiteralPath "vcpkg").ToString()
+          Write-Output ${VCPKG_ROOT} | `
+            Out-File -FilePath ${Env:GITHUB_PATH} -Encoding utf8 -Append
+          Write-Output "VCPKG_ROOT=${VCPKG_ROOT}" | `
+            Out-File -FilePath ${Env:GITHUB_ENV} -Encoding utf8 -Append
+      - name: Setup NuGet credentials for vcpkg caching
+        shell: bash
+        run: |
+          $(vcpkg fetch nuget | tail -n 1) \
+            sources add \
+            -source "https://nuget.pkg.github.com/$GITHUB_REPOSITORY_OWNER/index.json" \
+            -storepasswordincleartext \
+            -name "GitHub" \
+            -username "$GITHUB_REPOSITORY_OWNER" \
+            -password "${{ secrets.GITHUB_TOKEN }}"
+          $(vcpkg fetch nuget | tail -n 1) \
+            setapikey "${{ secrets.GITHUB_TOKEN }}" \
+            -source "https://nuget.pkg.github.com/$GITHUB_REPOSITORY_OWNER/index.json"
+      - name: Build
+        shell: cmd
+        run: |
+          set VCPKG_ROOT_KEEP=%VCPKG_ROOT%
+          call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" ${{ inputs.arch }}
+          set VCPKG_ROOT=%VCPKG_ROOT_KEEP%
+          bash -c "ci/scripts/cpp_build.sh $(pwd) $(pwd)/build"
+      - name: Register Flight SQL ODBC Driver
+        shell: cmd
+        run: |
+          call "cpp\src\arrow\flight\sql\odbc\tests\install_odbc.cmd" ${{ github.workspace }}\build\cpp\%ARROW_BUILD_TYPE%\arrow_flight_sql_odbc.dll
+      - name: Test
+        shell: cmd
+        run: |
+          set VCPKG_ROOT_KEEP=%VCPKG_ROOT%
+          call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" ${{ inputs.arch }}
+          # For ORC
+          set TZDIR=${{ steps.setup-msys2.outputs.msys2-location }}\usr\share\zoneinfo
+
+          # Convert VCPKG Windows path to MSYS path
+          for /f "usebackq delims=" %%I in (`bash -c "cygpath -u \"$VCPKG_ROOT_KEEP\""` ) do set VCPKG_ROOT=%%I
+
+          bash -c "ci/scripts/cpp_test.sh $(pwd) $(pwd)/build"
diff --git a/ci/scripts/cpp_test.sh b/ci/scripts/cpp_test.sh
@@ -144,8 +144,9 @@ if [ "${ARROW_USE_MESON:-OFF}" = "OFF" ] && \
       CMAKE_PREFIX_PATH+="/lib/cmake/"
       ;;
   esac
+  # Search vcpkg before <prefix>/lib/cmake.
   if [ -n "${VCPKG_ROOT}" ] && [ -n "${VCPKG_DEFAULT_TRIPLET}" ]; then
-    CMAKE_PREFIX_PATH+=";${VCPKG_ROOT}/installed/${VCPKG_DEFAULT_TRIPLET}"
+    CMAKE_PREFIX_PATH="${VCPKG_ROOT}/installed/${VCPKG_DEFAULT_TRIPLET};${CMAKE_PREFIX_PATH}"
   fi
   cmake \
     -S "${source_dir}/examples/minimal_build" \
diff --git a/cpp/cmake_modules/SetupCxxFlags.cmake b/cpp/cmake_modules/SetupCxxFlags.cmake
@@ -186,6 +186,7 @@ if(WIN32)
       #
       # ARROW-2986: Without /EHsc we get C4530 warning
       set(CXX_COMMON_FLAGS "/W3 /EHsc")
+      string(APPEND CMAKE_CXX_FLAGS " /EHsc")
     endif()
 
     # Disable C5105 (macro expansion producing 'defined' has undefined
diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt
@@ -129,13 +129,9 @@ if(WIN32)
                          system_dsn.h)
 endif()
 
-target_link_libraries(arrow_odbc_spi_impl PUBLIC arrow_flight_sql_shared
-                                                 arrow_compute_shared Boost::locale)
-
-# Link libraries on MINGW64 and macOS
-if(MINGW OR APPLE)
-  target_link_libraries(arrow_odbc_spi_impl PUBLIC ${ODBCINST})
-endif()
+target_link_libraries(arrow_odbc_spi_impl
+                      PUBLIC arrow_flight_sql_shared arrow_compute_shared Boost::locale
+                             ${ODBCINST})
 
 set_target_properties(arrow_odbc_spi_impl
                       PROPERTIES ARCHIVE_OUTPUT_DIRECTORY
diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection_test.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_connection_test.cc
@@ -24,6 +24,8 @@
 
 #include <optional>
 
+#include <optional>
+
 namespace arrow::flight::sql::odbc {
 
 TEST(AttributeTests, SetAndGetAttribute) {
diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/get_info_cache.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/get_info_cache.cc
@@ -19,8 +19,6 @@
 
 #include "arrow/flight/sql/odbc/odbc_impl/platform.h"
 
-#include <sql.h>
-#include <sqlext.h>
 #include "arrow/array.h"
 #include "arrow/array/array_nested.h"
 #include "arrow/flight/sql/api.h"
@@ -32,6 +30,10 @@
 #include "arrow/flight/sql/odbc/odbc_impl/scalar_function_reporter.h"
 #include "arrow/flight/sql/odbc/odbc_impl/util.h"
 
+// Include ODBC headers after arrow headers to avoid conflicts with sql_info_undef.h
+#include <sql.h>
+#include <sqlext.h>
+
 // Aliases for entries in SqlInfoOptions::SqlInfo that are defined here
 // due to causing compilation errors conflicting with ODBC definitions.
 #define ARROW_SQL_IDENTIFIER_CASE 503
diff --git a/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.cc b/cpp/src/arrow/flight/sql/odbc/odbc_impl/odbc_connection.cc
@@ -30,6 +30,7 @@
 #include "arrow/flight/sql/odbc/odbc_impl/spi/statement.h"
 #include "arrow/flight/sql/odbc/odbc_impl/util.h"
 
+// Include ODBC headers after arrow headers to avoid conflicts with sql_info_undef.h
 #include <odbcinst.h>
 #include <sql.h>
 #include <sqlext.h>
diff --git a/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt b/cpp/src/arrow/flight/sql/odbc/tests/CMakeLists.txt
@@ -53,3 +53,6 @@ add_arrow_test(flight_sql_odbc_test
                ${ODBCINST}
                ${SQLite3_LIBRARIES}
                arrow_odbc_spi_impl)
+
+# Disable unity build due to sqlite_sql_info.cc conflict with sql.h and sqlext.h headers.
+set_target_properties(arrow-flight-sql-odbc-test PROPERTIES UNITY_BUILD OFF)
diff --git a/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.h b/cpp/src/arrow/flight/sql/odbc/tests/odbc_test_suite.h
@@ -15,6 +15,8 @@
 // specific language governing permissions and limitations
 // under the License.
 
+#pragma once
+
 #include "arrow/testing/gtest_util.h"
 #include "arrow/util/io_util.h"
 #include "arrow/util/utf8.h"

Original file line number	Diff line number	Diff line change
`@@ -186,6 +186,7 @@ if(WIN32)`
`186`	`186`	`#`
`187`	`187`	`# ARROW-2986: Without /EHsc we get C4530 warning`
`188`	`188`	`set(CXX_COMMON_FLAGS "/W3 /EHsc")`
	`189`	`+ string(APPEND CMAKE_CXX_FLAGS " /EHsc")`
`189`	`190`	`endif()`
`190`	`191`
`191`	`192`	`# Disable C5105 (macro expansion producing 'defined' has undefined`