Enable ODBC build on MSVC CI

Code Clean up and enable ODBC tests in CI

Still need to modify ODBCUtilEnvironment if we decide to use it. Still need to add ODBC V2 support so a different env and conn is used for ODBC 2 tests.

Draft enable ODBC global setup/teardown

Run ODBC test once outside of test script

If ODBC test is run using MinGW Shell, segfault occurs.
I am not getting any seg fault on my local MSVC Windows when I run the tests without the bash script. But if Windows CI breaks from running the standalone exe then I will look into this

Prepend vcpkg to search vcpkg before `<prefix>/lib/cmake`

Since we are installing dependencies on vcpkg, if `lib/cmake` is searched first, then cmake will look into that directory and use the wrong paths.

Convert VCPKG Windows path to MSYS path

Set `VCPKG_ROOT` in test phase

Fix ODBC dll name

Use `arrow_flight_sql_odbc.dll` instead of `libarrow_flight_sql_odbc.dll` which is the naming convention used on MinGW Windows

Link ODBC library on all platforms

On my local MSVC Windows machine, Visual Studio is used to build without needing to link ` ${ODBCINST}` explicitly, its behavior might be different from Ninja which is what CI uses.
`${ODBCINST}` is likely needed by Linux as well, so adding it for all platforms.

Attempt to resolve `arrow-compute-grouper-benchmark` build issue

I think `if(ARROW_FLIGHT_TEST_LINKAGE STREQUAL "static")` might be more appropriate here, as I am seeing build issues due to both dynamic and static linking occurring. I see in apache@59903d0, this check is used for `arrow-filesystem-s3fs-benchmark`

Disable `UNITY_BUILD` for ODBC test due to conflict on `sqlite_sql_info.cc`

On some workflows, unity build is set to ON to make the build faster. This is an attempt to resolve the build issue.

Remove debug messages

Fix lint

Add `sql_info_undef.h` to sqlite_sql_info.cc

Undefine duplicates in SQLGetInfo

Add `#pragma once` to odbc_test_suite.h

To avoid redefinition error

Attempt to resolve conflict with sql/types.h

Attempt to include Arrow headers before ODBC headers to avoid conflict with arrow/flight/sql/types.h

[apacheGH-48084] Replace boost::optional with std::optional

Addresses comment apache#40939 (comment)
Replace boost::optional with std::optional in ODBC codebase

apache#48084

Fix lint

Remove `BOOST_SOURCE=BUNDLED` to use vcpkg's dynamic link to boost

Since we need to use link to boost dynamically, we need to remove the bundled boost library flag.

Add debug messages.

Remove `ARROW_BOOST_USE_SHARED = OFF`

Since we have a release build, can enable boost as shared.

With `ARROW_BOOST_USE_SHARED = OFF`, I am getting error
```
D:\a\arrow\arrow\build\cpp\vcpkg_installed\x64-windows\include\boost/filesystem/config.hpp(96): fatal error C1189: #error:  Must not define both BOOST_FILESYSTEM_DYN_LINK and BOOST_FILESYSTEM_STATIC_LINK
```

Also increased time limit, since 2 hours don't seem to be enough to finish the vcpkg build

Change to release build

Getting
```
orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for '_ITERATOR_DEBUG_LEVEL': value '0' doesn't match value '2' in unity_2_cxx.cxx.obj
orc.lib(Exceptions.cc.obj) : error LNK2038: mismatch detected for 'RuntimeLibrary': value 'MD_DynamicRelease' doesn't match value 'MDd_DynamicDebug' in unity_2_cxx.cxx.obj
```
when building with debug and presumably ARROW_ORC

Set ARROW_BOOST_USE_SHARED to OFF

`ARROW_BOOST_USE_SHARED` is restored to `OFF`, which according to Windows doc should help with the debug build now.

Retore value of `ARROW_BUILD_BENCHMARKS`, though noting it is set to `OFF` in GLib MSVC workflow.

Add VCPKG set up

Borrowed from the Glib workflow

Add `/EHsc` flag

* Add back `CMAKE_CXX_STANDARD: "17"`

Remove `CMAKE_CXX_STANDARD` 17

* reason: gtest can't be used with C++17. Arrow project doesn't support C++ 17 yet.

Extend run-time to 2hr

windows-mingw also has timeout of 2hr

Empty commit to trigger CI

Specify `VCPKG_BINARY_SOURCES` and `VCPKG_DEFAULT_TRIPLET`

Specify VCPKG_TARGET_TRIPLET as x64 windows

Set ARROW_DEPENDENCY_SOURCE to VCPKG

To make it easier to manage dependencies

Enable static build on MSVC

`ARROW_BUILD_BENCHMARKS` prevents Flight from being built

Remove `ARROW_BOOST_USE_SHARED`

Having static vs. shared issue

Enable Flight & Flight SQL for ODBC

Enable ODBC build on MSVC CI

Enable regular ctest tests

Author: alinaliBQ

.github/workflows/cpp_windows.yml

@@ -36,15 +36,18 @@ on:
 jobs:
   windows:
     runs-on: ${{ inputs.os }}
-    timeout-minutes: 60
+    timeout-minutes: 240
     env:
-      ARROW_BOOST_USE_SHARED: OFF
       ARROW_BUILD_BENCHMARKS: ON
       ARROW_BUILD_SHARED: ON
-      ARROW_BUILD_STATIC: OFF
+      ARROW_BUILD_STATIC: ON
       ARROW_BUILD_TESTS: ON
+      ARROW_BUILD_TYPE: release
       ARROW_DATASET: ON
-      ARROW_FLIGHT: OFF
+      ARROW_DEPENDENCY_SOURCE: VCPKG
+      ARROW_FLIGHT: ON
+      ARROW_FLIGHT_SQL: ON
+      ARROW_FLIGHT_SQL_ODBC: ON
       ARROW_HDFS: ON
       ARROW_HOME: /usr
       ARROW_JEMALLOC: OFF
@@ -62,11 +65,12 @@ jobs:
       ARROW_WITH_SNAPPY: ON
       ARROW_WITH_ZLIB: ON
       ARROW_WITH_ZSTD: ON
-      BOOST_SOURCE: BUNDLED
       CMAKE_CXX_STANDARD: "17"
       CMAKE_GENERATOR: Ninja
       CMAKE_INSTALL_PREFIX: /usr
       CMAKE_UNITY_BUILD: ON
+      VCPKG_BINARY_SOURCES: 'clear;nuget,GitHub,readwrite'
+      VCPKG_DEFAULT_TRIPLET: x64-windows
     steps:
       - name: Disable Crash Dialogs
         run: |
@@ -110,15 +114,53 @@ jobs:
           path: ${{ steps.ccache-info.outputs.cache-dir }}
           key: cpp-ccache-windows-${{ inputs.arch }}-${{ hashFiles('cpp/**') }}
           restore-keys: cpp-ccache-windows-${{ inputs.arch }}-
+      - name: Checkout vcpkg
+        uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0
+        with:
+          fetch-depth: 0
+          path: vcpkg
+          repository: microsoft/vcpkg
+      - name: Bootstrap vcpkg
+        run: |
+          vcpkg\bootstrap-vcpkg.bat
+          $VCPKG_ROOT = $(Resolve-Path -LiteralPath "vcpkg").ToString()
+          Write-Output ${VCPKG_ROOT} | `
+            Out-File -FilePath ${Env:GITHUB_PATH} -Encoding utf8 -Append
+          Write-Output "VCPKG_ROOT=${VCPKG_ROOT}" | `
+            Out-File -FilePath ${Env:GITHUB_ENV} -Encoding utf8 -Append
+      - name: Setup NuGet credentials for vcpkg caching
+        shell: bash
+        run: |
+          $(vcpkg fetch nuget | tail -n 1) \
+            sources add \
+            -source "https://nuget.pkg.github.com/$GITHUB_REPOSITORY_OWNER/index.json" \
+            -storepasswordincleartext \
+            -name "GitHub" \
+            -username "$GITHUB_REPOSITORY_OWNER" \
+            -password "${{ secrets.GITHUB_TOKEN }}"
+          $(vcpkg fetch nuget | tail -n 1) \
+            setapikey "${{ secrets.GITHUB_TOKEN }}" \
+            -source "https://nuget.pkg.github.com/$GITHUB_REPOSITORY_OWNER/index.json"
       - name: Build
         shell: cmd
         run: |
+          set VCPKG_ROOT_KEEP=%VCPKG_ROOT%
           call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" ${{ inputs.arch }}
+          set VCPKG_ROOT=%VCPKG_ROOT_KEEP%
           bash -c "ci/scripts/cpp_build.sh $(pwd) $(pwd)/build"
+      - name: Register Flight SQL ODBC Driver
+        shell: cmd
+        run: |
+          call "cpp\src\arrow\flight\sql\odbc\tests\install_odbc.cmd" ${{ github.workspace }}\build\cpp\%ARROW_BUILD_TYPE%\arrow_flight_sql_odbc.dll
       - name: Test
         shell: cmd
         run: |
+          set VCPKG_ROOT_KEEP=%VCPKG_ROOT%
           call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" ${{ inputs.arch }}
           # For ORC
           set TZDIR=${{ steps.setup-msys2.outputs.msys2-location }}\usr\share\zoneinfo
+
+          # Convert VCPKG Windows path to MSYS path
+          for /f "usebackq delims=" %%I in (`bash -c "cygpath -u \"$VCPKG_ROOT_KEEP\""` ) do set VCPKG_ROOT=%%I
+
           bash -c "ci/scripts/cpp_test.sh $(pwd) $(pwd)/build"

ci/scripts/cpp_test.sh

@@ -144,8 +144,9 @@ if [ "${ARROW_USE_MESON:-OFF}" = "OFF" ] && \
       CMAKE_PREFIX_PATH+="/lib/cmake/"
       ;;
   esac
+  # Search vcpkg before <prefix>/lib/cmake.
   if [ -n "${VCPKG_ROOT}" ] && [ -n "${VCPKG_DEFAULT_TRIPLET}" ]; then
-    CMAKE_PREFIX_PATH+=";${VCPKG_ROOT}/installed/${VCPKG_DEFAULT_TRIPLET}"
+    CMAKE_PREFIX_PATH="${VCPKG_ROOT}/installed/${VCPKG_DEFAULT_TRIPLET};${CMAKE_PREFIX_PATH}"
   fi
   cmake \
     -S "${source_dir}/examples/minimal_build" \

cpp/cmake_modules/SetupCxxFlags.cmake

@@ -186,6 +186,7 @@ if(WIN32)
       #
       # ARROW-2986: Without /EHsc we get C4530 warning
       set(CXX_COMMON_FLAGS "/W3 /EHsc")
+      string(APPEND CMAKE_CXX_FLAGS " /EHsc")
     endif()
 
     # Disable C5105 (macro expansion producing 'defined' has undefined

cpp/src/arrow/compute/registry.cc

@@ -127,6 +127,8 @@ class FunctionRegistry::FunctionRegistryImpl {
 
   const Function* cast_function() { return cast_function_; }
 
+  void ClearFunctioRegistry() { name_to_function_.clear(); }
+
  private:
   // must not acquire mutex
   Status CanAddFunctionName(const std::string& name, bool allow_overwrite) {
@@ -277,6 +279,8 @@ int FunctionRegistry::num_functions() const { return impl_->num_functions(); }
 
 const Function* FunctionRegistry::cast_function() const { return impl_->cast_function(); }
 
+void FunctionRegistry::ClearFunctioRegistry() { impl_->ClearFunctioRegistry(); }
+
 namespace internal {
 
 static std::unique_ptr<FunctionRegistry> CreateBuiltInRegistry() {

cpp/src/arrow/compute/registry.h

@@ -112,6 +112,11 @@ class ARROW_EXPORT FunctionRegistry {
   /// Helpful for get cast function as needed.
   const Function* cast_function() const;
 
+  /// \brief Clear function registry
+  ///
+  /// Helpful to avoid segfault from race condition. Call this function before DLL unload.
+  void ClearFunctioRegistry();
+
  private:
   FunctionRegistry();
 

cpp/src/arrow/compute/row/CMakeLists.txt

@@ -22,7 +22,7 @@ arrow_install_all_headers("arrow/compute/row")
 
 if(ARROW_BUILD_BENCHMARKS AND ARROW_COMPUTE)
   add_arrow_benchmark(grouper_benchmark PREFIX "arrow-compute")
-  if(ARROW_BUILD_STATIC)
+  if(ARROW_TEST_LINKAGE STREQUAL "static")
     target_link_libraries(arrow-compute-grouper-benchmark PUBLIC arrow_compute_static)
   else()
     target_link_libraries(arrow-compute-grouper-benchmark PUBLIC arrow_compute_shared)

cpp/src/arrow/flight/sql/example/sqlite_sql_info.cc

@@ -20,6 +20,8 @@
 #include "arrow/flight/sql/types.h"
 #include "arrow/util/config.h"
 
+#include "arrow/flight/sql/sql_info_undef.h"
+
 namespace arrow {
 namespace flight {
 namespace sql {

cpp/src/arrow/flight/sql/odbc/odbc_impl/CMakeLists.txt

@@ -129,13 +129,9 @@ if(WIN32)
                          system_dsn.h)
 endif()
 
-target_link_libraries(arrow_odbc_spi_impl PUBLIC arrow_flight_sql_shared
-                                                 arrow_compute_shared Boost::locale)
-
-# Link libraries on MINGW64 and macOS
-if(MINGW OR APPLE)
-  target_link_libraries(arrow_odbc_spi_impl PUBLIC ${ODBCINST})
-endif()
+target_link_libraries(arrow_odbc_spi_impl
+                      PUBLIC arrow_flight_sql_shared arrow_compute_shared Boost::locale
+                             ${ODBCINST})
 
 set_target_properties(arrow_odbc_spi_impl
                       PROPERTIES ARCHIVE_OUTPUT_DIRECTORY

cpp/src/arrow/flight/sql/odbc/odbc_impl/blocking_queue.h

@@ -18,9 +18,9 @@
 #pragma once
 
 #include <atomic>
-#include <boost/optional.hpp>
 #include <condition_variable>
 #include <mutex>
+#include <optional>
 #include <thread>
 #include <vector>
 
@@ -43,7 +43,7 @@ class BlockingQueue {
   std::atomic<bool> closed_{false};
 
  public:
-  typedef std::function<boost::optional<T>(void)> Supplier;
+  typedef std::function<std::optional<T>(void)> Supplier;
 
   explicit BlockingQueue(size_t capacity) : capacity_(capacity), buffer_(capacity) {}
 
@@ -58,7 +58,7 @@ class BlockingQueue {
 
         // Only one thread at a time be notified and call supplier
         auto item = supplier();
-        if (!item) break;
+        if (!item.has_value()) break;
 
         Push(*item);
       }

cpp/src/arrow/flight/sql/odbc/odbc_impl/flight_sql_auth_method.cc

@@ -24,6 +24,7 @@
 #include "arrow/result.h"
 #include "arrow/status.h"
 
+#include <optional>
 #include <utility>
 
 namespace arrow::flight::sql::odbc {
@@ -63,7 +64,7 @@ class UserPasswordAuthMethod : public FlightSqlAuthMethod {
   void Authenticate(FlightSqlConnection& connection,
                     FlightCallOptions& call_options) override {
     FlightCallOptions auth_call_options;
-    const boost::optional<Connection::Attribute>& login_timeout =
+    const std::optional<Connection::Attribute>& login_timeout =
         connection.GetAttribute(Connection::LOGIN_TIMEOUT);
     if (login_timeout && boost::get<uint32_t>(*login_timeout) > 0) {
       // ODBC's LOGIN_TIMEOUT attribute and FlightCallOptions.timeout use