GH-49537: [C++][FlightRPC] Windows CI to Support ODBC DLL & MSI Signing by alinaliBQ · Pull Request #49603 · apache/arrow

alinaliBQ · 2026-03-26T22:54:18Z

Rationale for this change

What changes are included in this PR?

Implement Windows CIs to:
1. CI A - odbc-msvc-upload-dll Upload unsigned DLL
2. CI B - odbc-msvc-upload-msi Download signed DLL and upload unsigned MSI
Remove odbc-release CI that is replaced by the new CIs.
Use composite action to reuse code for building ODBC Windows.
The release manager can run CI A first, then download the unsigned DLL and upload signed DLL.
Run CI B to upload unsigned MSI. Then sign the MSI locally, and upload signed MSI to GitHub release. Example Command to trigger CI B:

gh workflow run cpp_extra.yml --ref apache-arrow-test-24.0.0-rc0 -f odbc_release_step=true

Example of 07-flightsqlodbc-upload.sh script (not tested):
We need to either 1) implement a way to get RUN_ID and then call gh run watch,
or 2) enter each command manually and wait for the CI to finish.

# download unsigned DLL
gh release download $tag --pattern arrow_flight_sql_odbc_unsigned.dll

# sign ODBC DLL and upload to GitHub release
jsign arrow_flight_sql_odbc_unsigned.dll ...
mv arrow_flight_sql_odbc_unsigned.dll arrow_flight_sql_odbc.dll
gh release upload $tag --clobber arrow_flight_sql_odbc.dll

# trigger CI B
gh workflow run cpp_extra.yml --ref $tag -f odbc_upload=msi

# download unsigned MSI
gh release download $tag --pattern Apache-Arrow-Flight-SQL-ODBC-*-win64.msi

# sign ODBC MSI and upload to GitHub release
jsign Apache-Arrow-Flight-SQL-ODBC-*-win64.msi ...
gh release upload $tag --clobber Apache-Arrow-Flight-SQL-ODBC-*-win64.msi

# remove ODBC DLLs from GitHub release
gh release delete-asset $tag arrow_flight_sql_odbc_unsigned.dll --yes
gh release delete-asset $tag arrow_flight_sql_odbc.dll --yes

Are these changes tested?

The uploading and signing process is tested in my repo
Workflows are verified in CI

Are there any user-facing changes?

N/A

GitHub Issue: [C++][FlightRPC][ODBC] Add CI steps to support Windows DLL and MSI signing #49537

* Add draft code for CI A and CI B Attempt workflow dispatch Only ODBC Windows original workflow should run. Later need to add `github.event_name != 'workflow_dispatch' ||` to all existing workflows after uncomment Use `GITHUB_REF_NAME` directly via push Add `workflow_dispatch` definitions Add `ODBC Windows Upload DLL` Use common ODBC Windows environment variables Use ODBC as composite action Create cpp_odbc.yml Initial draft temp disable test step Temp disable non-ODBC Windows workflows * Clean Up Code * Remove comments * Fix Installer path for MSI

alinaliBQ · 2026-03-26T23:01:50Z

.github/workflows/cpp_extra.yml

+      - name: Upload the artifacts to GitHub Release
+        env:
+          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+        run: |
+          gh release upload ${GITHUB_REF_NAME} \
+            --clobber \
+            arrow_flight_sql_odbc_unsigned.dll


Since both DLL and MSI need to be signed and unsigned DLL is harder to catch, uploading as arrow_flight_sql_odbc_unsigned.dll to make it clear on GitHub release if the DLL is unsigned.

alinaliBQ · 2026-03-27T17:35:07Z

@amoeba PR is ready for review. We have an issue for flight-sql-tests ODBC Windows failure: #49465

alinaliBQ · 2026-03-27T22:34:19Z

I did an empty commit (a522393) and got this error:

Error: D:\a\arrow\arrow\./.github/actions/odbc-windows\action.yml (Line: 71, Col: 12): Unrecognized named-value: 'secrets'. Located at position 1 within expression: secrets.GITHUB_TOKEN
Error: GitHub.DistributedTask.ObjectTemplating.TemplateValidationException: The template is not valid. D:\a\arrow\arrow\./.github/actions/odbc-windows\action.yml (Line: 71, Col: 12): Unrecognized named-value: 'secrets'. Located at position 1 within expression: secrets.GITHUB_TOKEN
   at GitHub.DistributedTask.ObjectTemplating.TemplateValidationErrors.Check()
   at GitHub.Runner.Worker.ActionManifestManagerLegacy.ConvertRuns(IExecutionContext executionContext, TemplateContext templateContext, TemplateToken inputsToken, String fileRelativePath, MappingToken outputs)
   at GitHub.Runner.Worker.ActionManifestManagerLegacy.Load(IExecutionContext executionContext, String manifestFile)
Error: Failed to load D:\a\arrow\arrow\./.github/actions/odbc-windows\action.yml

The same implementation worked yesterday (see https://github.com/apache/arrow/actions/runs/23622018872/job/68803175375). Seems that GitHub might have updated runner, I will look into this.

alinaliBQ · 2026-03-27T22:48:57Z

Error: Failed to load D:\a\arrow\arrow./.github/actions/odbc-windows\action.yml

This issue should be resolved now by commit 95bc75b

alinaliBQ · 2026-03-31T18:40:18Z

@kou @raulcd This PR is ready for review. Please have a look if you have a chance, thank you

amoeba · 2026-03-31T23:09:13Z

.github/workflows/cpp_extra.yml

+          gh release download $env:GITHUB_REF_NAME `
+          --pattern arrow_flight_sql_odbc.dll `
+          --clobber
+      - name: Build ODBC Windows


Did you have any luck figuring out how to extract just the minimum set of files cpack needs to build the MSI? I see this job is rebuilding everything from scratch so we're building the ODBC driver twice. If it's not possible that's fine, and if we want to work on removing the extra full-build in a follow-up that's fine.

I am assuming we are referring to the comment here (#49404 (comment)). Yup the approach builds everything twice by design. Unfortunately getting specific files for a CPack build isn't doable as CPack only packages from a single build directory.

This workflow looks for cache, if a cache is hit, the build will be faster. I think this is the cleanest approach. And thanks for your understanding.

amoeba · 2026-03-31T23:10:37Z

.github/workflows/cpp_extra.yml

          remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}
          remote_host_key: ${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}

-  odbc-release:


Could we automatically build and upload the unsigned DLL when we tag a release? I think it could speed things up for the release managers. It looks like the way you have it here, the release manager has to manually run odbc-msvc-upload-dll. Is that just because of the renaming step?

We do it on the Linux packaging ones. We trigger the job on tag creation:

arrow/.github/workflows/package_linux.yml

Lines 38 to 39 in 9273467

tags:

- "apache-arrow-*-rc*"

We wait until the Release is created on GitHub so we can use the tar.gz from the release:

arrow/.github/workflows/package_linux.yml

Lines 228 to 235 in 9273467

- name: Wait for creating GitHub Release

if: github.ref_type == 'tag'

env:

GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

run: |

dev/release/utils-watch-gh-workflow.sh \

${GITHUB_REF_NAME} \

release_candidate.yml

And we only upload to the release:

arrow/.github/workflows/package_linux.yml

Lines 288 to 295 in 9273467

- name: Upload the artifacts to GitHub Release

if: github.ref_type == 'tag'

env:

GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

run: |

gh release upload ${GITHUB_REF_NAME} \

--clobber \

${{ matrix.id }}.tar.gz*

Some of those steps are only done in the case of tag trigger (so only on RCs).

I think we should use a similar pattern.

👍 thanks @raulcd

Yes I have changed odbc-msvc-upload-dll to be automatically triggered by the release tag. odbc-msvc-upload-dll will no longer be trigged via workflow dispatch.

From the discussion at #49404, I originally thought we wanted the odbc-msvc-upload-dll workflow to be triggered manually as well.

amoeba · 2026-03-31T23:16:08Z

.github/workflows/cpp_extra.yml

+        uses: ./.github/actions/odbc-windows
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+      - name: Name Unsigned ODBC DLL


Suggested change

- name: Name Unsigned ODBC DLL

- name: Rename Unsigned ODBC DLL

amoeba · 2026-03-31T23:56:14Z

.github/workflows/cpp_extra.yml

+
+  odbc-msvc-upload-msi:
+    needs: check-labels
+    name: ODBC Windows Upload Unsigned MSI


Suggested change

name: ODBC Windows Upload Unsigned MSI

name: ODBC Windows Build & Upload Unsigned MSI

amoeba · 2026-03-31T23:59:11Z

.github/workflows/cpp_extra.yml

        0 0 * * *
+  workflow_dispatch:
+    inputs:
+      odbc_upload:


Just a suggestion, this seems clearer as to the intent:

Suggested change

odbc_upload:

odbc_release_step:

yup sure, changed the name to odbc_release_step. I have changed the input to be boolean as well since odbc_release_step will only be used for triggering the MSI step.

amoeba · 2026-03-31T23:59:39Z

.github/workflows/cpp_extra.yml

+  workflow_dispatch:
+    inputs:
+      odbc_upload:
+        description: 'ODBC Component Upload'


Suggested change

description: 'ODBC Component Upload'

description: 'Which ODBC release step to run'

Changed to ODBC MSI release step

amoeba · 2026-04-01T00:00:12Z

.github/workflows/cpp_extra.yml

+    needs: check-labels
+    name: ODBC Windows Upload Unsigned DLL
+    runs-on: windows-2022
+    if: inputs.odbc_upload == 'dll'


Suggested change

if: inputs.odbc_upload == 'dll'

if: inputs.odbc_release_step == 'dll'

amoeba · 2026-04-01T00:00:22Z

.github/workflows/cpp_extra.yml

+    needs: check-labels
+    name: ODBC Windows Upload Unsigned MSI
+    runs-on: windows-2022
+    if: inputs.odbc_upload == 'msi'


Suggested change

if: inputs.odbc_upload == 'msi'

if: inputs.odbc_release_step == 'msi'

Change `odbc-msvc-upload-dll` to be triggered via rc tag and can be invoked manually

alinaliBQ

Addressed comments. Please have another look, thanks!

alinaliBQ · 2026-04-02T21:25:36Z

.github/workflows/cpp_extra.yml

+          gh release download $env:GITHUB_REF_NAME `
+          --pattern arrow_flight_sql_odbc.dll `
+          --clobber
+      - name: Build ODBC Windows


I am assuming we are referring to the comment here (#49404 (comment)). Yup the approach builds everything twice by design. Unfortunately getting specific files for a CPack build isn't doable as CPack only packages from a single build directory.

This workflow looks for cache, if a cache is hit, the build will be faster. I think this is the cleanest approach. And thanks for your understanding.

alinaliBQ · 2026-04-02T21:28:58Z

.github/workflows/cpp_extra.yml

          remote_key: ${{ secrets.NIGHTLIES_RSYNC_KEY }}
          remote_host_key: ${{ secrets.NIGHTLIES_RSYNC_HOST_KEY }}

-  odbc-release:


Yes I have changed odbc-msvc-upload-dll to be automatically triggered by the release tag. odbc-msvc-upload-dll will no longer be trigged via workflow dispatch.

From the discussion at #49404, I originally thought we wanted the odbc-msvc-upload-dll workflow to be triggered manually as well.

alinaliBQ · 2026-04-02T22:01:20Z

.github/workflows/cpp_extra.yml

+        uses: ./.github/actions/odbc-windows
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+      - name: Name Unsigned ODBC DLL


alinaliBQ · 2026-04-02T22:01:39Z

.github/workflows/cpp_extra.yml

+
+  odbc-msvc-upload-msi:
+    needs: check-labels
+    name: ODBC Windows Upload Unsigned MSI


alinaliBQ · 2026-04-02T22:03:12Z

.github/workflows/cpp_extra.yml

        0 0 * * *
+  workflow_dispatch:
+    inputs:
+      odbc_upload:


yup sure, changed the name to odbc_release_step. I have changed the input to be boolean as well since odbc_release_step will only be used for triggering the MSI step.

alinaliBQ · 2026-04-02T22:03:50Z

.github/workflows/cpp_extra.yml

+  workflow_dispatch:
+    inputs:
+      odbc_upload:
+        description: 'ODBC Component Upload'


Changed to ODBC MSI release step

alinaliBQ · 2026-04-02T22:04:07Z

.github/workflows/cpp_extra.yml

+    needs: check-labels
+    name: ODBC Windows Upload Unsigned MSI
+    runs-on: windows-2022
+    if: inputs.odbc_upload == 'msi'


alinaliBQ · 2026-04-02T22:06:56Z

.github/workflows/cpp_extra.yml

+    needs: check-labels
+    name: ODBC Windows Upload Unsigned DLL
+    runs-on: windows-2022
+    if: inputs.odbc_upload == 'dll'


github-actions bot added CI: Extra: C++ Run extra C++ CI awaiting review Awaiting review labels Mar 26, 2026

alinaliBQ mentioned this pull request Mar 26, 2026

[C++][FlightRPC][ODBC] Add CI steps to support Windows DLL and MSI signing #49537

Open

alinaliBQ requested a review from amoeba March 26, 2026 22:56

alinaliBQ commented Mar 26, 2026

View reviewed changes

github-actions bot added awaiting committer review Awaiting committer review and removed awaiting review Awaiting review labels Mar 26, 2026

alinaliBQ marked this pull request as ready for review March 27, 2026 17:35

alinaliBQ requested review from assignUser, jonkeane, kou and raulcd as code owners March 27, 2026 17:35

Trigger CI

a522393

Fix issue with secrets.GITHUB_TOKEN

95bc75b

amoeba reviewed Apr 1, 2026

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Apr 1, 2026

alinaliBQ added 2 commits April 2, 2026 15:00

Change odbc_msi_upload to boolean input

9ccebbc

Change `odbc-msvc-upload-dll` to be triggered via rc tag and can be invoked manually

Work on Bryce's code review comments

20fefd5

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Apr 2, 2026

alinaliBQ commented Apr 2, 2026

View reviewed changes

	- name: Wait for creating GitHub Release
	if: github.ref_type == 'tag'
	env:
	GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	run: \|
	dev/release/utils-watch-gh-workflow.sh \
	${GITHUB_REF_NAME} \
	release_candidate.yml

	- name: Upload the artifacts to GitHub Release
	if: github.ref_type == 'tag'
	env:
	GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
	run: \|
	gh release upload ${GITHUB_REF_NAME} \
	--clobber \
	${{ matrix.id }}.tar.gz*

	- name: Name Unsigned ODBC DLL
	- name: Rename Unsigned ODBC DLL

	name: ODBC Windows Upload Unsigned MSI
	name: ODBC Windows Build & Upload Unsigned MSI

	description: 'ODBC Component Upload'
	description: 'Which ODBC release step to run'

	if: inputs.odbc_upload == 'dll'
	if: inputs.odbc_release_step == 'dll'

	if: inputs.odbc_upload == 'msi'
	if: inputs.odbc_release_step == 'msi'

Conversation

alinaliBQ commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alinaliBQ commented Mar 27, 2026

Uh oh!

alinaliBQ commented Mar 27, 2026

Uh oh!

alinaliBQ commented Mar 27, 2026

Uh oh!

alinaliBQ commented Mar 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alinaliBQ left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

alinaliBQ commented Mar 26, 2026 •

edited

Loading