[Python][Packaging] Revisit Windows wheel Docker setup

[pitrou]

Describe the enhancement requested

Our Windows wheel build setup currently uses Docker images based on https://hub.docker.com/r/abrarov/msvc-2019

This image is old and contains unwanted software.

We should instead try to build our own images from scratch.
It's possible to install the VS Build Tools in a container image:
https://learn.microsoft.com/en-us/visualstudio/install/build-tools-container?view=vs-2022
https://learn.microsoft.com/en-us/visualstudio/install/workload-component-id-vs-build-tools?view=vs-2022

Initial research suggests this command:

vs_buildtools.exe --quiet --wait --norestart \
    --add Microsoft.VisualStudio.Component.VC.CoreBuildTools \
    --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 \
    --installPath c:\tt

Component(s)

Continuous Integration, Packaging, Python

[pitrou]

cc @raulcd @kou

[amoeba]

I'm looking at this today and have made some progress. My image so far is only 7.57GB whereas the existing wheel image is 26.8GB which is a nice savings. So far I have,

# escape=`

# build using
# docker build -t buildtools -m 2GB --file buildtools.dockerfile .

# Use the latest Windows Server Core 2022 image.
FROM mcr.microsoft.com/windows/servercore:ltsc2022

# Restore the default Windows shell for correct batch processing.
SHELL ["cmd", "/S", "/C"]

RUN `
  # Download the Build Tools bootstrapper.
  curl -SL --output vs_buildtools.exe https://aka.ms/vs/17/release/vs_buildtools.exe `
  `
  # Install Build Tools with the Microsoft.VisualStudio.Workload.AzureBuildTools workload, excluding workloads and components with known issues.
  && (start /w vs_buildtools.exe --quiet --wait --norestart --nocache `
  --installPath "%ProgramFiles(x86)%\Microsoft Visual Studio\2022\BuildTools" `
  --add Microsoft.VisualStudio.Component.VC.CoreBuildTools `
  --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 `
  || IF "%ERRORLEVEL%"=="3010" EXIT 0) `
  `
  # Cleanup
  && del /q vs_buildtools.exe

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

# Install choco CLI
# from https://chocolatey.org/install#completely-offline-install
RUN `
  Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))

# Install CMake and Ninja
# TODO: ARG cmake=3.31.2
RUN choco install --no-progress -r -y cmake --version=3.31.2 --installargs 'ADD_CMAKE_TO_PATH=System'
RUN choco install --no-progress -r -y gzip wget ninja

# Define the entry point for the docker container.
# This entry point starts the developer command prompt and launches the PowerShell shell.
ENTRYPOINT ["C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\Common7\\Tools\\VsDevCmd.bat", "&&", "powershell.exe", "-NoLogo", "-ExecutionPolicy", "Bypass"]

This currently starts with a warning "The system cannot find the file specified." which I think means more components are needed so I'm looking at that now.

[amoeba]

Installing a Win10/11 SDK fixes the warning at startup, --add Microsoft.VisualStudio.Component.Windows11SDK.

[pitrou]

Thanks a lot for working on this @amoeba ! Just a question: is there a reason to pin this issue?

[amoeba]

Hrm, no and I'm not sure how I did that. Maybe there's a shortcut? Either way, it's unpinned now.

[amoeba]

Issue resolved by pull request 45442
#45442