Skip to content

Commit a16e033

Browse files
Paulchen-PantherLord-Grey
Paulchen-Panther
and
Lord-Grey
authored
Build Hyperion on GitHub with pre-built dependencies (#1851)
* Preparation for compilation with pre-build dependencies * c&p typo * cleanup * typo * test run * superbuild libusb on windows * macos changes * typo * set libftdi branch to master * libftdi tests * edits * set libusb/libftdi prefix fix libftdi patch * set CMAKE_STATIC_LIBRARY_PREFIX to lib * update * libftdi changes * test * test 2 * test3 * test 4 * remove libftdi folder * test 5 * test 6 * test 7 * reset * Replace static array with dynamic size allocation * Go for default inline * Correct inline update * Replace FTDI_CHECK_RESULT macro by function * Enable DEFAULT_DEV_USB_HID on all platforms by default * parse prebuiltdeps json with jq * typo * pre built run test (linux) * typo * test run * typo * verbose output * verbose echo * typo * redirect * verbose * test * map /tmp/deps on docker * chmod pre-built binaries * link libudev under linux * link libudev on linux * debug * debug * debug * debug * debug * test * debug * pre-built test run * update * exclude windows * append macos pre-built cmake args * add windows * update windows path * Extraction of dxsdk and turbojpeg to speed up the step * use cmd shell * correct installer paths * use env:GITHUB_ENV on windows * another test * test * remove ninja parallel parameter * fix directx9 build * test run * test * change to hyperion-project * typo * typo 2 * use pre-built deps in the codeql cpp analysis --------- Co-authored-by: Lord-Grey <[email protected]>
1 parent 37c141a commit a16e033

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

59 files changed

+1567
-3815
lines changed

‎.github/actions/download-pre-built-deps/action.yml

+64
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
name: Download Pre-Built Dependencies
2+
description: Download Pre-Built Dependencies for Hyperion to speed-up the CI build
3+
inputs:
4+
os:
5+
description: Operating System
6+
required: true
7+
default: 'debian_buster'
8+
architecture:
9+
description: Linux architecture
10+
required: false
11+
default: 'amd64'
12+
qt_version:
13+
description: Qt version
14+
required: false
15+
default: '6'
16+
build_type:
17+
type: string
18+
description: Build type
19+
default: ''
20+
outputs:
21+
cmakeArgs:
22+
description: "CMake arguments"
23+
value: ${{ steps.download-pre-built-deps.outputs.cmake-args }}
24+
runs:
25+
using: composite
26+
steps:
27+
- name: Download latest specs
28+
uses: robinraju/release-downloader@v1
29+
with:
30+
repository: 'hyperion-project/hyperion-deps'
31+
latest: true
32+
fileName: 'buildspec.json'
33+
34+
- name: Download Deps
35+
id: download-pre-built-deps
36+
shell: bash
37+
run: |
38+
preBuiltDeps='buildspec.json'
39+
baseURL=$(jq -r '.baseUrl' $preBuiltDeps)
40+
version=$(jq -r '.version' $preBuiltDeps)
41+
downloadHash=$(jq -r '.hashes .${{ inputs.os }} .${{ inputs.architecture }} .qt${{ inputs.qt_version }} .${{ inputs.build_type }}' $preBuiltDeps)
42+
downloadFilename=${{ inputs.os }}-${{ inputs.architecture }}-qt${{ inputs.qt_version }}-${{ inputs.build_type }}-$version.tar.gz
43+
echo 💾 Download "$downloadFilename"...
44+
curl -OL "$baseURL"/"$version"/"$downloadFilename"
45+
if [[ ${{ inputs.os }} == 'windows' ]]; then
46+
computedHash=$(powershell -Command "(Get-FileHash -Path \"$downloadFilename\" -Algorithm SHA256).Hash.ToLower()")
47+
destDir=C:/tmp
48+
else
49+
computedHash=$(sha256sum "$downloadFilename" | awk '{print $1}')
50+
destDir=/tmp/deps
51+
fi
52+
echo 👀 Check Hash...
53+
if [[ "$computedHash" == "$downloadHash" ]]; then
54+
mkdir "$destDir"
55+
tar -xzf "$downloadFilename" -C "$destDir"
56+
if [[ ${{ inputs.os }} != 'windows' ]]; then
57+
chmod +x "$destDir"/bin/*
58+
fi
59+
echo ✅ Hash check passed.
60+
echo "cmake-args="-DUSE_PRE_BUILT_DEPS=ON -DPRE_BUILT_DEPS_DIR=$destDir"" >> $GITHUB_OUTPUT
61+
else
62+
echo ❌ Hash check failed.
63+
echo "cmake-args=" >> $GITHUB_OUTPUT
64+
fi

‎.github/workflows/codeql.yml

+30-18
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ name: 📊 CodeQL
22
run-name: |
33
${{ github.event_name == 'schedule' && '⏰ Scheduled CodeQL run' || '' }}
44
${{ github.event_name == 'push' && format('📊 Pushed CodeQL run - {0}', github.event.head_commit.message) || '' }}
5-
${{ github.event_name == 'pull_request' && format('📊 CodeQL run for PR {0} - {1}', github.event.pull_request.number, github.event.pull_request.title) || github.event.head_commit.message }}
5+
${{ github.event_name == 'pull_request' && format('📊 CodeQL run for PR {0} - {1}', github.event.pull_request.number, github.event.pull_request.title) || '' }}
66
77
on:
88
push:
@@ -26,33 +26,49 @@ jobs:
2626
matrix:
2727
language: [ python, javascript, cpp ]
2828

29+
container:
30+
image: ghcr.io/hyperion-project/debian:bullseye-qt6
31+
2932
steps:
3033
- name: ⬇ Checkout
3134
uses: actions/checkout@v4
3235
with:
3336
submodules: recursive
3437

35-
- name: 📥 Install Packages (cpp)
36-
if: ${{ matrix.language == 'cpp' }}
37-
run: |
38-
sudo apt-get update
39-
sudo apt-get install --yes git build-essential qtbase5-dev libqt5serialport5-dev libqt5websockets5-dev libqt5sql5-sqlite libqt5svg5-dev libqt5x11extras5-dev libusb-1.0-0-dev python3-dev libcec-dev libxcb-image0-dev libxcb-util0-dev libxcb-shm0-dev libxcb-render0-dev libxcb-randr0-dev libxrandr-dev libxrender-dev libavahi-core-dev libavahi-compat-libdnssd-dev libasound2-dev libturbojpeg0-dev libjpeg-dev libssl-dev libftdi1-dev
40-
41-
- name: Temporarily downgrade CMake to 3.28.3 # Please remove if GitHub has updated Cmake (greater than 3.30.0)
42-
uses: jwlawson/actions-setup-cmake@v2
43-
with:
44-
cmake-version: '3.28.3'
45-
4638
- name: 🔁 Initialize CodeQL
4739
uses: github/codeql-action/init@v3
4840
with:
4941
languages: ${{ matrix.language }}
5042
queries: +security-and-quality
5143
config-file: ./.github/config/codeql.yml
52-
53-
- name: 👷 Autobuild
44+
45+
- name: 👷 Build (Python & JavaScript)
46+
if: ${{ matrix.language != 'cpp' }}
5447
uses: github/codeql-action/autobuild@v3
5548

49+
- name: 📥 Install jq (C++)
50+
if: ${{ matrix.language == 'cpp' }}
51+
shell: bash
52+
run: apt-get update && apt-get install -y jq
53+
54+
- name: 💾 Download Pre-Build Dependencies (C++)
55+
if: ${{ matrix.language == 'cpp' }}
56+
id: dependencies
57+
uses: ./.github/actions/download-pre-built-deps
58+
with:
59+
os: 'debian_bullseye'
60+
architecture: 'amd64'
61+
qt_version: '6'
62+
build_type: ${{ github.event_name == 'pull_request' && 'debug' || 'release' }}
63+
64+
- name: 👷 Build (C++)
65+
if: ${{ matrix.language == 'cpp' }}
66+
run: |
67+
cmake --preset linux-${{ env.BUILD_TYPE }} ${{ steps.dependencies.outputs.cmakeArgs }}
68+
cmake --build --preset linux-${{ env.BUILD_TYPE }}
69+
env:
70+
BUILD_TYPE: ${{ github.event_name == 'pull_request' && 'debug' || 'release' }}
71+
5672
- name: 🏃 Perform CodeQL Analysis
5773
uses: github/codeql-action/analyze@v3
5874
with:
@@ -66,9 +82,6 @@ jobs:
6682
patterns: |
6783
-**/dependencies/**
6884
-**/moc_*.cpp
69-
-**/libsrc/flatbufserver/hyperion_request_generated.h
70-
-**/libsrc/protoserver/message.pb.cc
71-
-**/libsrc/protoserver/message.pb.h
7285
input: sarif-results/${{ matrix.language }}.sarif
7386
output: sarif-results/${{ matrix.language }}.sarif
7487

@@ -83,4 +96,3 @@ jobs:
8396
name: ${{ matrix.language }}.sarif
8497
path: sarif-results
8598
retention-days: 1
86-

‎.github/workflows/qt5_6.yml

+46-13
Original file line numberDiff line numberDiff line change
@@ -66,24 +66,35 @@ jobs:
6666
shell: bash
6767
run: tr -d '\n' < .version > temp && mv temp .version && echo -n "+PR${{ inputs.pull_request_number }}" >> .version
6868

69-
- name: 👷 Build
69+
- name: 💾 Download Pre-Build Dependencies
70+
id: dependencies
71+
uses: ./.github/actions/download-pre-built-deps
72+
with:
73+
os: ${{ matrix.os.distribution }}_${{ matrix.os.codename }}
74+
architecture: ${{ matrix.os.architecture[0] }}
75+
qt_version: ${{ inputs.qt_version }}
76+
build_type: ${{ inputs.event_name == 'pull_request' && 'debug' || 'release' }}
77+
78+
- name: 👷 Build ${{ env.HINT }}
7079
shell: bash
7180
run: |
7281
mkdir deploy
7382
docker run --rm --platform=${{ matrix.os.architecture[1] }} ${{ env.ENTRYPOINT }} \
7483
-v "${{ github.workspace }}/deploy:/deploy" \
7584
-v "${{ github.workspace }}:/source:rw" \
85+
-v "/tmp/deps:/tmp/deps:rw" \
7686
-w "/source" \
7787
ghcr.io/hyperion-project/${{ env.DOCKER_IMAGE }}:${{ env.DOCKER_TAG }} \
78-
/bin/bash -c "cmake --preset linux-${{ env.BUILD_TYPE }} -DPLATFORM=${{ matrix.os.platform }} ${{ env.CPACK_SYSTEM_PROCESSOR }} &&
79-
cmake --build --preset linux-${{ env.BUILD_TYPE }} --target package --parallel $(nproc) &&
88+
/bin/bash -c "cmake --preset linux-${{ env.BUILD_TYPE }} ${{ steps.dependencies.outputs.cmakeArgs }} -DPLATFORM=${{ matrix.os.platform }} ${{ env.CPACK_SYSTEM_PROCESSOR }} &&
89+
cmake --build --preset linux-${{ env.BUILD_TYPE }} --target package &&
8090
cp /source/build/Hyperion-* /deploy/ 2>/dev/null"
8191
env:
8292
DOCKER_IMAGE: ${{ matrix.os.distribution }}
8393
DOCKER_TAG: ${{ matrix.os.codename }}${{ inputs.qt_version == '6' && '-qt6' || '' }}
8494
BUILD_TYPE: ${{ inputs.event_name == 'pull_request' && 'debug' || 'release' }}
8595
ENTRYPOINT: ${{ matrix.os.architecture[0] != 'amd64' && '--entrypoint /usr/bin/env' || '' }}
8696
CPACK_SYSTEM_PROCESSOR: ${{ matrix.os.architecture[0] != 'amd64' && format('-DCPACK_SYSTEM_PROCESSOR={0}', matrix.target_lookup[format('{0}', matrix.os.architecture[0])]) || '' }}
97+
HINT: ${{ steps.dependencies.outputs.cmakeArgs != '' && '(with pre-built dependencies)' || '(full build)' }}
8798

8899
- name: 📦 Upload
89100
if: ${{ inputs.publish || inputs.event_name == 'pull_request' }}
@@ -120,23 +131,33 @@ jobs:
120131
- name: 📥 Install dependencies
121132
uses: tecolicom/actions-use-homebrew-tools@v1
122133
with:
123-
tools: qt@${{ inputs.qt_version }} vulkan-headers ninja libftdi
134+
tools: qt@${{ inputs.qt_version }} vulkan-headers coreutils
124135
key: ${{ runner.os }}-${{ matrix.architecture }}-homebrew-packages-${{ inputs.qt_version }}
125136

126-
- name: 👷 Build
137+
- name: 💾 Download Pre-Build Dependencies
138+
id: dependencies
139+
uses: ./.github/actions/download-pre-built-deps
140+
with:
141+
os: 'macos'
142+
architecture: ${{ matrix.architecture }}
143+
qt_version: ${{ inputs.qt_version }}
144+
build_type: ${{ inputs.event_name == 'pull_request' && 'debug' || 'release' }}
145+
146+
- name: 👷 Build ${{ env.HINT }}
127147
shell: bash
128148
run: |
129149
# Remove once issue with hombrew cache action is fixed
130150
brew reinstall --formula cmake
131151
# Build
132-
cmake --preset macos-${{ env.BUILD_TYPE }}
133-
cmake --build --preset macos-${{ env.BUILD_TYPE }} --parallel $(sysctl -n hw.ncpu)
152+
cmake --preset macos-${{ env.BUILD_TYPE }} ${{ steps.dependencies.outputs.cmakeArgs }}
153+
cmake --build --preset macos-${{ env.BUILD_TYPE }}
134154
# CPack workaround for macOS 13
135155
echo killing...; sudo pkill -9 XProtect >/dev/null || true;
136156
echo waiting...; while pgrep XProtect; do sleep 3; done;
137157
cd build; cpack
138158
env:
139159
BUILD_TYPE: ${{ inputs.event_name == 'pull_request' && 'debug' || 'release' }}
160+
HINT: ${{ steps.dependencies.outputs.cmakeArgs != '' && '(with pre-built dependencies)' || '(full build)' }}
140161

141162
- name: 📦 Upload
142163
if: ${{ inputs.publish || inputs.event_name == 'pull_request' }}
@@ -181,10 +202,12 @@ jobs:
181202
destination: .\installer\libjpeg-turbo.exe
182203
cache-key: libjpeg-turbo
183204

184-
- name: 📥 Install DirectX SDK & libjpeg-turbo
205+
- name: 📥 Extract DirectX SDK & libjpeg-turbo
185206
run: |
186-
cmd.exe /c start /wait .\installer\libjpeg-turbo.exe /S
187-
cmd.exe /c start /wait .\installer\dxsdk-jun10.exe /U /F
207+
7z x -aoa .\installer\dxsdk-jun10.exe DXSDK/Include DXSDK/Lib -oC:\
208+
echo "DXSDK_DIR=C:\DXSDK" | Out-File -FilePath $env:GITHUB_ENV -Append
209+
7z x -aoa .\installer\libjpeg-turbo.exe bin include lib -oC:\turbojpeg
210+
echo "TURBOJPEG_DIR=C:\turbojpeg" | Out-File -FilePath $env:GITHUB_ENV -Append
188211
189212
- name: 📥 Install Vulkan SDK
190213
if: ${{ inputs.qt_version == '6' }}
@@ -206,13 +229,23 @@ jobs:
206229
- name: 🛠️ Setup MSVC
207230
uses: ilammy/msvc-dev-cmd@v1
208231

209-
- name: 👷 Build
232+
- name: 💾 Download Pre-Build Dependencies
233+
id: dependencies
234+
uses: ./.github/actions/download-pre-built-deps
235+
with:
236+
os: 'windows'
237+
architecture: 'x64'
238+
qt_version: ${{ inputs.qt_version }}
239+
build_type: ${{ inputs.event_name == 'pull_request' && 'debug' || 'release' }}
240+
241+
- name: 👷 Build ${{ env.HINT }}
210242
shell: cmd
211243
run: |
212-
cmake --preset windows-${{ env.BUILD_TYPE }}
213-
cmake --build --preset windows-${{ env.BUILD_TYPE }} --target package --parallel %NUMBER_OF_PROCESSORS%
244+
cmake --preset windows-${{ env.BUILD_TYPE }} ${{ steps.dependencies.outputs.cmakeArgs }}
245+
cmake --build --preset windows-${{ env.BUILD_TYPE }} --target package
214246
env:
215247
BUILD_TYPE: ${{ inputs.event_name == 'pull_request' && 'debug' || 'release' }}
248+
HINT: ${{ steps.dependencies.outputs.cmakeArgs != '' && '(with pre-built dependencies)' || '(full build)' }}
216249

217250
- name: 📦 Upload
218251
if: ${{ inputs.publish || inputs.event_name == 'pull_request' }}

‎.gitignore

-3
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,6 @@ deploy/*
4343
# ccache/buildcache
4444
.*cache/
4545

46-
# release-deps/debug-deps
47-
*-deps/
48-
4946
# User defined CMake preset file.
5047
CMakeUserPresets.json
5148

‎.gitmodules

+7-1
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,10 @@
1414
url = https://github.com/nitroshare/qmdnsengine.git
1515
[submodule "dependencies/external/mbedtls"]
1616
path = dependencies/external/mbedtls
17-
url = ../../Mbed-TLS/mbedtls.git
17+
url = https://github.com/Mbed-TLS/mbedtls.git
18+
[submodule "dependencies/external/hidapi"]
19+
path = dependencies/external/hidapi
20+
url = https://github.com/libusb/hidapi
21+
[submodule "dependencies/external/libusb-cmake"]
22+
path = dependencies/external/libusb-cmake
23+
url = https://github.com/libusb/libusb-cmake

0 commit comments

Comments
 (0)