Skip to content

Commit 26fefba

Browse files
filipecosta90fcostaoliveira
filipecosta90
and
fcostaoliveira
authored
Add APT repository automation system for Ubuntu package distribution (#1)
* Add APT repository automation system - Add GitHub Actions workflow for automated .deb package creation and APT repository management - Create build script using FPM to generate architecture-independent .deb packages - Add comprehensive documentation for APT repository setup and release process - Update README.md with APT installation instructions for Ubuntu users - Add setup script for initializing gh-pages branch structure - Support Ubuntu 20.04, 22.04, and 24.04 across amd64 and arm64 architectures - Automated testing across multiple platform combinations - Repository hosted at https://redis-performance.github.io/perf_data_converter/ * updated the CI workflow to use latest checkout and trigger on any branch. * Fix build-deb.sh to properly use Bazel and package the correct binary - Build perf_to_profile binary using Bazel instead of looking for Python file - Package the actual C++ binary from bazel-bin/src/perf_to_profile - Add proper dependencies (libc6, libelf1, libcap2) - Include metadata (description, URL, license, maintainer) - Use temporary directory structure for proper packaging * Add .deb package build testing to CI workflow - Add test-deb-build job that runs on all PRs and pushes - Test .deb package creation using build-deb.sh script - Validate package contents and metadata - Test actual package installation and binary functionality - Upload test packages as artifacts for inspection - Ensures .deb packaging works before releases * removed spurious lines * Fix FPM installation permissions in build script - Add sudo to gem install command for proper permissions - Add FPM_SKIP_INSTALL environment variable to skip installation when FPM is already available - Update CI workflows to use FPM_SKIP_INSTALL=1 since FPM is pre-installed - Prevents permission errors when running gem install without sudo - Improves efficiency by skipping redundant FPM installation in CI --------- Co-authored-by: fcostaoliveira <[email protected]>
1 parent d75584e commit 26fefba

File tree

7 files changed

+729
-6
lines changed

7 files changed

+729
-6
lines changed

‎.github/workflows/apt-release.yml‎

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
name: APT Repository Release
2+
3+
on:
4+
release:
5+
types: [published]
6+
# Allow manual triggering for testing
7+
workflow_dispatch:
8+
inputs:
9+
release_tag:
10+
description: 'Release tag (e.g., v1.2.3)'
11+
required: true
12+
type: string
13+
14+
env:
15+
DEBIAN_FRONTEND: noninteractive
16+
17+
jobs:
18+
build:
19+
runs-on: ubuntu-latest
20+
steps:
21+
- name: Checkout code
22+
uses: actions/checkout@v4
23+
with:
24+
submodules: recursive
25+
26+
- name: Set up dependencies
27+
run: |
28+
sudo apt-get -qq update
29+
sudo apt-get install -y libelf-dev libssl-dev libcap-dev ruby-dev build-essential
30+
31+
- name: Install FPM
32+
run: |
33+
sudo gem install fpm
34+
35+
- name: Set release tag
36+
id: release_tag
37+
run: |
38+
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
39+
echo "tag=${{ github.event.inputs.release_tag }}" >> $GITHUB_OUTPUT
40+
else
41+
echo "tag=${{ github.ref_name }}" >> $GITHUB_OUTPUT
42+
fi
43+
44+
- name: Build .deb package
45+
run: |
46+
# FPM is already installed in CI, so skip gem install
47+
FPM_SKIP_INSTALL=1 ./scripts/build-deb.sh ${{ steps.release_tag.outputs.tag }}
48+
49+
- name: Upload .deb artifact
50+
uses: actions/upload-artifact@v4
51+
with:
52+
name: deb-package
53+
path: artifacts/*.deb
54+
retention-days: 30
55+
56+
publish:
57+
needs: build
58+
runs-on: ubuntu-latest
59+
steps:
60+
- name: Checkout gh-pages branch
61+
uses: actions/checkout@v4
62+
with:
63+
ref: gh-pages
64+
token: ${{ secrets.GITHUB_TOKEN }}
65+
66+
- name: Install reprepro
67+
run: |
68+
sudo apt-get update
69+
sudo apt-get install -y reprepro
70+
71+
- name: Download .deb artifact
72+
uses: actions/download-artifact@v4
73+
with:
74+
name: deb-package
75+
path: ./artifacts
76+
77+
- name: Set up repository structure
78+
run: |
79+
# Create directories if they don't exist
80+
mkdir -p conf pool/main/p/perf-data-converter dists
81+
82+
# Create reprepro configuration
83+
cat > conf/distributions << 'EOF'
84+
Origin: perf_data_converter
85+
Label: perf_data_converter
86+
Architectures: amd64 arm64 all
87+
Components: main
88+
Description: perf data converter APT repository
89+
SignWith: no
90+
91+
Codename: focal
92+
Suite: focal
93+
94+
Codename: jammy
95+
Suite: jammy
96+
97+
Codename: noble
98+
Suite: noble
99+
EOF
100+
101+
- name: Move .deb files to pool
102+
run: |
103+
mv artifacts/*.deb pool/main/p/perf-data-converter/
104+
105+
- name: Update repository for each suite
106+
run: |
107+
for suite in focal jammy noble; do
108+
echo "Processing suite: $suite"
109+
reprepro -b . includedeb $suite pool/main/p/perf-data-converter/*.deb
110+
done
111+
112+
- name: Commit and push changes
113+
run: |
114+
git config --local user.email "[email protected]"
115+
git config --local user.name "GitHub Action"
116+
git add .
117+
if git diff --staged --quiet; then
118+
echo "No changes to commit"
119+
else
120+
git commit -m "Update APT repository with new package"
121+
git push
122+
fi
123+
124+
test:
125+
needs: publish
126+
runs-on: ubuntu-latest
127+
strategy:
128+
matrix:
129+
distro: ['20.04', '22.04', '24.04']
130+
arch: ['amd64', 'arm64']
131+
steps:
132+
- name: Test package installation
133+
run: |
134+
# Map Ubuntu versions to codenames
135+
case "${{ matrix.distro }}" in
136+
"20.04") CODENAME="focal" ;;
137+
"22.04") CODENAME="jammy" ;;
138+
"24.04") CODENAME="noble" ;;
139+
*) echo "Unknown distro version"; exit 1 ;;
140+
esac
141+
142+
# Create and run Docker container
143+
docker run --rm --platform linux/${{ matrix.arch }} ubuntu:${{ matrix.distro }} bash -c "
144+
set -e
145+
apt-get update
146+
apt-get install -y apt-transport-https gnupg curl
147+
148+
# Add our APT repository
149+
echo 'deb [trusted=yes] https://redis-performance.github.io/perf_data_converter focal main' > /etc/apt/sources.list.d/perf_data_converter.list
150+
echo 'deb [trusted=yes] https://redis-performance.github.io/perf_data_converter jammy main' >> /etc/apt/sources.list.d/perf_data_converter.list
151+
echo 'deb [trusted=yes] https://redis-performance.github.io/perf_data_converter noble main' >> /etc/apt/sources.list.d/perf_data_converter.list
152+
153+
# Update package list and install our package
154+
apt-get update
155+
apt-get install -y perf-data-converter
156+
157+
# Test that the binary works
158+
perf-data-converter --help || /usr/local/bin/perf_to_profile --help
159+
160+
echo 'Package installation and basic functionality test passed!'
161+
"

‎.github/workflows/ci.yaml‎

Lines changed: 59 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ name: ci
22

33
on:
44
push:
5-
branches: [copybara_staging]
5+
branches: ['**'] # Trigger on any branch
66
pull_request:
77
branches: [master]
88
# Allows for manually triggering workflow runs.
@@ -14,15 +14,14 @@ jobs:
1414
runs-on: ubuntu-22.04
1515
steps:
1616
- name: Checkout the repo
17-
uses: actions/checkout@v2
17+
uses: actions/checkout@v4
1818
with:
19-
path: ${{ env.WORKING_DIR }}
19+
submodules: recursive
2020

2121
- name: Fetch dependencies
2222
run: |
2323
sudo apt-get -qq update
2424
sudo apt-get install -y libelf-dev libssl-dev libcap-dev linux-tools-`uname -r`
25-
git submodule update --init --recursive
2625
2726
- name: Update toolchain
2827
run: |
@@ -37,3 +36,59 @@ jobs:
3736

3837
- name: Test
3938
run: bazel test //src:all //src/quipper:all
39+
40+
test-deb-build:
41+
runs-on: ubuntu-latest
42+
steps:
43+
- name: Checkout code
44+
uses: actions/checkout@v4
45+
with:
46+
submodules: recursive
47+
48+
- name: Set up dependencies
49+
run: |
50+
sudo apt-get -qq update
51+
sudo apt-get install -y libelf-dev libssl-dev libcap-dev ruby-dev build-essential
52+
53+
- name: Install FPM
54+
run: |
55+
sudo gem install --no-document fpm
56+
57+
- name: Test .deb package build
58+
run: |
59+
# FPM is already installed in CI, so skip gem install
60+
FPM_SKIP_INSTALL=1 ./scripts/build-deb.sh v0.0.0-test
61+
62+
- name: Validate .deb package
63+
run: |
64+
# Check that the package was created
65+
ls -la artifacts/
66+
test -f artifacts/perf-data-converter_0.0.0-test_all.deb
67+
68+
# Check package contents
69+
dpkg-deb --contents artifacts/perf-data-converter_0.0.0-test_all.deb
70+
71+
# Check package info
72+
dpkg-deb --info artifacts/perf-data-converter_0.0.0-test_all.deb
73+
74+
# Verify the binary is included and executable
75+
dpkg-deb --contents artifacts/perf-data-converter_0.0.0-test_all.deb | grep "perf_to_profile"
76+
77+
- name: Test package installation
78+
run: |
79+
# Install the package
80+
sudo dpkg -i artifacts/perf-data-converter_0.0.0-test_all.deb || true
81+
sudo apt-get install -f -y # Fix any dependency issues
82+
83+
# Test that the binary works
84+
/usr/local/bin/perf_to_profile --help || echo "Binary help test completed"
85+
86+
# Test that it's in PATH (if /usr/local/bin is in PATH)
87+
which perf_to_profile || echo "Binary not in PATH (expected)"
88+
89+
- name: Upload test .deb artifact
90+
uses: actions/upload-artifact@v4
91+
with:
92+
name: test-deb-package
93+
path: artifacts/*.deb
94+
retention-days: 7

‎README.md‎

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,38 @@ For details on pprof, see https://github.com/google/pprof
1010

1111
**THIS IS NOT AN OFFICIAL GOOGLE PRODUCT**
1212

13-
# Prerequisites to build
13+
# Installation
14+
15+
## Option 1: APT Repository (Recommended for Ubuntu users)
16+
17+
For Ubuntu 20.04, 22.04, and 24.04 users, you can install `perf_data_converter` directly from our APT repository:
18+
19+
```bash
20+
# Add the APT repository
21+
echo "deb [trusted=yes] https://redis-performance.github.io/perf_data_converter focal main" | sudo tee /etc/apt/sources.list.d/perf_data_converter.list
22+
echo "deb [trusted=yes] https://redis-performance.github.io/perf_data_converter jammy main" | sudo tee -a /etc/apt/sources.list.d/perf_data_converter.list
23+
echo "deb [trusted=yes] https://redis-performance.github.io/perf_data_converter noble main" | sudo tee -a /etc/apt/sources.list.d/perf_data_converter.list
24+
25+
# Update and install
26+
sudo apt-get update
27+
sudo apt-get install perf-data-converter
28+
```
29+
30+
The binary will be installed as `/usr/local/bin/perf_to_profile`.
31+
32+
For detailed APT repository documentation, see [docs/APT_REPOSITORY.md](docs/APT_REPOSITORY.md).
33+
34+
## Option 2: Build from Source
35+
36+
### Prerequisites to build
1437
* Install dependencies
1538

1639
```
1740
sudo apt-get -y install g++ git libelf-dev libcap-dev
1841
```
1942
* At least g++-5 or clang-7
2043

21-
# Compile and Test
44+
### Compile and Test
2245
To install all dependences and build the binary, run the following commands.
2346
These were tested on Ubuntu 14.04 LTS:
2447

@@ -60,6 +83,13 @@ directory `bazel-bin/`.
6083
pprof -web perf.data
6184
```
6285

86+
* If you installed via APT, you can also use:
87+
88+
```
89+
perf_to_profile -i perf.data -o profile.pb
90+
pprof -web profile.pb
91+
```
92+
6393
# Contribution
6494
We appreciate your help!
6595

0 commit comments

Comments
 (0)