Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 150 additions & 4 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,15 @@ on:

permissions:
contents: write
id-token: write # Required for cosign keyless signing

jobs:
# Build for Linux (amd64 only for now)
# Build for Linux
build-linux:
runs-on: ubuntu-latest
strategy:
matrix:
goarch: [amd64] # arm64 can be added later with cross-compilation setup
goarch: [amd64, arm64]
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -42,7 +43,22 @@ jobs:
libsqlite3-dev \
libpcap-dev

- name: Install ARM64 cross-compiler
if: matrix.goarch == 'arm64'
run: |
sudo apt-get install -y \
gcc-aarch64-linux-gnu \
g++-aarch64-linux-gnu

- name: Cache Vectorscan build
id: cache-vectorscan
uses: actions/cache@v4
with:
path: build/vectorscan
key: vectorscan-${{ runner.os }}-${{ matrix.goarch }}-${{ hashFiles('scripts/build-vectorscan.sh') }}

- name: Build Vectorscan
if: steps.cache-vectorscan.outputs.cache-hit != 'true'
run: bash scripts/build-vectorscan.sh

- name: Build poltergeist
Expand Down Expand Up @@ -98,7 +114,15 @@ jobs:
- name: Install build dependencies
run: brew install cmake ragel boost pkg-config sqlite libpcap

- name: Cache Vectorscan build
id: cache-vectorscan
uses: actions/cache@v4
with:
path: build/vectorscan
key: vectorscan-${{ runner.os }}-${{ matrix.goarch }}-${{ hashFiles('scripts/build-vectorscan.sh') }}

- name: Build Vectorscan
if: steps.cache-vectorscan.outputs.cache-hit != 'true'
run: bash scripts/build-vectorscan.sh

- name: Build poltergeist
Expand Down Expand Up @@ -129,9 +153,102 @@ jobs:
name: darwin-${{ matrix.goarch }}
path: poltergeist_darwin_${{ matrix.goarch }}.tar.gz

# Build for Windows
build-windows:
runs-on: windows-latest
defaults:
run:
shell: msys2 {0}
strategy:
matrix:
goarch: [amd64]
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Setup MSYS2
uses: msys2/setup-msys2@v2
with:
msystem: MINGW64
update: true
cache: true
install: >-
base-devel
git
unzip
mingw-w64-x86_64-gcc
mingw-w64-x86_64-cmake
mingw-w64-x86_64-boost
mingw-w64-x86_64-ragel
mingw-w64-x86_64-pcre
mingw-w64-x86_64-sqlite3
mingw-w64-x86_64-pkg-config

- name: Set up Go in MSYS2
shell: msys2 {0}
run: |
# Download and install Go
GO_VERSION=$(cat go.mod | grep -oP 'go \K[0-9.]+')
echo "Installing Go ${GO_VERSION}..."
curl -L "https://go.dev/dl/go${GO_VERSION}.windows-amd64.zip" -o go.zip
unzip -q go.zip -d /c/
echo "/c/go/bin" >> $GITHUB_PATH

- name: Cache Go modules
uses: actions/cache@v4
with:
path: |
~/go/pkg/mod
~/.cache/go-build
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-

- name: Cache Vectorscan build
id: cache-vectorscan
uses: actions/cache@v4
with:
path: build/vectorscan
key: vectorscan-${{ runner.os }}-${{ matrix.goarch }}-${{ hashFiles('scripts/build-vectorscan-windows.sh') }}

- name: Build Vectorscan
if: steps.cache-vectorscan.outputs.cache-hit != 'true'
run: bash scripts/build-vectorscan-windows.sh

- name: Build poltergeist
run: |
export CGO_ENABLED=1
export PKG_CONFIG_PATH="$(pwd)/build/vectorscan/windows_amd64/lib/pkgconfig"
export CGO_CFLAGS="-I$(pwd)/build/vectorscan/windows_amd64/include"
export CGO_LDFLAGS="-L$(pwd)/build/vectorscan/windows_amd64/lib -lhs -static"

VERSION="${{ github.ref_name }}"
if [ "${{ github.event_name }}" = "pull_request" ]; then
VERSION="dev-$(git rev-parse --short HEAD)"
fi

/c/go/bin/go build \
-ldflags "-s -w -X main.version=$VERSION" \
-o poltergeist.exe \
./cmd/poltergeist

- name: Create archive
shell: pwsh
run: |
Compress-Archive -Path poltergeist.exe,README.md,LICENSE,NOTICE `
-DestinationPath poltergeist_windows_${{ matrix.goarch }}.zip

- name: Upload artifact
uses: actions/upload-artifact@v4
with:
name: windows-${{ matrix.goarch }}
path: poltergeist_windows_${{ matrix.goarch }}.zip

# Create the GitHub release
create-release:
needs: [build-linux, build-macos]
needs: [build-linux, build-macos, build-windows]
runs-on: ubuntu-latest
if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/')
steps:
Expand All @@ -146,10 +263,39 @@ jobs:
- name: Display structure
run: ls -R dist/

- name: Install cosign
uses: sigstore/cosign-installer@v3.0.4

- name: Sign release artifacts
run: |
# Sign all tar.gz files
for file in dist/**/*.tar.gz; do
if [ -f "$file" ]; then
echo "Signing $file"
cosign sign-blob --yes "$file" \
--output-signature "${file}.sig" \
--output-certificate "${file}.crt"
fi
done

# Sign all zip files
for file in dist/**/*.zip; do
if [ -f "$file" ]; then
echo "Signing $file"
cosign sign-blob --yes "$file" \
--output-signature "${file}.sig" \
--output-certificate "${file}.crt"
fi
done

- name: Create Release
uses: softprops/action-gh-release@v1
with:
files: dist/**/*.tar.gz
files: |
dist/**/*.tar.gz
dist/**/*.zip
dist/**/*.sig
dist/**/*.crt
generate_release_notes: true
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
16 changes: 15 additions & 1 deletion .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ builds:
goos:
- linux
- darwin
- windows
goarch:
- amd64
- arm64
Expand Down Expand Up @@ -75,9 +76,22 @@ builds:
- -s -w
- -X main.version={{.Version}}

- goos: windows
goarch: amd64
env:
- CGO_ENABLED=1
- CGO_CFLAGS=-I{{.Env.PWD}}/build/vectorscan/windows_amd64/include
- CGO_LDFLAGS=-L{{.Env.PWD}}/build/vectorscan/windows_amd64/lib -lhs -static
ldflags:
- -s -w
- -X main.version={{.Version}}

archives:
- id: default
formats: ['tar.gz']
format: tar.gz
format_overrides:
- goos: windows
format: zip
name_template: >-
{{ .ProjectName }}_
{{- .Os }}_
Expand Down
80 changes: 73 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,11 @@ Some decisions were made in the interest of performance and simplicity:
Download the latest release for your platform from [GitHub Releases](https://github.com/ghostsecurity/poltergeist/releases).

**Supported Platforms:**
- Linux (x86_64)
- macOS (Intel & Apple Silicon)
- Linux (x86_64, ARM64) - statically linked with Vectorscan
- macOS (Intel & Apple Silicon) - statically linked with Vectorscan
- Windows (x86_64) - statically linked with Vectorscan (built with MinGW)

Release binaries are statically linked with Vectorscan and have no external dependencies (except standard system libraries on macOS).
Release binaries have no external dependencies.

```bash
# Linux/macOS
Expand All @@ -52,6 +53,45 @@ tar xzf poltergeist_*.tar.gz
sudo mv poltergeist /usr/local/bin/
```

```powershell
# Windows (PowerShell)
Invoke-WebRequest -Uri "https://github.com/ghostsecurity/poltergeist/releases/download/v2.0.0/poltergeist_windows_amd64.zip" -OutFile "poltergeist.zip"
Expand-Archive -Path poltergeist.zip -DestinationPath .
.\poltergeist.exe --version
```

#### Verifying Release Signatures

All release artifacts are signed with [Sigstore cosign](https://github.com/sigstore/cosign) for supply chain security.

```bash
# Install cosign
brew install cosign # macOS
# or download from https://github.com/sigstore/cosign/releases

# Verify a release artifact
cosign verify-blob poltergeist_linux_amd64.tar.gz \
--signature poltergeist_linux_amd64.tar.gz.sig \
--certificate poltergeist_linux_amd64.tar.gz.crt \
--certificate-identity-regexp 'https://github.com/ghostsecurity/poltergeist/.github/workflows/release.yml' \
--certificate-oidc-issuer 'https://token.actions.githubusercontent.com'
```

#### Platform-Specific Notes

**macOS Security Warning:**

When running the binary on macOS, you may see a Gatekeeper warning. This is because the binary is not signed with an Apple Developer certificate. To bypass:

```bash
# Remove quarantine attribute
xattr -d com.apple.quarantine ./poltergeist

# Or right-click the binary in Finder and select "Open"
```

The binary is safe to run - verify with cosign signatures above.

#### Building from Source

If you want to build from source, you'll need Vectorscan installed:
Expand All @@ -70,20 +110,46 @@ sudo apt-get install cmake ragel libboost-dev pkg-config
bash scripts/build-vectorscan.sh
```

**Windows (requires MSYS2):**
```bash
# Install MSYS2 from https://www.msys2.org/
# Open MSYS2 MINGW64 terminal

# Install dependencies
pacman -S --needed base-devel git \
mingw-w64-x86_64-gcc \
mingw-w64-x86_64-cmake \
mingw-w64-x86_64-boost \
mingw-w64-x86_64-ragel \
mingw-w64-x86_64-pcre \
mingw-w64-x86_64-sqlite3 \
mingw-w64-x86_64-pkg-config

# Build Vectorscan
bash scripts/build-vectorscan-windows.sh

# Build standalone binary (in MINGW64 terminal)
CGO_ENABLED=1 \
CGO_CFLAGS="-I$(pwd)/build/vectorscan/windows_amd64/include" \
CGO_LDFLAGS="-L$(pwd)/build/vectorscan/windows_amd64/lib -lhs -static" \
go build -ldflags "-s -w" -o poltergeist.exe ./cmd/poltergeist
```

**Build:**
```bash
# Linux/macOS
make build
```

### About Vectorscan/Hyperscan
### About Vectorscan

Poltergeist uses Vectorscan (a portable fork of Intel's Hyperscan) for high-performance multi-pattern matching.
Poltergeist uses [Vectorscan](https://github.com/VectorCamp/vectorscan), a portable fork of Intel's Hyperscan, on all platforms (Linux, macOS, and Windows).

The main benefit of Vectorscan is multi-pattern matching and aggressive optimizations.
Vectorscan provides high-performance multi-pattern matching with aggressive optimizations.
Some advanced regex features (backtracking, lookbehind, lookahead, capture groups) are not supported.

Despite these limitations, rule patterns are written with extended regex syntax for
flexibility. While initial matches use Vectorscan, the final match location and capture
flexibility. While initial matches use Hyperscan/Vectorscan, the final match location and capture
groups are refined with Go regex for maximum compatibility.

## Build from Source
Expand Down
Loading
Loading