Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Independent container updates #1080

Open
wants to merge 86 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
81ee267
Add a `dangerzone-image` CLI script
almet Feb 11, 2025
3d28ae2
Download and verify cosign signatures
almet Feb 11, 2025
197325b
Add the ability to download diffoci for multiple platforms
almet Feb 11, 2025
f60c43f
Publish and attest multi-architecture container images
apyrgio Feb 11, 2025
af55d26
Add documentation for independent container updates
almet Feb 11, 2025
5c9a38d
(WIP) Check for container updates rather than using `image-id.txt`
almet Feb 11, 2025
27647cc
fixup! Download and verify cosign signatures
almet Feb 12, 2025
f6562ae
fixup! Download and verify cosign signatures
almet Feb 12, 2025
7002ab8
fixup! Download and verify cosign signatures
almet Feb 12, 2025
6aff845
fixup! Download and verify cosign signatures
almet Feb 12, 2025
db33038
fixup! Download and verify cosign signatures
almet Feb 12, 2025
5001328
fixup! Download and verify cosign signatures
almet Feb 12, 2025
22d235c
fixup! Download and verify cosign signatures
almet Feb 12, 2025
5a4ddb1
fixup! Download and verify cosign signatures
almet Feb 12, 2025
1e9e468
fixup! Download and verify cosign signatures
almet Feb 12, 2025
379c9f8
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
d667c28
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
431f0cb
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
aac6c63
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
ccae6c5
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
5202d79
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
9889710
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
668ee71
fixup! Add a `dangerzone-image` CLI script
almet Feb 12, 2025
0724f86
fixup! Publish and attest multi-architecture container images
almet Feb 12, 2025
537d23e
fixup! Publish and attest multi-architecture container images
almet Feb 12, 2025
5acb302
fixup! Publish and attest multi-architecture container images
almet Feb 12, 2025
e078e9b
fixup! 1e9e468e3766eb8fd518a94589f9acdd6b3081ac
almet Feb 12, 2025
60674ea
fixup! (WIP) Check for container updates rather than using `image-id.…
almet Feb 12, 2025
835970b
fixup! (WIP) Check for container updates rather than using `image-id.…
almet Feb 12, 2025
a540fc5
(WIP) Add tests
almet Feb 12, 2025
0f2d81d
(WIP) some more tests
almet Feb 13, 2025
b4818ce
fixup! (WIP) some more tests
almet Feb 25, 2025
35704b8
fixup! (WIP) some more tests
almet Feb 25, 2025
b37815a
fixup! (WIP) some more tests
almet Feb 25, 2025
c9c301d
fixup! (WIP) some more tests
almet Feb 25, 2025
df3efa8
fixup! 6aff84549364a548fbf738d5e62a00e8b35c9105
almet Feb 25, 2025
a4fa6aa
fixup! (WIP) Add tests
almet Feb 25, 2025
ecb3d87
fixup! (WIP) Add tests
almet Feb 25, 2025
fb89f00
fixup! (WIP) Add tests
almet Feb 25, 2025
83418f0
fixup! (WIP) Add tests
almet Feb 25, 2025
3e861cc
fixup! (WIP) Add tests
almet Feb 25, 2025
b5bfbb5
fixup! (WIP) Add tests
almet Feb 25, 2025
ab51a71
fixup! (WIP) Add tests
almet Feb 25, 2025
43cb02b
fixup! (WIP) Add tests
almet Feb 25, 2025
ec4028b
fixup! (WIP) Add tests
almet Feb 25, 2025
4621902
fixup! (WIP) Add tests
almet Feb 25, 2025
cf7a3db
fixup! (WIP) Add tests
almet Feb 25, 2025
9bf663f
fixup! (WIP) Add tests
almet Feb 25, 2025
01f7b37
fixup! (WIP) Add tests
almet Feb 25, 2025
0c063b5
fixup! (WIP) Add tests
almet Feb 25, 2025
7baddd0
fixup! (WIP) Add tests
almet Feb 25, 2025
8381b2f
fixup! (WIP) Add tests
almet Feb 25, 2025
7e28319
fixup! 35704b8a1854711720fe2bacd3416cb712c076a4
almet Feb 25, 2025
d5d3038
fixup! Download and verify cosign signatures
almet Feb 25, 2025
33ee158
fixup! Download and verify cosign signatures
almet Feb 25, 2025
7f83505
fixup! Download and verify cosign signatures
almet Feb 25, 2025
4073a62
fixup! Download and verify cosign signatures
almet Feb 25, 2025
30ec1f1
fixup! Download and verify cosign signatures
almet Feb 25, 2025
2476ed6
fixup! Download and verify cosign signatures
almet Feb 25, 2025
4a4bf7c
fixup! 3e861cc0cda83b5a58a4457918726835bc9fd68b
almet Feb 25, 2025
bba427d
fixup! 83418f09f20ea1fdd5e808bd0750d1dfb07b389a
almet Feb 25, 2025
43f6d89
fixup! b37815a96c044cbdea8fb50c8106f6a70dd11345
almet Feb 25, 2025
d93c99f
fixup! b4818ce854d191c5ecca93ef22ce5b2a3f6c8d07
almet Feb 25, 2025
7e4cd66
fixup! b4818ce854d191c5ecca93ef22ce5b2a3f6c8d07
almet Feb 25, 2025
22d01a4
fixup! c9c301d83330f07852856725fbe8b5502caa1371
almet Feb 25, 2025
49c4cee
make the signature tests pass
almet Feb 25, 2025
356d848
fixup! Add a `dangerzone-image` CLI script
almet Feb 25, 2025
3d579c8
fixup! Add a `dangerzone-image` CLI script
almet Feb 25, 2025
f175739
fixup! Add a `dangerzone-image` CLI script
almet Feb 25, 2025
3ea4917
fixup! Add a `dangerzone-image` CLI script
almet Feb 25, 2025
760948b
Add tests for registry
almet Feb 25, 2025
c313c6d
FIXUP: Use the digest when pulling the container
almet Feb 26, 2025
53a7028
Introduce a `subprocess_run` utility function
almet Feb 26, 2025
f00f962
FIXUP: Use exceptions to ease the flow
almet Feb 26, 2025
3f6c134
FIXUP: Use user data dir rather than config
almet Feb 26, 2025
49b54aa
FIXUP: throw rather than bools
almet Feb 26, 2025
a82ba28
FIXUP: Add a comment to update the DEFAULT_LOG_INDEX with releases
almet Feb 26, 2025
2aeb53a
fixup! Download and verify cosign signatures
almet Feb 26, 2025
cff3ac2
fixup! Download and verify cosign signatures
almet Feb 26, 2025
c405eb9
Provide an `is_update_available` function
almet Feb 26, 2025
f1dac59
FIXUP commit for signature tests
almet Feb 26, 2025
7eb54c3
Split updater GUI code from the code checking for release updates
almet Feb 27, 2025
264f1d1
Replace the `updater_check` setting by `updater_check_all`
almet Mar 1, 2025
052c352
Add a `dangerzone-image store-signature` CLI command
almet Mar 3, 2025
5bd5157
Display the `{podman,docker} pull` progress when installing a new image
almet Mar 3, 2025
bbac103
Allow a different runtime on `dangerzone-image` commands.
almet Mar 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
168 changes: 168 additions & 0 deletions .github/workflows/release-container-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
name: Release multi-arch container image

on:
workflow_dispatch:
push:
branches:
- main
- "test/**"
schedule:
- cron: "0 0 * * *" # Run every day at 00:00 UTC.

env:
REGISTRY: ghcr.io/${{ github.repository_owner }}
REGISTRY_USER: ${{ github.actor }}
REGISTRY_PASSWORD: ${{ github.token }}
IMAGE_NAME: dangerzone/dangerzone

jobs:
build:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
platform:
- linux/amd64
- linux/arm64
steps:
- uses: actions/checkout@v4

- name: Get current date
id: date
run: echo "date=$(date +'%Y%m%d')" >> $GITHUB_OUTPUT

- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV

- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}

- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

- name: Build and push by digest
id: build
uses: docker/build-push-action@v6
with:
context: ./dangerzone/
file: Dockerfile
build-args: |
DEBIAN_ARCHIVE_DATE=${{ steps.date.outputs.date }}
## Remove potentially incorrect Docker provenance.
#provenance: false
platforms: ${{ matrix.platform }}
labels: ${{ steps.meta.outputs.labels }}
outputs: type=image,"name=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}",push-by-digest=true,name-canonical=true,push=true

- name: Export digest
run: |
mkdir -p ${{ runner.temp }}/digests
digest="${{ steps.build.outputs.digest }}"
touch "${{ runner.temp }}/digests/${digest#sha256:}"

- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: ${{ runner.temp }}/digests/*
if-no-files-found: error
retention-days: 1

merge:
runs-on: ubuntu-latest
needs:
- build
outputs:
digest: ${{ steps.image.outputs.digest }}
image: ${{ steps.image.outputs.image }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Compute image tag
id: tag
run: |
DATE=$(date +'%Y%m%d')
TAG=$(git describe --long --first-parent | tail -c +2)
echo "tag=${DATE}-${TAG}" >> $GITHUB_OUTPUT

- name: Download digests
uses: actions/download-artifact@v4
with:
path: ${{ runner.temp }}/digests
pattern: digests-*
merge-multiple: true

- name: Login to GHCR
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3

#- name: Docker meta
# id: meta
# uses: docker/metadata-action@v5
# with:
# images: |
# ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
# tags: |
# type=ref,event=branch
# type=ref,event=pr
# type=semver,pattern={{version}}
# type=semver,pattern={{major}}.{{minor}}

- name: Create manifest list and push
working-directory: ${{ runner.temp }}/digests
run: |
IMAGE=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ steps.tag.outputs.tag }}
DIGESTS=$(printf '${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}@sha256:%s ' *)
docker buildx imagetools create -t ${IMAGE} ${DIGESTS}

- name: Inspect image
id: image
run: |
# NOTE: Set the image as an output because the `env` context is not
# available to the inputs of a reusable workflow call.
image_name="${REGISTRY}/${IMAGE_NAME}"
echo "image=$image_name" >> "$GITHUB_OUTPUT"
docker buildx imagetools inspect ${image_name}:${{ steps.tag.outputs.tag }}
digest=$(docker buildx imagetools inspect ${image_name}:${{ steps.tag.outputs.tag }} --format "{{json .Manifest}}" | jq -r '.digest')
echo "digest=$digest" >> "$GITHUB_OUTPUT"

# This step calls the container workflow to generate provenance and push it to
# the container registry.
provenance:
needs:
- merge
permissions:
actions: read # for detecting the Github Actions environment.
id-token: write # for creating OIDC tokens for signing.
packages: write # for uploading attestations.
uses: slsa-framework/slsa-github-generator/.github/workflows/[email protected]
with:
digest: ${{ needs.merge.outputs.digest }}
image: ${{ needs.merge.outputs.image }}
registry-username: ${{ github.actor }}
secrets:
registry-password: ${{ secrets.GITHUB_TOKEN }}
116 changes: 99 additions & 17 deletions dangerzone/container_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,25 @@
import platform
import shutil
import subprocess
from typing import List, Tuple
from typing import IO, Callable, List, Optional, Tuple

from . import errors
from .util import get_resource_path, get_subprocess_startupinfo

CONTAINER_NAME = "dangerzone.rocks/dangerzone"
OLD_CONTAINER_NAME = "dangerzone.rocks/dangerzone"
CONTAINER_NAME = "ghcr.io/almet/dangerzone/dangerzone" # FIXME: Change this to the correct container name
RUNTIME_NAME = "podman" if platform.system() == "Linux" else "docker"

log = logging.getLogger(__name__)


def subprocess_run(*args, **kwargs) -> subprocess.CompletedProcess:
"""subprocess.run with the correct startupinfo for Windows."""
return subprocess.run(*args, startupinfo=get_subprocess_startupinfo(), **kwargs)


def get_runtime_name() -> str:
if platform.system() == "Linux":
runtime_name = "podman"
else:
# Windows, Darwin, and unknown use docker for now, dangerzone-vm eventually
runtime_name = "docker"
return runtime_name
return RUNTIME_NAME


def get_runtime_version() -> Tuple[int, int]:
Expand All @@ -40,9 +42,8 @@ def get_runtime_version() -> Tuple[int, int]:

cmd = [runtime, "version", "-f", query]
try:
version = subprocess.run(
version = subprocess_run(
cmd,
startupinfo=get_subprocess_startupinfo(),
capture_output=True,
check=True,
).stdout.decode()
Expand Down Expand Up @@ -112,13 +113,7 @@ def delete_image_tag(tag: str) -> None:
)


def get_expected_tag() -> str:
"""Get the tag of the Dangerzone image tarball from the image-id.txt file."""
with open(get_resource_path("image-id.txt")) as f:
return f.read().strip()


def load_image_tarball() -> None:
def load_image_tarball_from_gzip() -> None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While working on reproducible builds, I started to realize that our container image becomes marginally smaller if we compress it with gzip. That could be because Buildkit compresses the files in each layer, but I'm not sure yet. Anyway, what I'm getting at is that this code path is a strong candidate for ✂️ .

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neat, let's cut these hairy hairs!

log.info("Installing Dangerzone container image...")
p = subprocess.Popen(
[get_runtime(), "load"],
Expand Down Expand Up @@ -147,3 +142,90 @@ def load_image_tarball() -> None:
)

log.info("Successfully installed container image from")


def load_image_tarball_from_tar(tarball_path: str) -> None:
cmd = [get_runtime(), "load", "-i", tarball_path]
subprocess_run(cmd, check=True)
log.info("Successfully installed container image from %s", tarball_path)


def tag_image_by_digest(digest: str, tag: str) -> None:
"""Tag a container image by digest.
The sha256: prefix should be omitted from the digest.
"""
image_id = get_image_id_by_digest(digest)
cmd = [get_runtime(), "tag", image_id, tag]
log.debug(" ".join(cmd))
subprocess_run(cmd, check=True)


def get_image_id_by_digest(digest: str) -> str:
"""Get an image ID from a digest.
The sha256: prefix should be omitted from the digest.
"""
cmd = [
get_runtime(),
"images",
"-f",
f"digest=sha256:{digest}",
"--format",
"{{.Id}}",
]
log.debug(" ".join(cmd))
process = subprocess_run(cmd, check=True, capture_output=True)
# In case we have multiple lines, we only want the first one.
return process.stdout.decode().strip().split("\n")[0]


def container_pull(image: str, manifest_digest: str, callback: Callable):
"""Pull a container image from a registry."""
cmd = [get_runtime_name(), "pull", f"{image}@sha256:{manifest_digest}"]
process = subprocess.Popen(
cmd,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
bufsize=1,
)

for line in process.stdout: # type: ignore
callback(line)

process.wait()
if process.returncode != 0:
raise errors.ContainerPullException(
f"Could not pull the container image: {process.returncode}"
)


def get_local_image_digest(image: str) -> str:
"""
Returns a image hash from a local image name
"""
# Get the image hash from the "podman images" command.
# It's not possible to use "podman inspect" here as it
# returns the digest of the architecture-bound image
cmd = [get_runtime_name(), "images", image, "--format", "{{.Digest}}"]
log.debug(" ".join(cmd))
try:
result = subprocess_run(
cmd,
capture_output=True,
check=True,
)
lines = result.stdout.decode().strip().split("\n")
if len(lines) != 1:
raise errors.MultipleImagesFoundException(
f"Expected a single line of output, got {len(lines)} lines"
)
image_digest = lines[0].replace("sha256:", "")
if not image_digest:
raise errors.ImageNotPresentException(
f"The image {image} does not exist locally"
)
Comment on lines +223 to +226
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I get what you're talking about, you're handling here the case where the image exists locally, but Podman/Docker somehow don't return a digest for it. That's a weird one indeed, I've seen it while testing things out, but never could figure out why it happens.

At any rate, I'd change the message here to show that the image exists locally, but the container engine reports no digest for it. Solely to not get confused with the error message below.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We rely on podman images --format "{{.Digest}}" to avoid the problem you're mentioning, yes, but this isn't why we're raising twice the same exception.

I agree this isn't ideal, but I don't really see a way around it. Maybe my eyes are too close to the screen: if you see another way to do it let me know :-)

return image_digest
except subprocess.CalledProcessError as e:
raise errors.ImageNotPresentException(
f"The image {image} does not exist locally"
)
20 changes: 16 additions & 4 deletions dangerzone/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,21 +122,33 @@ def wrapper(*args, **kwargs): # type: ignore
#### Container-related errors


class ImageNotPresentException(Exception):
class ContainerException(Exception):
pass


class ImageInstallationException(Exception):
class ImageNotPresentException(ContainerException):
pass


class NoContainerTechException(Exception):
class MultipleImagesFoundException(ContainerException):
pass


class ImageInstallationException(ContainerException):
pass


class NoContainerTechException(ContainerException):
def __init__(self, container_tech: str) -> None:
super().__init__(f"{container_tech} is not installed")


class NotAvailableContainerTechException(Exception):
class NotAvailableContainerTechException(ContainerException):
def __init__(self, container_tech: str, error: str) -> None:
self.error = error
self.container_tech = container_tech
super().__init__(f"{container_tech} is not available")


class ContainerPullException(ContainerException):
pass
11 changes: 6 additions & 5 deletions dangerzone/gui/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@
from ..isolation_provider.container import Container
from ..isolation_provider.dummy import Dummy
from ..isolation_provider.qubes import Qubes, is_qubes_native_conversion
from ..updater import errors as updater_errors
from ..updater import releases
from ..util import get_resource_path, get_version
from .logic import DangerzoneGui
from .main_window import MainWindow
Expand Down Expand Up @@ -161,16 +163,15 @@ def open_files(filenames: List[str] = []) -> None:
window.register_update_handler(updater.finished)

log.debug("Consulting updater settings before checking for updates")
if updater.should_check_for_updates():
should_check = updater.should_check_for_updates()

if should_check:
log.debug("Checking for updates")
updater.start()
else:
log.debug("Will not check for updates, based on updater settings")

# Ensure the status of the toggle updates checkbox is updated, after the user is
# prompted to enable updates.
window.toggle_updates_action.setChecked(bool(updater.check))

window.toggle_updates_action.setChecked(should_check)
if filenames:
open_files(filenames)

Expand Down
Loading
Loading