Building DEB packages with podman & pbuilder by qount25 · Pull Request #8 · postgres-pm/pgpm

qount25 · 2025-03-02T23:21:27Z

Nice things:

Podman containers and temporary files are removed after the build is completed.
Podman containers created for each build have unique names, so if something fails, it is clear from the name what time the container has been created and it's left running for us to attach and see what happened.
All minimal dependencies are installed inside the container already, almost nothing is pulled, therefore...
The build is very fast. Takes maybe 30 second to complete.

Gotchas:

Downloads large (6Gb) Podman image from quay.io (but only once!). The image can probably be shrinked, but at the moment, it is what it is.
Lots of stuff is hard-coded -- for example architecture (amd64).
Uses a more declarative API as compared to being more of a DSL-like. This makes it very clear what the steps are (you only need to look at lib/pgpm/deb/builder.rb#build() to get the full picture.
Errors you see during the build are fakeroot related, but do not affect the build. I had to use fakeroot pbuilder, because SELinux would not allow me to mount /dev/* in a rootless container.
Currently works only for building extensions with no extra-dependencies (however, it's merely a matter of adding appropriate info into each particular package file -- this needs to be discussed).

… step

…mp dir

exe/pgpm

lib/pgpm/deb/builder.rb

lib/pgpm/deb/spec.rb

lib/pgpm/deb/templates/control.erb

yrashk · 2025-03-04T15:30:37Z

lib/pgpm/deb/templates/copyright.erb

@@ -0,0 +1,48 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/


Some of this information should be extractable from the package.

Not sure what you mean. This line looks like some XML schema definition that should always be the same. No?

lib/pgpm/deb/spec.rb

lib/pgpm/deb/builder.rb

@Package

…es from @Package

…#image_name

…nd return its contents

packages/timescaledb.rb

lib/pgpm/package/pgxn.rb

lib/pgpm/deb/spec.rb

lib/pgpm/deb/builder.rb

…icense method

…ld not require dependencies install inside podman

… constructor

lib/pgpm/rpm/spec.rb

…ding deb This is instead of copying repository into the container.

…ckage name

…ackage in Pgpm::Deb::Builder

pgpm.gemspec

packages/timescale/timescaledb.rb

yrashk

Can you also please rebase?

lib/pgpm/deb/templates/rules.erb

lib/pgpm/package/building.rb

lib/pgpm/package/dependencies.rb

yrashk · 2025-03-19T09:12:32Z

lib/pgpm/package/dependencies.rb

+        when "debian", "ubuntu"
+          [
+            "build-essential",
+            "postgresql-#{postgres_major_version}",


Postgres dependencies are added in the RPM spec / DEB control preparation based on Distribution's methods.

lib/pgpm/package/pgxn.rb

… in Pgpm::Deb::Spec templates

yrashk

Some more comments. Also, there is a minor conflict to be resolved again (sorry!)

packages/timescale/timescaledb.rb

lib/pgpm/package/dependencies.rb

lib/pgpm/deb/builder.rb

…pendencies

yrashk · 2025-03-22T11:25:36Z

lib/pgpm/package/make.rb

      def install_steps
-        return [Pgpm::Commands::Make.new("install", "DESTDIR=$PGPM_BUILDROOT", "PG_CONFIG=$PG_CONFIG")] if makefile_present?
-
+        case Pgpm::OS.in_scope.class.name


And speaking of special-casing and the case I expanded on when it comes to configuring – this is the same issue – and an opportunity – to unify the procedure.

yrashk · 2025-03-22T11:25:46Z

lib/pgpm/package/make.rb

+        when "debian", "ubuntu"
+          return []
+        when "rocky+epel-9", "redhat", "fedora"
+          return [Pgpm::Commands::Make.new("PG_CONFIG=$PG_CONFIG")] if makefile_present?


And speaking of special-casing and the case I expanded on when it comes to configuring – this is the same issue – and an opportunity – to unify the procedure.

…ing `dh` in it (WIP)

…n deb rules + timescaledb package custom configure/install steps

…dependncies for rpm's side of timescaledb pkg

qount25 added 6 commits March 1, 2025 02:35

Working scaffold for Pgpm::Deb::Builder where every method is a build…

06df6f3

… step

Pgpm::Deb::Builder #prepare and #create_container methods

0a1c5f2

Generating debian files used when building a deb package with pbuilder

676ac4d

Pgpm::Deb::Buider#cleanup stops & removes podman container, removes t…

11cc4de

…mp dir

Pgpm::Deb::Builder build with pbuilder, then move .deb file to the host

0b2dd51

Pgpb::Deb:Builder pull image unless it exists

62748ff

yrashk reviewed Mar 4, 2025

View reviewed changes

qount25 added 6 commits March 6, 2025 00:47

Refactor: replace most hardcoded values for Pgpm::Deb::Spec with valu…

aa0d915

…es from @Package

Refactor: make build run all podman commands in one go, then exit

97d5506

Fix: hardocoded podman image name instead of using Pgpm::Deb::Builder…

e26594f

…#image_name

Choosing arch suffix for debian packages (it's not the same as in RPM!)

29b258b

WIP: Timescaledb package with #build_info_for() for Debian and RedHat

1f8b9b2

Add Package#licence which attempts to read LICENCE file in pkg root a…

3165c10

…nd return its contents

qount25 force-pushed the deb branch from 693288f to 3165c10 Compare March 7, 2025 06:46

yrashk reviewed Mar 7, 2025

View reviewed changes

packages/timescaledb.rb Outdated Show resolved Hide resolved

yrashk reviewed Mar 7, 2025

View reviewed changes

lib/pgpm/package/pgxn.rb Show resolved Hide resolved

lib/pgpm/deb/spec.rb Outdated Show resolved Hide resolved

lib/pgpm/deb/builder.rb Show resolved Hide resolved

lib/pgpm/deb/builder.rb Show resolved Hide resolved

qount25 added 3 commits March 8, 2025 05:11

Fix: email address domain name for debian packages

70cb28a

Refactor: use Pgpm::Arch.in_scope to determine architecture

7dec4dc

Rename Pgpm::Package::PGXN#license into #license_text, restore old #l…

998546d

…icense method

qount25 force-pushed the deb branch 3 times, most recently from 56f8b9a to 59d72f8 Compare March 8, 2025 06:36

Specify default & custom dependencies in packages

4ac7c0f

qount25 force-pushed the deb branch from 59d72f8 to 4ac7c0f Compare March 8, 2025 06:37

qount25 added 3 commits March 8, 2025 07:19

Fix: dpkg-buildpackage --build=source step in Pgpm::Deb::Builder shou…

97581e7

…ld not require dependencies install inside podman

Proper changelog template for pbuilder packaging

3b02c92

Assign .os and .postgres_major_version to @Package in Pgpm::RPM::Spec…

0a84b1d

… constructor

yrashk reviewed Mar 9, 2025

View reviewed changes

lib/pgpm/rpm/spec.rb Outdated Show resolved Hide resolved

yrashk reviewed Mar 9, 2025

View reviewed changes

lib/pgpm/rpm/spec.rb Outdated Show resolved Hide resolved

Refactor: simpler way to extract postgres major version

d39f047

qount25 force-pushed the deb branch from 59551ad to a1ca046 Compare March 11, 2025 06:18

qount25 added 7 commits March 11, 2025 07:18

Fix: use Pgpm::OS.in_scope to determine OS name at the package level

0b62193

Remove 'require "debug"'

0e9ce56

Refactor: use @spec.sources to download and extract sources when buil…

f2339f8

…ding deb This is instead of copying repository into the container.

Rename & package versioned files into a DEB package, assign proper pa…

71bcfdd

…ckage name

Create second .deb package (default) which depends on the versioned p…

cb58e0a

…ackage in Pgpm::Deb::Builder

Refactor: prepare_artifacts in deb/

8de62ed

Merge branch 'master' into deb

6b56207

yrashk reviewed Mar 19, 2025

View reviewed changes

pgpm.gemspec Show resolved Hide resolved

yrashk reviewed Mar 19, 2025

View reviewed changes

packages/timescale/timescaledb.rb Outdated Show resolved Hide resolved

yrashk reviewed Mar 19, 2025

View reviewed changes

qount25 added 4 commits March 20, 2025 20:11

Restore accidentally deleted pgpm.spec

7b2aee8

Use configure_steps, make_steps & install_steps instead of build_info…

775da7c

… in Pgpm::Deb::Spec templates

Remove build-essential from the list of default build dependencies

4bd0cbf

Accept a larger diversity of filenames for License

a99f00b

yrashk reviewed Mar 21, 2025

View reviewed changes

packages/timescale/timescaledb.rb Show resolved Hide resolved

lib/pgpm/package/dependencies.rb Show resolved Hide resolved

lib/pgpm/deb/builder.rb Show resolved Hide resolved

qount25 added 3 commits March 21, 2025 13:36

Oops: spelling of "ubuntu"

2a4a708

Merge branch 'master' into deb

de2e49d

Use Package#native? as condition to add "build-essential" to build_de…

e14be75

…pendencies

yrashk reviewed Mar 22, 2025

View reviewed changes

qount25 added 7 commits March 26, 2025 03:17

Refactor: use completely customized debian rules file -- without call…

8d0c175

…ing `dh` in it (WIP)

Fix: better way to handle custom configure, build and install steps i…

8eb2766

…n deb rules + timescaledb package custom configure/install steps

Fix: timescaledb #build_steps method with custom steps for debian

6fb0482

Fix: deb package names should not include _, replace them with -

a9ac637

Fix: os-specific dependency names for pgsodium package

7f4217c

Oops: remove require "debug"

d0a7907

Fix: remove default dependencies for rocky/redheat packages + update …

79e1172

…dependncies for rpm's side of timescaledb pkg

qount25 changed the title ~~WIP: building DEB packages with podman & pbuilder~~ Building DEB packages with podman & pbuilder Mar 29, 2025

Fixing rubocop issues

3073f73

yrashk merged commit bbea903 into postgres-pm:master Mar 29, 2025
2 checks passed

		@@ -0,0 +1,48 @@
		Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/

Conversation

qount25 commented Mar 2, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yrashk Mar 4, 2025

Choose a reason for hiding this comment

Uh oh!

qount25 Mar 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yrashk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yrashk Mar 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

yrashk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yrashk Mar 22, 2025

Choose a reason for hiding this comment

Uh oh!

yrashk Mar 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants