ARM-64 build

[derdaele]

I'm currently running Apple Silicon machine, which is using the ARM-64 ISA.
Currently the postgis image is only built for amd64.

Would it be possible to export ARM-64 builds?

The current workaround is to use community built images (e.g. https://github.com/Duvel/docker-postgis).

[derdaele]

Note: If I build the image from this repository, I can successfully build on ARM-64. This means that the current Dockerfile(s) are already ARM-64 friendly, it's just a matter of uploading the built image to docker hub.

[phillipross]

I believe there is already an open issue around this, but it hasn't seen any activity for a few months:
#144

The state of things has changed a bit since then... most notably, travis is no longer the primary CI for the the docker images. Due to changes in the Travis-CI offering, the CI system was ported over to Github Actions. While Github Actions runners are x86 only, I believe there are actions that have been developed by the community to use qemu or other hypervisors to enable building the arm images using x86 platforms, but I have yet to look into it.

I believe the issue I referenced above mentions raspberry pi explicitly, but building out this repo to work on multiple arm platforms (windows, apple) should probably be the goal. The docker-postgis repo was initially build on top of the official postgres repo, but I have yet to look at how the official postgres repo is doing their arm builds either.

Probably the next logical step should checking out the official postgres repo and seeing how they're doing arm builds, and investigating how to make Github Actions build arm images.

[phillipross]

@derdaele also, are you actually running docker on your Apple Silicon machine? If so, I'd like to know more about how this is accomplished. Thanks!

[derdaele]

Yes, I'm running the developer preview [1].

I tried to look into how the official postgres image is built and it looks like it's using the brewbash project, I'm not sure how easy it would be to integrate this here.

However I stumbled upon this [2] GitHub action that seems to be able to do multi-arch builds.

I'll give it a try and send a PR if it works.

[1] docker/roadmap#142 (comment)

[2] https://github.com/docker/setup-buildx-action

[phillipross]

Right, while we don't necessary need to integrate with brewbash, what we probably want to do is analyze things that prevent us from integrating in the future. They have set of patterns and guidelines for the official images (a bit lengthy actually) that we try to keep in mind as we're making changes to this repo.

If the dev preview of docker desktop for M1 is capable of running buildx, then they're further along than I thought. Seeing the progress on this is very encouraging.

I believe this github action from docker was also the one I saw for doing the multiplatform builds. I would say we don't necessarily have to use it, but only that it does exist so building multiplatform should be possible without having to setup any custom github runner. At the moment, there's only one github action flow in this repo which was directly ported from travis, and uses shell scripts that are invoked by github actions rather than using things that are tightly bound to github actions. This prevents us from committing too heavily to the CI product, and allows folks to run the same processes in local linux vms (or to some extent, a macos or windows environment) without the need for some local github actions emulation framework.

It looks as if modifying things to get buildx in the mix is what the next task is to tackle multiplatform builds for this repo. If you'd like to take a stab at that, it would be much appreciated!

[derdaele]

I ran a few tests and results are actually not so great.

• Workflow Run: https://github.com/derdaele/docker-postgis/runs/1552700456?check_suite_focus=true
• Compiled results: https://docs.google.com/spreadsheets/d/12_hTPovi4EH5DjT5DoDiHM0BBKulrT331YX4sDwcBvg/edit#gid=1837538205

The ARM-64 build is only successful for the following combination:

postgres version	postgis version	variant
12	2.5	default
12	3	default
13	3	default

I identified two error sources:

• arm64 packages for some postgresql-V-postgis-V are not available.
• alpine builds are failing due to an issue that I suspect being from qemu.

However the workflow is somewhat cleaner than the current setup (cf https://github.com/derdaele/docker-postgis/blob/master/.github/workflows/main.yml) as it doesn't require maintaining parts of the current Makefile.

I'm happy to start a PR with this workflow and manually include in the matrix the combinations above to have some ARM build if you are good with that.

[phillipross]

@derdaele It's actually good news that ANY builds worked on a github actions runner! It indicates that building arm64 will be possible within the CI. Good work, and thanks!

I think it's already been identified that arm64 packages on debian aren't available for some postgres+postgis combinations, but there's not much that can be done about that since that is handled upstream. I think we can simply exclude them from the build on a per-version-combination basis.

As for alpine, if memory serves, there were reports of successful builds on alpine on raspberry pis (issue #144) but they may be outdated. And I can't remember if this was for the "master branch" builds, release builds, or some other combination. There may be some more work necessary to get it working.

What would be ideal would be a prescribed (scriptable) way to install/configure qemu on an x86 ubuntu environment with buildx and any other necessary tooling to build and push arm images to dockerhub. This would allow allow us to script it on a VM, add it to the setup scripts for github actions, then write/modify github actions workflow(s) to build the images. But contributors would also be able to use the same scripts on a local environment to accomplish the same thing. This will help especially for people who want to contribute PRs without breaking the CI builds too much.

So I guess my immediate question is: would you be willing to attempt to tackle a shell script method of accomplishing docker/buildx/qemu installation... basically duplicating what github actions does with the actions/setup and/or docker buildx setup actions? Integration with the Makefile or templates is not necessary, but just something that is runnable from a shell script.

[phillipross]

@derdaele unless you've already started messing with shell scripts, I'd say it's not actually necessary. There's a little more to it than I thought, so I won't ask you to bother with it.

What I'm seeing now is that the github actions runners might be setup by default with a docker environment that is not sufficient for doing multiplatform builds, so some experimentation is needed to find a good way to get the environment setup. The github action that docker provides to do this seems to accomplish this, but it would be more beneficial to have the setup encapsulated in a script that people can use in their own VMs outside of the github actions environment. I'm in the process of working all this out now, so I won't ask you to duplicate the effort.

When do get these multiplatform images building, i'll push to a test repo and let you know so you can test on the M1 platform. I signed up for the docker development program i hopes of getting access to their M1 builds, but I'm not guaranteed to get approved and get access, so I won't be able to test for myself.

[derdaele]

Hey @phillipross, I did not start working on the shell script already.

SGTM, I'd be happy to give a try to the test images.

[kohenkatz]

I have been trying to troubleshoot the ARM64 builds failing in buildx/QEMU for 11-3.0-alpine (which just got changed to 3.1, but I don't think that matters), and I discovered something very interesting - it works perfectly when building natively on Amazon Graviton (t4g.large) and on Raspberry Pi (4B 2G running a Raspberry Pi OS 64-bit beta from August 2020) but fails in buildx/QEMU (on x86_64) with the following error:

...
#11 1042. /usr/bin/perl -pi -e 's,\$libdir,/usr/src/postgis/regress/00-regress-install/lib,g' /usr/src/postgis/regress/00-regress-install/share/contrib/postgis/*.sql
#11 1043. #/usr/bin/make -C ../loader REGRESS=1 DESTDIR=/usr/src/postgis/regress/00-regress-install install
#11 1043. /usr/bin/make -C core check
#11 1043. make[1]: Entering directory '/usr/src/postgis/regress/core'
#11 1043. /usr/bin/perl ../run_test.pl --extension ../loader/Point ../loader/PointM ../loader/PointZ ../loader/MultiPoint ../loader/MultiPointM ../loader/MultiPointZ ../loader/Arc ../loader/ArcM ../loader/ArcZ ../loader/Polygon ../loader/PolygonM ../loader/PolygonZ ../loader/TSTPolygon ../loader/TSIPolygon ../loader/TSTIPolygon ../loader/PointWithSchema ../loader/NoTransPoint ../loader/NotReallyMultiPoint ../loader/MultiToSinglePoint ../loader/ReprojectPts ../loader/ReprojectPtsD ../loader/ReprojectPtsGeog ../loader/ReprojectPtsGeogD ../loader/Latin1 ../loader/Latin1-implicit ../loader/mfile ../dumper/literalsrid ../dumper/realtable ../dumper/nullsintable ../dumper/null3d affine bestsrid binary boundary chaikin filterm cluster concave_hull ctors curvetoline dump dumppoints empty estimatedextent forcecurve geography geometric_median hausdorff in_geohash in_gml in_kml in_encodedpolyline iscollection legacy long_xact lwgeom_regress measures minimum_bounding_circle normalize operators orientation out_geometry out_geography polygonize polyhedralsurface postgis_type_name quantize_coordinates regress regress_bdpoly regress_buffer_params regress_gist_index_nd regress_index regress_index_nulls regress_management regress_selectivity regress_lrs regress_ogc regress_ogc_cover regress_ogc_prep regress_proj relate remove_repeated_points removepoint reverse setpoint simplify simplifyvw size snaptogrid split sql-mm-serialize sql-mm-circularstring sql-mm-compoundcurve sql-mm-curvepoly sql-mm-general sql-mm-multicurve sql-mm-multisurface swapordinates summary temporal temporal_knn tickets twkb typmod wkb wkt wmsservers offsetcurve relatematch isvaliddetail sharedpaths snap node unaryunion clean relate_bnr delaunaytriangles clipbybox2d subdivide voronoi regress_brin_index regress_brin_index_3d regress_brin_index_geography minimum_clearance oriented_envelope point_coordinates out_geojson frechet geos38 in_geojson regress_spgist_index_2d regress_spgist_index_3d regress_spgist_index_nd mvt mvt_jsonb geobuf
#11 1044. PATH is /usr/local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
#11 1044. Checking for shp2pgsql ... found
#11 1044. Checking for pgsql2shp ... found
#11 1044. TMPDIR is /tmp/pgis_reg
#11 1044. Creating database 'postgis_reg'
#11 1046. Preparing db 'postgis_reg' using: CREATE EXTENSION postgis
#11 1053. PostgreSQL 11.10 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
#11 1053.   Postgis 3.0.3 - r0 - 2020-12-21 06:58:19
#11 1053.   scripts 3.0.3 0
#11 1053.   GEOS: 3.8.1-CAPI-1.13.3
#11 1053.   PROJ: 7.0.1
#11 1053.
#11 1053. Running tests
#11 1053.
#11 1053.  ../loader/Point ...2020-12-21 07:15:40.054 UTC [23208] PANIC:  stuck spinlock detected at LWLockWaitListLock, lwlock.c:833
#11 1189. 2020-12-21 07:15:40.054 UTC [23208] STATEMENT:  COMMIT;
#11 1189. qemu: uncaught target signal 6 (Aborted) - core dumped
#11 1189. 2020-12-21 07:15:40.062 UTC [22616] LOG:  server process (PID 23208) was terminated by signal 6: Aborted
#11 1189. 2020-12-21 07:15:40.062 UTC [22616] DETAIL:  Failed process was running: COMMIT;
#11 1189. 2020-12-21 07:15:40.063 UTC [22616] LOG:  terminating any other active server processes
#11 1189. 2020-12-21 07:15:40.065 UTC [22628] WARNING:  terminating connection because of crash of another server process
#11 1189. 2020-12-21 07:15:40.065 UTC [22628] DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
#11 1189. 2020-12-21 07:15:40.065 UTC [22628] HINT:  In a moment you should be able to reconnect to the database and repeat your command.
#11 1189. 2020-12-21 07:15:40.065 UTC [23144] WARNING:  terminating connection because of crash of another server process
#11 1189. 2020-12-21 07:15:40.065 UTC [23144] DETAIL:  The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
#11 1189. 2020-12-21 07:15:40.065 UTC [23144] HINT:  In a moment you should be able to reconnect to the database and repeat your command.
#11 1189.  failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
#11 1189. 2020-12-21 07:15:40.066 UTC [23144] LOG:  could not send data to client: Broken pipe
#11 1189. 2020-12-21 07:15:40.073 UTC [22616] LOG:  all server processes terminated; reinitializing
#11 1189. 2020-12-21 07:15:40.098 UTC [23214] LOG:  database system was interrupted; last known up at 2020-12-21 07:13:17 UTC
#11 1189. 2020-12-21 07:15:40.128 UTC [23216] FATAL:  the database system is in recovery mode
#11 1189. psql: error: FATAL:  the database system is in recovery mode
#11 1189. Can't return outside a subroutine at ../run_test.pl line 519.
#11 1189.  failed (PostGIS object count pre-test () != post-test (8500))
#11 1189. make[1]: *** [Makefile:231: check] Error 22
#11 1189. make[1]: Leaving directory '/usr/src/postgis/regress/core'
#11 1189. make: *** [Makefile:47: check-regress] Error 2

I wonder what the significance of this is, but I don't really know enough about the PostGIS build process to have a good guess yet.

[phillipross]

I'm seeing the same thing and I believe it's something to do with QEMU. Thanks for extra datapoint though. Thus far I've been trying with an M1 running ubuntu 20.04.01 in parallels and wasn't sure if it was an issue with the parallels (it's a preview build) or something else.

As it stands, I'm able to build, test, and run natively, but trying to do cross-platform with QEMU continues breaking at lower levels that I'm not so familiar with. I've thought about trying to build QEMU from source and seeing if that makes a difference.

[notmartinnot]

Any updates on this issue?

[danilo-znamerovszkij]

docker/for-mac#5122

a comment from stephen-turner
This is a qemu bug, which is the upstream component we use for running Intel (amd64) containers on M1 (arm64) chips, and is unfortunately not something we control. In general we recommend running arm64 containers on M1 chips because (even ignoring any crashes) they will always be faster and use less memory.

Please encourage the author of this container to supply an arm64 or multi-arch image, not just an Intel one. Now that M1 is a mainstream platform, we think that most container authors will be keen to do this.

[kohenkatz]

Unfortunately, saying "it's a QEMU bug so use a multi-arch image" without providing any details is not useful, considering that docker buildx requires using QEMU to do the multi-arch build in the first place.