supabase/postgres v0.0.11 (#12)

v0.0.11
- max_replication_slots = 5
- readonly role for public schema
- plpython3
- plv8

Author: dragarcia

README.md

@@ -1,4 +1,4 @@
-# Supabase Postgres 
+# Postgres + goodies
 
 Unmodified Postgres with some useful plugins. Our goal with this repo is not to modify Postgres, but to provide some of the most common extensions with a one-click install.
 
@@ -7,12 +7,12 @@ Unmodified Postgres with some useful plugins. Our goal with this repo is not to
 - ✅ Postgres [12](https://www.postgresql.org/about/news/1976/). Includes [generated columns](https://www.postgresql.org/docs/12/ddl-generated-columns.html) and [JSON path](https://www.postgresql.org/docs/12/functions-json.html#FUNCTIONS-SQLJSON-PATH) support
 - ✅ Ubuntu 18.04 (Bionic) 
 - ✅ [pg-contrib-12](https://www.postgresql.org/docs/12/contrib.html). Because everyone should enable `pg_stat_statements`.
-- ⏳ **readonly** role. A readonly role set up by default for the public schema.
-- ✅ [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) = logical. Ready for replication.
+- ✅ **readonly** role. A readonly role set up by default for the public schema.
+- ✅ [wal_level](https://www.postgresql.org/docs/current/runtime-config-wal.html) = logical and [max_replication_slots](https://www.postgresql.org/docs/current/runtime-config-replication.html) = 5. Ready for replication.
 - ✅ [PostGIS](https://postgis.net/). Postgres' most popular extension - support for geographic objects.
 - ✅ [pgTAP](https://pgtap.org/). Unit Testing for Postgres
-- ⏳ [plv8](https://github.com/plv8/plv8) - [coming soon](https://github.com/supabase/postgres/issues/5#issuecomment-621129147). Write in Javascript functions in Postgres.
-- ⏳ [plpython3u](https://www.postgresql.org/docs/current/plpython-python23.html) - [coming soon](https://github.com/supabase/postgres/issues/5#issuecomment-621129797). Python3 enabled by default. Write in Python functions in Postgres.
+- ✅ [plv8](https://github.com/plv8/plv8) - Write in Javascript functions in Postgres.
+- ✅ [plpython3u](https://www.postgresql.org/docs/current/plpython-python23.html) - Python3 enabled by default. Write in Python functions in Postgres.
 
 ## Install
 
@@ -24,11 +24,11 @@ See all installation instructions in the [repo wiki](https://github.com/supabase
 
 ## Motivation
 
-After talking to a lot of techies, we've found that most believe Postgres is the best (operational) database but they *still* choose other databases. This is overwhelmingly because "the other one was quicker/easier". Our goal is to make it quick and simple to get started with Postgres, so that we never hear that excuse again. 
+After talking to a lot of techies, we've found that most believe Postgres is the best (operational) database but they *still* choose other databases. This is overwhelmingly because "the other one was quicker/easier". Our goal is to make it fast and simple to get started with Postgres, so that we never hear that excuse again. 
 
-Our secondary goal is to show off some of the features that are particularly exciting about Postgres to convince new developers to choose it over other database (a decision which we hope they will appreciate when they start scaling).
+Our secondary goal is to show off a few of Postgres' most exciting features. This is to convince new developers to choose it over other database (a decision we hope they'll appreciate once they start scaling).
 
-This is also the same build we offer at [Supabase](https://supabase.io), and everything we do is opensource. This repo makes it easy to *install* Postgres, Supabase makes it easy to *use* Postgres.
+Finally, this is the same build we offer at [Supabase](https://supabase.io), and everything we do is opensource. This repo makes it easy to *install* Postgres, Supabase makes it easy to *use* Postgres.
 
 ## Roadmap
 

ansible/playbook.yml

@@ -13,6 +13,23 @@
   tasks:
     - include_tasks: tasks/setup-extensions.yml
 
+    - name: Dump SQL script
+      copy:
+        dest: /tmp/00-schema.sql
+        src: ../docker/mnt/00-schema.sql
+
+    - name: Set up readonly user for the public schema
+      become: yes
+      become_user: postgres
+      postgresql_query:
+        db: postgres
+        path_to_script: /tmp/00-schema.sql
+
+    - name: Delete SQL script
+      file:
+        path: /tmp/00-schema.sql
+        state: absent
+
     - name: Set up password for superadmin postgres
       become: yes
       become_user: postgres

ansible/tasks/setup-extensions.yml

@@ -18,7 +18,7 @@
     cache_valid_time: 3600
   when: postgresql_version >= 10
 
-- name: pgTAP - download latest releas
+- name: pgTAP - download latest release
   get_url:
     url: "https://github.com/theory/pgtap/archive/{{ pgtap_release }}.tar.gz"
     dest: /tmp
@@ -35,4 +35,66 @@
   make:
     chdir: /tmp/pgtap-1.1.0
     target: install
-  become: yes
+  become: yes
+
+- name: Install plpython
+  apt:
+    pkg: postgresql-plpython3-12
+    update_cache: yes
+    cache_valid_time: 3600
+
+- name: plv8 - download & install dependencies
+  apt:
+    pkg:
+      - build-essential
+      - ca-certificates
+      - curl
+      - git-core
+      - python
+      - gpp
+      - cpp
+      - pkg-config
+      - apt-transport-https
+      - cmake
+      - libc++-dev
+      - libc++abi-dev
+      - postgresql-server-dev-12
+      - libc++1
+      - libtinfo5
+      - libc++abi1
+    update_cache: yes
+    install_recommends: no
+
+- name: plv8 - download latest release
+  git:
+    repo: https://github.com/plv8/plv8.git
+    dest: /tmp/plv8
+    version: r3.0alpha
+  become: yes
+
+- name: plv8 - build
+  make: 
+    chdir: /tmp/plv8
+  become: yes
+
+- name: plv8 - install
+  make:
+    chdir: /tmp/plv8
+    target: install
+  become: yes
+
+- name: plv8 - remove build dependencies
+  apt:
+    pkg:
+      - ca-certificates
+      - curl
+      - git-core
+      - python
+      - gpp
+      - cpp
+      - pkg-config
+      - apt-transport-https
+      - cmake
+      - libc++-dev
+      - libc++abi-dev
+      - postgresql-server-dev-12

ansible/tasks/setup-system.yml

@@ -4,15 +4,9 @@
   apt: update_cache=yes upgrade=yes
   # SEE http://archive.vn/DKJjs#parameter-upgrade
 
-- name: add universe repository for bionic
-  apt_repository: 
-    repo: deb http://archive.ubuntu.com/ubuntu bionic universe
-    state: present
-
 - name: Install essentials
   apt:
     pkg:
-      - build-essential
       - ufw
     update_cache: yes
     cache_valid_time: 3600

ansible/vars.yml

@@ -1,5 +1,7 @@
 postgresql_version: 12
 postgresql_wal_level: "logical"
+postgresql_max_wal_senders: 10
+postgresql_max_replication_slots: 5
 postgresql_listen_addresses:
   - "*"
 

digitalOcean.json

@@ -8,9 +8,9 @@
     "api_token": "{{user `do_token`}}",
     "image": "ubuntu-18-04-x64",
     "region": "{{user `region`}}",
-    "size": "512mb",
+    "size": "s-1vcpu-1gb",
     "ssh_username": "root",
-    "snapshot_name": "supabase-postgresql-0.0.8"
+    "snapshot_name": "supabase-postgresql-0.0.11"
   }],
   "provisioners": [
     {

docker/Dockerfile

@@ -0,0 +1,61 @@
+FROM postgres:12
+
+# install postgis
+ENV POSTGIS_MAJOR 3
+ENV POSTGIS_VERSION 3.0.0+dfsg-2~exp1.pgdg100+1
+RUN apt-get update \
+      && apt-cache showpkg postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR \
+      && apt-get install -y --no-install-recommends \
+           postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR \
+           postgresql-$PG_MAJOR-postgis-$POSTGIS_MAJOR-scripts \
+      && apt-get install software-properties-common -y \
+      && apt-get install git -y \
+      && apt-get install build-essential -y \
+      && rm -rf /var/lib/apt/lists/*
+
+# install pgtap
+ENV PGTAP_VERSION v1.1.0
+RUN git clone git://github.com/theory/pgtap.git \
+    && cd pgtap && git checkout tags/$PGTAP_VERSION \
+    && make install
+
+# install plpython3
+RUN apt-get update \ 
+      && apt-get install postgresql-plpython3-12 -y
+
+# install plv8
+ENV PLV8_VERSION=r3.0alpha
+
+RUN buildDependencies="build-essential \
+    ca-certificates \
+    curl \
+    git-core \
+    python \
+    gpp \
+    cpp \
+    pkg-config \
+    apt-transport-https \
+    cmake \
+    libc++-dev \
+    libc++abi-dev \
+    postgresql-server-dev-$PG_MAJOR" \
+  && runtimeDependencies="libc++1 \
+    libtinfo5 \
+    libc++abi1" \
+  && apt-get update \
+  && apt-get install -y --no-install-recommends ${buildDependencies} ${runtimeDependencies} \
+  && mkdir -p /tmp/build \
+  && cd /tmp/build \
+  && git clone https://github.com/plv8/plv8.git \
+  && cd plv8 \
+  && git checkout ${PLV8_VERSION} \
+  && make static \
+  && make install \
+  && rm -rf /root/.vpython_cipd_cache /root/.vpython-root \
+  && apt-get clean \
+  && apt-get remove -y ${buildDependencies} \
+  && apt-get autoremove -y \
+  && rm -rf /tmp/build /var/lib/apt/lists/*
+
+RUN mkdir -p /docker-entrypoint-initdb.d
+ADD ./mnt /docker-entrypoint-initdb.d/

docker/docker-compose.yml

@@ -0,0 +1,9 @@
+version: '3'
+
+services:
+  db:
+    image: supabase/postgres
+    ports:
+      - "6543:5432"
+    environment:
+      POSTGRES_PASSWORD: postgres

docker/mnt/00-schema.sql

@@ -0,0 +1,14 @@
+-- Strip everyone on rights to the public schema except for the user postgres
+REVOKE ALL ON schema public FROM public;
+GRANT ALL ON schema public TO postgres;
+
+
+-- Provide read only access to the schema and its current content
+CREATE ROLE public_readonly;
+GRANT CONNECT ON DATABASE postgres TO public_readonly;
+GRANT USAGE ON SCHEMA public TO public_readonly;
+GRANT SELECT ON ALL TABLES IN SCHEMA public TO public_readonly;
+
+-- Provide read only access to future tables in the schema
+ALTER DEFAULT PRIVILEGES IN SCHEMA public
+GRANT SELECT ON TABLES TO public_readonly;

docker/mnt/init-permissions.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+set -e
+
+echo "host replication $POSTGRES_USER 0.0.0.0/0 trust" >> $PGDATA/pg_hba.conf
+echo "shared_preload_libraries = 'pg_stat_statements'" >> $PGDATA/postgresql.conf
+echo "pg_stat_statements.max = 10000" >> $PGDATA/postgresql.conf
+echo "pg_stat_statements.track = all" >> $PGDATA/postgresql.conf
+echo "wal_level=logical" >> $PGDATA/postgresql.conf
+echo "max_replication_slots=5" >> $PGDATA/postgresql.conf
+echo "max_wal_senders=10" >> $PGDATA/postgresql.conf