feat(example): sport tracker app initial commit#6
Merged
danielebriggi merged 2 commits intomainfrom Jul 28, 2025
Merged
Conversation
danielebriggi
force-pushed
the
sport-tracker-app-example
branch
from
f5b6f88 to
9861622
Compare
danielebriggi
force-pushed
the
sport-tracker-app-example
branch
3 times, most recently
from
6aa925d to
60b11ac
Compare
danielebriggi
force-pushed
the
sport-tracker-app-example
branch
from
60b11ac to
145b548
Compare
Gioee
added a commit
that referenced
this pull request
Mar 24, 2026
…cloudsync_network_init to use managedDatabaseId and sync functions to return JSON # Changelog All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ## [1.0.0] - 2026-03-24 ### Added - **PostgreSQL support**: The CloudSync extension can now be built and loaded on PostgreSQL, so both SQLiteCloud and PostgreSQL are supported as the cloud backend database of the sync service. The core CRDT functions are shared by the SQLite and PostgreSQL extensions. Includes support for PostgreSQL-native types (UUID primary keys, composite PKs with mixed types, and automatic type casting). - **Row-Level Security (RLS)**: Sync payloads are now fully compatible with SQLiteCloud and PostgreSQL Row-Level Security policies. Changes are buffered per primary key and flushed as complete rows, so RLS policies can evaluate all columns at once. - **Block-level LWW for text conflict resolution**: Text columns can now be tracked at block level (lines by default) using Last-Writer-Wins. Concurrent edits to different parts of the same text are preserved after sync. New functions: `cloudsync_set_column()` to write individual blocks and `cloudsync_text_materialize()` to reconstruct the full text. ### Changed - **BREAKING: `cloudsync_network_init` now accepts a `managedDatabaseId` instead of a connection string.** The `managedDatabaseId` is returned by the CloudSync service when a new database is registered for sync. For SQLiteCloud projects, it can be obtained from the project's OffSync page on the dashboard. Before: ```sql SELECT cloudsync_network_init('sqlitecloud://myproject.sqlite.cloud:8860/mydb.sqlite?apikey=KEY'); ``` After: ```sql SELECT cloudsync_network_init('your-managed-database-id'); ``` - **BREAKING: Sync functions now return structured JSON.** `cloudsync_network_send_changes`, `cloudsync_network_check_changes`, and `cloudsync_network_sync` return a JSON object instead of a plain integer. This provides richer status information including sync state, version numbers, row counts, and affected table names. Before: ```sql SELECT cloudsync_network_sync(); -- 3 (number of rows received) ``` After: ```sql SELECT cloudsync_network_sync(); -- '{"send":{"status":"synced","localVersion":5,"serverVersion":5},"receive":{"rows":3,"tables":["tasks"]}}' ``` - **Batch merge replaces column-by-column processing**: During sync, changes to the same row are now applied in a single SQL statement instead of one statement per column. This eliminates the previous behavior where UPDATE triggers fired multiple times per row during synchronization. - **Network endpoints updated for the CloudSync v2 HTTP service**: Internal network layer now targets the new CloudSync service endpoints, including support for multi-organization routing. - **NULL primary key rejection at runtime**: The extension now enforces NULL primary key rejection at runtime, so the explicit `NOT NULL` constraint on primary key columns is no longer a schema requirement. ### Fixed - **Improved error reporting**: Sync network functions now surface the actual server error message instead of generic error codes. - **Schema hash verification**: Normalized schema comparison now uses only column name (lowercase), type (SQLite affinity), and primary key flag, preventing false mismatches caused by formatting differences. - **SQLite trigger safety**: Internal functions used inside triggers are now marked with `SQLITE_INNOCUOUS`, fixing `unsafe use of` errors when initializing tables that have triggers. - **NULL column binding**: Column value parameters are now correctly bound even when NULL, preventing sync failures on rows with NULL values. - **Stability and reliability improvements** across the SQLite and PostgreSQL codebases, including fixes to memory management, error handling, and CRDT version tracking. # Commit History * Added support for Postgres database (#1) * New architecture WP 1 * New architecture WP 2 * WIP 3 * Optimized cloudsync_table_context interaction * New architecture WP 4 * Refactored begin/commit ALTER * New architecture WP 6 * New architecture (WP 5) * Minor changes * New architecture WP 6 * New architecture WP 7 * Small compilation issue fixed * New architecture WP 9 * fix: minor compilation issue * fix: database_* functions must call cloudsync_memory_* functions instead of direct sqlite3_* memory function to support the memory debugger module call cloudsync_memory_mprintf and cloudsync_memory_free instead of direct sqlite3_mprintf and sqlite3_free memory functions from database_create_insert_trigger otherwise the memory debugger would report "Pointer being freed was not previously allocated." * fix(memdebug): fix pointer returned by memdebug_zeroalloc * fix(unittest): avoid a memory leak from do_test_dbutils * test: add unittest target to Makefile Introduces a 'unittest' target to run only unit tests and updates the help output accordingly. * Update unit.c * Refactored SQL in dbutils (related to settings) * Updated static statements in cloudsync.c (WP 1) * Replaced name escape with a function (WP 2) * Completed SQL refactoring * fix: minor compilation issue * test: re-add integration test use `make unittest` to run only the unittest * fix(network): fix the network layer after database-api refactoring * Several compilation warnings fixed * Cleaned-up 64bit types * fix(lcov): exclude sql_sqlite.c from code coverage, it just contains query strings * Update cloudsync_sqlite.c * Use int64_t for version variables in network.c Replaces sqlite3_int64 with int64_t for new_db_version and new_seq variables to standardize integer type usage and improve portability. Was giving a compile error on linux musl * Update vtab.c * chore: remove warnings * Update vtab.c * test: fix compile errors on linux musl * Replaced all %lld (except one) * fix: minor compilation error * refactor: new directory structure to separate multi-platform code from database-specific implementations Refactor the codebase to separate multi-platform code from database-specific implementations, preparing for PostgreSQL extension development. vtab.c/h has been renamed to sqlite/cloudsync_changes_sqlite.c/h * fix(workflow): add CFLAGS for CURL Android builds and clean up Android build files * fix(android): add -fPIC to CFLAGS * fix(android): use OpenSSL specific version * fix(android): update OpenSSL install path to a local one, instead of a system wide path * db_version must be int64_t in network.c * fix: avoid crash on postgres when these functions are called with NULL values for db and data * improved error logs, fix some debug messages * New postgresql extension WIP 1 * Updated SQL_DATA_VERSION and added a new sql_build_select_nonpk_by_pk function to database.h * fix: remove obsolete property * chore * fix: improved SQL queries (WIP) * implement SQL_SCHEMA_VERSION with app_schema_version table and event trigger * fix: fix PG SQL queries used in cloudsync_init * fix: avoid a segfault crash in cloudsync_init Allocate in TopMemoryContext to survive SPI cleanup * test: calling twice the cloudsync_init function on postgresql is failing (WIP) * Code simplification and memory cleanup (wp) * Minor fixes * test: add debug PostgreSQL devcontainer and Docker setup Introduces a VS Code devcontainer configuration for PostgreSQL development with CloudSync, including a debug Dockerfile, docker-compose file, and Makefile changes to support debug builds. This setup enables easier debugging and development of the CloudSync extension for PostgreSQL in a containerized environment. * chore: update docker/README.md with VS Code Dev Container Debugging instructions * test: add .vscode/launch.json with the "Attach to Postgres (gdb)" configuration * Added more explicit string_dup functions * Fixed network * Error returned by sqlite3_extension_init function must be dynamically allocated * test: add debug symbols and src code of postgresql server to dev container * fix: add SPI_connect and SPI_finish to _PG_init function * dbutils.c removed db_t * Minor changes * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) * Refactoring (wp) Removed cloudsync_private.h and db_t * Refactoring (pg wp) * Various PostgreSQL fixes * Refactoring PG code * chore: minor fixes for when debug macros are enabled * fix: use global memory when the DBFLAG_PERSISTENT flag is set to avoid crash for double free * fix(cloudsync): guard against errmsg aliasing in error formatting It was fixed by snapshotting db_error into a local buffer when it aliases data->errmsg, then formatting from the copy. This avoids undefined behavior. * fix(postgresql): fix error message in cloudsync_init_internal, the error message must be copied before database_rollback_savepoint reset the error message * fix(postgresql): skip SPI_cursor_open for non-cursorable plans - avoid trying to open a cursor on INSERT/UPDATE/DELETE plans without RETURNING - use SPI_is_cursor_plan to decide whether to use a portal or execute once - fixes “cannot open INSERT query as cursor” during cloudsync_init paths (for example with SQL_INSERT_SITE_ID_ROWID) * Minor fixes * Improved SPI’s memory ownership rule and prevents accumulation across large result sets * dbmem apis now use pg native memory functions * More memory related fixes * Apparently repalloc doesn't like a NULL ptr * Update database_postgresql.c * get_cloudsync_context must be allocated in a global context * Revert "get_cloudsync_context must be allocated in a global context" This reverts commit 4e614f1. * Revert "Update database_postgresql.c" This reverts commit 13367c1. * Revert "Apparently repalloc doesn't like a NULL ptr" This reverts commit ffd587a. * Revert "More memory related fixes" This reverts commit f44c161. * Revert "dbmem apis now use pg native memory functions" This reverts commit 4523e42. * Several memory related issues fixed * Fixed memory allocations in PG BLOB functions * pgvalue_vec_push can fails (it now return a bool) * Improved memory handling for dbvalue_t * fix(sql_postgresql): fix placeholder for cloudsync_memory_mprintf in SQL_PRAGMA_TABLEINFO_LIST_NONPK_NAME_CID * fix(database_postgresql): refactor error handling in PostgreSQL database functions to always call PG_END_TRY, this fix a SIGSEGV error for corrupted stack * fix(dbutils): dbutils_table_settings_get_value and dbutils_settings_get_value must return NULL in case no rows or error Caller code can check if the return value is NULL or not * fix: remove unnecessary switch to top memory context for text_to_cstring * fix: if databasevm_bind_text size argurmnt is negative, then the length of the string is the number of bytes up to the first zero terminator * refactor(pgvalue): always alloc values owned by pgvalue struct in the PG memory context to simplify the memory management * chore * chore: remove unused SQL queries from sql_postgresql.c * fix(sql_postgresql): fix SQL_PRAGMA_TABLEINFO_LIST_NONPK_NAME_CID * fix(sql_postgresql): SQL_PRAGMA_TABLEINFO_LIST_NONPK_NAME_CID * fix(postgresql): implement triggers and functions called by triggers, fix metatable's schemas, fix cloudsync_pk_encode/decode functions (use bytea instead of text for pk col values) * test(pg smoke test): add tests for cloudsync_pk_encode and for insert/delete triggers and for the content of the metatable * Minor changes * Payload encoding sanity check added * Improved dbutils_settings_get_value * fix(cloudsync_postgresql): cloudsync_init returns the site_id as bytea type * chore * fix(postgresql): lazy-init cloudsync context per call Avoid relcache/snapshot leaks during CREATE EXTENSION by moving SPI-dependent init to normal function calls. Add cloudsync_pg_ensure_initialized helper, drop SPI work from _PG_init, and wire initialization into SQL entry points so context loads on demand. * Improved dbutils_table_settings_get_value * fix: solve a compile error. cloudsync_context is opaque in this compilation unit, so data->site_id isn’t accessible, use the public accessor instead. * feat(postgresql): implement cloudsync_update * chore * fix(postgresql): fix the PG_TRY/PG_CATCH exception stack, must not return inside PG_TRY block * Various improvements to encoding/decoding functions * Improved cloudsync_pg_context_init * Better network memory management * Implemented cloudsync_changes * fix: sql_build_rekey_pk_and_reset_version_except_col has different parameters for sqlite and postgresql * fix(cloudsync_postgresql.c): fix implementation of cloudsync_update aggregate function and cloudsync_delete * ci: add new target to build and run the debug version for vscode and the standalone asan version * fix(cloudsync_postgresql): fix memory context for cloudsync_changes (read) and change the col_value type to bytea to store pk_encoded value preserve SQLite-compatible payloads by encoding `col_value` with the same pk wire format before it reaches the SRF/view layer. With the bytea value for col_value we can reuse the same existing columns from the sqlite extension to encode/decode the type of the value and the value itself, and reuse the same query `SELECT cloudsync_payload_encode(tbl, pk, col_name, col_value, ...) FROM cloudsync_changes`, otherwise we should add a new column for the type and use a cast to the specific type * Added new pk_encode_value * fix(sql_postgresql): fix SQL_BUILD_UPSERT_PK_AND_COL query used for col_merge_stmt * add a skip_decode_idx argument to pk_decode, used in postgresql * fix(cloudsync_postgresql): fix cloudsync_changes_insert_trg to use col_value as bytea with the pk encoded value * test: add read and write tests for cloudsync_changes * Renamed PG changes functions and removed unused variable * No real changes * Improved cloudsync_changes SELECT * Updated cloudsync_changes (wp) * Finished implementing cloudsync_changes * fix(pk): the original blob value for the undecoded col_value for the skipped column was missing the first type byte * fix(pk): add the skip_idx argument for the pk_encode and pk_encode_size just like I already added to pk_decode, needed by cloudsync_payload_encode on postgresql * fix(cloudsync): fix the buffer len value (blen) after decompressing a compressed payload * fix(sql_postgresql): fix the SQL_CLOUDSYNC_UPSERT_RAW_COLVERSION for postgresql * fix(sql_postgresql): fix placeholder from ? to $<n> notation for postgresql * refactor(postgresql): add pgvalue_free function to make it clear how to free pgvalue object from internal functions * chore * fix(postgresql/coydsync--1.0.sql): fix arguments for cloudsync_payload_encode aggregate function * Update cloudsync--1.0.sql * fix(postgresql): fix cloudsync_changes_insert_trigger for TOMBSTONE rows * fix(postgresql): trying to fix relcache/plancache/snapshot leaks that occurs when an exception is thrown and catched inside cloudsync_changes_insert_trigger (WIP) * test(postgresql/smoke_test): add a test for payload roundtrip to another database * test: minor changes to smoke_test.sql * Added bounds check to pk and checksum to payload * test: update include directive for integration test when run with CLOUDSYNC_LOAD_FROM_SOURCES from Xcode project * Checksum is checked only if payload version is >= 2 * Fixed compilation issue * fix(android): renamed endian.h to cloudsync_endian.h to avoid android ndk clang to have conflicts with sys/endian.h * ci: update the dockerfile configurations to use postgresql 17 instead of 16, the same version used by supabase * build(supabase): build a custom supabase/postgres:17.6.1.071 docker image to be used from the `supabase start` stack * test(postgres/smoke_test): update the test to create different databases to simulate different peers * Several issues fixed and optimizations added (reported by Claude) * skip the schema hash check for now, we cannot compare the hash between sqlite and postgres like we were doing for sqlite * feat(network)!: support the v2 endpoints exposed by the new sqlite-sync-server * fix: free SPI_tuptable (if exists) after each invocation of SPI_execute to avoid memory leaks and to optimize memory usage * fix: update the return type for the cloudsync_payload_apply function, it returns the number of applied rows * Update database_postgresql.c * fix(supabase): prevent a unhealthy status during the restart of the supabase stack The error occurs because the event trigger function inserts into app_schema_version without schema qualification, failing due to missing table in the search_path used by Supabase realtime (which connects to the "postgres" database with a different schema context). Always create app_schema_version in public and updating the function’s insert statement to reference public.app_schema_version * docs(docker/README.md): added a troubleshooting note about the app_schema_version/Realtime migration error * Several memory related issues fixed * fix(network): token size when calling cloudsync_network_set_token before cloudsync_network_init, if the token was greater that 256 chars it was truncated * fix: bind null values for col_value column in INSERT INTO cloudsync_changes with type bytea to avoid "failed to find conversion function from unknown to bytea" error * fix(postgresql): return raw column in SQL_BUILD_SELECT_COLS_BY_PK_FMT instead of the encoded value This query is used during merge conflict resolution to compare local values against incoming changes. Returning encoded bytea caused type mismatches and order-dependent winners in multi-db tests, failing 03_3db_multiple_roundtrip.sql. * test(postgresql): move the smoke test to the test/postgresql dir Also split the smoke test into different test files, all these test files are called by smoke_test.sql * bump version * test(postgresql): new multi-db tests * chore: add docs with analysis on the open issues * test(postgresql): make 02_roundript.sql test executable as a standalone test * Update ISSUE_WARNING_resource_was_not_closed.md * Added support for schema * release wip-pg-extension branch node and expo packages to npmjs with the "pg" tag * renamed workflows for OIDC publishing issues, to revert before merging to main * Added new database_internal_table_exists function to make sure to check for system tables in the public schema (PG only) * fix(workflow): node packages to use pg tagged version * fix(packages/node): wrong fat binary artifact folder * Bump version to 0.9.63 * fix(database_postgresql): fix database_select1_value to make it work with text result of type name (for example the result of SELECT current_schema();) * fix(cloudsync_postgresql): only free quoted identifiers if they're different from the input * test(postgresql): add tests for multi-schema scenario * fix(postgresql): prevent duplicate primary keys when tables exist in multiple schemas When a table name exists in multiple schemas (e.g., public.users and auth.users), SQL queries joining information_schema.table_constraints with information_schema.key_column_usage were returning duplicate primary key columns. Solution: Added "AND tc.table_schema = kcu.table_schema" to all JOIN conditions to ensure primary key information is only retrieved from the target schema specified by cloudsync_schema() or current_schema(). * fix(cloudsync): avoid a crash when setting the cloudsync_set_schema to the the same previous pointer * test(postgresql): improved tests * Added new define for schema literal * fix: skip the decode step regardless of the data type (for the col idx specified by skip_decode_idx) we still need to parse the value depending on the data type to increment the bseek value for the next loop * Several minor issues fixed * Several other issues fixed * Update network.m * fix: preserve prepared plan across databasevm_reset() in PostgreSQL backend Previously, databasevm_reset() called databasevm_clear_bindings() which destroyed the SPIPlanPtr on every reset, forcing a full SPI_prepare on each bind/step cycle. This negated the benefit of caching statements in cloudsync_table_context. Now reset() only clears parameter values while keeping the plan, types, and nparams intact for reuse. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * Added new CLOUDSYNC_CHANGES_NCOLS constant * database_value_text returns NULL also in PostgreSQL implementation * fix(postgres): fix current memory context to avoid crashes on PG_CATCH, the CopyErrorData func must not be called from a error context * test(postgres): add a test similar to the sport_tracker app * Fixed allocation in value returned after SPI_finish * Updated databasevm_step0, databasevm_step and databasevm_clear_bindings. Fixed warnings in test (due to uint64_t usage in a signed BIGINT) * Fix for 11_multi_table_rounds.sql * Several minor issues fixed * fix(packages/node): broken dynamic import for platform specific package in ESM * test(postgresql): improved test for multi_table_multi_columns_rounds and move the test for repeated_table on multiple schemas to a separated test file * Fixed cloudsync_network_logout * fix(network): cleanup the network configuration during network logout * fix: use the correct schema for previously initialized tables on new connections to the database * test(postgres): build the debug image with no-optimization flag * Fixed some issues related to escaping * Quoting and memory issues fixed Several quoting issues fixed. Added pfree(elems) and pfree(nulls) after the loop to free memory allocated by deconstruct_array. Moved CStringGetTextDatum allocations before PG_TRY and pfree calls after PG_END_TRY. A need_schema_param flag determines whether 1 or 2 Datums are allocated/freed. This ensures the Datum memory is cleaned up on both the success path and the PG_CATCH error path. * Removed unused files * Delete .github/workflows/main.yml * Rename rename_to_main_before_merge_to_main_branch.yml to main.yml --------- Co-authored-by: Andrea Donetti <andinux@gmail.com> Co-authored-by: Gioele Cantoni <gioele.cantoni@gmail.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> * fix(makefile): skip integration test * enable release job to publish on dev tag or -dev naming * fix(release): change android package name * fix(workflow): skip gh pages deploy * Bump version to 0.9.91 * Remove unused file * build(postgres): add SUPABASE_POSTGRES_TAG support for image builds Introduces the SUPABASE_POSTGRES_TAG variable to allow explicit control over the Supabase Postgres base image tag in Docker builds. Updates the Makefile, Dockerfile, and documentation to support and document this new build argument, improving flexibility for building custom images. * test(postgres): improved tests * fix(packages): change npmjs package names to *-dev * fix(packages/expo): npmjs sigstore provenance bundle error * fix(workflow): replace npm token with oidc auth * Update README.md * fix(postgres): support mixed-type composite primary keys with VARIADIC "any" Changed PostgreSQL functions from VARIADIC anyarray to VARIADIC "any" to support composite primary keys with mixed data types (e.g., TEXT + INTEGER). PostgreSQL arrays require all elements to be the same type, causing errors like "function cloudsync_insert(unknown, text, integer) does not exist" for heterogeneous composite keys. VARIADIC "any" accepts arguments of any type. This ensures pk_encode_prikey() receives correct type information for encoding, maintaining compatibility with SQLite. * test(postgres): add tests for unmapped types and composite PK roundtrip Added 15_datatype_roundtrip_unmapped.sql to test roundtrip encoding/decoding of unmapped PostgreSQL types (JSONB, UUID, INET, CIDR, RANGE) and 16_composite_pk_text_int_roundtrip.sql to test roundtrip and bidirectional sync with composite primary keys mixing TEXT and INTEGER. Updated full_test.sql to include the new composite PK test. 15_datatype_roundtrip_unmapped.sql still fails, so it is not yet included to full_test.sql * Bump version to 0.9.95 to force sqlite-wasm rebuild * Revert last commit * Added support for non explicitly mapped PG type * Fixed SQLite unit test * Update CloudSyncSetup.js * featpostgres): allow to use any token * test(postgres): minor changes * Added PG documentation * Update README.md * Update CLIENT.md * Update "Conversion Between SQLite and PostgreSQL Tables" in docs/postgresql/CLIENT.md * chore: typos readme * fix(postgres): return uuid type from cloudsync_uuid() for cross-database sync * update docs/postgresql markdowns (#2) * add new @sqliteai/sqlite-sync-react-native library to the release job and to the docs * Bump version to 0.9.98 * Bump version to 0.9.99 * feat: support to https connection string and JWT token * chore: update .env.example * feat: add support for UUID primary keys in PG (#3) * Fix for PG UUID used as PK * build(postgres): update test target for the current test files * docs(docs/postgresql/CLIENT.md): clarify primary key requirements for PostgreSQL and SQLite, added support for UUID primary keys * test(claude): add custom command for claude code to run sqlite-to-pg tests for the specified table schema --------- Co-authored-by: Marco Bambini <marco@creolabs.com> * refactor(postgres): use palloc in TopMemoryContext for memory allocation Replace malloc/free with palloc/pfree in the PostgreSQL dbmem_* abstraction layer. This integrates cloudsync memory management with PostgreSQL's memory context system. Benefits: - Memory tracking via MemoryContextStats() for debugging - Automatic cleanup when PostgreSQL connection terminates - Consistent memory management across the extension - Respects PostgreSQL memory configuration and limits * Removed some duplicated code * perf(postgres): use SQL template casting for non-PK column types (#4) * perf(postgres): use SQL template casting for non-PK column types Refactor type conversion for non-PK columns to use SQL template casting ($n::typename) instead of per-value SPI_execute_with_args("SELECT $1::typename") calls. This matches the pattern already used for primary key columns. Changes: - sql_postgresql.c: Add format_type() lookup and $n::coltype casting to SQL_BUILD_UPSERT_PK_AND_COL for non-PK column values - cloudsync_postgresql.c: Simplify cloudsync_decode_bytea_to_pgvalue() to return base types only (INT8, FLOAT8, TEXT, BYTEA) without SPI casting - cloudsync_postgresql.c: Remove lookup_column_type_oid() function - Add test 19 (19_uuid_pk_with_unmapped_cols.sql) covering UUID PK with unmapped non-PK types (JSONB, UUID, INET, CIDR) and all CRUD operations: INSERT, UPDATE non-PK, UPDATE mapped cols, DELETE, RESURRECT, UPDATE PK Performance benefit: Eliminates one SPI_execute call per non-PK column value during payload application, reducing overhead for unmapped PostgreSQL types. * test(postgres): new tests * fix(workflow): enable github pages * fix README coverage badge * Increased unit testing and code coverage * fix(examples/to-do-app): missing dependencies, upgrade react-native, fix Android to use different connection string for localhost usage and replaced old cloudsync_init * apis with new one with explicit table names * fix(examples/to-do-app): migrate to new icon package * fix(examples/to-do-app): missing material-design-icons package * Bump examples/to-do-app version * Add EXPO example to docs/postgresql * fix(examples/to-do-app): iOS manually add icons fonts to Info.plist * Update EXPO.md * fix(examples/to-do-app): create 'Work' and 'Personal' tags without randomUUID * Update EXPO.md * fix(schema_hash): build normalized schema string using only column name (lowercase), type (SQLite affinity), pk flag (#5) Build normalized schema string using only: column name (lowercase), type (SQLite affinity), pk flag Format: tablename:colname:affinity:pk,... (ordered by table name, then column id). This makes the hash resilient to formatting, quoting, case differences and portable across databases. * Fix/bind column value parameters also if null (#6) * fix(sync): always bind column value parameters in merge_insert_col Fix parameter binding bug in merge_insert_col that caused SQLite-to-PostgreSQL sync to fail with "there is no parameter $3" when NULL values were synced before non-NULL values for the same column. * fix(network): fix the value of the seq variable in cloudsync_payload_get when the last db_version is not related to a local change * test(postgres): new test for null value * fix(postgres): ensure NULL values use consistent decoded types for SPI plan caching When syncing NULL values first, PostgreSQL's SPI caches the prepared plan with the NULL's type. If a subsequent non-NULL value decodes to a different type, the plan fails. The fix maps column types to their decoded equivalents so NULL and non-NULL values always use consistent types (e.g., all integers use INT8OID, all floats use FLOAT8OID, most others use TEXTOID). Add map_column_oid_to_decoded_oid() to map column types to their decoded equivalents (INT2/4/8 → INT8, FLOAT4/8/NUMERIC → FLOAT8, BYTEA → BYTEA, others → TEXT). This ensures NULL and non-NULL values bind with the same type, preventing "there is no parameter $N" errors when NULL is synced before non-NULL values for the same column. Add tests 23 and 24 for UUID columns and comprehensive nullable type coverage (INT2/4/8, FLOAT4/8, NUMERIC, BYTEA, TEXT, VARCHAR, CHAR, UUID, JSON, JSONB, DATE, TIMESTAMP). * fix(postgres): add bigint to boolean cast for BOOLEAN column sync BOOLEAN values are encoded as INT8 in sync payloads for SQLite interoperability, but PostgreSQL has no built-in cast from bigint to boolean. Add a custom ASSIGNMENT cast that enables BOOLEAN columns to sync correctly. The cast uses ASSIGNMENT context (not IMPLICIT) to avoid unintended conversions in WHERE clauses while still enabling INSERT/UPDATE operations used by merge_insert. The write direction (BOOL → INT encoding flow) "just works" because DatumGetBool() naturally returns 0 or 1. The problem was only on the read side where PostgreSQL refused to cast the decoded INT8 back to BOOLEAN without our custom cast. * feat(commands): add sync roundtrip RLS test guide * bump version * ci: add postgres-test job to main workflow * ci(Makefile.postgresql): replaced all docker-compose commands with docker compose (v2 plugin syntax) Fix the execution in the github actions runner: The GitHub Actions runner has docker compose (v2 plugin) but not the standalone docker-compose (v1) * ci: fix the "run postgresql tests" step * Added the ability to perform a perform a sync only if a column expression is satisfied (#7) * Added the ability to perform a perform a sync only if a column expression is satisfied --------- Co-authored-by: Marco Bambini <marco@creolabs.com> * Fix 35 bugs and bump version to 0.9.111 (#9) * Fix 35 bugs across CloudSync SQLite/PostgreSQL sync extension Comprehensive audit identified and fixed 35 bugs (1 CRITICAL, 7 HIGH, 18 MEDIUM, 9 LOW) across the entire codebase. All 84 SQLite tests and 26 PostgreSQL tests pass with 0 failures and 0 memory leaks. ## src/cloudsync.c (13 fixes) - [HIGH] Guard NULL db_version_stmt in cloudsync_dbversion_rerun — set db_version = CLOUDSYNC_MIN_DB_VERSION and return 0 when stmt is NULL, preventing NULL dereference after schema rebuild failure - [MEDIUM] Add early return for NULL stmt in dbvm_execute to prevent crash when called with uninitialized statement pointer - [MEDIUM] Change (bool)dbvm_count() to (dbvm_count() > 0) in table_pk_exists — prevents negative return values being cast to true, giving false positive "pk exists" results - [MEDIUM] Add NULL check on database_column_text result in cloudsync_refill_metatable before calling strlen — prevents crash on corrupted or empty column data - [MEDIUM] Route early returns in cloudsync_payload_apply through goto cleanup so CLEANUP callback and vm finalize always run — prevents resource leaks and callback contract violation - [MEDIUM] Change return false to goto abort_add_table when ROWIDONLY rejected — ensures table_free runs on the partially allocated table, preventing memory leak - [MEDIUM] Initialize *persistent = false at top of cloudsync_colvalue_stmt — prevents use of uninitialized value when table_lookup returns NULL - [LOW] Add NULL check on database_column_blob in merge_did_cid_win — prevents memcmp with NULL pointer on corrupted cloudsync table - [LOW] Handle partial failure in table_add_to_context_cb — clean up col_name, col_merge_stmt, col_value_stmt at index on error instead of leaving dangling pointers - [LOW] Remove unused pragma_checked field from cloudsync_context - [LOW] Change pointer comparison to strcmp in cloudsync_set_schema — pointer equality missed cases where different string pointers had identical content - [LOW] Fix cloudsync_payload_get NULL check: blob == NULL (always false for char** arg) changed to *blob == NULL - [LOW] Pass extra meta_ref args to SQL_CLOUDSYNC_UPSERT_COL_INIT_OR_BUMP_VERSION mprintf call to match updated PostgreSQL format string ## src/sqlite/cloudsync_sqlite.c (5 fixes) - [HIGH] Split DEFAULT_FLAGS into FLAGS_PURE (SQLITE_UTF8 | SQLITE_INNOCUOUS | SQLITE_DETERMINISTIC) and FLAGS_VOLATILE (SQLITE_UTF8). Pure functions: cloudsync_version, cloudsync_pk_encode, cloudsync_pk_decode. All others volatile — fixes cloudsync_uuid() returning identical values within the same query when SQLite cached deterministic results - [HIGH] Fix realloc inconsistency in dbsync_update_payload_append: on second realloc failure, state was inconsistent (new_values resized, old_values not, capacity not updated). Both reallocs now checked before updating pointers and capacity - [MEDIUM] Move payload->count++ after all database_value_dup NULL checks in dbsync_update_payload_append — prevents count increment when allocation failed, which would cause use-after-free on cleanup - [MEDIUM] Add dbsync_update_payload_free(payload) before 3 early returns in dbsync_update_final — prevents memory leak of entire aggregate payload on error paths - [MEDIUM] Clean up partial database_value_dup allocations on OOM in dbsync_update_payload_append — free dup'd values at current index when count is not incremented to prevent leak ## src/sqlite/database_sqlite.c (6 fixes) - [MEDIUM] Replace fixed 4096-byte trigger WHEN clause buffers with dynamic cloudsync_memory_mprintf — prevents silent truncation for tables with long filter expressions - [MEDIUM] Check cloudsync_memory_mprintf return for NULL before storing in col_names[] in database_build_trigger_when — prevents strlen(NULL) crash in filter_is_column under OOM - [LOW] Use consistent PRId64 format with (int64_t) cast for schema hash in database_check_schema_hash and database_update_schema_hash — prevents format string mismatch on platforms where uint64_t and int64_t have different printf specifiers - [LOW] Fix DEBUG_DBFUNCTION using undeclared variable 'table' instead of 'table_name' in database_create_metatable (line 568) and database_create_triggers (line 782) — compile error when debug macros enabled - [LOW] Remove dead else branch in database_pk_rowid — unreachable code after sqlite3_prepare_v2 success check ## src/sqlite/sql_sqlite.c (1 fix) - [MEDIUM] Change %s to %q in SQL_INSERT_SETTINGS_STR_FORMAT — prevents SQL injection via malformed setting key/value strings ## src/dbutils.c (1 fix) - [MEDIUM] Change snprintf to cloudsync_memory_mprintf for settings insert using the new %q format — ensures proper SQL escaping ## src/utils.c (1 fix) - [MEDIUM] Fix integer overflow in cloudsync_blob_compare: (int)(size1 - size2) overflows for large size_t values, changed to (size1 > size2) ? 1 : -1 ## src/network.c (1 fix) - [MEDIUM] Remove trailing semicolon from savepoint name "cloudsync_logout_savepoint;" — semicolon in name caused savepoint/release mismatch ## src/postgresql/sql_postgresql.c (1 fix) - [CRITICAL] Replace EXCLUDED.col_version with %s.col_version (table reference) in SQL_CLOUDSYNC_UPSERT_COL_INIT_OR_BUMP_VERSION — PostgreSQL EXCLUDED refers to the proposed INSERT row (always col_version=1), not the existing row. This caused col_version to never increment correctly on conflict, breaking CRDT merge logic ## src/postgresql/cloudsync_postgresql.c (4 fixes) - [HIGH] Change PG_RETURN_INT32(rc) to PG_RETURN_BOOL(rc == DBRES_OK) in pg_cloudsync_terminate — SQL declaration returns BOOLEAN but code returned raw integer, causing protocol mismatch - [HIGH] Copy blob data with palloc+memcpy before databasevm_reset in cloudsync_col_value, and fix PG_RETURN_CSTRING to PG_RETURN_BYTEA_P — reset invalidates SPI tuple memory, causing use-after-free; wrong return type caused type mismatch with SQL declaration - [MEDIUM] Use palloc0 instead of cloudsync_memory_alloc+memset in aggregate context — palloc0 is lifetime-safe in PG aggregate memory context; cloudsync_memory_alloc uses wrong allocator - [MEDIUM] Free SPI_tuptable in all paths of get_column_oid — prevents SPI tuple table leak on early return ## src/postgresql/database_postgresql.c (3 fixes) - [MEDIUM] Use sql_escape_identifier for table_name/schema in CREATE INDEX — prevents SQL injection via specially crafted table names - [MEDIUM] Use sql_escape_literal for table_name in trigger WHEN clause — prevents SQL injection in trigger condition - [LOW] Use SPI_getvalue instead of DatumGetName for type safety in database_pk_names — DatumGetName assumes Name type which may not match the actual column type from information_schema ## test/unit.c (3 new tests) - do_test_blob_compare_large_sizes: verifies overflow fix for large size_t values in cloudsync_blob_compare - do_test_deterministic_flags: verifies cloudsync_uuid() returns different values in same query (non-deterministic flag working) - do_test_schema_hash_consistency: verifies int64 hash format roundtrip through cloudsync_schema_versions table Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Bump version to 0.9.111 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * Update .gitignore * fix(sqlite): PRIVATE functions used inside triggers require SQLITE_INNOCUOUS ... otherwise the cloudsync_init function returns this error: table_add_stmts error: 1 unsafe use of cloudsync_is_sync() Runtime error: An error occurred while adding test_sync table information to global context (unsafe use of cloudsync_is_sync()) (21) * Update README.md * Dev (#14) * fix(network): cloudsync_network_check_changes must not return the nrows value in case of error `SELECT cloudysnc_network_check_changes();` was returning "Runtime error: 0" in case of error response from the cloudsync microservice instead of the real error message * feat(rls): add complete support for RLS with batch merge in cloudsync_payload_apply * Feat/add support for status endpoint (#10) * feat(network): add support for new status endpoint * refactor(network): structured JSON responses for sync functions. Example: {"send":{"status":"synced","localVersion":5,"serverVersion":5},"receive":{"rows":3,"tables":["tasks"]}} * Feat/network support for multi org cloudsync (#11) * Disable notnull prikey constraints (#12) * The cloudsync extension now enforces NULL primary key rejection at runtime (any write with a NULL PK returns an error), so the explicit NOT NULL constraint on primary key columns is no longer a schema requirement * test: add null primary key rejection tests for SQLite and PostgreSQL * docs: remove NOT NULL requirement from primary key definitions The extension now enforces NULL primary key rejection at runtime, so the explicit NOT NULL constraint on PK columns is no longer a schema requirement. Replace the "must be NOT NULL" guidance with a note about runtime enforcement. * docs: add first draft of PERFORMANCE.md and CHANGELOG.md * fix(postgresql): resolve commit_alter crash and BYTEA handling in column_text Guard savepoint commit/rollback against missing subtransactions to prevent segfault in autocommit mode. Add BYTEA support to database_column_text so encoded PKs are readable during refill_metatable after ALTER TABLE. Enable alter table sync test (31). * test: new alter table test for postgres * feat: update endpoints to use databaseMangedId for /v2/cloudsync api * feat(network)!: replace URL connection string with a UUID (managedDatabaseId) BREAKING CHANGE: cloudsync_network_init now accepts a UUID string instead of the previous URL string. URL connection strings are no longer accepted. The managed database identifier returned by the CloudSync service when a new database is registered for sync. For SQLiteCloud projects, this value can be obtained from the project's OffSync page on the dashboard. * docs: update docs for the new managedDatabaseId arg for cloudsync_network_init * docs(examples): update example for the new managedDatabaseId arg for cloudsync_network_init * feat: add block-level LWW for fine-grained text conflict resolution (#16) * feat: add block-level LWW for fine-grained text conflict resolution Implements block-level Last-Writer-Wins for text columns across SQLite and PostgreSQL. Text is split into blocks (lines by default) and each block is tracked independently, so concurrent edits to different parts of the same text are preserved after sync. - Add block.c/block.h with split, diff, position, and materialize logic - Add fractional-indexing submodule for stable block ordering - Add cloudsync_set_column() and cloudsync_text_materialize() functions - Add cross-platform SQL abstractions for blocks table (SQLite/PostgreSQL) - Add block handling to PG insert, update, col_value, and set_column - Move network code to src/network/ directory - Bump version to 0.9.200 - Add 36 SQLite block LWW unit tests and 7 PostgreSQL test files - Update README and API docs with block-level LWW documentation * fix(ci): checkout submodules in GitHub Actions workflow * docs: update CHANGELOG.md * fix(ci): remove artifact size increase check to be less than 5% from previous release * test: improved stress test command * refactor: move all savepoint management from shared layer to platform wrappers (#17) * refactor: move all savepoint management from shared layer to platform wrappers Move database_begin_savepoint, database_commit_savepoint, and database_rollback_savepoint calls out of cloudsync_begin_alter and cloudsync_commit_alter (shared CRDT logic) into the platform-specific wrappers in cloudsync_sqlite.c and cloudsync_postgresql.c. This fixes the PostgreSQL "subtransaction left non-empty SPI stack" warning by ensuring SPI_connect() is called before the savepoint boundary, and creates architectural symmetry where shared code is pure business logic and all transaction management lives in platform wrappers. * chore * test: improved stress test command * docs: add SUPABASE_FLYIO.md * fix: update schema hash on extension version change When upgrading from an older version, the schema hash algorithm may differ. Call cloudsync_update_schema_hash before updating the stored version to ensure hash compatibility across synced devices. * test: add "Payload Apply Lock Test" * test: add edge-case tests for CRDT sync correctness and error handling Add 7 new SQLite unit tests and 7 new PostgreSQL test files covering: - DWS/AWS algorithm rejection (unsupported CRDT algos return clean errors) - Corrupted payload handling (empty, garbage, truncated, bit-flipped) - Payload apply idempotency (3x apply produces identical results) - Causal-length tie-breaking determinism (3-way concurrent update convergence) - Delete/resurrect with out-of-order payload delivery - Large composite primary key (5-column mixed-type PK roundtrip) - PostgreSQL-specific type roundtrips (JSONB, TIMESTAMPTZ, NUMERIC, BYTEA) - Schema hash mismatch detection (ALTER TABLE without cloudsync workflow) * test: improved stress test command * feat(ci): add PostgreSQL extension builds for Linux, macOS, and Windows Add a postgres-build matrix job that compiles the PostgreSQL extension for 5 platform/arch combinations (linux-x86_64, linux-arm64, macos-arm64, macos-x86_64, windows-x86_64) and includes them as release assets. Add postgres-package Makefile target and Windows platform support (PG_EXTENSION_LIB, -lpostgres linking). * fix(ci): resolve PostgreSQL extension build failures on macOS and Windows macOS: install gettext for libintl.h and pass include path via PG_EXTRA_CFLAGS. Windows: skip POSIX defines (_POSIX_C_SOURCE, _GNU_SOURCE) so PG headers use Winsock instead of netinet/in.h. * fix(ci): resolve PostgreSQL extension build on macOS, drop Windows macOS: guard Security.h include behind !CLOUDSYNC_POSTGRESQL_BUILD to avoid type conflicts (Size, uint64) with PostgreSQL headers, use getentropy instead. Windows: dropped from postgres-build matrix — MSYS2 PostgreSQL headers expect Unix includes incompatible with MinGW. * fix(ci): use _DARWIN_C_SOURCE on macOS for PostgreSQL extension build Replace _GNU_SOURCE with _DARWIN_C_SOURCE on macOS to expose preadv/pwritev declarations required by PostgreSQL headers. _GNU_SOURCE is now Linux-only. * fix(ci): add -undefined dynamic_lookup for macOS PostgreSQL extension linking macOS linker requires explicit handling of undefined symbols. PostgreSQL extensions resolve symbols at load time, so -undefined dynamic_lookup is needed. Also consolidate UNAME_S detection to a single location. * fix(ci): use macos-15 for PostgreSQL x86_64 build with cross-compilation macos-13 runners are deprecated. Cross-compile x86_64 on macos-15 ARM runner via -arch x86_64 passed through PG_EXTRA_CFLAGS. Also pass PG_EXTRA_CFLAGS to linker for arch flag propagation. * fix: make begin_alter and commit_alter idempotent Add is_altering flag to cloudsync_table_context to prevent errors when alter functions are called multiple times on the same table. begin_alter returns early if already altering; commit_alter returns early if not. * docs: remove outdated documentation files * fix: restore old sqlite-sync main packages references * fix: update broken gitignore after sync from main repo * fix(test): update environment variables * fix(workflow): update environment variables for test and restore token for release job * chore: restore old packages non dev names * chore: update packages example versions to new major version * chore: restore sqlite-sync-dev urls to sqlite-sync main repo * chore: update link on docs/postgresql/sport app * chore: update docs and references to main repo * chore: temp disable release job * Bump version to 1.0.0 --------- Co-authored-by: Marco Bambini <marco@creolabs.com> Co-authored-by: Andrea Donetti <andinux@gmail.com> Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Daniele Briggi <=> Co-authored-by: Andrea Donetti <andrea@sqlitecloud.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.