-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
Description
Feature Request
Currently, for one-time uploads (e.g. via the Dataset Uploader app or by running one of the connector scripts like GeoJSON to Postgres), we do not provide a robust way to append data to an existing dataset:
- The Dataset Uploader app explicitly prevents you from uploading a new dataset if the database table already exists. This is by design for now to keep scope for an MVP of this app, but there is a TODO in the app's README to make it possible to modify this app to update an existing dataset.
- The connector scripts, eventually calling
StructureDBWriter._safe_insert
, will try to update existing records based on primary key. But sometimes we need to generate a unique ID to use as primary key if there is none, and it becomes tricky to do that deterministically across multiple uploads, e.g. if the user uploads different versions of the same dataset to add new data that was created since the last upload. For this, see discussion in Generate unique ID if not present in GeoJSON feature #136.
For now, this is just a placeholder issue for a future epic, to ensure we don't forget about this.