Decide how deleting (and restoring) forms impact deleting datasets

[ktuite]

Decision:
Deleting a form will count as "unlinking it" from a dataset, allowing the dataset to be deleted.

As for restoring a deleted form that references a deleted dataset (or deleted dataset property), there were too many questions related to re-creating or re-linking forms to deleted datasets that we decided to just restrict form restoration. If you delete a form, and then delete any of the datasets or properties it was linked to, you will be unable to restore the form.

Testing note for QA:
Verify that a deleted form can't be restored if any of the datasets or properties it was attached to have been deleted.

---

First brought up in

• Dataset attachments replaced with static CSVs still show up linked to forms #1626

Does the dataset metadata stuff check for deleted forms?

If you're able to delete a dataset after deleting a form, what does that mean for restoring the form? If you're not able to do so, what is it like to be stuck with a dataset you can't delete because a soft-deleted form still has a reference to it?

[ktuite]

What I've observed so far:

• Delete form (that writes to a dataset) -> ok to delete dataset

Once both form and dataset are soft-deleted:

• If both are purged at the same time, that's fine.

• If the form is purged first, that's fine, the dataset can't be restored.

• If the dataset is purged, and the form is restored, it just doesn't have a record of being attached to the dataset.

  • Submissions won't create entities and won't even log an error.

• If the dataset is in the trash and the form is restored, a submission that tries to create an entity will have an error that says something like "The dataset (trees) specified in the submission does not exist."

• If the dataset is in the trash and the form is restored and then the form is edited, the new draft will make a new dataset that then works for entity creation.

---

Draft datasets

Make one draft form that writes to "trees"
Make another form and publish that writes to "trees" (creates published dataset)

Delete published form
Delete "trees" dataset

Draft form still references old deleted dataset and some things don't know it's been deleted

• draft page shows reference to dataset
• publishing has 500 error

Draft form referencing published, then deleted dataset

Error on publishing form

...
Error: Option value not present on get.
    at None.get (/Users/ktuite/Desktop/code/odk/central-backend/lib/util/option.js:70:17)
    at /Users/ktuite/Desktop/code/odk/central-backend/lib/model/query/forms.js:355:66
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
...

The error, while not providing any useful information, does block publishing a form that links to a deleted dataset.

Downloading the form definition and reuploading it reprocesses it and triggers redoing the dataset to make a new one.

Some things I'm thinking of:

1. Frontend could potentially show that the draft form is trying to do something to a deleted dataset and show an error panel instead of the dataset details. It could also disable to form publish button while that error is present. The user is working on a draft form anyway so it kinda makes sense if they're deleting their related datasets, they need to take some extra action on their draft form before publishing.

2. Do something with the form/dataset link when the dataset is soft deleted? Like in addition to setting deletedAt to the time, we'd also alter the dataset form def table? I don't like this approach of having to do more than just setting the deletedAt when deleting a dataset. It could remove all indications on the form draft page about a dataset, which could let someone more easily publish the form, but they might prefer a big "error - your form is in a bad state!" message over seeing less information.

This is going to come up with dataset properties, too.

If you have a draft form that mentions a dataset property that has been deleted in some other way, there should probably be a big error that says "this form writes to a deleted property and you need to change that before you can publish it."