Skip to content

VFS pin state (OnlineOnly) silently lost when the sync journal is rebuilt — client re-hydrates entire trees with no warning (Linux suffix VFS, 4.0.6) #10705

Description

@gklewis83

Summary

On Linux suffix-VFS, the per-path pin state in flags is the only record that a subtree was set to OnlineOnly. If that table is lost — the journal is rebuilt, recreated, or replaced — there is no warning and no recovery: the root default takes over and the client silently re-hydrates every placeholder it finds, filling the disk and saturating the sync queue for hours.

The failure is invisible while it runs, because VFS still looks engaged. The .nextcloud placeholders are all still visibly there, right up until each one is consumed.

This has now happened twice on the same machine, 11 days apart. The second time it went unnoticed for about five hours and ~20 GB.

Environment

  • Client: 4.0.6-1 (Debian/Ubuntu archive build)
  • OS: Ubuntu 26.04 LTS, KDE Plasma 6.6.5, Wayland
  • VFS: suffix (virtualFilesMode=suffix)
  • Roughly 125,000 placeholders across ~600 GB of intentionally-dehydrated content

Expected behaviour

Subtrees set to "Free up local space" stay dehydrated across client restarts and journal maintenance, or — if that state cannot be preserved — the client says so rather than silently re-downloading hundreds of gigabytes.

Actual behaviour

After the journal was recreated, flags contained:

  • zero rows for any top-level subtree that had been set OnlineOnly
  • zero rows with pinState = 2 at all
  • 17,387 rows, every one of them pinState = 1 (AlwaysLocal) on an individual file path

Those 17,387 rows were written by the client itself as it hydrated each file. So the end state is not merely "the settings are missing" — it is inverted: every file it pulled down now carries an explicit "keep local" entry, which will resist a subsequent subtree-level OnlineOnly unless wipeForPathAndBelow clears them.

Measured effect on one tree: 118 GB → 138 GB, every placeholder consumed, and the sync queue was 100% that tree for ~5 hours — during which no other folder made progress. On the first occurrence the same fault took a tree from 85 GB to 139 GB and starved unrelated folders for most of a day.

How the state was read

The journal is under an exclusive lock by the running client for long stretches (eleven consecutive read-only attempts failed), so it was read from a snapshot copy taken with the -wal and -shm sidecars — a copy of the .db alone omits whatever has not been checkpointed:

SELECT DISTINCT pinState FROM flags;              -- 1
SELECT count(*) FROM flags WHERE path NOT LIKE '%/%';  -- 0
SELECT count(*) FROM flags;                       -- 17387

What I have not been able to establish

Why the journal was rebuilt. That is the part I would most like help with. It was not a version transition — SyncJournalDb::checkConnect() has no forward-compatibility guard and simply overwrites the stored version stamp, and the schemas are identical across the two release channels, so opening a differently-stamped journal does not trigger a rebuild. Candidates I could not test: a client crash or kill during a sync pass, an integrity-check failure, or an operation that removes and re-adds the folder connection.

Is there any client-side signal when flags is empty but placeholders exist on disk? That combination seems inherently suspicious: it means "the user dehydrated this content, and the record of that intent is gone." Detecting it looks cheap.

Why I am filing despite two related closures

I am not re-litigating either. This is a distinct report: not that pin state cannot be exported, and not that a hand-written journal value is ignored, but that pin state set through the supported UI is lost without warning and with large, silent consequences, on a supported platform.

What would help, roughly in order of value

  1. A warning when the journal has no OnlineOnly entries but dehydrated placeholders exist on disk — this alone would turn a silent 20 GB event into a visible one.
  2. Confirmation before mass re-hydration — "this will download N files / X GB", with a way to decline.
  3. Durability for pin state — persisted outside the journal, or rebuilt from the placeholders that are still present on disk, which already encode the intent.

Happy to provide further journal queries, logs, or to test a patch.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions