Skip to content

Disk backend ranged reads are not revision-atomic (open fd vs. metadata) #348

Description

@alecthomas

Summary

Disk.Open opens the file descriptor before loading the object's metadata (ETag) from BoltDB, so the served body and the returned ETag are not read atomically. If the same key is replaced between the two reads, the descriptor points at the old (renamed-away) file while the headers/ETag come from the new revision.

Impact

This breaks IfRange/range correctness on the disk backend: rangeShortCircuit validates IfRange against the (new) ETag while f still serves the old file's bytes. With ParallelGet, chunk 0 returns the new ETag (so the consistency check passes), but later chunks re-Open and can read the new file — silently splicing old and new bytes within a single download. The per-chunk ETag pinning added in #346 therefore does not actually bind the opened body to the returned metadata on the disk backend.

Location

internal/cache/disk.go, Disk.Open:

  • os.Open(fullPath) (~L268) runs before d.db.getHeaders(...) (~L283).

Suggested fix

Make the disk body + metadata read revision-atomic — e.g. read the headers/ETag and open the file under a consistent view, or re-validate the ETag after opening and retry/fail on mismatch before honoring ranges.

Context

Pre-existing issue surfaced by the byte-range work in #346 (Codex review), tracked separately per discussion: #346 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions