fix(mem-wal): reject zero manifest scan batch size

[u70b3]

Summary

• reject manifest_scan_batch_size=0 at ShardWriter::open before manifest or epoch side effects
• defensively reject zero inside manifest discovery so direct store callers cannot trust a stale hint
• add public-boundary and stale-hint regression coverage with contextual InvalidInput assertions

Fixes #7928.

Test Plan

• cargo fmt --all -- --check
• cargo test -p lance manifest_scan_batch_size --lib -- --nocapture
• cargo test -p lance dataset::mem_wal::manifest::tests --lib -- --nocapture
• cargo clippy --all --tests --benches -- -D warnings

[coderabbitai]

📝 Walkthrough

Walkthrough

Changes

Manifest scan validation

Layer / File(s)	Summary
Manifest reader validation rust/lance/src/dataset/mem_wal/manifest.rs	Adds shared validation before manifest hint lookup or version scanning, with coverage for zero batch sizes and stale hints.
Shard writer open validation rust/lance/src/dataset/mem_wal/write.rs	Documents and enforces the positive batch-size requirement when opening a shard writer, with invalid-input test coverage.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: hamersaw, touch-of-grey

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: rejecting zero manifest scan batch size.
Description check	✅ Passed	The description matches the implemented fix and regression coverage for zero-sized manifest scans.
Linked Issues check	✅ Passed	The change rejects zero in ShardWriter::open and manifest discovery, matching #7928's required behavior.
Out of Scope Changes check	✅ Passed	No unrelated changes are evident beyond the zero-batch-size validation and tests.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments (2)

rust/lance/src/dataset/mem_wal/write.rs-6130-6149 (1)

6130-6149: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Assert that rejected open leaves no manifest or epoch side effects.

The PR objective requires validation before claiming the shard, but this test only checks the returned error. A future regression that claims an epoch before validating would still pass. Reuse the store after the failed open and assert that read_latest() returns None.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rust/lance/src/dataset/mem_wal/write.rs` around lines 6130 - 6149, Extend
test_open_rejects_zero_manifest_scan_batch_size to reuse the store after the
rejected ShardWriter::open and assert that read_latest() returns None, verifying
validation leaves no manifest or epoch side effects while preserving the
existing error assertions.

rust/lance/src/dataset/mem_wal/manifest.rs-697-699 (1)

697-699: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert that the stale hint was actually persisted.

write_version_hint is best-effort and returns no error, so this call can fail silently; the test would still pass because zero is rejected regardless of the hint. Verify the fixture before constructing the zero-batch reader.

Suggested test assertion

 manifest_store.write_version_hint(1).await;
+assert_eq!(manifest_store.read_version_hint().await, Some(1));

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rust/lance/src/dataset/mem_wal/manifest.rs` around lines 697 - 699, Update
the test around ShardManifestStore::write_version_hint to verify that version
hint 1 was persisted before constructing the zero-batch reader. Read or inspect
the manifest store’s persisted hint using the existing fixture/assertion
mechanism, and assert it equals 1 so the test cannot pass solely because the
zero-batch reader rejects zero.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rust/lance/src/dataset/mem_wal/write.rs`:
- Around line 134-139: Complete the documentation for the public field
manifest_scan_batch_size and its builder method with a synchronized example
using ShardWriterConfig::new(...) followed by
with_manifest_scan_batch_size(...), keeping the example aligned with the actual
signatures. Add cross-links between the field and method, and preserve the
documented greater-than-zero constraint.

---

Other comments:
In `@rust/lance/src/dataset/mem_wal/manifest.rs`:
- Around line 697-699: Update the test around
ShardManifestStore::write_version_hint to verify that version hint 1 was
persisted before constructing the zero-batch reader. Read or inspect the
manifest store’s persisted hint using the existing fixture/assertion mechanism,
and assert it equals 1 so the test cannot pass solely because the zero-batch
reader rejects zero.

In `@rust/lance/src/dataset/mem_wal/write.rs`:
- Around line 6130-6149: Extend test_open_rejects_zero_manifest_scan_batch_size
to reuse the store after the rejected ShardWriter::open and assert that
read_latest() returns None, verifying validation leaves no manifest or epoch
side effects while preserving the existing error assertions.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: QUIET

Plan: Pro Plus

Run ID: 65160c7f-2148-41f5-81ba-6346f6051df5

📥 Commits

Reviewing files that changed from the base of the PR and between 9726f88 and d4391fc.

📒 Files selected for processing (2)

• rust/lance/src/dataset/mem_wal/manifest.rs
• rust/lance/src/dataset/mem_wal/write.rs

[coderabbitai]

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Complete the public API documentation.

The changed public field and builder method state the > 0 contract but do not provide the required synchronized example and links to the corresponding configuration API. Add a small example using ShardWriterConfig::new(...).with_manifest_scan_batch_size(...) and cross-link the field and method.

As per coding guidelines, **/*.{rs,md,rst}: Document all public APIs with examples and links to relevant structs and methods; keep examples synchronized with actual signatures.

Also applies to: 331-336

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rust/lance/src/dataset/mem_wal/write.rs` around lines 134 - 139, Complete the
documentation for the public field manifest_scan_batch_size and its builder
method with a synchronized example using ShardWriterConfig::new(...) followed by
with_manifest_scan_batch_size(...), keeping the example aligned with the actual
signatures. Add cross-links between the field and method, and preserve the
documented greater-than-zero constraint.

Source: Coding guidelines

[codecov]

Codecov Report

❌ Patch coverage is 93.33333% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance/src/dataset/mem_wal/write.rs	86.66%	1 Missing and 1 partial ⚠️
rust/lance/src/dataset/mem_wal/manifest.rs	96.66%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!