Conversation
read_range_utf8 used the infer-based ForgeFS::is_binary (MIME sniff, 8192 bytes, assumes text when type is unknown) while fs_search and all other callers used the BOM + zero-byte detector in binary_detection.rs. The same file could be skipped as binary by search but read as text by read_range. Route read_range_utf8 through the same is_binary(path) detector and remove the now-unused infer-based implementation and its dependency. Fixes tailcallhq#3633
|
Action required: PR inactive for 5 days. |
|
The fix is complete and the branch is up to date. This PR has been open for 5 days awaiting review, happy to rebase or adjust if anything needs changing. |
|
Action required: PR inactive for 5 days. |
|
Independent review of head Remaining acceptance gap: the current public read tool does not call #3848 interaction: its known-source-extension precedence is a separate MIME-routing fix and does not supersede this consolidation. It leaves the nonvisual strict-UTF8 read path intact, so both changes can coexist but neither closes that service-level NUL gap by itself. No changes made to that work. Coverage/validation: an isolated rustc probe using the exact pure detector functions passed 15 cases: empty/plain, isolated NUL, UTF-8 BOM with/without NUL, UTF-16 with/without BOM, short PNG headers, unknown non-UTF8 bytes, and NUL at indices 511/512. This confirms existing heuristic limitations (UTF-16 classification is not decoding; short no-NUL binary headers and NUL beyond the sample remain nonbinary), NOT crate or service test success. The added regression should assert
Co-Authored-By: ForgeCode noreply@forgecode.dev |
read_range_utf8used theinfer-basedForgeFS::is_binary(MIME sniff over 8192 bytes, assumes text when the type is unknown), whilefs_search,fs_write,plan_create, andimage_readall used the BOM + zero-byte detector inbinary_detection.rs. A file with a stray0x00in the first 512 bytes was skipped as binary by search but read as text byread_range_utf8.This PR routes
read_range_utf8through the sameis_binary(path)detector used by every other caller, and removes the now-unusedinfer-based implementation, its tests, and theinferdependency.Files changed:
crates/forge_fs/src/read_range.rs: useis_binary(path_ref)instead ofSelf::is_binary(&mut file); add regression testcrates/forge_fs/src/is_binary.rs: deleted (no remaining callers)crates/forge_fs/src/lib.rs: removemod is_binary;crates/forge_fs/Cargo.toml: removeinferdependencyFixes #3633
Verification
cargo test -p forge_fs --libpasses (10 tests, including newtest_stray_zero_byte_classified_as_binary)cargo clippy -p forge_fs --libis cleancargo check -p forge_infraandcargo check -p forge_apppass (no downstream breakage)