Repository navigation
decompress: make ZSTD_decompressBound() a true upper bound - #4832
ilyas-mallah wants to merge 2 commits into
Conversation
|
Hi @ilyas-mallah! Thank you for your pull request and welcome to our community. Action RequiredIn order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you. ProcessIn order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA. Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks! |
|
Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks! |
What and why
ZSTD_decompressBound()is documented as an upper bound on the decompressed size, but two kinds of input make it return less thanZSTD_decompress()produces. The decoder still honorsdstCapacity, so a caller that allocates the bound getsdstSize_tooSmallon input that otherwise decodes.ZSTD_findDecompressedSize()already guards the same sum, andZSTD_decompressBound()now does too.blockSizeMax(spec: 0-sized compressed blocks in frames with window_size = 0 #3482), but the bound counts every block asblockSizeMax. A 10-byte frame with one RLE block decodes to 17,597 bytes against a bound of 6,144. Such blocks now count at their declared size, and valid frames keep the same bound.The other way to fix the second case is to reject these blocks in
ZSTD_decompressFrame(), as streaming does and as the spec says (#4669). That changes which inputs decode, so this PR keeps them decodable and only fixes the bound. I can switch to rejecting them if that's preferred.A new unit test fails on
dev, and two fuzz targets now check the bound, skipping legacy frames since the frozen legacy decoders don't keep it.tests/fuzzer(64-bit and 32-bit),test-fuzzerandtest-legacypass, and 300 s per fuzz target with ASan and UBSan found nothing.Checklist
Tools used
AI usage: Claude Code helped with the fuzzing runs and test builds. I traced both cases, made the fix, and validated every change.