Repository navigation
Skip stray TEM markers and reserved markers before restart markers when decoding JPEGs - #3225
Merged
Merged
Conversation
…en decoding jpegs A TEM marker (FF 01) between header segments was read as a marker with a length field, so the next segment header was consumed as its length and decoding failed. TEM stands alone like RSTn, so skip it the same way. Some producers write extraneous bytes before restart markers. Those that form a TEM or reserved marker (FF 01 to FF BF) were taken as the end of the scan, which corrupted the rest of the image or made the marker parser fail. Skip them while looking for the marker that ends a restart interval and resync at the next marker, as libjpeg does.
3 of 4 tasks
4 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prerequisites
Description
JPEG header parsing treated the standalone TEM marker as a segment with a length field. It now skips TEM without consuming bytes from the next segment. This applies to both identification and decoding under every integrity policy. ITU T.81 Table B.1 defines TEM as a standalone marker.
At restart boundaries, TEM and reserved marker codes could end a scan early. The shared
JpegBitReadernow scans past those codes only whenSegmentIntegrityHandling.IgnoreImageDatapermits recovery.StrictandIgnoreAncillaryreject them. Real segment markers remain available to the segment parser. Both Huffman and arithmetic scans use this path. Arithmetic restart handling also classifies the detected marker before resetting the reader, so it does not lose the next segment marker.The cases are in the existing
JpegDecoderTests.cs, beside the extraneous-marker regression. The added standalone test file is removed. Tests load stored inputs; they do not encode or modify images at runtime. Local libjpeg-turbo 3.1.91 generated the clean inputs. Its decoder produces identical pixels for each clean/corrupted pair, with warnings for the corrupted inputs. ImageSharp checks exact pixel equality for recovery and checks rejection under the other policies. Header tests also check that an invalid required DQT length remains fatal in every mode.Arithmetic decoding now uses the JPEG defaults for omitted DAC entries: DC bounds L = 0 and U = 1, and AC threshold Kx = 5. The existing table type initializes those values. Missing entries are cached once per image, and later explicit DAC entries still replace them. Three stored regression inputs omit all conditioning entries or only the AC entries. Libjpeg decodes each file without warnings and produces exactly the same pixels as the explicit-table input. ImageSharp requires exact equality under every integrity policy.
Validation: Release JPEG and TIFF tests pass on .NET 10 and .NET 11: 1,574 passed on .NET 10 and 1,573 passed on .NET 11, with one existing debug-only skip and zero failures on each target. All three new arithmetic-default cases also pass in a focused .NET 11 run. No tolerances, reference images, or
BufferedReadStreamcode changed.V4 backport: #3258.
Fixes #3237