Skip to content

Fix periodic intra frames and feature reset for chunk-based models - #169

Open
DucTu (DucTu2k4) wants to merge 1 commit into
microsoft:mainfrom
DucTu2k4:fix-intra-period-chunk
Open

DucTu (DucTu2k4) wants to merge 1 commit into
microsoft:mainfrom
DucTu2k4:fix-intra-period-chunk

Conversation

@DucTu2k4

Copy link
Copy Markdown

Problem

With --model_structure hts (or htl) and a periodic intra setting such as
--force_intra_period 128, test_video.py inserts only one periodic intra frame
and then stops both intra insertion and feature-memory reset for the rest of the
sequence.

In run_one_point_with_stream, an intra frame is inserted when
frame_idx % intra_period == 1, checked at the start of each chunk. With
g_frame_delay = 8 the chunk starts are 1, 9, ..., 129, so frame 129 is hit.
The intra frame consumes a single frame, so the following chunks start at
130, 138, ... and the condition is never true again. The reset condition
(frame_idx + g_frame_delay) % reset_interval == 1 is affected in the same way.

For a 750-frame sequence this gives intra frames at [0, 129] only, and the last
620 frames are coded without any intra frame or reset.

Fix

  • Insert an intra frame when the GOP index max(frame_idx - 1, 0) // intra_period
    increases, instead of testing frame_idx directly.
  • Trigger the reset when the current chunk covers a multiple of reset_interval.

Behaviour is unchanged for the LD model (g_frame_delay = 1) and for HT models
without periodic intra frames (intra_period = -1); I checked this by simulating
the loop for several frame counts, intra periods and reset intervals.

Note: since an intra frame shifts the chunk alignment by one frame, the distance
between intra frames is intra_period + 1 for most GOPs (0, 129, 258, 387, ...),
and one GOP is shortened by g_frame_delay every g_frame_delay GOPs so that the
average period stays intra_period. I am happy to change this if you prefer a
different convention.

Results

HT-S, QP 40, 1920x1080, 750 frames, 8-bit YUV420, --reset_interval 128,
--skip_thres 0.15, RTX 3090.

Setting I frames Avg bpp P-frame PSNR-Y
main, --force_intra_period 128 2 0.0740 25.01 dB
This PR, --force_intra_period 128 6 0.0040 41.59 dB
Reference, intra_period = -1 1 0.0034 41.50 dB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant