fix(repl): heal scroll cap and terminal blanking [SCR-555]#28
Merged
Conversation
Summary - Fix REPL scrollback so PgUp/Ctrl+Home can reach content above long wrapped output (e.g. google preview warnings), because scroll_offset was capped in logical lines while rendering uses visual (post-wrap) rows - Add automatic full-repaint when the terminal blanks alt-screen cells after window occlusion (macOS Terminal.app), because prompt_toolkit's differential renderer never notices externally cleared cells - Repaint on focus-in (xterm mode 1004) for terminals that support it, plus Ctrl+L as a manual escape hatch - Add unit and PTY regression tests so both bugs stay fixed Details: src/scrapingbee_cli/interactive.py - Lines 478–503 (`ScrollbackBuffer.__init__`, `_total_visual_rows_locked`): Track `_last_wrap_width` and count total visual rows at that width (ceil(chars/width) per logical line). `scroll_offset` is consumed in visual rows by `get_visible_visual_with_meta`, but `scroll_up`/`scroll_to_top` previously capped using `len(self.lines)`, so a single 4000-char preview line (~44 visual rows at 92 cols) made everything above it unreachable. - Lines 645–647 (`get_visible_visual_with_meta`): Record the wrap width on each visual render so scroll methods cap in the same units the renderer uses. - Lines 685–705 (`scroll_up`, `scroll_to_top`): Replace logical-line cap with `_total_visual_rows_locked() - 1`. `get_visible_visual_with_meta` still clamps to `total - height` on the next frame. - Line 1713 (`_print_help`): Document Ctrl+L as a repaint shortcut for terminal rendering glitches. - Lines 4541–4564 (`run_repl` / `_run_raw`): Wrap `tutorial`/`auth` `run_in_terminal` calls to disable mode 1004 (`ESC[?1004l`) while the app is suspended for `click.prompt()`, then re-enable it. Without this, focus reports would be read as literal input. - Lines 5170–5203 (`_force_full_repaint`, Ctrl+L, F24 bindings): Clear `app.renderer._last_screen` and invalidate so the next frame repaints every cell in place. Avoids `renderer.reset()` (leaves alt screen / disables mouse) and `renderer.clear()` (visible flash). Ctrl+L overrides prompt_toolkit's default clear-screen; F24 handles focus-in repaint. - Lines 5234–5245 (ANSI sequence registration): Map `ESC[I` → F24 and `ESC[O` → Ignore before the app starts so focus events are not inserted into the input buffer as `[I`/`[O` text. - Lines 5284, 5319–5352 (`_ticker`): Add 1 Hz automatic full repaint as a universal fallback for terminals (e.g. Terminal.app) that do not report focus. Heals blanked cells without user action; up to ~1s delay if the process was napped while backgrounded. - Lines 5476–5484 (`_pre_run`): Enable xterm focus reporting (`ESC[?1004h`) at REPL startup. - Lines 5543–5548 (`finally`): Disable focus reporting on exit so `ESC[I`/`ESC[O` do not leak into the parent shell. tests/unit/test_scrollback_selection.py - Lines 131–182 (`TestVisualRowScrollCap`): Unit regression for the visual-row scroll cap — wheel-style `scroll_up`, `scroll_to_top`, render-time clamp, pre-render logical fallback, and scroll-down back to bottom, using the google-preview shape (one 4000-char line). tests/unit/test_repl_pty.py - Lines 40–72 (`_SCROLL_*` markers, `_setup_exec_home`, `_spawn` extensions): PTY helpers — unique scroll markers, unsafe-shell config for `!` commands, configurable terminal size. - Lines 125–137 (`_scrollback_lines`, `_marker_in_scrollback`): Assert on scrollback rows only, excluding the input echo (which contains marker strings from the shell command text). - Lines 216–230 (`_scroll_keys_top`, `_scroll_keys_bottom`): Send real xterm key sequences (`Ctrl+Home`, PgUp, PgDn) through the PTY. - Lines 289–304 (`test_ctrl_l_repaints_without_clearing`): PTY check that Ctrl+L repaints without losing scrollback or the prompt. - Lines 307–325 (`test_focus_reports_do_not_leak_into_input`): PTY check that mode 1004 is enabled and injected focus reports are consumed, not typed into the UI. - Lines 329–389 (`test_scroll_up_reaches_content_above_wrapped_line`): End-to-end scroll regression at 92×30 — `!python3` prints top marker + 4000-char wrapped line + bottom marker; Ctrl+Home, PgDn, and repeated PgUp must reach the top marker in scrollback.
| except Exception: | ||
| pass | ||
|
|
||
| # ── Full-repaint escape hatch ─────────────────────────────────────────── |
Contributor
There was a problem hiding this comment.
I'm not sure if I'm supposed to read all these very verbose comments
kostas-jakeliunas-sb
approved these changes
Jul 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Details:
src/scrapingbee_cli/interactive.py
Lines 478–503 (
ScrollbackBuffer.__init__,_total_visual_rows_locked): Track_last_wrap_widthand count total visual rows at that width (ceil(chars/width) per logical line).scroll_offsetis consumed in visual rows byget_visible_visual_with_meta, butscroll_up/scroll_to_toppreviously capped usinglen(self.lines), so a single 4000-char preview line (~44 visual rows at 92 cols) made everything above it unreachable.Lines 645–647 (
get_visible_visual_with_meta): Record the wrap width on each visual render so scroll methods cap in the same units the renderer uses.Lines 685–705 (
scroll_up,scroll_to_top): Replace logical-line cap with_total_visual_rows_locked() - 1.get_visible_visual_with_metastill clamps tototal - heighton the next frame.Line 1713 (
_print_help): Document Ctrl+L as a repaint shortcut for terminal rendering glitches.Lines 4541–4564 (
run_repl/_run_raw): Wraptutorial/authrun_in_terminalcalls to disable mode 1004 (ESC[?1004l) while the app is suspended forclick.prompt(), then re-enable it. Without this, focus reports would be read as literal input.Lines 5170–5203 (
_force_full_repaint, Ctrl+L, F24 bindings): Clearapp.renderer._last_screenand invalidate so the next frame repaints every cell in place. Avoidsrenderer.reset()(leaves alt screen / disables mouse) andrenderer.clear()(visible flash). Ctrl+L overrides prompt_toolkit's default clear-screen; F24 handles focus-in repaint.Lines 5234–5245 (ANSI sequence registration): Map
ESC[I→ F24 andESC[O→ Ignore before the app starts so focus events are not inserted into the input buffer as[I/[Otext.Lines 5284, 5319–5352 (
_ticker): Add 1 Hz automatic full repaint as a universal fallback for terminals (e.g. Terminal.app) that do not report focus. Heals blanked cells without user action; up to ~1s delay if the process was napped while backgrounded.Lines 5476–5484 (
_pre_run): Enable xterm focus reporting (ESC[?1004h) at REPL startup.Lines 5543–5548 (
finally): Disable focus reporting on exit soESC[I/ESC[Odo not leak into the parent shell.tests/unit/test_scrollback_selection.py
TestVisualRowScrollCap): Unit regression for the visual-row scroll cap — wheel-stylescroll_up,scroll_to_top, render-time clamp, pre-render logical fallback, and scroll-down back to bottom, using the google-preview shape (one 4000-char line).tests/unit/test_repl_pty.py
Lines 40–72 (
_SCROLL_*markers,_setup_exec_home,_spawnextensions): PTY helpers — unique scroll markers, unsafe-shell config for!commands, configurable terminal size.Lines 125–137 (
_scrollback_lines,_marker_in_scrollback): Assert on scrollback rows only, excluding the input echo (which contains marker strings from the shell command text).Lines 216–230 (
_scroll_keys_top,_scroll_keys_bottom): Send real xterm key sequences (Ctrl+Home, PgUp, PgDn) through the PTY.Lines 289–304 (
test_ctrl_l_repaints_without_clearing): PTY check that Ctrl+L repaints without losing scrollback or the prompt.Lines 307–325 (
test_focus_reports_do_not_leak_into_input): PTY check that mode 1004 is enabled and injected focus reports are consumed, not typed into the UI.Lines 329–389 (
test_scroll_up_reaches_content_above_wrapped_line): End-to-end scroll regression at 92×30 —!python3prints top marker + 4000-char wrapped line + bottom marker; Ctrl+Home, PgDn, and repeated PgUp must reach the top marker in scrollback.