Skip to content

fix(repl): heal scroll cap and terminal blanking [SCR-555]#28

Merged
sahilsunny merged 1 commit into
mainfrom
fix/repl/display-fixes-SCR-555
Jul 10, 2026
Merged

fix(repl): heal scroll cap and terminal blanking [SCR-555]#28
sahilsunny merged 1 commit into
mainfrom
fix/repl/display-fixes-SCR-555

Conversation

@sahilsunny

Copy link
Copy Markdown
Collaborator

Summary

  • Fix REPL scrollback so PgUp/Ctrl+Home can reach content above long wrapped output (e.g. google preview warnings), because scroll_offset was capped in logical lines while rendering uses visual (post-wrap) rows
  • Add automatic full-repaint when the terminal blanks alt-screen cells after window occlusion (macOS Terminal.app), because prompt_toolkit's differential renderer never notices externally cleared cells
  • Repaint on focus-in (xterm mode 1004) for terminals that support it, plus Ctrl+L as a manual escape hatch
  • Add unit and PTY regression tests so both bugs stay fixed

Details:

src/scrapingbee_cli/interactive.py

  • Lines 478–503 (ScrollbackBuffer.__init__, _total_visual_rows_locked): Track _last_wrap_width and count total visual rows at that width (ceil(chars/width) per logical line). scroll_offset is consumed in visual rows by get_visible_visual_with_meta, but scroll_up/scroll_to_top previously capped using len(self.lines), so a single 4000-char preview line (~44 visual rows at 92 cols) made everything above it unreachable.

  • Lines 645–647 (get_visible_visual_with_meta): Record the wrap width on each visual render so scroll methods cap in the same units the renderer uses.

  • Lines 685–705 (scroll_up, scroll_to_top): Replace logical-line cap with _total_visual_rows_locked() - 1. get_visible_visual_with_meta still clamps to total - height on the next frame.

  • Line 1713 (_print_help): Document Ctrl+L as a repaint shortcut for terminal rendering glitches.

  • Lines 4541–4564 (run_repl / _run_raw): Wrap tutorial/auth run_in_terminal calls to disable mode 1004 (ESC[?1004l) while the app is suspended for click.prompt(), then re-enable it. Without this, focus reports would be read as literal input.

  • Lines 5170–5203 (_force_full_repaint, Ctrl+L, F24 bindings): Clear app.renderer._last_screen and invalidate so the next frame repaints every cell in place. Avoids renderer.reset() (leaves alt screen / disables mouse) and renderer.clear() (visible flash). Ctrl+L overrides prompt_toolkit's default clear-screen; F24 handles focus-in repaint.

  • Lines 5234–5245 (ANSI sequence registration): Map ESC[I → F24 and ESC[O → Ignore before the app starts so focus events are not inserted into the input buffer as [I/[O text.

  • Lines 5284, 5319–5352 (_ticker): Add 1 Hz automatic full repaint as a universal fallback for terminals (e.g. Terminal.app) that do not report focus. Heals blanked cells without user action; up to ~1s delay if the process was napped while backgrounded.

  • Lines 5476–5484 (_pre_run): Enable xterm focus reporting (ESC[?1004h) at REPL startup.

  • Lines 5543–5548 (finally): Disable focus reporting on exit so ESC[I/ESC[O do not leak into the parent shell.

tests/unit/test_scrollback_selection.py

  • Lines 131–182 (TestVisualRowScrollCap): Unit regression for the visual-row scroll cap — wheel-style scroll_up, scroll_to_top, render-time clamp, pre-render logical fallback, and scroll-down back to bottom, using the google-preview shape (one 4000-char line).

tests/unit/test_repl_pty.py

  • Lines 40–72 (_SCROLL_* markers, _setup_exec_home, _spawn extensions): PTY helpers — unique scroll markers, unsafe-shell config for ! commands, configurable terminal size.

  • Lines 125–137 (_scrollback_lines, _marker_in_scrollback): Assert on scrollback rows only, excluding the input echo (which contains marker strings from the shell command text).

  • Lines 216–230 (_scroll_keys_top, _scroll_keys_bottom): Send real xterm key sequences (Ctrl+Home, PgUp, PgDn) through the PTY.

  • Lines 289–304 (test_ctrl_l_repaints_without_clearing): PTY check that Ctrl+L repaints without losing scrollback or the prompt.

  • Lines 307–325 (test_focus_reports_do_not_leak_into_input): PTY check that mode 1004 is enabled and injected focus reports are consumed, not typed into the UI.

  • Lines 329–389 (test_scroll_up_reaches_content_above_wrapped_line): End-to-end scroll regression at 92×30 — !python3 prints top marker + 4000-char wrapped line + bottom marker; Ctrl+Home, PgDn, and repeated PgUp must reach the top marker in scrollback.

Summary
- Fix REPL scrollback so PgUp/Ctrl+Home can reach content above long wrapped output (e.g. google preview warnings), because scroll_offset was capped in logical lines while rendering uses visual (post-wrap) rows
- Add automatic full-repaint when the terminal blanks alt-screen cells after window occlusion (macOS Terminal.app), because prompt_toolkit's differential renderer never notices externally cleared cells
- Repaint on focus-in (xterm mode 1004) for terminals that support it, plus Ctrl+L as a manual escape hatch
- Add unit and PTY regression tests so both bugs stay fixed

Details:

src/scrapingbee_cli/interactive.py

- Lines 478–503 (`ScrollbackBuffer.__init__`, `_total_visual_rows_locked`): Track `_last_wrap_width` and count total visual rows at that width (ceil(chars/width) per logical line). `scroll_offset` is consumed in visual rows by `get_visible_visual_with_meta`, but `scroll_up`/`scroll_to_top` previously capped using `len(self.lines)`, so a single 4000-char preview line (~44 visual rows at 92 cols) made everything above it unreachable.

- Lines 645–647 (`get_visible_visual_with_meta`): Record the wrap width on each visual render so scroll methods cap in the same units the renderer uses.

- Lines 685–705 (`scroll_up`, `scroll_to_top`): Replace logical-line cap with `_total_visual_rows_locked() - 1`. `get_visible_visual_with_meta` still clamps to `total - height` on the next frame.

- Line 1713 (`_print_help`): Document Ctrl+L as a repaint shortcut for terminal rendering glitches.

- Lines 4541–4564 (`run_repl` / `_run_raw`): Wrap `tutorial`/`auth` `run_in_terminal` calls to disable mode 1004 (`ESC[?1004l`) while the app is suspended for `click.prompt()`, then re-enable it. Without this, focus reports would be read as literal input.

- Lines 5170–5203 (`_force_full_repaint`, Ctrl+L, F24 bindings): Clear `app.renderer._last_screen` and invalidate so the next frame repaints every cell in place. Avoids `renderer.reset()` (leaves alt screen / disables mouse) and `renderer.clear()` (visible flash). Ctrl+L overrides prompt_toolkit's default clear-screen; F24 handles focus-in repaint.

- Lines 5234–5245 (ANSI sequence registration): Map `ESC[I` → F24 and `ESC[O` → Ignore before the app starts so focus events are not inserted into the input buffer as `[I`/`[O` text.

- Lines 5284, 5319–5352 (`_ticker`): Add 1 Hz automatic full repaint as a universal fallback for terminals (e.g. Terminal.app) that do not report focus. Heals blanked cells without user action; up to ~1s delay if the process was napped while backgrounded.

- Lines 5476–5484 (`_pre_run`): Enable xterm focus reporting (`ESC[?1004h`) at REPL startup.

- Lines 5543–5548 (`finally`): Disable focus reporting on exit so `ESC[I`/`ESC[O` do not leak into the parent shell.

tests/unit/test_scrollback_selection.py

- Lines 131–182 (`TestVisualRowScrollCap`): Unit regression for the visual-row scroll cap — wheel-style `scroll_up`, `scroll_to_top`, render-time clamp, pre-render logical fallback, and scroll-down back to bottom, using the google-preview shape (one 4000-char line).

tests/unit/test_repl_pty.py

- Lines 40–72 (`_SCROLL_*` markers, `_setup_exec_home`, `_spawn` extensions): PTY helpers — unique scroll markers, unsafe-shell config for `!` commands, configurable terminal size.

- Lines 125–137 (`_scrollback_lines`, `_marker_in_scrollback`): Assert on scrollback rows only, excluding the input echo (which contains marker strings from the shell command text).

- Lines 216–230 (`_scroll_keys_top`, `_scroll_keys_bottom`): Send real xterm key sequences (`Ctrl+Home`, PgUp, PgDn) through the PTY.

- Lines 289–304 (`test_ctrl_l_repaints_without_clearing`): PTY check that Ctrl+L repaints without losing scrollback or the prompt.

- Lines 307–325 (`test_focus_reports_do_not_leak_into_input`): PTY check that mode 1004 is enabled and injected focus reports are consumed, not typed into the UI.

- Lines 329–389 (`test_scroll_up_reaches_content_above_wrapped_line`): End-to-end scroll regression at 92×30 — `!python3` prints top marker + 4000-char wrapped line + bottom marker; Ctrl+Home, PgDn, and repeated PgUp must reach the top marker in scrollback.
@sahilsunny sahilsunny self-assigned this Jul 9, 2026
except Exception:
pass

# ── Full-repaint escape hatch ───────────────────────────────────────────

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if I'm supposed to read all these very verbose comments

@sahilsunny sahilsunny merged commit b2613dd into main Jul 10, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants