fix: preserve UTF-8 in search_code source - #1253
Conversation
|
Thanks for opening the focused split requested in #1209. The exact patch stays in post-match serialization, reuses the existing |
Signed-off-by: anfedoro <anfedoro@gmail.com>
57032ca to
05574ff
Compare
Signed-off-by: anfedoro <anfedoro@gmail.com>
Signed-off-by: anfedoro <anfedoro@gmail.com>
|
A quick note so this does not sit here looking like your problem: your red CI is not caused by your change — and a rebase will not fix it either. The failure is I have flagged it internally as a production-race candidate rather than a test to tune: under our own testing doctrine a test whose verdict depends on scheduler timing is a defect, and the right response is a deterministic repro plus a fix, not a wider budget. So: nothing is needed from you on the CI front. Your PR is queued for a proper content review and I will come back to you on the change itself. Apologies that a failure of ours has been sitting on your PR looking like yours. |
Summary
search_codefull results and raw match content.?with the existing UTF-8-safe lossy sanitizer.Root cause
search_codeconstructed result fields throughsanitize_ascii(). Because UTF-8 characters are encoded as bytes >= 0x80, the sanitizer replaced their individual bytes with?after the file and line match had already been found correctly.Scope
This PR addresses result serialization after a successful match. It intentionally does not address the separate Windows CP_ACP pattern-matching issue discussed in #1209.
Related to #1209
Validation
make -f Makefile.cbm test-focused TEST_SUITES=mcp— 179 passed, 2 skipped.scripts/build.shon macOS arm64.search_code --mode fullsmoke test confirmed Cyrillic is preserved insource.