Skip to content

Conversation

@loremmoqi
Copy link

Bug: lex_string flush ignores characters at start of input

Description

In lex_string, the flush function was originally written as:

if start > 0 && end > start {
    buf.write_substring(ctx.input, start, end - start)
}

When parsing a string literal that starts at the beginning of the input (ctx.offset = 0), the first call to flush does not add any characters to the buffer, causing the returned string to be empty. This issue only occurs when the string starts at the beginning of the input or when start = 0 during the first flush.

Fix

Remove the start > 0 condition and use:

if end > start {
    buf.write_substring(ctx.input, start, end - start)
}

This ensures that characters at the start of the string are correctly added to the buffer.

@peter-jerry-ye-code-review
Copy link

Fix for string lexing bug that ignores characters at start of input

Category
Correctness
Code Snippet
Line 20: if start > 0 && end > start {
buf.write_substring(ctx.input, start, end - start)
}
Recommendation
Change condition to: if end > start {
buf.write_substring(ctx.input, start, end - start)
}
Reasoning
The original condition start > 0 prevents flushing characters when parsing starts at the beginning of input (offset 0), causing the lexer to skip the initial characters of string literals that appear at the start of the input buffer.

@peter-jerry-ye
Copy link
Collaborator

Can you please construct a test case?

@loremmoqi
Copy link
Author

Can you please construct a test case?
Sure,I have added the test.
But I don't know why the nightly-builds of three platforms including window failed.

@peter-jerry-ye
Copy link
Collaborator

You added a test using internal API. What I would like to see is a test using external API because the "bug" you mentioned could have been an undocumented invariant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants