Command lines that end with a backtick cause history recall issues

[msftrncs]

Environment

Windows 10
PowerShell 7.1.2
PSReadLine 2.2.0 beta2

Reproduction steps

It is possible to submit a command line that ends in a backtick. When these lines are added to the history file, they conflict with the history file's encoding where a backtick indicates a multiline command line.

# treat each line separately
ACommand`
Echo Hello`
Echo Hello``

The result of the two commands above will be that ACommand` is not a valid command and the echo of hello`. In both cases, the backtick is not considered as a line continuation, and are instead part of the token they terminate.

UpdateHistoryFromFile fails to read all lines into history, if the last line in the history being read ends with a backtick. This is because there is an assumption that this will not happen, and so there is no checking to see if sb.Length > 0 after all the lines have been processed. This is only evident with a new session right after a prior session has saved such a command. Eventually, the command line in the history that ends with a backtick will be appended to other lines, corrupting the history, due to how the history file uses the lines ending in backticks to indicate a multiline history item.

I do not see any way to compatibly fix this, and there certainly isn't a fix for the existing occurrences as the original intent cannot be detected. The existing history file would need to be converted, and probably saved with a new name, in order to provide a remedy.

However, here is some PowerShell scripting that demonstrates a working solution after demonstrating a basis.

# get PSReadLine command history items from history file, return the number of items read.
($a = (Get-Content (Get-PSReadLineOption).HistorySavePath -Raw) -split '(?<=(?<!`\r?)\n)(?=.|\n)' -replace '\r?\n(?!.|\n)' -replace '`(?=\r?\n)|\\(?=`(?!.|\n))').Length

# the above (and PSReadLine as well) fail to handle commands that ended with a backtick, since backticks are used to indicate continued lines.
# to form a fully reversable encoding consider this possibility:
# when writing history
#   if a sub-line ends with either a backtick or a backslash, it must be terminated with two backticks
#   if the final line ends with a backtick, a backslash must be prepended before the backtick.
# when reading history
#   if line ends with two backticks, remove both and append the line.
#   if line ends with backslash backtick, remove backslash, append line, and finish.
#   if line ends with a backtick, remove the backtick and append the line.
#   else append the line and finish.

# to encode the history correctly in the first place:
($b = ($a = [Microsoft.PowerShell.PSConsoleReadLine]::GetHistoryItems().CommandLine) -replace '(?<=[\\`])(?=\n)', '`' -replace '(?=\n)', '`' -replace '`(?!.|\n)', '\`').Length

# demonstrating reversal
($c = (($b -join "`r`n") + "`r`n") -split '(?<=(?<!(?<!\\)`\r?)\n)(?=.|\n)' -replace '\r?\n(?!.|\n)' -replace '`?`(?=\r?\n)|\\(?=`(?!.|\n))').Length

Comparing between $a and $c can confirm that the process fully reversed.

There are probably plenty of other possibly remedies.