Backslashes in single quotes should not be treated as escape characters.

[wolfmanx]

While Javascript (as well as Python) took a lazy approach to string escape sequences by treating single and double quotes the same:

$ set -x; node -e 'console.log(".\\\\shellcheck.exe --version");'
+ node -e 'console.log(".\\\\shellcheck.exe --version");'
.\\shellcheck.exe --version

$ set -x; node -e 'console.log('\''.\\\\shellcheck.exe --version'\'');'
+ node -e 'console.log('\''.\\\\shellcheck.exe --version'\'');'
.\\shellcheck.exe --version

The POSIX shell interpolates variables and backticks in double quoted strings, which requires the backslash escape syntax to be able to prevent that. However, in single quoted strings, the POSIX shell does not do any interpolation or backtick expansion, therefore, no escape mechanism is needed. Consequently, a backslash is not a special character in single quoted strings and should not be removed by a shell-compatible tokenizer.

In the supplied pull request #5, there is a test that constructs a string of all characters in the ASCII code range 1-127 escaped, which is then supplied as unquoted, double and single quoted argument to argsTokenizer. The results are checked against the actual output from /bin/sh.

That test fails against the unmodified version of args-tokenizer.