Bug with how [:representation :charset] slot is getting populated in the context at the 'charset-available?' decision

[evanspa]

The liberator.conneg/best-allowed-charset function contains an "or" expression in which the 3rd clause explicitly tests for "iso-8859-1" (and returning a quality value of 1), and this is causing some trouble.

For example, if I have a request with an "Accept-Charset" value of: "ISO-2022-CN", best-allowed-charset returns "ISO-8859-1". But if I have request with value: "UTF-8", best-allowed-charset returns "UTF-8". This is because "UTF-8" appears after "ISO-8859-1" from a lexicographic sort order, and since the 'select-best' function does a sort, reverse, first, UTF-8 is returned. But for ISO-2022-CN, ISO-8859-1 is returned.

FYI, (best-allowed-charset "ISO-2022-CN;q=1,ISO-8859-1;q=0.8") correctly returns ISO-2022-CN because 'select-best' does the sort based on the parsed quality number first.

I feel the solution is simply to remove the explicit 3rd clause of the aforementioned 'best-allowed-charset' function; I just don't see why an explicit clause should be there for the "ISO-8859-1" charset.

Thank you,

-Paul

[ordnungswidrig]

RFC 2616 mentiones ISO-8859-1 as a default charset. Nevertheless, what you describe looks like a bug. I will have to check the RFC again. Have you had a look at liberator's tests? Can you provide additional tests as a patch?

Paul Evans notifications@github.com schrieb:

The liberator.conneg/best-allowed-charset function contains an "or" expression in which the 3rd clause explicitly tests for "iso-8859-1" (and returning a quality value of 1), and this is causing some trouble.

For example, if I have a request with an "Accept-Charset" value of: "ISO-2022-CN", best-allowed-charset returns "ISO-8859-1". But if I have request with value: "UTF-8", best-allowed-charset returns "UTF-8". This is because "UTF-8" appears after "ISO-8859-1" from a lexicographic sort order, and since the 'select-best' function does a sort, reverse, first, UTF-8 is returned. But for ISO-2022-CN, ISO-8859-1 is returned.

FYI, (best-allowed-charset "ISO-2022-CN;q=1,ISO-8859-1;q=0.8") correctly returns ISO-2022-CN because 'select-best' does the sort based on the parsed quality number.

I feel the solution is simply to remove the explicit 3rd clause of the aforementioned 'best-allowed-charset' function; I just don't see why an explicit clause should be there for the "ISO-8859-1" charset.

Thank you,

-Paul

—
Reply to this email directly or view it on GitHub.

[evanspa]

I just re-checked the http spec, and you're right, ISO-8859-1 is considered as the default! It mentions it in section 14.2 (at http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html) That said, I don't think the behavior I mentioned should be considered a bug. If a client truly wants to use UTF-8, and only use UTF-8, it looks like it should explicitly list the ISO-8859-1 charset with a quality number less than 1.

Regards,

-Paul

[ordnungswidrig]

I keep this bug open as a reminder to improve the documentation at this point.