Parsing breaks because it is locale sensitive

[michihenning]

I'm seeing this with version 1.6.5 on Ubuntu 16.04 (xenial).

Below is a trivial test program . (It requires the German locale to be installed.)

With the line that sets the locale commented out (assuming your are in a locale that uses '.' as the decimal point), everything works fine. With the locale setting enabled, the call fails and the error message is

'3.14' is not a number.

There is a related discussion in bug #283, but the issue below is different, I believe.

#include <jsoncpp/json/reader.h>
#include <iostream>
#include <locale>

int main(int, char **)
{
    using namespace std;

    locale::global(locale("de_DE.utf8"));

    Json::Value root;
    Json::CharReaderBuilder rbuilder;
    rbuilder["collectComments"] = false;

    string json_string = "3.14";

    unique_ptr<Json::CharReader> reader(rbuilder.newCharReader());
    auto begin = json_string.c_str();
    auto end = json_string.c_str() + json_string.size();
    string errors;
    bool ok = reader->parse(begin, end, &root, &errors);
    if (!ok)
    {
        cerr << errors << endl;
    }
}

[cdunn2001]

One idea is in #285. You can see the long discussion at #283, referring to much older discussions.

I'd be fine with using std::imbue() to adjust the locale during reading/writing. I just don't want to break anything else.

Floating-point conversions are a huge issue whenever they come up. We're always open to suggestions.

[michihenning]

std::imbue() will work fine. All reading and writing of JSON should be locale-independent, so setting the classic locale on the relevant input and output stream should do the trick, and do it cleanly.

[cdunn2001]

The problem on another pull-request was that some tests broke. And the issue there was that, ideally, we would like to preserve round-trip conversions.

I'm not sure I can find time to work on this in the next few weeks. Help is definitely appreciated.

[michihenning]

By round-trip conversion, you mean that you want to write a floating-point value out the same it was in the original file? If so, the only way to do that would be to remember the original string representation of the floating-point value.

[cdunn2001]

That's not true. It's true that for arbitrary input, the output will not necessarily match the input. But it is not true that we cannot find round-trippable inputs.

People have put a lot of effort into this. I only hit the "Merge" button. If you want to write the code and update the tests, then I think we could probably get away with only a minor version bump, since floating-point representation is not part of the JSON standard. But the tests do need to pass, or their removal would need to be justified. I definitely do not have time for this myself right now. Sorry.

[martinitus]

Hey, I'just leave a note here:
I took me about 5 hours to figure out, why sometimes parsing floats fails. It boiled down that Qt's QApplication changes the locale and hence parsing floats/doubles failed in the main application, but not in my test cases. Maybe you can add some

BIG WARNING

lights to the docs that hint users to those kind of problems. So long, I hope that this comment gets found be people with similar problems. Anyway: keep up the nice work!
HtH Martin

[cdunn2001]

Yes, comments on where locale is set are helpful, but jsoncpp does not do that. Comments on where locale is used but not set might be too verbose. Not sure. I think Qt is really the culprit, setting the locale for a callback.