Skip to content

[Fix] sort inconsistent comparator - #1416

Open
dimensionscape wants to merge 2 commits into
HaxeFoundation:masterfrom
dimensionscape:fix/sort-inconsistent-comparator
Open

dimensionscape wants to merge 2 commits into
HaxeFoundation:masterfrom
dimensionscape:fix/sort-inconsistent-comparator

Conversation

@dimensionscape

Copy link
Copy Markdown
Contributor

Array.sort uses std::stable_sort, which requires the comparator to be a strict weak ordering. Haxe code doesn't always give it one. The common "shuffle" idiom is enough:

array.sort((a, b) -> Std.random(3) - 1);

With a comparator like that, libstdc++'s insertion step can walk off the front of the range it is sorting. In release builds that ends in a crash, a corrupted heap or a hang, for both value arrays and object arrays.

This replaces std::stable_sort in both sort paths with a small merge sort (hx::StableSort) that stays in bounds whatever the comparator returns. A bad comparator just gets an unspecified order, as it would in JS.

For valid comparators nothing changes: it is still a stable, in-place sort, so the result is exactly what std::stable_sort produced. It is no slower on random data, needs about 4% fewer comparator calls, and is 4 to 5 times faster on already-sorted or reversed input.

The new testInconsistentComparator hangs on current master and passes with this change. testSizes checks order and stability around the merge sort's insertion runs.

std::stable_sort requires a strict weak ordering. A comparator that breaks
it, such as one returning random results, made libstdc++'s insertion step
run off the front of the range: a crash, a corrupted heap or a hang.

Sort with a merge sort that stays in range whatever the comparator
returns. It is stable too, so a valid comparator gives the same order as
before.
Also check order and stability at sizes around the merge sort's
insertion runs, with random, ascending and descending keys.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant