Fix for dissect processor overeager delimiter consumption

[lukewhiting]

This PR addresses #119264 and the overly greedy consumption of delimiters by the ES implementation of the dissect processor.

This code modifies the implementation to correctly match the spec and only attempts to greedily consume repeated delimiters if the skip right padding operator is used.

In the process of this, 2 new unit tests were created to highlight the current incorrect behaviour and one tests which did not match the specification was modified to correctly reflect the desired behaviour.

To ensure our implementation aligns with the rest of the product stack, logstash-plugins/logstash-filter-dissect#92 was merged to add the new units tests to Logstash's implimentation of Disect where it passed without the issues raised in #119264, further indicating that our current implementation is faulty.

[Copilot]

Pull Request Overview

This PR addresses #119264 by fixing the overly greedy consumption of delimiters in the dissect processor implementation to correctly adhere to the spec.

• Updated the delimiter consumption logic in DissectParser.java to only greedily consume delimiters when the skip right padding operator is active.
• Modified and added unit tests in DissectParserTests.java to reflect and validate the corrected behavior.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
libs/dissect/src/test/java/org/elasticsearch/dissect/DissectParserTests.java	Updated test assertions and added tests for delimiter consumption edge cases.
libs/dissect/src/main/java/org/elasticsearch/dissect/DissectParser.java	Modified while loop logic to incorporate skipRightPadding() checks for delimiter consumption.

Comments suppressed due to low confidence (2)

libs/dissect/src/main/java/org/elasticsearch/dissect/DissectParser.java:230

• [nitpick] Consider adding a clarifying comment to explain that the loop only processes consecutive delimiters when the skip right padding operator is enabled, ensuring the behavior aligns with the updated spec.

while (dissectPair.key.skipRightPadding() && i < input.length) {

libs/dissect/src/test/java/org/elasticsearch/dissect/DissectParserTests.java:319

• [nitpick] Update expected output has been modified to reflect the revised delimiter consumption behavior; consider adding a comment to detail the rationale behind this test change.

assertMatch("%{+a/1},%{+a/3}-%{+a/2} %{b}", "foo,bar----baz lol", Arrays.asList("a", "b"), Arrays.asList("foo---bazbar", "lol"));