[#1150] Read the benchmark JTL with a CSV parser, so that an error whose message holds a comma reaches the step log - #1151
Merged
vharseko merged 1 commit intoOct 2, 2026
Conversation
… so that an error whose message holds a comma reaches the step log compare-opendj.sh split the JMeter result file with awk -F','. JMeter quotes a responseMessage that contains a comma, and every message naming a DN does, so the columns after it shifted and such rows were never matched as failures. On the PDB side this hid the result-80 and result-32 rows of OpenIdentityPlatform#1149 and left only BIND 49, the last consequence of a failed ADD. Parse the JTL with python3's csv module instead, and replace digits before grouping, so that the per-row DN and elapsed time no longer turn each failed row into a kind of its own. Print the number of failed rows and kinds, and say how many kinds the top ten leaves out. Fixes OpenIdentityPlatform#1150
maximthomas
approved these changes
Oct 2, 2026
maximthomas
left a comment
Contributor
There was a problem hiding this comment.
praise: The listing now reads the JTL the way JMeter writes it, and the CI run of this head shows the rows #1150 was missing.
csv.DictReaderkeyed by the header (compare-opendj.sh:112): the build-docker job of this head lists2 COMPARE | 32 | The specified entry mail=u_N_N@test.com,ou=People,dc=example,dc=com does not exist ...and2 MODIFY | 32, rows thatawk -F','split on the DN's commas.- The listing stays on stderr (
>&2,:108), sobench_one's stdout still carries only the version.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1150
Problem
compare-opendj.shlisted the errors of each benchmark side withawk -F','. JMeter quotes aresponseMessagethat contains a comma, and every message naming a DN does, so the columns after it shifted and such rows were
never matched as failures. On the PDB side this hid the result-80 and result-32 rows of #1149 and left only
BIND | 49, the last consequence of a failed ADD.Change
The error listing in
bench_onenow reads the JTL withpython3'scsvmodule, which is already used by thesame Docker jobs in
build.yml.N. Themessages carry the entry DN or the elapsed milliseconds, and without this each failed row would be a kind of its
own.
more, how many are left out.
bench_onecarries the server version.|| truekeeps afailure of the listing from stopping the benchmark.
The
errcolumns of the step summary come fromstatistics.jsonthroughsummary.shand are unchanged.Verification
The block, cut verbatim from the script, run on the
benchmark-pdb-vs-jeartifacts named in the issue:2 BIND | 495131 BIND | 800,2 BIND | 49Output for run 36845050319, PDB:
A synthetic JTL with 13 kinds of error, messages holding commas and one message spanning two lines printed ten
kinds and
... 3 more kinds not shown. When the JTL is missing or malformed, the block exits with 0. In every casenothing was written to stdout.
bash -npasses.#1145 edits the same file in a different place (the
sysctlnear the top), so the two do not conflict.