[Vert.x] Incorrect trace propagation during message transmission

[skipkolch]

Describe the bug

I integrated tracing into my Vert.x project using the vertx-tracing
library. For instrumentation, I use the @WithSpan annotation together with your Java Agent to create and export traces to collectors.

However, I noticed an issue: with this setup, different independent requests are merged into a single trace (this behavior was also described in the old issue #93).

The Vert.x team suggested not using the Java Agent and instead creating spans manually — this indeed solves the problem. But using the annotation is much more convenient, and I’d like to keep this option available.

Could you please take a look at my reproducer and advise how this can be fixed while still using the agent?

Steps to reproduce

https://github.com/skipkolch/vertx-tracer-example

1. Download reproducer project
2. Start WorkerTraceMain.java
   $ gradle startWorkerTraceMain
3. Start VertxHttpServer.java
   $ gradle startHttpServer
4. Start sending http requests in parallel
   $ bash curl_requests.sh
5. Observe mixed traces in log/Tempo/OTLP collector

Expected behavior

• Each request is processed in its own trace.
• Trace context propagates correctly between worker and event loop verticles

Actual behavior

• Some independent requests appear merged in the same trace
• This behavior is consistently repeated with a different number of requests.

Javaagent or library instrumentation version

2.19.0

Environment

• Vert.x: 4.5.12
• Vert.x OpenTelemetry: 4.5.12
• OpenTelemetry Instrumentation: 1.32.0
• OpenTelemetry Java-agent: 2.19.0

Additional context

No response

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[laurit]

If I understand correctly this issue is about context propagation with vert.x event bus. Since we don't have any tests for the event bus context propagation not working there is not unexpected.

The Vert.x team suggested not using the Java Agent and instead creating spans manually — this indeed solves the problem. But using the annotation is much more convenient, and I’d like to keep this option available.

Could you elaborate how you would do this manually. Perhaps knowing how it is done manually will help us in figuring out how to do this in the agent.

[skipkolch]

Could you elaborate how you would do this manually. Perhaps knowing how it is done manually will help us in figuring out how to do this in the agent.

Certainly. Below is the main code snippet that demonstrates how to create spans manually — using this approach, context propagation works correctly.

    final var spanBuilder = tracer.spanBuilder(spanName);
    if (rootSpan) {
        spanBuilder.setNoParent();
    } else {
        spanBuilder.setParent(Context.current());
    }
    final var span = spanBuilder.startSpan();
    try (var ignore = span.makeCurrent()) {
        final var result = runnable.apply(Span.current());
        if (result instanceof Future<?> future) {
            return (T) future
                    .onComplete(__ -> spanEndSuccess(span))
                    .onFailure(it -> spanEndFail(it, span));
        }
        spanEndSuccess(span);
        return result;
    } catch (Throwable throwable) {
        spanEndFail(throwable, span);
        throw throwable;
    }

For more details, I’ve published a separate branch of my reproducer [tree/exporter].

[Munken]

We are seeing something similar.
We are using a async resteasy filter and when switching to the event loop thread it looses the span id.