Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

StackOverflowError when using PrometheusOptions with startEmbeddedServer = true and -Dvertx.metrics.options.enabled=true #251

Closed
afeskoff-godaddy opened this issue Jan 10, 2025 · 3 comments
Closed

StackOverflowError when using PrometheusOptions with startEmbeddedServer = true and -Dvertx.metrics.options.enabled=true #251

afeskoff-godaddy opened this issue Jan 10, 2025 · 3 comments
Assignees
@tsegismont
Labels
bug Something isn't working
Milestone
4.5.14

Comments

@afeskoff-godaddy
Copy link

afeskoff-godaddy commented Jan 10, 2025 •
edited
Loading

Version

4.2.6(Although it looks like all 4.. releases are impacted)

Context

Setting startEmbeddedServer to true

return new VertxPrometheusOptions()
        .setStartEmbeddedServer(true)
        .setEmbeddedServerOptions(new HttpServerOptions().setPort(port));

and passing -Dvertx.metrics.options.enabled=true results in StackOverFlowException. This is a fragment of the stacktrace that contains 8 stack long repeating fragment.

```
at java.base/java.lang.String.decodeASCII(Unknown Source)
at java.base/java.lang.System$2.decodeASCII(Unknown Source)
at java.base/sun.nio.cs.UTF_8$Decoder.decodeArrayLoop(Unknown Source)
at java.base/sun.nio.cs.UTF_8$Decoder.decodeLoop(Unknown Source)
at java.base/java.nio.charset.CharsetDecoder.decode(Unknown Source)
at java.base/sun.nio.cs.StreamDecoder.implRead(Unknown Source)
at java.base/sun.nio.cs.StreamDecoder.read(Unknown Source)
at java.base/java.io.InputStreamReader.read(Unknown Source)
at java.base/java.io.BufferedReader.fill(Unknown Source)
at java.base/java.io.BufferedReader.readLine(Unknown Source)
at java.base/java.io.BufferedReader.readLine(Unknown Source)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.parseLine(Unknown Source)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.parse(Unknown Source)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(Unknown Source)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(Unknown Source)
at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(Unknown Source)
at java.base/java.util.ServiceLoader$2.hasNext(Unknown Source)
at java.base/java.util.ServiceLoader$3.hasNext(Unknown Source)
at io.vertx.core.ServiceHelper.loadFactories(ServiceHelper.java:55)
at io.vertx.core.ServiceHelper.loadFactories(ServiceHelper.java:42)
at io.vertx.core.impl.VertxBuilder.init(VertxBuilder.java:270)
at io.vertx.core.Vertx.vertx(Vertx.java:86)
at io.vertx.core.Vertx.vertx(Vertx.java:76)
at io.vertx.micrometer.backends.PrometheusBackendRegistry.init(PrometheusBackendRegistry.java:73)
at io.vertx.micrometer.impl.VertxMetricsImpl.init(VertxMetricsImpl.java:102)
at io.vertx.micrometer.impl.VertxMetricsFactoryImpl.metrics(VertxMetricsFactoryImpl.java:48)
at io.vertx.core.spi.VertxMetricsFactory.init(VertxMetricsFactory.java:50)
at io.vertx.core.impl.VertxBuilder.initProviders(VertxBuilder.java:280)
at io.vertx.core.impl.VertxBuilder.init(VertxBuilder.java:271)
at io.vertx.core.Vertx.vertx(Vertx.java:86)
at io.vertx.core.Vertx.vertx(Vertx.java:76)
at io.vertx.micrometer.backends.PrometheusBackendRegistry.init(PrometheusBackendRegistry.java:73)
at io.vertx.micrometer.impl.VertxMetricsImpl.init(VertxMetricsImpl.java:102)
at io.vertx.micrometer.impl.VertxMetricsFactoryImpl.metrics(VertxMetricsFactoryImpl.java:48)
at io.vertx.core.spi.VertxMetricsFactory.init(VertxMetricsFactory.java:50)
at io.vertx.core.impl.VertxBuilder.initProviders(VertxBuilder.java:280)
at io.vertx.core.impl.VertxBuilder.init(VertxBuilder.java:271)
at io.vertx.core.Vertx.vertx(Vertx.java:86)
at io.vertx.core.Vertx.vertx(Vertx.java:76)
at io.vertx.micrometer.backends.PrometheusBackendRegistry.init(PrometheusBackendRegistry.java:73)
at io.vertx.micrometer.impl.VertxMetricsImpl.init(VertxMetricsImpl.java:102)
at io.vertx.micrometer.impl.VertxMetricsFactoryImpl.metrics(VertxMetricsFactoryImpl.java:48)
at io.vertx.core.spi.VertxMetricsFactory.init(VertxMetricsFactory.java:50)
at io.vertx.core.impl.VertxBuilder.initProviders(VertxBuilder.java:280)
at io.vertx.core.impl.VertxBuilder.init(VertxBuilder.java:271)
at io.vertx.core.Vertx.vertx(Vertx.java:86)
at io.vertx.core.Vertx.vertx(Vertx.java:76)


Without `startEmbeddedServer = true` verticle starts successfully. Problem also disappears if we stop 
passing -`Dvertx.metrics.options.enabled=true`.

It looks like in 3.* `Vertx.vertx()` was using factory https://github.com/eclipse-vertx/vert.x/blob/3.9/src/main/java/io/vertx/core/Vertx.java#L85. 
In 4.* this was changed to use Builder which leads to a loop https://github.com/eclipse-vertx/vert.x/blob/master/vertx-core/src/main/java/io/vertx/core/Vertx.java#L159

### Do you have a reproducer?

-

### Steps to reproduce

-
### Extra

We can reproduce it on linux and mac, java version - 17. Although, none of this seems to matter.
@afeskoff-godaddy afeskoff-godaddy added the bug Something isn't working label Jan 10, 2025
@tsegismont tsegismont added this to the 4.5.12 milestone Jan 16, 2025
@tsegismont tsegismont self-assigned this Jan 16, 2025
@tsegismont
Copy link
Contributor

Can you please share a small reproducer with 4.5.11? I failed to create one.

@tsegismont tsegismont removed this from the 4.5.12 milestone Jan 20, 2025
@afeskoff-godaddy
Copy link
Author

@tsegismont Here you go. Running VerticleStartup.main() throws StackOverflowError that I described above.
simple-vertx-project.zip

@tsegismont tsegismont added this to the 4.5.14 milestone Feb 25, 2025
tsegismont added a commit to tsegismont/vertx-micrometer-metrics that referenced this issue Mar 5, 2025
@tsegismont
StackOverflowError when using PrometheusOptions
7ae87e8 
Fixes vert-x3#251

When the embedded server is enabled, the backend registry creates an internal Vert.x instance, for the sole purpose of serving metrics data over HTTP.

In general, it's not a problem because the default Vert.x options don't enable metrics.
But if metrics are enabled with the vertx.metrics.options.enabled system property, the internal Vert.x instance will have metrics enabled too (because of code introduced to retrofit Vert.x builder in Vert.x 4).
And then the application fail to start with a StackOverflow error (because the default PrometheusBackendRegistry is reused and reinitialized).

So, we can avoid that situation by making sure the internal Vert.x instance uses dummy metrics.

This problem doesn't apply to Vert.x 5.

Signed-off-by: Thomas Segismont <[email protected]>
@tsegismont tsegismont mentioned this issue Mar 5, 2025
Merged
tsegismont added a commit to tsegismont/vertx-micrometer-metrics that referenced this issue Mar 5, 2025
@tsegismont
StackOverflowError when using PrometheusOptions
49ca62d 
Fixes vert-x3#251

When the embedded server is enabled, the backend registry creates an internal Vert.x instance, for the sole purpose of serving metrics data over HTTP.

In general, it's not a problem because the default Vert.x options don't enable metrics.
But if metrics are enabled with the vertx.metrics.options.enabled system property, the internal Vert.x instance will have metrics enabled too (because of code introduced to retrofit Vert.x builder in Vert.x 4).
And then the application fail to start with a StackOverflow error (because the default PrometheusBackendRegistry is reused and reinitialized).

So, we can avoid that situation by making sure the internal Vert.x instance uses dummy metrics.

This problem doesn't apply to Vert.x 5.

Signed-off-by: Thomas Segismont <[email protected]>
tsegismont added a commit that referenced this issue Mar 7, 2025
@tsegismont
StackOverflowError when using PrometheusOptions (#254)
8850c5e 
Fixes #251

When the embedded server is enabled, the backend registry creates an internal Vert.x instance, for the sole purpose of serving metrics data over HTTP.

In general, it's not a problem because the default Vert.x options don't enable metrics.
But if metrics are enabled with the vertx.metrics.options.enabled system property, the internal Vert.x instance will have metrics enabled too (because of code introduced to retrofit Vert.x builder in Vert.x 4).
And then the application fail to start with a StackOverflow error (because the default PrometheusBackendRegistry is reused and reinitialized).

So, we can avoid that situation by making sure the internal Vert.x instance uses dummy metrics.

This problem doesn't apply to Vert.x 5.

Signed-off-by: Thomas Segismont <[email protected]>
@tsegismont
Copy link
Contributor

Fixed by #254

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tsegismont tsegismont

Labels
bug Something isn't working
Milestone
4.5.14
Development

No branches or pull requests
2 participants
@tsegismont @afeskoff-godaddy