Support for appdynamics

[bjartek]

It is possible to hook micrometer into https://www.appdynamics.com/ in some way?

Relevant links from slack
https://docs.appdynamics.com/display/PRO41/Build+a+Monitoring+Extension+Using+Java

https://docs.appdynamics.com/display/PRO42/Analytics+Events+API#AnalyticsEventsAPI-AbouttheAnalyticsEventsAPI

[checketts]

I'll be working with a company that uses AppDynamics shortly and should be able to dig into this.

[bjartek]

Has there been any progress on this issue?

[checketts]

Yes. So I'm finally actually to the point of working on it.

Here are my findings:

• The recommended extension mechanism relies on extending a base class, but the dependency isn't available in any Maven repos
• I found a good DropWizard based example, so I have a pattern I can copy, but it suffers from the missing Maven dep too
• The agent can expose an HTTP port that you push metrics to, so I believe I'll pursue that approach, bypassing the missing dep
• I have a running app that is being monitored, so I'll be iterating in that project before porting it out into a PR.

I'm still so new to AppDynamics, that I'm not certain how to query the metrics that are pushed in. It doesn't seem to support metric tags, but instead follows the hierarchical model, so I'm going to be pushing metrics in various ways to see how cross-instance grouping and aggregations work, so I can try to publish the metrics in a way that is easy to query.

[jkschneider]

@checketts Is there anything we can do from a partner perspective to reach out and ask AppDynamics to publish a jar to maven central?

[checketts]

Here is the DropWizard extension: https://www.appdynamics.com/community/exchange/extension/dropwizard-monitoring-extension/

I haven't been able to find appd-exts-commons in any maven repos as listed here https://github.com/Appdynamics/dropwizard-monitoring-extension/blob/master/pom.xml#L18

At the least if they documented where it can be found would be the first step. however, their http support is simple enough that I'm comfortable writing to it directly: https://docs.appdynamics.com/display/PRO44/Standalone+Machine+Agent+HTTP+Listener

[pjfanning]

The appd-exts-commons jar is here.
https://github.com/Appdynamics/maven-repo/tree/master/releases/com/appdynamics/appd-exts-commons/1.6.5

In gradle, you can add a setting like (and other build tools have similar extensibility)

repositories {
    maven {
        url "https://github.com/Appdynamics/maven-repo/tree/master/releases"
    }
}

[jkschneider]

@checketts Anything you can share on this?

[checketts]

I'm pretty disappointed with AppDynamics. It has a very hierarchical metrics concept, so I've paused on it and instead have been working toward Splunk metrics support via HEC (http event collector).

Progress has been slow on that one too, but I am making progress when I have time to work on it.

[jkschneider]

I was looking at AQDL from the outside in today and it looks like you might be able to select dimensions using WHERE clauses. Combining this with analytics functions seems like it could work. Did you go down the same path?

[checketts]

No, I didn't look into that part. I didn't know it had a query language.

I was just ramping up on AppD and trying to get the metrics in when I decided to switch focus.

[bjartek]

Any progress on this?

[checketts]

No I've tabled my work on this for now.

[jkschneider]

@bjartek I did have a chat with an AppD engineer at Oracle CodeOne recently. Looks like we'll essentially have two implementations: a hierarchical one passing through the Java agent and a dimensional one going to the Analytics Event API. Evidently, licenses for the agent-based approach are separate from licenses for the Analytics Event API.

[febinrejoe]

Any update on this issue?

[shakuzen]

Any update on this issue?

Only the information available as comments on this issue. Which license do users requesting this have so we can know what solution would work for them? Another solution would be for AppDynamics customers to request AppDynamics to support collecting Micrometer metrics from the AppDynamics Java agent.
Do users want Micrometer metrics in AppDynamics for the purpose of custom metrics in their application or because of some metrics Micrometer provides but AppDynamics does not?

[dermoritz]

I just stumbled across this. For me the main use case is, that you can develop your metrics or insights independently of an metric application like AppDynamics.
It is like logging - nowadays no one uses a specific log-Implementation but Slf4J.

[checketts]

@dermoritz Are you saying you are interested in contributing AppDynamics support? I would be happy to mentor the effort.

[rjmveloso]

I came across this issue when searching for a micrometer exporter to AppDynamics.
Despite all the magic that AppDynamics provides out-of-the-box this integration could be handy for custom metrics.
I've been working in an monitor extension to retrieve statistics from the actuator endpoint: https://github.com/rjmveloso/actuator-monitoring-extension
However I think a better solution could be a micrometer exporter that makes use of the machine agent rest API (if enabled).

[ameyg7]

Team,
What is the update on this?
Do we have full support for AppDynamics integration?
Which all features of AppD can be integrated with Micrometer?

[checketts]

No changes that I am aware of. There is no AppD exporter yet.

[jonatan-ivanov]

It seems though that AppD has support for Prometheus, StatsD, and Elasticsearch but for these you will need extra infrastructure.

[rjmveloso]

If possible to use a monitoring extension, I believe my implementation is a good starting point.
I will provide shortly a draft for an AppDynamicsMeterRegistry. I will be glad if anyone could help on contribution.
By now, one of my main concerns is providing the min value required by AppD metrics. I’m following the same approach used by the implementation of Dynatrace, with a DistributionStatisticConfig with the percentile 0 for that purpose.
Need some time to get it working and tested.

[shakuzen]

We appreciate the offer for a contribution @rjmveloso. Let us know if you have something for us to take a look at. I'm not sure if we have access to AppDynamics for the Micrometer team to test. We can check into that.
Another option that may be available at some point is that I see OpenTelemetry Protocol support in AppDynamics for traces now. If they add support for metrics, you could try our micrometer-registry-otlp module for sending metrics to AppDynamics via OTLP format.

[rjmveloso]

Hello all, a draft approach can be found at the fork https://github.com/rjmveloso/micrometer/tree/micrometer-registry-appdynamics/implementations/micrometer-registry-appdynamics
Please feel free to comment, contribute, fork, as you pleased.

AppDynamics Agent API documented that already aggregated metrics may be reported (which is the case with Micrometer) but requires a min value. As stated, this point has to be implemented since Micrometer does not keep min values.
A recent investigation on Dynatrace implementation, they have replaced the "artifical" zero percentile with a custom DistributionSummary that keeps track of the min value. I will try to implement the same principle in the following days. Currently min value is being reported as "0".

[rjmveloso]

Hello everyone,
Implementation is done and it was tested with our own proper cases.
Anyone who was willing for this would please have a try? @ameyg7 , @bjartek , @dermoritz maybe.
@shakuzen before propose the implementation to Micrometer is it possible for you to have a look in my repo?