Kafka UI produces a lot of API calls

[42kbit]

Hi, our organization experiences a following problem - it seems like Kafka UI, passively, about every 30 seconds updates metrics and causes a lot of IAM API calls (we are using MSK), for which we are billed by GuardDuty, is there a way to disable metric updates or decrease number of API calls?

CloudTrail statistics

Action	Number of events
DescribeTopicDynamicConfiguration	624575
CreateTopic	312329
DeleteTopic	312329
AlterTopicDynamicConfiguration	312329
AlterTopic	312329
DescribeClusterDynamicConfiguration	3794

No one was using this Kafka-UI instance, and it was scaled-out during business hours and scaled-in during non-business-hours

This is the statistics for around 3-4 days

Config

Sensitive values are removed

kafka:
  clusters:
  - bootstrapServers: "..."
    name: ...
    properties:
      security.protocol: SASL_SSL
      sasl.mechanism: AWS_MSK_IAM
      sasl.client.callback.handler.class: software.amazon.msk.auth.iam.IAMClientCallbackHandler
      sasl.jaas.config: software.amazon.msk.auth.iam.IAMLoginModule required awsRoleArn="arn:aws:iam::...:role/..." awsRoleSessionName="...";

    serde:
      - name: GlueSchemaRegistry
        className: io.kafbat.ui.serde.glue.GlueSerde
        filePath: /opt/glue-serde-jar-with-dependencies.jar
        properties:
          region: ...
          registry: ...
          keySchemaNameTemplate: '%s-key'
          valueSchemaNameTemplate: '%s-value'

          roleArn: ...

Thanks in advance

[germanosin]

Hi. sure
yaml kafka.update-metrics-rate-millis
or env
KAFKA_UPDATE_METRICS_RATE_MILLIS

[42kbit]

Hi, thanks we'll try it out

A couple of questions, if you will:

1. Is it described somewhere in the docs, I've searched through them but haven't found it :(
2. Is there a way to disable those metrics entirely? Or for now, as a workaround, we should set an absurdly large value
3. Just wondering, why does it execute CreateTopic and DeleteTopic?

Thank you very much!

[germanosin]

Hi, thanks we'll try it out

A couple of questions, if you will:

1. Is it described somewhere in the docs, I've searched through them but haven't found it :(
2. Is there a way to disable those metrics entirely? Or for now, as a workaround, we should set an absurdly large value
3. Just wondering, why does it execute CreateTopic and DeleteTopic?

Thank you very much!

I'm 100% sure that it is not executing CreateTopic/DeleteTopic/AlterTopicDynamicConfiguration/AlterTopic, but only
DescribeTopicDynamicConfiguration
DescribeClusterDynamicConfiguration

[42kbit]

We've set the update-metrics-rate-millis to 86,400,000 ms (once every 24 hours) for now to check if that would decrease the number of API calls, we'll continue to monitor it and keep you posted 😄

Mind if I ask, are you planning to optionally disable metrics update or describe it in the docs somewhere for someone with similar issue? I think increased GuardDuty bills would be a potential drawback to other large organizations using kafka-ui. Thanks a lot for your work!

Also related GitHub discussion that I've found regarding this topic - #1425