社区版支持基于流量吞吐压力调度的问题

[linux-time]

我正在试用automq社区版，看到社区版支持基于流量吞吐压力调度。

发现流量相差蛮大的，但是没有触发controller调度。

这是auto-balancer.log的一段日志
[2025-02-11 10:54:47,245] INFO NetworkInUsageDistributionGoal expected dist bound: NWIn=10.48MB/s-NWIn=12.8MB/s (kafka.autobalancer) [2025-02-11 10:54:47,245] INFO NetworkOutUsageDistributionGoal expected dist bound: NWOut=10.78MB/s-NWOut=13.18MB/s (kafka.autobalancer) [2025-02-11 10:54:47,245] INFO [AnomalyDetector id=0] Expected load change: brokerId=0, NWIn=11.47MB/s (-0Bytes/s), NWOut=11.7MB/s (-0Bytes/s) (kafka.autobalancer) [2025-02-11 10:54:47,245] INFO [AnomalyDetector id=0] Expected load change: brokerId=1000, NWIn=11.45MB/s (-0Bytes/s), NWOut=11.62MB/s (-0Bytes/s) (kafka.autobalancer) [2025-02-11 10:54:47,245] INFO [AnomalyDetector id=0] Expected load change: brokerId=1001, NWIn=11.99MB/s (-0Bytes/s), NWOut=12.61MB/s (-0Bytes/s) (kafka.autobalancer) [2025-02-11 10:54:47,245] INFO [AnomalyDetector id=0] Total actions num: 0, split to 0 batches, estimated time to complete: 0ms (kafka.autobalancer) [2025-02-11 10:54:47,245] INFO [AnomalyDetector id=0] Detect finished, next detect will be after 60000 ms (kafka.autobalancer) [2025-02-11 10:55:47,245] INFO [AnomalyDetector id=0] Start detect (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Broker status: Broker{brokerId=0, outOfDate=false, slow=false, timestamp=1739242537544, Loads={NWIn=11.79MB/s (trusted), NWOut=11.9MB/s (trusted)}} (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Broker status: Broker{brokerId=1000, outOfDate=false, slow=true, timestamp=1739242543103, Loads={NWIn=11.15MB/s (trusted), NWOut=10.89MB/s (trusted)}} (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Broker status: Broker{brokerId=1001, outOfDate=false, slow=false, timestamp=1739242543375, Loads={NWIn=12.17MB/s (trusted), NWOut=12.4MB/s (trusted)}} (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO NetworkInUsageDistributionGoal expected dist bound: NWIn=10.53MB/s-NWIn=12.87MB/s (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO NetworkOutUsageDistributionGoal expected dist bound: NWOut=10.56MB/s-NWOut=12.9MB/s (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Expected load change: brokerId=0, NWIn=11.79MB/s (-0Bytes/s), NWOut=11.9MB/s (-0Bytes/s) (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Expected load change: brokerId=1000, NWIn=11.15MB/s (-0Bytes/s), NWOut=10.89MB/s (-0Bytes/s) (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Expected load change: brokerId=1001, NWIn=12.17MB/s (-0Bytes/s), NWOut=12.4MB/s (-0Bytes/s) (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Total actions num: 0, split to 0 batches, estimated time to complete: 0ms (kafka.autobalancer) [2025-02-11 10:55:47,250] INFO [AnomalyDetector id=0] Detect finished, next detect will be after 60000 ms (kafka.autobalancer)

Partitions skew和Leader skew已相差很大

controller已配置了autobalancer.controller.enable=true

[SCNieh]

你好，从 autobalancer.log 来看，其统计到的各 broker 流量均处于期望的均衡范围内，所以并没有发起任何分区迁移，日志上显示各 broker 的发送和消费流量都大约处于 11MB/s ~ 12MB/s 区间，该流量数据可与指标 kafka_broker_network_io_bytes_total 进行交叉验证。此外社区版的 autobalancer 仅支持流量负载均衡，并不考虑分区数量的均衡，所以 partitions skew 并不会导致调度。如果对调度相关的行为还有更多问题的话，我们可以拉一个微信群详细探讨

[linux-time]

你好，从 autobalancer.log 来看，其统计到的各 broker 流量均处于期望的均衡范围内，所以并没有发起任何分区迁移，日志上显示各 broker 的发送和消费流量都大约处于 11MB/s ~ 12MB/s 区间，该流量数据可与指标 kafka_broker_network_io_bytes_total 进行交叉验证。此外社区版的 autobalancer 仅支持流量负载均衡，并不考虑分区数量的均衡，所以 partitions skew 并不会导致调度。如果对调度相关的行为还有更多问题的话，我们可以拉一个微信群详细探讨

谢谢