[QUESTION] partition offline or error occurs frequently

[linshgun]

Version & Environment

• AutoMQ version v1.3.0

What went wrong?

When no traffic limit is configured and the machine load is not high, partition offline often occurs and error increases. LEADER_NOT_AVAILABLE and NOT_OR_follower account for the largest proportion of followers, leading to frequent group rebalancing. Consumption instability, error reporting, such as “commit failed”, what factors may be responsible, and what parameters can I optimize

[SCNieh]

Hi, the error you're seeing typically happens when partition gets reassigned. AutoMQ has a built-in AutoBalancer that continuously monitors the cluster status and tries to balance the traffic by moving partitions around. To verify if the partition reassignment is caused by the AutoBalancer, please check the auto-balancer.log. Additionally, there are several parameters you can adjust to better fit your conditions. For detailed explanations on the relevant parameters, please refer to kafka.autobalancer.config.AutoBalancerControllerConfig.

[daniel-y]

Hi @linshgun, AutoMQ is a data-intensive software. Even if you can obtain all the source codes from GitHub, running AutoMQ at scale in production remains a challenge. If you need any support from our team, we're happy to help and can set up a WeChat group or Slack channel for you.