Skip to content

Conversation

j-santander
Copy link

This PR adds three admin operations that were not previously supported:

  • AdminClient::list_offsets: List offsets for the specified topic_partitions.
  • AdminClient::describe_consumer_groups: Provides a description on the requested consumer groups.
  • AdminClient::list_consumer_group_offsets: List offset information for the topics subscribed by a consumer group.

Implementation notes:

  • This mostly my first Rust code, so please comment and critique. The work started because I was in search of something to do after having read rust tutorials and documents. I decided for re-doing a python tool I've recently coded for Kafka administration tasks and I found that someof the operations I was using in python were missing from the admin
    interface in rdkafka. Although the implementation I did in my rust tool was different, I decided to try my hand at enhancing rdkafka and this PR is the result.
  • For all three operations I've used a similar pattern for the result: KafkaResult<Vec<Result<TYPE, (KEY, KafkaError)>. This is similar to the existing GroupResult. The alternative was to insert the error value in the TYPE record, following closer the librdkafka structure, but I felt it was less in line with rust conventions.
  • I felt that I was adding many types and that it was best to define them in a different module/file for each operation. What I've done is re-exporting them from the admin module.
  • I've defined types::RDKafkaXXX for all the bindings::rd_kafka_xxxx types, using the RDKafkaXXX alias in the code, keeping the function calls to rdsys::rd_kafka_xxxx
  • Regarding GroupResult and ConsumerGroupResult: Technically they both come from the same rd_kafka_group_result_t, GroupResul just extracting the consumer group name. The GroupResult type was already used by the delete_groups operation. In order not to break the existing API, I'm maintaining both. Perhaps in the future they could be enhanced (or perhaps it is well as it is, as the additional information is not meaningful in the delete_groups case).
  • Added use cases covering the different operations. I've probably not been exhaustive, but I hope the coverage is enough.
  • Documentation and field and structures descriptions I've taken from a mixture of the java, python and the protocol documentation. Hope it is in line with the rest of the documentation provided.

This PR adds three admin operations that were not previously supported:
- AdminClient::list_offsets: List offsets for the specified
topic_partitions.
- AdminClient::describe_consumer_groups: Provides a description on the
  requested consumer groups.
- AdminClient::list_consumer_group_offsets: List offset information for
  the topics subscribed by a consumer group.

Implementation notes:
- This mostly my first Rust code, so please comment and critique. The
work started because I was in search of something to do after having
read rust tutorials and documents. I decided for re-doing a python tool
I've recently coded for Kafka administration tasks and I found that some
of the operations I was using in python were missing from the admin
interface in rdkafka. Although the implementation I did in my rust tool
was different, I decided to try my hand at enhancing rdkafka and this PR
 is the result.
- For all three operations I've used a similar pattern for the result:
`KafkaResult<Vec<Result<TYPE, (KEY, KafkaError)>`. This is similar to
the existing GroupResult. The alternative was to insert the error value
in the TYPE record, following closer the librdkafka structure, but I
felt it was less in line with rust conventions.
- I felt that I was adding many types and that it was best to define
them in a different module/file for each operation. What I've done is
re-exporting them from the admin module.
- I've defined types::RDKafkaXXX for all the bindings::rd_kafka_xxxx
types, using the RDKafkaXXX alias in the code, keeping the function
calls to rdsys::rd_kafka_xxxx
- Regarding GroupResult and ConsumerGroupResult: Technically they both
come from the same rd_kafka_group_result_t, GroupResul just extracting
the consumer group name. The GroupResult type was already used by the
delete_groups operation. In order not to break the  existing API, I'm
maintaining both. Perhaps in the future they could be enhanced (or
perhaps it is well as it is, as the additional information is not
meaningful in the delete_groups case).
- Added use cases covering the different operations. I've probably not
been exhaustive, but I hope the coverage is enough.
- Documentation and field and structures descriptions I've taken from a
mixture of the java, python and the protocol documentation. Hope it
is in line with the rest of the documentation provided.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant