KAFKA-19858: Move the __remote_log_metadata to internal #20822
Conversation
Signed-off-by: stroller <[email protected]>
github-actions
bot
added
triage
AndrewJSchofield
added
kip
| public static final String LEGAL_CHARS = "[a-zA-Z0-9._-]"; | ||
|
|
||
| private static final Set<String> INTERNAL_TOPICS = Set.of(GROUP_METADATA_TOPIC_NAME, TRANSACTION_STATE_TOPIC_NAME, SHARE_GROUP_STATE_TOPIC_NAME); | ||
| private static final Set<String> INTERNAL_TOPICS = Set.of(GROUP_METADATA_TOPIC_NAME, TRANSACTION_STATE_TOPIC_NAME, SHARE_GROUP_STATE_TOPIC_NAME, REMOTE_LOG_METADATA_TOPIC_NAME); |
There was a problem hiding this comment.
I believe internal topics are those fundamental to kafka's core and are handled uniquely, not via standard producer/consumer logic. Therefore, REMOTE_LOG_METADATA_TOPIC_NAME is similar to connector topics rather than an internal topic
@AndrewJSchofield @jiafu1115 WDYT?
There was a problem hiding this comment.
You may be correct, but I think the main difference is that internal topics are filtered out of listing topics by default. I do think that we need to tread carefully here. If there is any doubt about making this an internal topic, we should not do it.
There was a problem hiding this comment.
@AndrewJSchofield @chia7712 I agree. I just created this PR as an example to show what the code changes would look like if we decided to proceed. I think we can leave it here for future discussion: It involve how we define the internal topic: not allow to see or not allow to be write by standard producer.
Thanks
| sendResponseCallback(Map.empty) | ||
| else { | ||
| val internalTopicsAllowed = request.header.clientId == "__admin_client" | ||
| val internalTopicsAllowed = request.header.clientId != null && request.header.clientId.startsWith("__") |
There was a problem hiding this comment.
we should not touch this "backdoor" - I have posted a discussion on the https://issues.apache.org/jira/browse/KAFKA-5246
3 participants
No description provided.