Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-14484: Decouple UnifiedLog and RemoteLogManager #18460

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from

Conversation

mimaison
Copy link
Member

@mimaison mimaison commented Jan 9, 2025

This introduces an interface, AsyncOffsetReader, to decouple UnifiedLog and RemoteLogManager.

This removes the last dependency UnifiedLog has on core so we can finally move it to the storage module.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@github-actions github-actions bot added core Kafka Broker storage Pull requests that target the storage module tiered-storage Related to the Tiered Storage feature labels Jan 9, 2025

public interface AsyncOffsetReader {

AsyncOffsetReadFutureHolder<OffsetResultHolder.FileRecordsOrError> asyncOffsetRead(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we think this interface is useful long-term or is this just a temporary thing while we move things around?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately the goal is to also move RemoteLogManager to the storage module, so in theory we could remove it once that is done.
RemoteLogManager still depends on a bunch of core classes (I think we'll need to introduce a few interfaces to decouple it from ReplicaManager/Partition too) so it not quite ready to move yet, hopefully in the next few months.

I've got to admit I've not really thought a lot about it yet, but it may make sense to keep the interface to have a boundary between UnifiedLog and RemoteLogManager as even if they are both in storage, they'll be in different packages.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either way, I'll add some javadoc to the interface to describe its (current) role.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Kafka Broker storage Pull requests that target the storage module tiered-storage Related to the Tiered Storage feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants