Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DVC] Add RequestBasedMetaRepository to enable metadata retrieval directly from server #1467

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

pthirun
Copy link
Contributor

@pthirun pthirun commented Jan 23, 2025

[DVC] Add RequestBasedMetaRepository to enable metadata retrieval directly from server

This PR introduces the use of the /store_properties/{store_name} endpoint to retrieve and cache store metadata for the DaVinci client. The endpoint was implemented in the Venice Server in this PR: #1374.

The RequestBasedMetaRepository leverages this endpoint to maintain a cache of Store and SchemaData objects for requested stores. The cache is refreshed by the NativeMetadataRepository, which this new class extends.

By default, this feature is disabled in the ClientConfig passed to the NativeMetadataRepository. The default configuration preserves the current behavior of using the ThinClientMetaStoreBasedRepository. The relevant configuration field is ClientConfig.useRequestBasedMetaRepository. For more details, see NativeMetadataRepository.getInstance.

How was this PR tested?

Unit and Integration test included in PR.

Does this PR introduce any user-facing changes?

  • No. You can skip the rest of this section.
  • Yes. Make sure to explain your proposed changes and call out the behavior change.

NOTE: behavior will not change by default, feature flag in ClientConfig must be turned on.

@pthirun pthirun requested a review from xunyin8 January 23, 2025 00:46
@pthirun pthirun force-pushed the feature/dvc-request-based-metadata-01-22-2025 branch from 62ff169 to cfa6434 Compare January 23, 2025 22:06
@pthirun pthirun force-pushed the feature/dvc-request-based-metadata-01-22-2025 branch from cfa6434 to 80f0745 Compare January 23, 2025 22:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant