feat(lazer): improve metadata in protobuf #2856

Riateche · 2025-07-16T10:18:41Z

Summary

This PR splits old content of FeedMetadata into two categories:

Essential feed properties that are used for feed aggregation or other functions within Lazer core system (relayer, router, history service): name, exponent, min_publishers, min_rate, expiry_time, market_schedule, state. These are now properties of the Feed message.
Extra metadata used for feed search, documentation, feed catalogs (symbol, description, asset_type, quote_currency), and for matching with external systems (cmc_id, hermes_id). These fields may be used by publishers and by history service (for feed search API), but are otherwise treated as opaque in the Lazer core system. They are now stored as a collection of dynamically typed key-value pairs. This gives the operations team more freedom to extend and change metadata.

Other relevant changes in this PR:

Rename is_activated to is_enabled_in_shard (same for other related fields) to clarify the meaning and intended usage, and to avoid confusion with Feed.state.
Add DynamicValue Rust type to deal with metadata without loss of information.

Rationale

For more flexible metadata.

How has this been tested?

Current tests cover my changes
Added new tests
Manually tested the code

vercel · 2025-07-16T10:18:46Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
api-reference	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am
component-library	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am
developer-hub	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am
entropy-debugger	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am
entropy-explorer	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am
insights	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am
proposals	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am
staking	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jul 17, 2025 0:43am

lazer/publisher_sdk/proto/state.proto

ali-behjati · 2025-07-16T12:08:19Z

lazer/publisher_sdk/proto/state.proto

+    optional FeedState state = 107;
+
+
+    // [required] Feed status in the current shard. Disabled feeds will not be visible in


sorry i got a bit lost here with feedState. what cases does it cover that feed state is not :? is it like we add a same feed to all the shards that we need this?

Feed state is a consumer-facing and publisher-facing property that's also used to limit access to the feeds. is_enabled_in_shard is a purely internal thing for moving live feeds between shards.

ali-behjati · 2025-07-16T12:09:47Z

lazer/publisher_sdk/rust/src/dynamic_value.rs

+}
+
+impl From<&DynamicValue> for ProtobufDynamicValue {
+    fn from(value: &DynamicValue) -> Self {


i think generally the expectation from From is to consume the object.

Well, it consumes the object, but the object is a reference. I don't think it's particularly unusual. If you look at std impls, there is a bunch of implementations on references. I think it's reasonable to impl From for references when conversion from a reference is more efficient than clone() followed by a conversion. In this case we can avoid cloning Vec and BTreeMap unnecessarily.

keyvankhademi · 2025-07-16T14:28:11Z

lazer/publisher_sdk/proto/state.proto

    STABLE = 1;
+    // Inactive feeds are not available to consumers or publishers.


it is technically available to publishers now.

What do you mean by "technically available"? Can you suggest a proper description of this state?

I think he means publishers can publish to them regardless of the state of the feed? We should change this in the future though.

Perhaps you can specify they are feeds which can/can't be subscribed to?

Suggested change

// Inactive feeds are not available to consumers or publishers.

// Inactive feeds are not available to consumers.

I think the only difference in INACTIVE and COMING_SOON is to inform the end user that inactive feed will never become active.

keyvankhademi · 2025-07-16T14:29:39Z

lazer/publisher_sdk/proto/state.proto

+    optional string market_schedule = 106;
+    // [required] Feed state
+    optional FeedState state = 107;
+


I think we should add asset_type here too. Because we will be using asset_type to determine what kind of feed type is expected (e.g funding-rate vs regular price).

let's call it feed type.

@ali-behjati are you saying it's different from asset_type and we have both? Because I was thinking we could infer feed type based on asset_type. and multiple asset_types can have the same feed type.

I also prefer to have feed type (Price/FundingRate) because those options are what we care about in the implementation. (It's already like this in the transaction protobuf). Asset type has a lot of options and I think it's better to leave it in the dynamic metadata.

I called it FeedKind because type is annoying to use in Rust 😅

…o protocol

darunrs

LGTM but will let Keyvan/Ali give final approval since they left their own comments already. Just a handful of small questions.

darunrs · 2025-07-17T15:48:30Z

lazer/publisher_sdk/proto/governance_instruction.proto

 message AddFeed {
-    // [required] ID of the feed. Must be unique (within the shard).
+    // [required]


Why was this description removed? Is it no longer accurate?

If a feed ID needs to change as a result of moving to another shard which already has that ID (further, how do we prevent this from happening?), how would that impact subscribing to the feed? Is this ID separate from the customer/publisher facing feed ID?

darunrs · 2025-07-17T15:51:00Z

lazer/publisher_sdk/proto/governance_instruction.proto

-    // IDs of publishers enabled for this feed.
-    repeated uint32 permissioned_publishers = 3;
+    // [required]
+    optional DynamicValue.Map metadata = 3;


Any reason we skip field ID 2 if this is not in use yet?

darunrs · 2025-07-17T15:54:14Z

lazer/publisher_sdk/proto/state.proto

    STABLE = 1;
+    // Inactive feeds are not available to consumers or publishers.


I think he means publishers can publish to them regardless of the state of the feed? We should change this in the future though.

Perhaps you can specify they are feeds which can/can't be subscribed to?

darunrs · 2025-07-17T15:55:20Z

lazer/publisher_sdk/proto/state.proto

+// Feed kind determines the set of data fields available in the feed.
+// It also determines the kind of data accepted from publishers for this feed
+// (`PriceUpdate` or `FundingRateUpdate`).
+enum FeedKind {


I intended to introduce this in some form. I will use the state enum to guard rail what updates a feed is able to accept.

we should use this for guard railing updates

darunrs · 2025-07-17T16:00:41Z

lazer/publisher_sdk/proto/state.proto

    // multiple shards, it must only be active in one of them at each time.
-    // To enforce this, `pending_activation` and `pending_deactivation` fields
+    // To enforce this, `enable_in_shard_timestamp` and `disable_in_shard_timestamp` fields


To confirm my understanding, Aggregator pulls all shards. So, it is able to handle this switch in logic?

keyvankhademi · 2025-07-17T16:23:09Z

Does this break the current agent transactions?
cc: @bplatak

keyvankhademi · 2025-07-17T16:23:52Z

lazer/publisher_sdk/proto/governance_instruction.proto

+        UpdateFeedProperties update_feed_properties = 101;
+        UpdateFeedMetadata update_feed_metadata = 102;
+        EnableFeedInShard enable_feed_in_shard = 103;
+        DisableFeedInShard disable_feed_in_shard = 104;
        google.protobuf.Empty remove_feed = 199;


we probably don't need this, since we will make feeds inactive instead of deleting.

Riateche added 2 commits July 15, 2025 13:44

feat(lazer): update metadata proto, rename and clarify docs

48521b5

feat(lazer): add DynamicValue parsed type

b1e808e

test(lazer): add DynamicValue tests

1cb5d89

Riateche marked this pull request as ready for review July 16, 2025 10:47

Riateche requested a review from darunrs as a code owner July 16, 2025 10:47

vercel bot deployed to Preview – staking July 16, 2025 10:50 View deployment

vercel bot deployed to Preview – developer-hub July 16, 2025 10:50 View deployment

vercel bot deployed to Preview – entropy-explorer July 16, 2025 10:50 View deployment

vercel bot deployed to Preview – insights July 16, 2025 10:51 View deployment

vercel bot deployed to Preview – entropy-debugger July 16, 2025 10:52 View deployment

vercel bot deployed to Preview – component-library July 16, 2025 10:52 View deployment

vercel bot deployed to Preview – api-reference July 16, 2025 10:53 View deployment

vercel bot deployed to Preview – proposals July 16, 2025 10:54 View deployment

ali-behjati reviewed Jul 16, 2025

View reviewed changes

keyvankhademi reviewed Jul 16, 2025

View reviewed changes

feat(lazer): add FeedKind to protocol

0129da1

vercel bot deployed to Preview – component-library July 17, 2025 09:51 View deployment

vercel bot deployed to Preview – entropy-debugger July 17, 2025 09:52 View deployment

vercel bot deployed to Preview – entropy-explorer July 17, 2025 09:52 View deployment

vercel bot deployed to Preview – insights July 17, 2025 09:53 View deployment

vercel bot deployed to Preview – api-reference July 17, 2025 09:53 View deployment

vercel bot deployed to Preview – developer-hub July 17, 2025 09:54 View deployment

vercel bot deployed to Preview – staking July 17, 2025 09:55 View deployment

vercel bot deployed to Preview – proposals July 17, 2025 09:55 View deployment

feat(lazer): add feed kind to protocol

f8c719b

vercel bot deployed to Preview – entropy-debugger July 17, 2025 10:01 View deployment

vercel bot deployed to Preview – entropy-explorer July 17, 2025 10:01 View deployment

vercel bot deployed to Preview – proposals July 17, 2025 10:02 View deployment

vercel bot deployed to Preview – component-library July 17, 2025 10:02 View deployment

vercel bot deployed to Preview – developer-hub July 17, 2025 10:03 View deployment

vercel bot deployed to Preview – staking July 17, 2025 10:04 View deployment

vercel bot deployed to Preview – api-reference July 17, 2025 10:04 View deployment

vercel bot deployed to Preview – insights July 17, 2025 10:04 View deployment

fix(lazer): expose dynamic_value protobuf as rust module, move type t…

1054759

…o protocol

vercel bot deployed to Preview – component-library July 17, 2025 12:39 View deployment

vercel bot deployed to Preview – entropy-explorer July 17, 2025 12:40 View deployment

vercel bot deployed to Preview – developer-hub July 17, 2025 12:40 View deployment

vercel bot deployed to Preview – staking July 17, 2025 12:40 View deployment

vercel bot deployed to Preview – insights July 17, 2025 12:41 View deployment

vercel bot deployed to Preview – entropy-debugger July 17, 2025 12:42 View deployment

vercel bot deployed to Preview – api-reference July 17, 2025 12:42 View deployment

vercel bot deployed to Preview – proposals July 17, 2025 12:43 View deployment

darunrs reviewed Jul 17, 2025

View reviewed changes

keyvankhademi reviewed Jul 17, 2025

View reviewed changes

		optional FeedState state = 107;


		// [required] Feed status in the current shard. Disabled feeds will not be visible in

		STABLE = 1;
		// Inactive feeds are not available to consumers or publishers.

feat(lazer): improve metadata in protobuf #2856

Are you sure you want to change the base?

feat(lazer): improve metadata in protobuf #2856

Conversation

Riateche commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Rationale

How has this been tested?

Uh oh!

vercel bot commented Jul 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darunrs left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

keyvankhademi commented Jul 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Riateche commented Jul 16, 2025 •

edited

Loading

vercel bot commented Jul 16, 2025 •

edited

Loading