Skip to content

Commit 29b205a

Browse files
luis-herasmePJColombo
luis-herasme
and
PJColombo
authored
Eth price syncer using Chainlink oracle (#731)
* feat(eth-price): add initial implementation for eth-price package with client and price retrieval functions * fix(test): use contractAddress variable in getPriceByTimestamp test * feat(eth-price): update package name and structure, add index file for exports * refactor(eth-price): rename timestamp parameter to targetTimestamp for clarity * refactor(eth-price): clarify parameter description for targetTimestamp in getPriceByTimestamp function * refactor(eth-price): rename Data type to PriceData for clarity in getPriceByTimestamp function * feat(db): add ETHPrice model and migration for eth_price table * feat(env): add ETH_PRICE_SYNCER_CRON_PATTERN to environment configuration * feat(syncers): add ETHPriceSyncer for fetching and storing ETH price data * docs(env): clarify description for ETH_PRICE_SYNCER_CRON_PATTERN in documentation * docs(env): update comment for ETH_PRICE_SYNCER_CRON_PATTERN to include default frequency * fix(get-price): improve error message for missing ETH price data * feat(env): add RPC_URL for Ethereum RPC endpoint configuration * feat(db): add EthUsdPrice table and update syncer to use new model * fix(get-price): rename variable for clarity in getPriceByTimestamp function * feat(docs): add detailed JSDoc comments for binary search and phase aggregator functions * feat(docs): enhance JSDoc comments for getPhaseAggregators and getPriceByTimestamp functions * refactor: rename timestamp variables for clarity and consistency across functions * fix(get-price): rename targetTimestamp variable to targetTimestampSeconds for consistency in getPriceByTimestamp tests * chore: rename test file * feat: implement getPriceInRange function and corresponding tests * feat(db): set eth usd price model timestamp field to unique * refactor(eth-price): centralized and unified all ETH price getter logic into a new class called PriceFeedFinder * test(eth-price): spin up a local forked anvil instance to run price feed tests against * ci: set up foundry before running tests * test: add anvil to test package * chore(eth-price): remove pnpm lock package * feat: add the following: - Add support for parametrizing eth price syncer chain id - Fix price feed finder issue where binary search stops at first valid price value, it should continue until it finds the closest one - Improve price feed finder by loading aggregators only once when creating an instance * refactor(eth-price): polish returning types, improve private functions and improve comments * chore(eth-price): remove unused functions from abis * style(eth-price): simplify `PriceFeedFinder` name * chore: rename `eth-price` package * chore(price-feed): add version, lint and ci missing commands * chore(price-feed): include missing ts files to `tsconfig` file * chore: apply lint fixes * chore: update pnpm lock file * feat(db): add price feed state model * fix(price-feed): return formatted prices * feat(syncers): update ETH price syncer to record prices based on configurable granularity * feat(docs): add eth price syncer missing env vars * chore: add missing env vars to turbo.json * test(syncers): add eth price syncer tests * chore: add `VITEST_MAINNET_FORK_URL` to `turbo.json` * test: try to get `VITEST_MAINNET_FORK_URL` from nodejs envs * test(syncer): coerce chain id * style(api): remove `console.log` * chore: resolve lint issues Sort deps and standarize viem version across all packages * test(test): add and expose test viem client * fix(syncers): inject price feed instance into eth price syncer instead of creating internally * style(price-feed): fix abi file name * fix(docs): remove eth price syncer time tolerance default value * fix(price-feed): fix abi import route * chore(price-feed): remove redundant `@blobscan/test` dep * chore: add changesets --------- Co-authored-by: PJColombo <[email protected]>
1 parent b6217e8 commit 29b205a

File tree

39 files changed

+1904
-99
lines changed

39 files changed

+1904
-99
lines changed

‎.changeset/flat-cobras-lay.md

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
"@blobscan/rest-api-server": minor
3+
"@blobscan/syncers": minor
4+
---
5+
6+
Added ETH price syncer

‎.changeset/tame-gorillas-think.md

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"@blobscan/db": minor
3+
---
4+
5+
Added ETH price feed state and eth usd price models

‎.env.test

+6
Original file line numberDiff line numberDiff line change
@@ -33,3 +33,9 @@ WEAVEVM_API_KEY=weavevm-api-key
3333

3434
BLOB_PROPAGATOR_TMP_BLOB_STORAGE=FILE_SYSTEM
3535
REDIS_URI=redis://localhost:6379/1
36+
37+
38+
ETH_PRICE_SYNCER_ETH_USD_PRICE_FEED_CONTRACT_ADDRESS=0x5f4eC3Df9cbd43714FE2740f5E3616155c5b8419
39+
ETH_PRICE_SYNCER_CHAIN_ID=1
40+
ETH_PRICE_SYNCER_CHAIN_JSON_RPC_URL=http://127.0.0.1:8545
41+
ETH_PRICE_SYNCER_CRON_PATTERN="0 * * * *"

‎.github/workflows/ci.yml

+7
Original file line numberDiff line numberDiff line change
@@ -133,8 +133,15 @@ jobs:
133133
- name: Generate Prisma Client
134134
run: pnpm db:generate
135135

136+
- name: Set up Foundry
137+
uses: foundry-rs/foundry-toolchain@v1
138+
with:
139+
version: stable
140+
136141
- name: Test
137142
run: pnpm coverage
143+
env:
144+
VITEST_MAINNET_FORK_URL: ${{ secrets.VITEST_MAINNET_FORK_URL }}
138145

139146
- name: Upload coverage reports to Codecov
140147
uses: codecov/codecov-action@v4

‎apps/docs/src/app/docs/environment/page.md

+41-35
Large diffs are not rendered by default.

‎apps/rest-api-server/package.json

+3-1
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
"@blobscan/env": "workspace:^0.1.0",
2222
"@blobscan/logger": "workspace:^0.1.2",
2323
"@blobscan/open-telemetry": "workspace:^0.0.9",
24+
"@blobscan/price-feed": "workspace:^0.0.1",
2425
"@blobscan/syncers": "workspace:^0.3.5",
2526
"@blobscan/zod": "workspace:^0.1.0",
2627
"@opentelemetry/instrumentation-express": "^0.33.0",
@@ -30,7 +31,8 @@
3031
"express": "^4.18.2",
3132
"morgan": "^1.10.0",
3233
"swagger-ui-express": "^4.6.2",
33-
"trpc-openapi": "^1.2.0"
34+
"trpc-openapi": "^1.2.0",
35+
"viem": "^2.17.4"
3436
},
3537
"devDependencies": {
3638
"@types/cors": "^2.8.13",

‎apps/rest-api-server/src/index.ts

+57-53
Original file line numberDiff line numberDiff line change
@@ -27,60 +27,64 @@ collectDefaultMetrics();
2727

2828
printBanner();
2929

30-
const closeSyncers = setUpSyncers();
31-
32-
const app = express();
33-
34-
app.use(cors());
35-
app.use(bodyParser.json({ limit: "3mb" }));
36-
app.use(morganMiddleware);
37-
38-
app.get("/metrics", metricsHandler);
39-
40-
// Serve Swagger UI with our OpenAPI schema
41-
app.use("/", swaggerUi.serve);
42-
app.get("/", swaggerUi.setup(openApiDocument));
43-
44-
// Handle incoming OpenAPI requests
45-
app.use(
46-
"/",
47-
createOpenApiExpressMiddleware({
48-
router: appRouter,
49-
createContext: createTRPCContext({
50-
scope: "rest-api",
51-
}),
52-
onError({ error }) {
53-
Sentry.captureException(error);
54-
55-
logger.error(error);
56-
},
57-
})
58-
);
59-
60-
const server = app.listen(env.BLOBSCAN_API_PORT, () => {
61-
logger.info(`Server started on http://0.0.0.0:${env.BLOBSCAN_API_PORT}`);
62-
});
63-
64-
async function gracefulShutdown(signal: string) {
65-
logger.debug(`Received ${signal}. Shutting down...`);
66-
67-
await apiGracefulShutdown()
68-
.finally(async () => {
69-
await closeSyncers();
30+
async function main() {
31+
const closeSyncers = await setUpSyncers();
32+
33+
const app = express();
34+
35+
app.use(cors());
36+
app.use(bodyParser.json({ limit: "3mb" }));
37+
app.use(morganMiddleware);
38+
39+
app.get("/metrics", metricsHandler);
40+
41+
// Serve Swagger UI with our OpenAPI schema
42+
app.use("/", swaggerUi.serve);
43+
app.get("/", swaggerUi.setup(openApiDocument));
44+
45+
// Handle incoming OpenAPI requests
46+
app.use(
47+
"/",
48+
createOpenApiExpressMiddleware({
49+
router: appRouter,
50+
createContext: createTRPCContext({
51+
scope: "rest-api",
52+
}),
53+
onError({ error }) {
54+
Sentry.captureException(error);
55+
56+
logger.error(error);
57+
},
7058
})
71-
.finally(() => {
72-
server.close(() => {
73-
logger.debug("Server shut down successfully");
59+
);
60+
61+
const server = app.listen(env.BLOBSCAN_API_PORT, () => {
62+
logger.info(`Server started on http://0.0.0.0:${env.BLOBSCAN_API_PORT}`);
63+
});
64+
65+
async function gracefulShutdown(signal: string) {
66+
logger.debug(`Received ${signal}. Shutting down...`);
67+
68+
await apiGracefulShutdown()
69+
.finally(async () => {
70+
await closeSyncers();
71+
})
72+
.finally(() => {
73+
server.close(() => {
74+
logger.debug("Server shut down successfully");
75+
});
7476
});
75-
});
76-
}
77+
}
78+
79+
// Listen for TERM signal .e.g. kill
80+
process.on("SIGTERM", async () => {
81+
await gracefulShutdown("SIGTERM");
82+
});
7783

78-
// Listen for TERM signal .e.g. kill
79-
process.on("SIGTERM", async () => {
80-
await gracefulShutdown("SIGTERM");
81-
});
84+
// Listen for INT signal e.g. Ctrl-C
85+
process.on("SIGINT", async () => {
86+
await gracefulShutdown("SIGINT");
87+
});
88+
}
8289

83-
// Listen for INT signal e.g. Ctrl-C
84-
process.on("SIGINT", async () => {
85-
await gracefulShutdown("SIGINT");
86-
});
90+
main();

‎apps/rest-api-server/src/syncers.ts

+44-2
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,13 @@
1+
import type { PublicClient } from "viem";
2+
import { createPublicClient, http } from "viem";
3+
import * as chains from "viem/chains";
4+
15
import { env } from "@blobscan/env";
6+
import { PriceFeed } from "@blobscan/price-feed";
27
import type { BaseSyncer } from "@blobscan/syncers";
38
import {
49
DailyStatsSyncer,
10+
ETHPriceSyncer,
511
OverallStatsSyncer,
612
SwarmStampSyncer,
713
createRedisConnection,
@@ -10,7 +16,7 @@ import {
1016
import { logger } from "./logger";
1117
import { getNetworkDencunForkSlot } from "./utils";
1218

13-
export function setUpSyncers() {
19+
export async function setUpSyncers() {
1420
const connection = createRedisConnection(env.REDIS_URI);
1521
const syncers: BaseSyncer[] = [];
1622

@@ -49,7 +55,43 @@ export function setUpSyncers() {
4955
})
5056
);
5157

52-
Promise.all(syncers.map((syncer) => syncer.start()));
58+
if (
59+
env.ETH_PRICE_SYNCER_CHAIN_ID &&
60+
env.ETH_PRICE_SYNCER_CHAIN_JSON_RPC_URL &&
61+
env.ETH_PRICE_SYNCER_ETH_USD_PRICE_FEED_CONTRACT_ADDRESS
62+
) {
63+
const chain = Object.values(chains).find(
64+
(c) => c.id === env.ETH_PRICE_SYNCER_CHAIN_ID
65+
);
66+
67+
if (!chain) {
68+
throw new Error(
69+
`Can't initialize ETH price syncer: chain with id ${env.ETH_PRICE_SYNCER_CHAIN_ID} not found`
70+
);
71+
}
72+
73+
const client = createPublicClient({
74+
chain,
75+
transport: http(env.ETH_PRICE_SYNCER_CHAIN_JSON_RPC_URL),
76+
});
77+
78+
const ethUsdPriceFeed = await PriceFeed.create({
79+
client: client as PublicClient,
80+
dataFeedContractAddress:
81+
env.ETH_PRICE_SYNCER_ETH_USD_PRICE_FEED_CONTRACT_ADDRESS as `0x${string}`,
82+
timeTolerance: env.ETH_PRICE_SYNCER_TIME_TOLERANCE,
83+
});
84+
85+
syncers.push(
86+
new ETHPriceSyncer({
87+
cronPattern: env.ETH_PRICE_SYNCER_CRON_PATTERN,
88+
redisUriOrConnection: connection,
89+
ethUsdPriceFeed,
90+
})
91+
);
92+
}
93+
94+
await Promise.all(syncers.map((syncer) => syncer.start()));
5395

5496
return () => {
5597
let teardownPromise = Promise.resolve();

‎package.json

+1
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,7 @@
5050
"@types/prettier": "^2.7.3",
5151
"@vitest/coverage-v8": "^0.34.3",
5252
"@vitest/ui": "^0.34.1",
53+
"dotenv": "^16.4.7",
5354
"dotenv-cli": "^7.2.1",
5455
"msw": "^2.3.1",
5556
"prettier": "^2.8.8",

‎packages/api/test/helpers.ts

-1
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,6 @@ export function runFiltersTestsSuite(
166166
rollups: "optimism",
167167
});
168168

169-
console.log(result);
170169
expect(result).toMatchSnapshot();
171170
});
172171

‎packages/db/prisma/migrations/20250309165444_add_eth_usd_price/migration.sql

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
-- CreateTable
2+
CREATE TABLE "eth_usd_price" (
3+
"id" SERIAL NOT NULL,
4+
"price" DECIMAL(65,30) NOT NULL,
5+
"timestamp" TIMESTAMP(3) NOT NULL,
6+
7+
CONSTRAINT "eth_usd_price_pkey" PRIMARY KEY ("id")
8+
);

‎packages/db/prisma/migrations/20250324110813_make_timestamp_unique/migration.sql

+12
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
/*
2+
Warnings:
3+
4+
- You are about to alter the column `price` on the `eth_usd_price` table. The data in that column could be lost. The data in that column will be cast from `Decimal(65,30)` to `Decimal(100,0)`.
5+
- A unique constraint covering the columns `[timestamp]` on the table `eth_usd_price` will be added. If there are existing duplicate values, this will fail.
6+
7+
*/
8+
-- AlterTable
9+
ALTER TABLE "eth_usd_price" ALTER COLUMN "price" SET DATA TYPE DECIMAL(100,0);
10+
11+
-- CreateIndex
12+
CREATE UNIQUE INDEX "eth_usd_price_timestamp_key" ON "eth_usd_price"("timestamp");

‎packages/db/prisma/migrations/20250328122843_add_eth_usd_price_feed_state_model/migration.sql

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
/*
2+
Warnings:
3+
4+
- You are about to alter the column `price` on the `eth_usd_price` table. The data in that column could be lost. The data in that column will be cast from `Decimal(100,0)` to `Decimal(18,8)`.
5+
6+
*/
7+
-- AlterTable
8+
ALTER TABLE "eth_usd_price" ALTER COLUMN "price" SET DATA TYPE DECIMAL(18,8);
9+
10+
-- CreateTable
11+
CREATE TABLE "eth_usd_price_feed_state" (
12+
"id" SERIAL NOT NULL,
13+
"latest_round_id" DECIMAL(100,0) NOT NULL,
14+
"updated_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
15+
16+
CONSTRAINT "eth_usd_price_feed_state_pkey" PRIMARY KEY ("id")
17+
);

‎packages/db/prisma/schema.prisma

+16
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,14 @@ model BlobStoragesState {
129129
@@map("blob_storages_state")
130130
}
131131

132+
model EthUsdPriceFeedState {
133+
id Int @id @default(autoincrement())
134+
latestRoundId Decimal @map("latest_round_id") @db.Decimal(100, 0)
135+
updatedAt DateTime @default(now()) @map("updated_at")
136+
137+
@@map("eth_usd_price_feed_state")
138+
}
139+
132140
model Address {
133141
address String @id
134142
rollup Rollup?
@@ -334,6 +342,14 @@ model OverallStats {
334342
@@map("overall_stats")
335343
}
336344

345+
model EthUsdPrice {
346+
id Int @id @default(autoincrement())
347+
price Decimal @db.Decimal(18, 8)
348+
timestamp DateTime @unique
349+
350+
@@map("eth_usd_price")
351+
}
352+
337353
// NextAuth.js Models
338354
// NOTE: When using postgresql, mysql or sqlserver,
339355
// uncomment the @db.Text annotations below

‎packages/env/index.ts

+21-1
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,21 @@ export const env = createEnv({
8383
.url()
8484
.default("http://localhost:4318"),
8585

86-
/**
86+
// ETH Price (default: every hour)
87+
ETH_PRICE_SYNCER_CRON_PATTERN: z
88+
.enum(["0 * * * *", "0 0 * * *", "* * * * *"])
89+
.default("0 * * * *"),
90+
ETH_PRICE_SYNCER_CHAIN_ID: z.coerce.number().optional(),
91+
ETH_PRICE_SYNCER_CHAIN_JSON_RPC_URL: z.string().url().optional(),
92+
ETH_PRICE_SYNCER_ETH_USD_PRICE_FEED_CONTRACT_ADDRESS: z
93+
.string()
94+
.optional(),
95+
ETH_PRICE_SYNCER_TIME_TOLERANCE: z.coerce
96+
.number()
97+
.positive()
98+
.default(3600),
99+
100+
/*
87101
* =====================
88102
* STORAGE PROVIDERS
89103
* =====================
@@ -135,6 +149,12 @@ export const env = createEnv({
135149
.string()
136150
.optional()
137151
.superRefine(requireIfEnvEnabled("WEAVEVM_STORAGE_ENABLED")),
152+
153+
VITEST_MAINNET_FORK_URL: z
154+
.string()
155+
.url()
156+
.optional()
157+
.default("https://eth.llamarpc.com"),
138158
},
139159

140160
...presetEnvOptions,

‎packages/price-feed/package.json

+32
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
{
2+
"name": "@blobscan/price-feed",
3+
"private": true,
4+
"version": "0.0.1",
5+
"main": "./src/index.ts",
6+
"types": "./src/index.ts",
7+
"scripts": {
8+
"clean": "rm -rf .turbo node_modules",
9+
"lint": "eslint .",
10+
"lint:fix": "pnpm lint --fix",
11+
"type-check": "tsc --noEmit",
12+
"test": "pnpm with-env:test vitest",
13+
"test:ui": "pnpm with-env:test vitest --ui",
14+
"with-env:test": "dotenv -e ../../.env.test --"
15+
},
16+
"dependencies": {
17+
"@blobscan/env": "workspace:^0.1.0",
18+
"@blobscan/logger": "workspace:^0.1.2"
19+
},
20+
"peerDependencies": {
21+
"viem": "^2.17.4"
22+
},
23+
"devDependencies": {
24+
"viem": "^2.17.4"
25+
},
26+
"eslintConfig": {
27+
"root": true,
28+
"extends": [
29+
"@blobscan/eslint-config/base"
30+
]
31+
}
32+
}

0 commit comments

Comments
 (0)