Skip to content

Commit 295088f

Browse files
DenhamPreenDenhamPreen
authored
subgraph migration guide (#662)
* docs: add section on environment variables * feat: subgraph migration guide
1 parent eb3b2fa commit 295088f

File tree

7 files changed

+275
-20
lines changed

7 files changed

+275
-20
lines changed

‎docs/HyperIndex/Guides/configuration-file.mdx

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -208,22 +208,24 @@ Since `[email protected]`, environment variable interpolation is supported for flexibi
208208

209209
```yaml
210210
networks:
211-
- id: ${CHAIN_ID:-ethereum-mainnet}
211+
- id: ${ENVIO_CHAIN_ID:-ethereum-mainnet}
212212
contracts:
213213
- name: Greeter
214-
address: ${GREETER_ADDRESS}
214+
address: ${ENVIO_GREETER_ADDRESS}
215215
```
216216

217217
Run your indexer with custom environment variables:
218218

219219
```bash
220-
CHAIN_ID=optimism ENVIO_ADDRESS=0xYourContractAddress pnpm dev
220+
ENVIO_CHAIN_ID=optimism ENVIO_GREETER_ADDRESS=0xYourContractAddress pnpm dev
221221
```
222222

223223
**Interpolation syntax:**
224224

225-
- `${VAR}` – Use the value of `VAR`
226-
- `${VAR:-default}` – Use `VAR` if set, otherwise use `default`
225+
- `${ENVIO_VAR}` – Use the value of `ENVIO_VAR`
226+
- `${ENVIO_VAR:-default}` – Use `ENVIO_VAR` if set, otherwise use `default`
227+
228+
For more detailed information about environment variables, see our [Environment Variables Guide](./environment-variables).
227229

228230
---
229231

‎docs/HyperIndex/Guides/environment-variables.md

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
id: environment-variables
3+
title: Environment Variables
4+
sidebar_label: Environment Variables
5+
slug: /environment-variables
6+
---
7+
8+
# Environment Variables
9+
10+
Environment variables are a crucial part of configuring your Envio indexer. They allow you to manage sensitive information and configuration settings without hardcoding them in your codebase.
11+
12+
## Naming Convention
13+
14+
All environment variables used by Envio must be prefixed with `ENVIO_`. This naming convention:
15+
- Prevents conflicts with other environment variables
16+
- Makes it clear which variables are used by the Envio indexer
17+
- Ensures consistency across different environments
18+
19+
## Example Environment Variables
20+
21+
Here are some commonly used environment variables:
22+
23+
```bash
24+
# Blockchain RPC URL
25+
ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
26+
27+
# Starting block number for indexing
28+
ENVIO_START_BLOCK=12345678
29+
30+
# Number of blocks to process in each batch
31+
ENVIO_COINGECKO_API_KEY=api-key
32+
33+
```
34+
35+
## Setting Environment Variables
36+
37+
### Local Development
38+
39+
For local development, you can set environment variables in several ways:
40+
41+
1. Using a `.env` file in your project root:
42+
```bash
43+
# .env
44+
ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
45+
ENVIO_START_BLOCK=12345678
46+
```
47+
48+
2. Directly in your terminal:
49+
```bash
50+
export ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
51+
```
52+
53+
### Hosted Service
54+
55+
When using the Envio Hosted Service, you can configure environment variables through the Envio platform's dashboard. Remember that all variables must still be prefixed with `ENVIO_`.
56+
57+
For more information about environment variables in the hosted service, see the [Hosted Service documentation](../HyperIndex/hosted-service).
58+
59+
## Configuration File
60+
61+
For use of environment variables in your configuration file, read the docs here: [Configuration File](../HyperIndex/configuration-file).
62+
63+
## Best Practices
64+
65+
1. **Never commit sensitive values**: Always use environment variables for sensitive information like API keys and database credentials
66+
1. **Never commit or use private keys**: Never commit or use private keys in your codebase
67+
1. **Use descriptive names**: Make your environment variable names clear and descriptive
68+
1. **Document your variables**: Keep a list of required environment variables in your project's README
69+
1. **Use different values**: Use different environment variables for development, staging, and production environments
70+
1. **Validate required variables**: Check that all required environment variables are set before starting your indexer
71+
72+
## Troubleshooting
73+
74+
If you encounter issues with environment variables:
75+
76+
1. Verify that all required variables are set
77+
2. Check that variables are prefixed with `ENVIO_`
78+
3. Ensure there are no typos in variable names
79+
4. Confirm that the values are correctly formatted
80+
81+
For more help, see our [Troubleshooting Guide](../HyperIndex/logging).

‎docs/HyperIndex/migration-guide.md

Lines changed: 180 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,21 @@
11
---
22
id: migration-guide
3-
title: Migrate from TheGraph and Other Indexers to HyperIndex
3+
title: Migrate from a Subgraph to HyperIndex
44
sidebar_label: Migrate from TheGraph
55
slug: /migration-guide
66
---
77

8-
# Migrate from TheGraph and Other Indexers to HyperIndex
8+
# Migrate from TheGraph to HyperIndex
99

10-
:::caution
11-
**This guide is currently under construction**
12-
13-
We're actively working on comprehensive migration documentation. In the meantime, please reach out to our team on [Discord](https://discord.gg/envio) for personalized migration assistance.
10+
:::info
11+
Please reach out to our team on [Discord](https://discord.gg/envio) for personalized migration assistance.
1412
:::
1513

1614
## Introduction
1715

18-
Migrating to HyperIndex from other indexing solutions is generally straightforward, and our team is actively working to reduce friction even further. Whether you're coming from TheGraph, Subsquid, Ponder, or a custom-built indexer, we have tools and processes to make your transition smooth and efficient.
16+
Migrating from a subgraph to HyperIndex is designed to be a developer-friendly process. HyperIndex draws strong inspiration from TheGraph’s subgraph architecture, which makes the migration simple, especially with the help of coding assistants like Cursor and AI tools (don't forget to use our ai friendly docs [llm-docs.envio.dev](https://llm-docs.envio.dev)).
17+
18+
The process is simple but requires a good understanding of the underlying concepts. If you are new to HyperIndex, we recommend starting with the [Getting Started](../HyperIndex/getting-started) guide.
1919

2020
## Why Migrate to HyperIndex?
2121

@@ -25,10 +25,181 @@ Migrating to HyperIndex from other indexing solutions is generally straightforwa
2525
- **Advanced Features**: Access to capabilities not available in other indexing solutions
2626
- **Seamless Integration**: Easy integration with existing GraphQL APIs and applications
2727

28+
## Subgraph to HyperIndex Migration Overview
29+
30+
Migration consists of three major steps:
31+
1. Subgraph.yaml migration
32+
1. Schema migration - near copy paste
33+
1. Event handler migration
34+
35+
At any point in the migration run
36+
37+
`pnpm envio codegen`
38+
39+
to verify the `config.yaml` and `schema.graphql` files are valid.
40+
41+
or run
42+
43+
`pnpm dev`
44+
45+
to verify the indexer is running and indexing correctly.
46+
47+
### 0.5 Use `npx envio init` to generate a boilerplate
48+
49+
As a first step, we recommend using `npx envio init` to generate a boilerplate for your project. This will handle the creation of the `config.yaml` file and a basic `schema.graphql` file with generic handler functions.
50+
51+
### 1. `subgraph.yaml` → `config.yaml`
52+
53+
`npx envio init` will generate this for you. It's a simple configuration file conversion. Effectively specifying which contracts to index, which networks to index (multiple networks can be specified with envio) and which events from those contracts to index.
54+
55+
Take the following conversion as an example, where the `subgraph.yaml` file is converted to `config.yaml` the below comparisons is for the Uniswap v4 pool manager subgraph.
56+
57+
58+
<div className="row">
59+
<div className="col col--6">
60+
theGraph - `subgraph.yaml`
61+
```yaml
62+
specVersion: 0.0.4
63+
description: Uniswap is a decentralized protocol for automated token exchange on Ethereum.
64+
repository: https://github.com/Uniswap/v4-subgraph
65+
schema:
66+
file: ./schema.graphql
67+
features:
68+
- nonFatalErrors
69+
- grafting
70+
- kind: ethereum/contract
71+
name: PositionManager
72+
network: mainnet
73+
source:
74+
abi: PositionManager
75+
address: "0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e"
76+
startBlock: 21689089
77+
mapping:
78+
kind: ethereum/events
79+
apiVersion: 0.0.7
80+
language: wasm/assemblyscript
81+
file: ./src/mappings/index.ts
82+
entities:
83+
- Position
84+
abis:
85+
- name: PositionManager
86+
file: ./abis/PositionManager.json
87+
eventHandlers:
88+
- event: Subscription(indexed uint256,indexed address)
89+
handler: handleSubscription
90+
- event: Unsubscription(indexed uint256,indexed address)
91+
handler: handleUnsubscription
92+
- event: Transfer(indexed address,indexed address,indexed uint256)
93+
handler: handleTransfer
94+
```
95+
</div>
96+
<div className="col col--6">
97+
HyperIndex - `config.yaml`
98+
```yaml
99+
# yaml-language-server: $schema=./node_modules/envio/evm.schema.json
100+
name: uni-v4-indexer
101+
networks:
102+
- id: 1
103+
start_block: 21689089
104+
contracts:
105+
- name: PositionManager
106+
address: 0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e
107+
handler: src/EventHandlers.ts
108+
events:
109+
- event: Subscription(uint256 indexed tokenId, address indexed subscriber)
110+
- event: Unsubscription(uint256 indexed tokenId, address indexed subscriber)
111+
- event: Transfer(address indexed from, address indexed to, uint256 indexed id)
112+
```
113+
</div>
114+
</div>
115+
116+
For any potential hurdles, please refer to the [Configuration File](../HyperIndex/configuration-file) documentation.
117+
118+
## 2. Schema migration
119+
120+
`copy` & `paste` the schema from the subgraph to the HyperIndex config file.
121+
122+
Small nuance differences:
123+
124+
- You can remove the `@entity` directive
125+
- [Enums](../HyperIndex/schema#enum-types)
126+
- [BigDecimals](../HyperIndex/schema#working-with-bigdecimal)
127+
128+
## 3. Event handler migration
129+
130+
This consists of two parts
131+
1. Converting assemblyscript to typescript
132+
1. Converting the subgraph syntax to HyperIndex syntax
133+
134+
### 3.1 Converting Assemblyscript to Typescript
135+
136+
The subgraph uses assemblyscript to write event handlers. The HyperIndex syntax is usually in typescript. Since assemblyscript is a subset of typescript, it's quite simple to copy and paste the code, especially so for pure functions.
137+
138+
### 3.2 Converting the subgraph syntax to HyperIndex syntax
139+
140+
There are some subtle differences in the syntax of the subgraph and HyperIndex. Including but not limited to the following:
141+
142+
- Replace Entity.save() with context.Entity.set()
143+
- Convert to async handler functions
144+
- Use `await` for loading entities `const x = await context.Entity.get(id)`
145+
- Use [dynamic contract registration](../HyperIndex/configuration-file#dynamic-contracts) to register contracts
146+
147+
The below code snippets can give you a basic idea of what this difference might look like.
148+
149+
<div className="row">
150+
<div className="col col--6">
151+
theGraph - `eventHandler.ts`
152+
```typescript
153+
export function handleSubscription(event: SubscriptionEvent): void {
154+
const subscription = new Subscribe(event.transaction.hash + event.logIndex)
155+
156+
subscription.tokenId = event.params.tokenId
157+
subscription.address = event.params.subscriber.toHexString()
158+
subscription.logIndex = event.logIndex
159+
subscription.blockNumber = event.block.number
160+
subscription.position = event.params.tokenId
161+
162+
subscription.save()
163+
}
164+
```
165+
</div>
166+
<div className="col col--6">
167+
HyperIndex - `eventHandler.ts`
168+
```typescript
169+
PoolManager.Subscription.handler( async (event, context) => {
170+
const entity = {
171+
id: event.transaction.hash + event.logIndex,
172+
tokenId: event.params.tokenId,
173+
address: event.params.subscriber,
174+
blockNumber: event.block.number,
175+
logIndex: event.logIndex,
176+
position: event.params.tokenId
177+
}
178+
179+
context.Subscription.set(entity);
180+
})
181+
```
182+
</div>
183+
</div>
184+
185+
186+
## Extra tips
187+
188+
HyperIndex is a powerful tool that can be used to index any contract. There are some features that are especially powerful that go above subgraph implementations and so in some cases you may want to optimise your migration to HyperIndex further to take advantage of these features. Here are some useful tips:
189+
190+
- Use the `field_selection` option to add additional fields to your index. Doc here: [field selection](../HyperIndex/configuration-file#field-selection)
191+
- Use the `unordered_multichain_mode` option to enable unordered multichain mode, this is the most common need for multichain indexing. However comes with tradeoffs worth understanding. Doc here: [unordered multichain mode](../HyperIndex/configuration-file#unordered-multichain-mode)
192+
- Use wildcard indexing to index by event signatures rather than by contract address.
193+
- HyperIndex uses the standard graphql query language, where as the subgraph uses a custom query language. You can read about the slight nuances [here](https://docs.sablier.com/api/caveats). (We are working on a basic tool to help with backwards compatibility, please check in with us on discord for it's current status).
194+
- Loaders are a powerful feature to optimize historical sync performance. You can read more about them [here](../HyperIndex/loaders).
195+
- HyperIndex is very flexible and can be used to index offchain data too or send messages to a queue etc for fetching external data, you can further optimise the fetching by using the [effects api](../HyperIndex/loaders#effect-api-experimental)
196+
197+
## Share Your Learnings
198+
199+
If you discover helpful tips during your migration, we’d love contributions! Open a [PR](https://github.com/enviodev/docs) to this guide and help future developers.
200+
28201
## Getting Help
29202

30203
**Join Our Discord**: The fastest way to get personalized help is through our [Discord community](https://discord.gg/envio).
31204

32-
## Stay Tuned
33205

34-
We're working hard to provide detailed, step-by-step migration guides for all major indexing solutions. Stay tuned for more comprehensive documentation coming soon!

‎docs/HyperIndex/supported-networks/chiliz.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ slug: /chiliz
1111

1212
| **Field** | **Value** |
1313
|-------------------------------|----------------------------------------------------------------------------------------------------|
14-
| **Chiliz Chain ID** | 8888 |
15-
| **HyperSync URL Endpoint** | [https://chiliz.hypersync.xyz](https://chiliz.hypersync.xyz) or [https://8888.hypersync.xyz](https://8888.hypersync.xyz) |
16-
| **HyperRPC URL Endpoint** | [https://chiliz.rpc.hypersync.xyz](https://chiliz.rpc.hypersync.xyz) or [https://8888.rpc.hypersync.xyz](https://8888.rpc.hypersync.xyz) |
14+
| **Chiliz Chain ID** | 88888 |
15+
| **HyperSync URL Endpoint** | [https://chiliz.hypersync.xyz](https://chiliz.hypersync.xyz) or [https://88888.hypersync.xyz](https://88888.hypersync.xyz) |
16+
| **HyperRPC URL Endpoint** | [https://chiliz.rpc.hypersync.xyz](https://chiliz.rpc.hypersync.xyz) or [https://88888.rpc.hypersync.xyz](https://88888.rpc.hypersync.xyz) |
1717

1818
---
1919

@@ -39,7 +39,7 @@ To get started, see our documentation or follow our quickstart [guide](/docs/Hyp
3939
name: IndexerName # Specify indexer name
4040
description: Indexer Description # Include indexer description
4141
networks:
42-
- id: 8888 # Chiliz
42+
- id: 88888 # Chiliz
4343
start_block: START_BLOCK_NUMBER # Specify the starting block
4444
contracts:
4545
- name: ContractName

‎docs/HyperSync/HyperRPC/hyperrpc-url-endpoints.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@ Here is a table of the currently supported networks on HyperRPC and their respec
3838
| Bsc | 56 | https://bsc.rpc.hypersync.xyz or https://56.rpc.hypersync.xyz | |
3939
| Bsc Testnet | 97 | https://bsc-testnet.rpc.hypersync.xyz or https://97.rpc.hypersync.xyz | |
4040
| Celo | 42220 | https://celo.rpc.hypersync.xyz or https://42220.rpc.hypersync.xyz | |
41-
| Chiliz | 8888 | https://chiliz.rpc.hypersync.xyz or https://8888.rpc.hypersync.xyz | |
41+
| Chiliz | 88888 | https://chiliz.rpc.hypersync.xyz or https://88888.rpc.hypersync.xyz | |
4242
| Citrea Testnet | 5115 | https://citrea-testnet.rpc.hypersync.xyz or https://5115.rpc.hypersync.xyz | |
4343
| Curtis | 33111 | https://curtis.rpc.hypersync.xyz or https://33111.rpc.hypersync.xyz | |
4444
| Cyber | 7560 | https://cyber.rpc.hypersync.xyz or https://7560.rpc.hypersync.xyz | |

‎docs/HyperSync/hypersync-supported-networks.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ If you are a network operator or user and would like improved service support or
4646
| Bsc | 56 | https://bsc.hypersync.xyz or https://56.hypersync.xyz | 🥉 | |
4747
| Bsc Testnet | 97 | https://bsc-testnet.hypersync.xyz or https://97.hypersync.xyz | 🎒 | |
4848
| Celo | 42220 | https://celo.hypersync.xyz or https://42220.hypersync.xyz | 🪨 | |
49-
| Chiliz | 8888 | https://chiliz.hypersync.xyz or https://8888.hypersync.xyz | 🪨 | |
49+
| Chiliz | 88888 | https://chiliz.hypersync.xyz or https://88888.hypersync.xyz | 🪨 | |
5050
| Citrea Testnet | 5115 | https://citrea-testnet.hypersync.xyz or https://5115.hypersync.xyz | 🪨 | |
5151
| Curtis | 33111 | https://curtis.hypersync.xyz or https://33111.hypersync.xyz | 🪨 | |
5252
| Cyber | 7560 | https://cyber.hypersync.xyz or https://7560.hypersync.xyz | 🪨 | |

‎sidebarsHyperIndex.js

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,7 @@ module.exports = {
5555
// "Guides/subgraph-mPigration",
5656
"Guides/testing",
5757
"Guides/navigating-hasura",
58+
"Guides/environment-variables",
5859
],
5960
},
6061
{

0 commit comments

Comments
 (0)