Skip to content

subgraph migration guide #662

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 7 additions & 5 deletions docs/HyperIndex/Guides/configuration-file.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -208,22 +208,24 @@ Since `[email protected]`, environment variable interpolation is supported for flexibi

```yaml
networks:
- id: ${CHAIN_ID:-ethereum-mainnet}
- id: ${ENVIO_CHAIN_ID:-ethereum-mainnet}
contracts:
- name: Greeter
address: ${GREETER_ADDRESS}
address: ${ENVIO_GREETER_ADDRESS}
```

Run your indexer with custom environment variables:

```bash
CHAIN_ID=optimism ENVIO_ADDRESS=0xYourContractAddress pnpm dev
ENVIO_CHAIN_ID=optimism ENVIO_GREETER_ADDRESS=0xYourContractAddress pnpm dev
```

**Interpolation syntax:**

- `${VAR}` – Use the value of `VAR`
- `${VAR:-default}` – Use `VAR` if set, otherwise use `default`
- `${ENVIO_VAR}` – Use the value of `ENVIO_VAR`
- `${ENVIO_VAR:-default}` – Use `ENVIO_VAR` if set, otherwise use `default`

For more detailed information about environment variables, see our [Environment Variables Guide](./environment-variables).

---

Expand Down
81 changes: 81 additions & 0 deletions docs/HyperIndex/Guides/environment-variables.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
---
id: environment-variables
title: Environment Variables
sidebar_label: Environment Variables
slug: /environment-variables
---

# Environment Variables

Environment variables are a crucial part of configuring your Envio indexer. They allow you to manage sensitive information and configuration settings without hardcoding them in your codebase.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!


## Naming Convention

All environment variables used by Envio must be prefixed with `ENVIO_`. This naming convention:
- Prevents conflicts with other environment variables
- Makes it clear which variables are used by the Envio indexer
- Ensures consistency across different environments

## Example Environment Variables

Here are some commonly used environment variables:

```bash
# Blockchain RPC URL
ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key

# Starting block number for indexing
ENVIO_START_BLOCK=12345678

# Number of blocks to process in each batch
ENVIO_COINGECKO_API_KEY=api-key
Comment on lines +23 to +31
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix misaligned example comment
The comment # Number of blocks to process in each batch does not describe ENVIO_COINGECKO_API_KEY.

- # Number of blocks to process in each batch
- ENVIO_COINGECKO_API_KEY=api-key
+ # CoinGecko API key for price data retrieval
+ ENVIO_COINGECKO_API_KEY=your-coingecko-api-key
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
```bash
# Blockchain RPC URL
ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
# Starting block number for indexing
ENVIO_START_BLOCK=12345678
# Number of blocks to process in each batch
ENVIO_COINGECKO_API_KEY=api-key
🤖 Prompt for AI Agents
In docs/HyperIndex/Guides/environment-variables.md around lines 23 to 31, the
comment above ENVIO_COINGECKO_API_KEY is incorrect and describes the wrong
variable. Update the comment to correctly describe ENVIO_COINGECKO_API_KEY, such
as indicating it is the API key for CoinGecko, and ensure the comment about the
number of blocks to process in each batch is placed above the appropriate
environment variable if it exists.


```

## Setting Environment Variables

### Local Development

For local development, you can set environment variables in several ways:

1. Using a `.env` file in your project root:
```bash
# .env
ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
ENVIO_START_BLOCK=12345678
```

2. Directly in your terminal:
```bash
export ENVIO_RPC_URL=https://arbitrum.direct.dev/your-api-key
```

### Hosted Service

When using the Envio Hosted Service, you can configure environment variables through the Envio platform's dashboard. Remember that all variables must still be prefixed with `ENVIO_`.

For more information about environment variables in the hosted service, see the [Hosted Service documentation](../HyperIndex/hosted-service).
Comment on lines +53 to +57
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Correct link to Hosted Service docs
Update the relative path to point to the Hosted_Service/hosted-service guide.

- see the [Hosted Service documentation](../HyperIndex/hosted-service)
+ see the [Hosted Service documentation](../Hosted_Service/hosted-service)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
### Hosted Service
When using the Envio Hosted Service, you can configure environment variables through the Envio platform's dashboard. Remember that all variables must still be prefixed with `ENVIO_`.
For more information about environment variables in the hosted service, see the [Hosted Service documentation](../HyperIndex/hosted-service).
### Hosted Service
When using the Envio Hosted Service, you can configure environment variables through the Envio platform's dashboard. Remember that all variables must still be prefixed with `ENVIO_`.
For more information about environment variables in the hosted service, see the [Hosted Service documentation](../Hosted_Service/hosted-service).
🤖 Prompt for AI Agents
In docs/HyperIndex/Guides/environment-variables.md around lines 53 to 57, the
link to the Hosted Service documentation uses an incorrect relative path. Update
the link to point to ../HyperIndex/Hosted_Service/hosted-service instead of
../HyperIndex/hosted-service to correctly reference the Hosted Service guide.


## Configuration File

For use of environment variables in your configuration file, read the docs here: [Configuration File](../HyperIndex/configuration-file).

Comment on lines +59 to +62
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix Configuration File link
The link should reference the local guide in the same directory.

- [Configuration File](../HyperIndex/configuration-file)
+ [Configuration File](./configuration-file)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
## Configuration File
For use of environment variables in your configuration file, read the docs here: [Configuration File](../HyperIndex/configuration-file).
## Configuration File
For use of environment variables in your configuration file, read the docs here: [Configuration File](./configuration-file).
🤖 Prompt for AI Agents
In docs/HyperIndex/Guides/environment-variables.md around lines 59 to 62, the
link to the Configuration File currently points to
../HyperIndex/configuration-file, which is incorrect. Update the link to
reference the local guide file in the same directory by changing the URL to
./configuration-file or the correct relative path within the same folder.

## Best Practices

1. **Never commit sensitive values**: Always use environment variables for sensitive information like API keys and database credentials
1. **Never commit or use private keys**: Never commit or use private keys in your codebase
1. **Use descriptive names**: Make your environment variable names clear and descriptive
1. **Document your variables**: Keep a list of required environment variables in your project's README
1. **Use different values**: Use different environment variables for development, staging, and production environments
1. **Validate required variables**: Check that all required environment variables are set before starting your indexer

## Troubleshooting

If you encounter issues with environment variables:

1. Verify that all required variables are set
2. Check that variables are prefixed with `ENVIO_`
3. Ensure there are no typos in variable names
4. Confirm that the values are correctly formatted

For more help, see our [Troubleshooting Guide](../HyperIndex/logging).
Comment on lines +79 to +81
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Correct Troubleshooting Guide link
Update the path to ../Troubleshoot/logging to match the sidebar configuration.

- For more help, see our [Troubleshooting Guide](../HyperIndex/logging).
+ For more help, see our [Troubleshooting Guide](../Troubleshoot/logging).
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
4. Confirm that the values are correctly formatted
For more help, see our [Troubleshooting Guide](../HyperIndex/logging).
4. Confirm that the values are correctly formatted
For more help, see our [Troubleshooting Guide](../Troubleshoot/logging).
🤖 Prompt for AI Agents
In docs/HyperIndex/Guides/environment-variables.md around lines 79 to 81, the
link to the Troubleshooting Guide is incorrect. Update the link path from
../HyperIndex/logging to ../Troubleshoot/logging to match the sidebar
configuration and ensure the link directs to the correct documentation page.

189 changes: 180 additions & 9 deletions docs/HyperIndex/migration-guide.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
---
id: migration-guide
title: Migrate from TheGraph and Other Indexers to HyperIndex
title: Migrate from a Subgraph to HyperIndex
sidebar_label: Migrate from TheGraph
slug: /migration-guide
---

# Migrate from TheGraph and Other Indexers to HyperIndex
# Migrate from TheGraph to HyperIndex
Comment on lines +3 to +8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Inconsistent title and header
The frontmatter title ("Migrate from a Subgraph to HyperIndex") and the H1 header ("Migrate from TheGraph to HyperIndex") differ. Standardize the phrasing for clarity.

--- title: Migrate from a Subgraph to HyperIndex
+++ title: Migrate from TheGraph subgraph to HyperIndex

--- # Migrate from TheGraph to HyperIndex
+++ # Migrate from TheGraph subgraph to HyperIndex
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
title: Migrate from a Subgraph to HyperIndex
sidebar_label: Migrate from TheGraph
slug: /migration-guide
---
# Migrate from TheGraph and Other Indexers to HyperIndex
# Migrate from TheGraph to HyperIndex
title: Migrate from TheGraph subgraph to HyperIndex
sidebar_label: Migrate from TheGraph
slug: /migration-guide
---
# Migrate from TheGraph subgraph to HyperIndex
🤖 Prompt for AI Agents
In docs/HyperIndex/migration-guide.md around lines 3 to 8, the frontmatter title
and the H1 header have inconsistent phrasing. Update either the frontmatter
title or the H1 header so both use the same wording, ensuring consistency and
clarity in the document's main topic.


:::caution
**This guide is currently under construction**

We're actively working on comprehensive migration documentation. In the meantime, please reach out to our team on [Discord](https://discord.gg/envio) for personalized migration assistance.
:::info
Please reach out to our team on [Discord](https://discord.gg/envio) for personalized migration assistance.
:::

## Introduction

Migrating to HyperIndex from other indexing solutions is generally straightforward, and our team is actively working to reduce friction even further. Whether you're coming from TheGraph, Subsquid, Ponder, or a custom-built indexer, we have tools and processes to make your transition smooth and efficient.
Migrating from a subgraph to HyperIndex is designed to be a developer-friendly process. HyperIndex draws strong inspiration from TheGraph’s subgraph architecture, which makes the migration simple, especially with the help of coding assistants like Cursor and AI tools (don't forget to use our ai friendly docs [llm-docs.envio.dev](https://llm-docs.envio.dev)).

The process is simple but requires a good understanding of the underlying concepts. If you are new to HyperIndex, we recommend starting with the [Getting Started](../HyperIndex/getting-started) guide.

## Why Migrate to HyperIndex?

Expand All @@ -25,10 +25,181 @@ Migrating to HyperIndex from other indexing solutions is generally straightforwa
- **Advanced Features**: Access to capabilities not available in other indexing solutions
- **Seamless Integration**: Easy integration with existing GraphQL APIs and applications

## Subgraph to HyperIndex Migration Overview

Migration consists of three major steps:
1. Subgraph.yaml migration
1. Schema migration - near copy paste
1. Event handler migration

At any point in the migration run

`pnpm envio codegen`

to verify the `config.yaml` and `schema.graphql` files are valid.

or run

`pnpm dev`

to verify the indexer is running and indexing correctly.

### 0.5 Use `npx envio init` to generate a boilerplate

As a first step, we recommend using `npx envio init` to generate a boilerplate for your project. This will handle the creation of the `config.yaml` file and a basic `schema.graphql` file with generic handler functions.

### 1. `subgraph.yaml` → `config.yaml`

`npx envio init` will generate this for you. It's a simple configuration file conversion. Effectively specifying which contracts to index, which networks to index (multiple networks can be specified with envio) and which events from those contracts to index.

Take the following conversion as an example, where the `subgraph.yaml` file is converted to `config.yaml` the below comparisons is for the Uniswap v4 pool manager subgraph.


<div className="row">
<div className="col col--6">
theGraph - `subgraph.yaml`
```yaml
specVersion: 0.0.4
description: Uniswap is a decentralized protocol for automated token exchange on Ethereum.
repository: https://github.com/Uniswap/v4-subgraph
schema:
file: ./schema.graphql
features:
- nonFatalErrors
- grafting
- kind: ethereum/contract
name: PositionManager
network: mainnet
source:
abi: PositionManager
address: "0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e"
startBlock: 21689089
mapping:
kind: ethereum/events
apiVersion: 0.0.7
language: wasm/assemblyscript
file: ./src/mappings/index.ts
entities:
- Position
abis:
- name: PositionManager
file: ./abis/PositionManager.json
eventHandlers:
- event: Subscription(indexed uint256,indexed address)
handler: handleSubscription
- event: Unsubscription(indexed uint256,indexed address)
handler: handleUnsubscription
- event: Transfer(indexed address,indexed address,indexed uint256)
handler: handleTransfer
```
</div>
<div className="col col--6">
HyperIndex - `config.yaml`
```yaml
# yaml-language-server: $schema=./node_modules/envio/evm.schema.json
name: uni-v4-indexer
networks:
- id: 1
start_block: 21689089
contracts:
- name: PositionManager
address: 0xbD216513d74C8cf14cf4747E6AaA6420FF64ee9e
handler: src/EventHandlers.ts
events:
- event: Subscription(uint256 indexed tokenId, address indexed subscriber)
- event: Unsubscription(uint256 indexed tokenId, address indexed subscriber)
- event: Transfer(address indexed from, address indexed to, uint256 indexed id)
```
</div>
</div>

For any potential hurdles, please refer to the [Configuration File](../HyperIndex/configuration-file) documentation.

## 2. Schema migration

`copy` & `paste` the schema from the subgraph to the HyperIndex config file.

Small nuance differences:

- You can remove the `@entity` directive
- [Enums](../HyperIndex/schema#enum-types)
- [BigDecimals](../HyperIndex/schema#working-with-bigdecimal)

## 3. Event handler migration

This consists of two parts
1. Converting assemblyscript to typescript
1. Converting the subgraph syntax to HyperIndex syntax

### 3.1 Converting Assemblyscript to Typescript

The subgraph uses assemblyscript to write event handlers. The HyperIndex syntax is usually in typescript. Since assemblyscript is a subset of typescript, it's quite simple to copy and paste the code, especially so for pure functions.

### 3.2 Converting the subgraph syntax to HyperIndex syntax

There are some subtle differences in the syntax of the subgraph and HyperIndex. Including but not limited to the following:

- Replace Entity.save() with context.Entity.set()
- Convert to async handler functions
- Use `await` for loading entities `const x = await context.Entity.get(id)`
- Use [dynamic contract registration](../HyperIndex/configuration-file#dynamic-contracts) to register contracts

The below code snippets can give you a basic idea of what this difference might look like.

<div className="row">
<div className="col col--6">
theGraph - `eventHandler.ts`
```typescript
export function handleSubscription(event: SubscriptionEvent): void {
const subscription = new Subscribe(event.transaction.hash + event.logIndex)

subscription.tokenId = event.params.tokenId
subscription.address = event.params.subscriber.toHexString()
subscription.logIndex = event.logIndex
subscription.blockNumber = event.block.number
subscription.position = event.params.tokenId

subscription.save()
}
```
</div>
<div className="col col--6">
HyperIndex - `eventHandler.ts`
```typescript
PoolManager.Subscription.handler( async (event, context) => {
const entity = {
id: event.transaction.hash + event.logIndex,
tokenId: event.params.tokenId,
address: event.params.subscriber,
blockNumber: event.block.number,
logIndex: event.logIndex,
position: event.params.tokenId
}

context.Subscription.set(entity);
})
```
</div>
</div>


## Extra tips

HyperIndex is a powerful tool that can be used to index any contract. There are some features that are especially powerful that go above subgraph implementations and so in some cases you may want to optimise your migration to HyperIndex further to take advantage of these features. Here are some useful tips:

- Use the `field_selection` option to add additional fields to your index. Doc here: [field selection](../HyperIndex/configuration-file#field-selection)
- Use the `unordered_multichain_mode` option to enable unordered multichain mode, this is the most common need for multichain indexing. However comes with tradeoffs worth understanding. Doc here: [unordered multichain mode](../HyperIndex/configuration-file#unordered-multichain-mode)
- Use wildcard indexing to index by event signatures rather than by contract address.
- HyperIndex uses the standard graphql query language, where as the subgraph uses a custom query language. You can read about the slight nuances [here](https://docs.sablier.com/api/caveats). (We are working on a basic tool to help with backwards compatibility, please check in with us on discord for it's current status).
- Loaders are a powerful feature to optimize historical sync performance. You can read more about them [here](../HyperIndex/loaders).
- HyperIndex is very flexible and can be used to index offchain data too or send messages to a queue etc for fetching external data, you can further optimise the fetching by using the [effects api](../HyperIndex/loaders#effect-api-experimental)

## Share Your Learnings

If you discover helpful tips during your migration, we’d love contributions! Open a [PR](https://github.com/enviodev/docs) to this guide and help future developers.

## Getting Help

**Join Our Discord**: The fastest way to get personalized help is through our [Discord community](https://discord.gg/envio).

## Stay Tuned

We're working hard to provide detailed, step-by-step migration guides for all major indexing solutions. Stay tuned for more comprehensive documentation coming soon!
8 changes: 4 additions & 4 deletions docs/HyperIndex/supported-networks/chiliz.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ slug: /chiliz

| **Field** | **Value** |
|-------------------------------|----------------------------------------------------------------------------------------------------|
| **Chiliz Chain ID** | 8888 |
| **HyperSync URL Endpoint** | [https://chiliz.hypersync.xyz](https://chiliz.hypersync.xyz) or [https://8888.hypersync.xyz](https://8888.hypersync.xyz) |
| **HyperRPC URL Endpoint** | [https://chiliz.rpc.hypersync.xyz](https://chiliz.rpc.hypersync.xyz) or [https://8888.rpc.hypersync.xyz](https://8888.rpc.hypersync.xyz) |
| **Chiliz Chain ID** | 88888 |
| **HyperSync URL Endpoint** | [https://chiliz.hypersync.xyz](https://chiliz.hypersync.xyz) or [https://88888.hypersync.xyz](https://88888.hypersync.xyz) |
| **HyperRPC URL Endpoint** | [https://chiliz.rpc.hypersync.xyz](https://chiliz.rpc.hypersync.xyz) or [https://88888.rpc.hypersync.xyz](https://88888.rpc.hypersync.xyz) |

---

Expand All @@ -39,7 +39,7 @@ To get started, see our documentation or follow our quickstart [guide](/docs/Hyp
name: IndexerName # Specify indexer name
description: Indexer Description # Include indexer description
networks:
- id: 8888 # Chiliz
- id: 88888 # Chiliz
start_block: START_BLOCK_NUMBER # Specify the starting block
contracts:
- name: ContractName
Expand Down
2 changes: 1 addition & 1 deletion docs/HyperSync/HyperRPC/hyperrpc-url-endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Here is a table of the currently supported networks on HyperRPC and their respec
| Bsc | 56 | https://bsc.rpc.hypersync.xyz or https://56.rpc.hypersync.xyz | |
| Bsc Testnet | 97 | https://bsc-testnet.rpc.hypersync.xyz or https://97.rpc.hypersync.xyz | |
| Celo | 42220 | https://celo.rpc.hypersync.xyz or https://42220.rpc.hypersync.xyz | |
| Chiliz | 8888 | https://chiliz.rpc.hypersync.xyz or https://8888.rpc.hypersync.xyz | |
| Chiliz | 88888 | https://chiliz.rpc.hypersync.xyz or https://88888.rpc.hypersync.xyz | |
| Citrea Testnet | 5115 | https://citrea-testnet.rpc.hypersync.xyz or https://5115.rpc.hypersync.xyz | |
| Curtis | 33111 | https://curtis.rpc.hypersync.xyz or https://33111.rpc.hypersync.xyz | |
| Cyber | 7560 | https://cyber.rpc.hypersync.xyz or https://7560.rpc.hypersync.xyz | |
Expand Down
2 changes: 1 addition & 1 deletion docs/HyperSync/hypersync-supported-networks.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ If you are a network operator or user and would like improved service support or
| Bsc | 56 | https://bsc.hypersync.xyz or https://56.hypersync.xyz | 🥉 | |
| Bsc Testnet | 97 | https://bsc-testnet.hypersync.xyz or https://97.hypersync.xyz | 🎒 | |
| Celo | 42220 | https://celo.hypersync.xyz or https://42220.hypersync.xyz | 🪨 | |
| Chiliz | 8888 | https://chiliz.hypersync.xyz or https://8888.hypersync.xyz | 🪨 | |
| Chiliz | 88888 | https://chiliz.hypersync.xyz or https://88888.hypersync.xyz | 🪨 | |
| Citrea Testnet | 5115 | https://citrea-testnet.hypersync.xyz or https://5115.hypersync.xyz | 🪨 | |
| Curtis | 33111 | https://curtis.hypersync.xyz or https://33111.hypersync.xyz | 🪨 | |
| Cyber | 7560 | https://cyber.hypersync.xyz or https://7560.hypersync.xyz | 🪨 | |
Expand Down
1 change: 1 addition & 0 deletions sidebarsHyperIndex.js
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@ module.exports = {
// "Guides/subgraph-mPigration",
"Guides/testing",
"Guides/navigating-hasura",
"Guides/environment-variables",
],
},
{
Expand Down