Skip to content

Add info about timelines and usage #609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 2, 2025
Merged

Add info about timelines and usage #609

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
193fcca
Add info about timelines and usage
JonoPrest Apr 2, 2025
9ac3b6f
Add api token info to hyperindex docs
JonoPrest Apr 2, 2025
45e231c
Add note about placement in .env
JonoPrest Apr 2, 2025
8ce200d
Add full stops
JonoPrest Apr 2, 2025
5734e25
Add note about reaching out to us for introductory pricing
JonoPrest Apr 2, 2025
6317316
Expand on credit calculation
JonoPrest Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/HyperIndex/overview.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,20 @@ While HyperIndex gives you a complete indexing solution with schema management a

---

## Hypersync API Token Requirements

Starting from **9 May 2024**, HyperSync (the data engine powering HyperIndex) will implement rate limits for requests without API tokens. Here's what you need to know:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't mention at all what the rate limits might be. We might want to mention, its rate limited to appox 100 req per minute and this means potentially your indexer may be 2x slower without a token. To avoid simply add a token?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't decided on this yet.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I thought leave this vague. But maybe be more direct in saying that if you want guaranteed uninterupted support use a token.

It is also hard to be precise. It is local rate limitting per hypersync instance, so for ETH we have 4 replicas now (we will reduce this in the future) - but that means 4x the rate limit. So best we can say is between 100 and 200 requests per minute or something like that.

For an indexer it'll only affect the height polling, don't think it'll make things slower for historical sync (at least at the levels of rate limiting we will have to start)


- **Local Development**: No API token is required for local development, though requests will be rate limited.
- **Self-Hosted Deployments**: API tokens are required for unlimited HyperSync access in self-hosted deployments. The token can be set via the `ENVIO_API_TOKEN` environment variable in your indexer configuration. This can be read from the `.env` file in the root of your HyperIndex project.
- **Hosted Service**: Indexers deployed to our hosted service will have special access that doesn't require a custom API token.
- **Free Usage**: The service remains free to use until the end of May 2024.
- **Future Pricing**: From June 2024 onwards, we will introduce tiered packages based on usage. For preferred introductory pricing based on your specific use case, reach out to us on [Discord](https://discord.gg/Q9qt8gZ2fX).

For more details about API tokens, including how to generate and implement them, see our [API Tokens documentation](/docs/HyperSync/api-tokens).

---

## Feature Roadmap

Upcoming features on our development roadmap:
Expand Down
9 changes: 9 additions & 0 deletions docs/HyperSync/api-tokens.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ import TabItem from "@theme/TabItem";

API tokens provide authenticated access to HyperSync services, enabling enhanced capabilities and usage tracking. While currently optional, they will become mandatory in the future and offer significant advantages for serious usage.

**Starting from 9 May**, you will still be able to use HyperSync without API tokens, but **rate limits will be applied**. The service will remain **free to use until the end of May**. From **June onwards**, we will introduce a set of **tiered packages based on usage**. Indexers deployed to our hosted service will have special access to HyperSync that does not require a custom API token.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably update a message in /hyperindex/messages

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea 👍🏼

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should add some kind of note about API tokens in the HyperIndex docs also.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 👍🏼


## Table of Contents

- [Benefits of Using API Tokens](#benefits-of-using-api-tokens)
Expand Down Expand Up @@ -78,6 +80,13 @@ let client = Client::new(ClientConfig {
</TabItem>
</Tabs>

## Understanding Usage

To understand your current month's usage, visit [https://envio.dev/app/api-tokens](https://envio.dev/app/api-tokens). Usage is composed of two main components:

- **Number of Requests**: The total count of API requests made.
- **Credits**: A calculation based on the endpoint used and the size of the query.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now this calculation is too opaque and scary for a user IMO.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you suggest? You can see it on the usage dashboard. We can create a separate doc on the calculation if you like but it's probably TMI for a user and subject to change during the course of this month.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still haven't even decided on the calc tbh as I know it varies a lot per use case. Maybe just to say the calc takes into account, data bandwidth, disk read etc and is a best representation of usage, we happy to expand upon detail on request?

I know if I was a user and a pricing calc was completley opaque it would be quite nerve wracking so I think we do need to share in more detail somewhere at somestage

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool I'll add that 👍🏼 sounds good.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks so much!


## Security Best Practices

When working with API tokens:
Expand Down