Skip to content

Commit 65ceab9

Browse files
rafaelugolinirafaelugolini
committed
feat: process transfers
0 parents  commit 65ceab9

28 files changed

+8347
-0
lines changed

‎.dockerignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
/.git
2+
/node_modules
3+
/lib
4+
/*Versions.json
5+
npm-debug.log
6+
7+
# OS Files
8+
.DS_Store

‎.env

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
DB_NAME=squid
2+
DB_PORT=23798
3+
GQL_PORT=4350
4+
# JSON-RPC node endpoint, both wss and https endpoints are accepted
5+
RPC_ENDPOINT=

‎.gitignore

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
/node_modules
2+
/lib
3+
/builds
4+
5+
/**Versions.json
6+
7+
# IDE files
8+
/.idea

‎.gitpod.yml

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# This configuration file was automatically generated by Gitpod.
2+
# Please adjust to your needs (see https://www.gitpod.io/docs/introduction/learn-gitpod/gitpod-yaml)
3+
# and commit this file to your remote git repository to share the goodness with others.
4+
5+
# Learn more from ready-to-use templates: https://www.gitpod.io/docs/introduction/getting-started/quickstart
6+
github:
7+
prebuilds:
8+
# enable for the master/default branch (defaults to true)
9+
master: true
10+
# enable for all branches in this repo (defaults to false)
11+
branches: false
12+
# enable for pull requests coming from this repo (defaults to true)
13+
pullRequests: true
14+
# add a check to pull requests (defaults to true)
15+
addCheck: true
16+
# add a "Review in Gitpod" button as a comment to pull requests (defaults to false)
17+
addComment: false
18+
19+
tasks:
20+
- init: |
21+
npm i
22+
npm i -g @subsquid/cli
23+
docker compose pull
24+
gp sync-done setup
25+
- name: DB
26+
command: |
27+
gp sync-await setup
28+
sqd up
29+
- name: GraphQL API
30+
command: |
31+
gp sync-await setup
32+
sqd serve
33+
- command: |
34+
gp ports await 4350
35+
gp preview $(gp url 4350)/graphql
36+
- name: Squid procesor
37+
command: |
38+
gp open src/processor.ts
39+
gp sync-await setup
40+
sqd build
41+
gp ports await 23798
42+
sqd process

‎LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 Subsquid Labs
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎README.md

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/subsquid/squid-evm-template)
2+
3+
# Minimal EVM squid
4+
5+
This is a starter template of a squid indexer for EVM networks (Ethereum, Polygon, BSC, etc.). See [Squid SDK docs](https://docs.subsquid.io/) for a complete reference.
6+
7+
To extract EVM logs and transactions by a topic or a contract address, use `EvmBatchProcessor.addLog()` and `EvmBatchProcessor.addTransaction()` methods of the `EvmBatchProcessor` instance defined in `src/processor.ts`.
8+
9+
The requested data is transformed in batches by a single handler provided to the `processor.run()` method.
10+
11+
For a full list of supported networks and config options,
12+
check the [`EvmBatchProcessor` overview](https://docs.subsquid.io/develop-a-squid/evm-processor/) and the [configuration page](https://docs.subsquid.io/develop-a-squid/evm-processor/configuration/).
13+
14+
For a step-by-step migration guide from TheGraph, see [the dedicated docs page](https://docs.subsquid.io/migrate/migrate-subgraph/).
15+
16+
Dependencies: Node.js, Docker.
17+
18+
## Quickstart
19+
20+
```bash
21+
# 0. Install @subsquid/cli a.k.a. the sqd command globally
22+
npm i -g @subsquid/cli
23+
24+
# 1. Retrieve the template
25+
sqd init my_squid_name -t evm
26+
cd my_squid_name
27+
28+
# 2. Install dependencies
29+
npm ci
30+
31+
# 3. Start a Postgres database container and detach
32+
sqd up
33+
34+
# 4. Build and start the processor
35+
sqd process
36+
37+
# 5. The command above will block the terminal
38+
# being busy with fetching the chain data,
39+
# transforming and storing it in the target database.
40+
#
41+
# To start the graphql server open the separate terminal
42+
# and run
43+
sqd serve
44+
```
45+
A GraphiQL playground will be available at [localhost:4350/graphql](http://localhost:4350/graphql).
46+
47+
## Dev flow
48+
49+
### 1. Define database schema
50+
51+
Start development by defining the schema of the target database via `schema.graphql`.
52+
Schema definition consists of regular graphql type declarations annotated with custom directives.
53+
Full description of `schema.graphql` dialect is available [here](https://docs.subsquid.io/basics/schema-file).
54+
55+
### 2. Generate TypeORM classes
56+
57+
Mapping developers use TypeORM [EntityManager](https://typeorm.io/#/working-with-entity-manager)
58+
to interact with target database during data processing. All necessary entity classes are
59+
generated by the squid framework from `schema.graphql`. This is done by running `sqd codegen`
60+
command.
61+
62+
### 3. Generate database migrations
63+
64+
All database changes are applied through migration files located at `db/migrations`.
65+
`squid-typeorm-migration(1)` tool provides several commands to drive the process.
66+
67+
```bash
68+
## drop create the database
69+
sqd down
70+
sqd up
71+
72+
## replace any old schemas with a new one made from the entities
73+
sqd migration:generate
74+
```
75+
See [docs on database migrations](https://docs.subsquid.io/basics/db-migrations) for more details.
76+
77+
### 4. Import ABI contract and generate interfaces to decode events
78+
79+
It is necessary to import the respective ABI definition to decode EVM logs. One way to generate a type-safe facade class to decode EVM logs is by placing the relevant JSON ABIs to `./abi`, then using `squid-evm-typegen(1)` via an `sqd` script:
80+
81+
```bash
82+
sqd typegen
83+
```
84+
85+
See more details on the [`squid-evm-typegen` doc page](https://docs.subsquid.io/evm-indexing/squid-evm-typegen).
86+
87+
## Project conventions
88+
89+
Squid tools assume a certain [project layout](https://docs.subsquid.io/basics/squid-structure):
90+
91+
* All compiled js files must reside in `lib` and all TypeScript sources in `src`.
92+
The layout of `lib` must reflect `src`.
93+
* All TypeORM classes must be exported by `src/model/index.ts` (`lib/model` module).
94+
* Database schema must be defined in `schema.graphql`.
95+
* Database migrations must reside in `db/migrations` and must be plain js files.
96+
* `sqd(1)` and `squid-*(1)` executables consult `.env` file for environment variables.

‎abi/README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# ABI folder
2+
3+
This is a dedicated folder for ABI files. Place you contract ABI here and generate facade classes for type-safe decoding of the event, function data and contract state queries with
4+
5+
```sh
6+
sqd typegen
7+
```
8+
9+
This `typegen` command is defined in `commands.json`.

‎abi/oeth.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
[{"anonymous":false,"inputs":[{"indexed":true,"internalType":"address","name":"owner","type":"address"},{"indexed":true,"internalType":"address","name":"spender","type":"address"},{"indexed":false,"internalType":"uint256","name":"value","type":"uint256"}],"name":"Approval","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"internalType":"address","name":"previousGovernor","type":"address"},{"indexed":true,"internalType":"address","name":"newGovernor","type":"address"}],"name":"GovernorshipTransferred","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"internalType":"address","name":"previousGovernor","type":"address"},{"indexed":true,"internalType":"address","name":"newGovernor","type":"address"}],"name":"PendingGovernorshipTransfer","type":"event"},{"anonymous":false,"inputs":[{"indexed":false,"internalType":"uint256","name":"totalSupply","type":"uint256"},{"indexed":false,"internalType":"uint256","name":"rebasingCredits","type":"uint256"},{"indexed":false,"internalType":"uint256","name":"rebasingCreditsPerToken","type":"uint256"}],"name":"TotalSupplyUpdatedHighres","type":"event"},{"anonymous":false,"inputs":[{"indexed":true,"internalType":"address","name":"from","type":"address"},{"indexed":true,"internalType":"address","name":"to","type":"address"},{"indexed":false,"internalType":"uint256","name":"value","type":"uint256"}],"name":"Transfer","type":"event"},{"inputs":[],"name":"_totalSupply","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_owner","type":"address"},{"internalType":"address","name":"_spender","type":"address"}],"name":"allowance","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_spender","type":"address"},{"internalType":"uint256","name":"_value","type":"uint256"}],"name":"approve","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_account","type":"address"}],"name":"balanceOf","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"account","type":"address"},{"internalType":"uint256","name":"amount","type":"uint256"}],"name":"burn","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"uint256","name":"_newTotalSupply","type":"uint256"}],"name":"changeSupply","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"claimGovernance","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_account","type":"address"}],"name":"creditsBalanceOf","outputs":[{"internalType":"uint256","name":"","type":"uint256"},{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_account","type":"address"}],"name":"creditsBalanceOfHighres","outputs":[{"internalType":"uint256","name":"","type":"uint256"},{"internalType":"uint256","name":"","type":"uint256"},{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"decimals","outputs":[{"internalType":"uint8","name":"","type":"uint8"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_spender","type":"address"},{"internalType":"uint256","name":"_subtractedValue","type":"uint256"}],"name":"decreaseAllowance","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"governor","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_spender","type":"address"},{"internalType":"uint256","name":"_addedValue","type":"uint256"}],"name":"increaseAllowance","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"string","name":"_nameArg","type":"string"},{"internalType":"string","name":"_symbolArg","type":"string"},{"internalType":"address","name":"_vaultAddress","type":"address"},{"internalType":"uint256","name":"_initialCreditsPerToken","type":"uint256"}],"name":"initialize","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"isGovernor","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"","type":"address"}],"name":"isUpgraded","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_account","type":"address"},{"internalType":"uint256","name":"_amount","type":"uint256"}],"name":"mint","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"name","outputs":[{"internalType":"string","name":"","type":"string"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"","type":"address"}],"name":"nonRebasingCreditsPerToken","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"nonRebasingSupply","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"rebaseOptIn","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"rebaseOptOut","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"","type":"address"}],"name":"rebaseState","outputs":[{"internalType":"enum OUSD.RebaseOptions","name":"","type":"uint8"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"rebasingCredits","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"rebasingCreditsHighres","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"rebasingCreditsPerToken","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"rebasingCreditsPerTokenHighres","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"symbol","outputs":[{"internalType":"string","name":"","type":"string"}],"stateMutability":"view","type":"function"},{"inputs":[],"name":"totalSupply","outputs":[{"internalType":"uint256","name":"","type":"uint256"}],"stateMutability":"view","type":"function"},{"inputs":[{"internalType":"address","name":"_to","type":"address"},{"internalType":"uint256","name":"_value","type":"uint256"}],"name":"transfer","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_from","type":"address"},{"internalType":"address","name":"_to","type":"address"},{"internalType":"uint256","name":"_value","type":"uint256"}],"name":"transferFrom","outputs":[{"internalType":"bool","name":"","type":"bool"}],"stateMutability":"nonpayable","type":"function"},{"inputs":[{"internalType":"address","name":"_newGovernor","type":"address"}],"name":"transferGovernance","outputs":[],"stateMutability":"nonpayable","type":"function"},{"inputs":[],"name":"vaultAddress","outputs":[{"internalType":"address","name":"","type":"address"}],"stateMutability":"view","type":"function"}]

‎assets/README.MD

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Assets
2+
3+
`assets` is the designated folder for any additional files to be used by the squid, for example a static data file. The folder is added by default to `Dockerfile` and is kept when the squid is deployed to the Aquairum.

‎commands.json

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
{
2+
"$schema": "https://cdn.subsquid.io/schemas/commands.json",
3+
"commands": {
4+
"clean": {
5+
"description": "delete all build artifacts",
6+
"cmd": ["npx", "--yes", "rimraf", "lib"]
7+
},
8+
"build": {
9+
"description": "Build the squid project",
10+
"deps": ["clean"],
11+
"cmd": ["tsc"]
12+
},
13+
"up": {
14+
"description": "Start a PG database",
15+
"cmd": ["docker-compose", "up", "-d"]
16+
},
17+
"down": {
18+
"description": "Drop a PG database",
19+
"cmd": ["docker-compose", "down"]
20+
},
21+
"migration:apply": {
22+
"description": "Apply the DB migrations",
23+
"cmd": ["squid-typeorm-migration", "apply"]
24+
},
25+
"migration:generate": {
26+
"description": "Generate a DB migration matching the TypeORM entities",
27+
"deps": ["build", "migration:clean"],
28+
"cmd": ["squid-typeorm-migration", "generate"],
29+
},
30+
"migration:clean": {
31+
"description": "Clean the migrations folder",
32+
"cmd": ["npx", "--yes", "rimraf", "./db/migrations"],
33+
},
34+
"migration": {
35+
"deps": ["build"],
36+
"cmd": ["squid-typeorm-migration", "generate"],
37+
"hidden": true
38+
},
39+
"codegen": {
40+
"description": "Generate TypeORM entities from the schema file",
41+
"cmd": ["squid-typeorm-codegen"]
42+
},
43+
"typegen": {
44+
"description": "Generate data access classes for an ABI file(s) in the ./abi folder",
45+
"cmd": ["squid-evm-typegen", "./src/abi", {"glob": "./abi/*.json"}, "--multicall"]
46+
},
47+
"process": {
48+
"description": "Load .env and start the squid processor",
49+
"deps": ["build", "migration:apply"],
50+
"cmd": ["node", "--require=dotenv/config", "lib/main.js"]
51+
},
52+
"process:prod": {
53+
"description": "Start the squid processor",
54+
"deps": ["migration:apply"],
55+
"cmd": ["node", "lib/main.js"],
56+
"hidden": true
57+
},
58+
"serve": {
59+
"description": "Start the GraphQL API server",
60+
"cmd": ["squid-graphql-server"]
61+
},
62+
"serve:prod": {
63+
"description": "Start the GraphQL API server with caching and limits",
64+
"cmd": ["squid-graphql-server",
65+
"--dumb-cache", "in-memory",
66+
"--dumb-cache-ttl", "1000",
67+
"--dumb-cache-size", "100",
68+
"--dumb-cache-max-age", "1000" ]
69+
},
70+
"check-updates": {
71+
"cmd": ["npx", "--yes", "npm-check-updates", "--filter=/subsquid/", "--upgrade"],
72+
"hidden": true
73+
},
74+
"bump": {
75+
"description": "Bump @subsquid packages to the latest versions",
76+
"deps": ["check-updates"],
77+
"cmd": ["npm", "i", "-f"]
78+
},
79+
"open": {
80+
"description": "Open a local browser window",
81+
"cmd": ["npx", "--yes", "opener"]
82+
}
83+
}
84+
}

‎db/migrations/1674844779131-Data.js

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
module.exports = class Data1674844779131 {
2+
name = 'Data1674844779131'
3+
4+
async up(db) {
5+
await db.query(`CREATE TABLE "burn" ("id" character varying NOT NULL, "block" integer NOT NULL, "address" text NOT NULL, "value" numeric NOT NULL, "tx_hash" text NOT NULL, CONSTRAINT "PK_dcb4f14ee4534154b31116553f0" PRIMARY KEY ("id"))`)
6+
await db.query(`CREATE INDEX "IDX_fc3726cbc7f5d4edf4340ae298" ON "burn" ("address") `)
7+
}
8+
9+
async down(db) {
10+
await db.query(`DROP TABLE "burn"`)
11+
await db.query(`DROP INDEX "public"."IDX_fc3726cbc7f5d4edf4340ae298"`)
12+
}
13+
}

‎docker-compose.yml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
version: "3"
2+
3+
services:
4+
db:
5+
image: postgres:15
6+
environment:
7+
POSTGRES_DB: squid
8+
POSTGRES_PASSWORD: postgres
9+
ports:
10+
- "${DB_PORT}:5432"
11+
# command: ["postgres", "-c", "log_statement=all"]
12+
shm_size: 1gb

0 commit comments

Comments
 (0)