Skip to content

Homogenize extractor crates: consistent structure, naming, and modules #1791

@LNSD

Description

@LNSD

Goal

Homogenize the three extractor crates (firehose, evm-rpc, solana) to have consistent structure, module names, function names, and type names. The end result should look as if all extractors were created from the same cookie-cutter template. Aligning them will improve maintainability, make it easier to onboard new extractors, and is a prerequisite for the dataset↔extractor decoupling needed for SQL DDL execution.

Current State

The three extractor crates have inconsistent structure:

Aspect firehose evm-rpc solana
Client module client.rsClient client.rsJsonRpcClient rpc_client.rs + of1_client.rs + extractor.rsSolanaExtractor
Error handling Error enum in lib.rs error.rsProviderError error.rsExtractorError
Dataset kind FirehoseDatasetKind EvmRpcDatasetKind SolanaDatasetKind
Metrics metrics.rs (pub) metrics.rs (pub) metrics.rs (private)
Tables evm/tables/ (nested) tables/ tables.rs + tables/
Provider config N/A (inline) provider.rs (pub) re-exports from amp-providers-solana
Extra modules proto/, evm/

Module structure comparison

firehose/src/             evm-rpc/src/              solana/src/
├── client.rs             ├── client.rs             ├── rpc_client.rs
├── dataset_kind.rs       ├── dataset_kind.rs       ├── of1_client.rs
├── dataset.rs            ├── dataset.rs            ├── extractor.rs
├── evm/                  ├── error.rs              ├── dataset_kind.rs
│   ├── mod.rs            ├── lib.rs                ├── dataset.rs
│   ├── pb_to_rows.rs     ├── metrics.rs            ├── error.rs
│   └── tables/           ├── provider.rs           ├── lib.rs
│       ├── mod.rs        └── tables/               ├── metrics.rs
│       ├── calls.rs          ├── mod.rs            ├── tables.rs
│       └── transactions.rs   └── transactions.rs   └── tables/
├── lib.rs                                              ├── block_headers.rs
├── metrics.rs                                          ├── block_rewards.rs
└── proto/                                              ├── instructions.rs
                                                        ├── messages.rs
                                                        └── transactions.rs

Naming inconsistencies

  • Error types: Error vs ProviderError vs ExtractorError
  • Client types: Client vs JsonRpcClient vs SolanaExtractor
  • Module visibility: mixed pub vs private for equivalent modules

Desired Outcome

All extractor crates should follow a consistent template. The end result should look as if they were all generated from the same cookie-cutter:

  • Same module names for equivalent functionality
  • Same type naming conventions (e.g., error type name pattern, client type name pattern)
  • Same module visibility for equivalent modules
  • Same structure for tables/ organization
  • Consistent approach to provider config, error handling, and metrics

Notes

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions