Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Separate location for ancillary for cardano database v1 #2380

Open
wants to merge 17 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
8b34a07
feat: add `ancillary_locations` and `ancillary_size` to all snapshot …
Alenar Mar 17, 2025
84b70e8
feat(aggregator): add `snapshot_all_completed_immutables` to Snapshotter
Alenar Mar 18, 2025
523e3a2
feat(aggregator): snapshot only completed immutables in cardano db v1
Alenar Mar 18, 2025
4cbb607
refactor(aggregator): compute archive name separatly from the archive…
Alenar Mar 18, 2025
34c3f7a
test(aggregator): make `DumbUploader` store multiple uploads instead …
Alenar Mar 18, 2025
4b94663
feat(aggregator): snapshot & upload ancillary in immutable files full…
Alenar Mar 18, 2025
c3e2888
refactor(aggregator): use `LocalUploader` for cardano database v1 upl…
Alenar Mar 18, 2025
9468b49
test(client-lib): reorganize integration tests infrastructure
Alenar Mar 18, 2025
a2ab589
test(client-lib): only add last ledger file and remove volatile from …
Alenar Mar 18, 2025
76f965c
test(client-lib): update cardano db v1 test to ancillary archive split
Alenar Mar 18, 2025
9065e6c
fix(client-cli): incorrect genesis vkey in shared dev.json param file
Alenar Mar 19, 2025
4ea7f74
feat(client-lib+cli): download ancillary archive if found in cardano …
Alenar Mar 19, 2025
e582efe
feat(client-cli): include ancillary size in prerequisite disk space c…
Alenar Mar 19, 2025
23bf434
chore: update changelog
Alenar Mar 19, 2025
1af6d60
refactor: rephrasings, change ancillary archive naming by moving 'anc…
dlachaume Mar 21, 2025
f2da5a6
refactor(client-lib): rename `fake` module to `fake_aggregator`
dlachaume Mar 21, 2025
80668fe
chore(client-lib): add comment to clarify underscore prefix usage for…
dlachaume Mar 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,13 @@ As a minor extension, we have adopted a slightly different versioning convention

- End support for **macOS x64 pre-built binaries** for the client CLI.

- Cardano database certification: creation of two separate archives, one for the immutable files and another for the ancillary files.

- **UNSTABLE** Implement a slave signer registration mode in the aggregator.

- **UNSTABLE** Cardano database incremental certification:

- Implement the client library for the the signed entity type `CardanoDatabase` (download and prove snapshot).
- Implement the client library for the signed entity type `CardanoDatabase` (download and prove snapshot).
- Implement the client CLI commands for the signed entity type `CardanoDatabase` (snapshot list, snapshot show and download commands).
- Implement an example crate for the signed entity type `CardanoDatabase`.
- Lighter ancillary archive by only including what's strictly necessary: the latest ledger file and the last immutable file trio.
Expand Down
44 changes: 39 additions & 5 deletions examples/client-cardano-database/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ async fn main() -> MithrilResult<()> {
pub struct IndicatifFeedbackReceiver {
progress_bar: MultiProgress,
download_pb: RwLock<Option<ProgressBar>>,
ancillary_download_pb: RwLock<Option<ProgressBar>>,
certificate_validation_pb: RwLock<Option<ProgressBar>>,
}

Expand All @@ -111,9 +112,19 @@ impl IndicatifFeedbackReceiver {
Self {
progress_bar: progress_bar.clone(),
download_pb: RwLock::new(None),
ancillary_download_pb: RwLock::new(None),
certificate_validation_pb: RwLock::new(None),
}
}

fn new_download_bytes_progress_bar(size: u64) -> ProgressBar {
let pb = ProgressBar::new(size);
pb.set_style(ProgressStyle::with_template("{spinner:.green} [{elapsed_precise}] [{wide_bar:.cyan/blue}] {bytes}/{total_bytes} ({eta})")
.unwrap()
.with_key("eta", |state: &ProgressState, w: &mut dyn Write| write!(w, "{:.1}s", state.eta().as_secs_f64()).unwrap())
.progress_chars("#>-"));
pb
}
}

#[async_trait]
Expand All @@ -126,11 +137,7 @@ impl FeedbackReceiver for IndicatifFeedbackReceiver {
size,
} => {
println!("Starting download of snapshot '{digest}'");
let pb = ProgressBar::new(size);
pb.set_style(ProgressStyle::with_template("{spinner:.green} [{elapsed_precise}] [{wide_bar:.cyan/blue}] {bytes}/{total_bytes} ({eta})")
.unwrap()
.with_key("eta", |state: &ProgressState, w: &mut dyn Write| write!(w, "{:.1}s", state.eta().as_secs_f64()).unwrap())
.progress_chars("#>-"));
let pb = Self::new_download_bytes_progress_bar(size);
self.progress_bar.add(pb.clone());
let mut download_pb = self.download_pb.write().await;
*download_pb = Some(pb);
Expand All @@ -152,6 +159,33 @@ impl FeedbackReceiver for IndicatifFeedbackReceiver {
}
*download_pb = None;
}
MithrilEvent::SnapshotAncillaryDownloadStarted {
download_id: _,
size,
} => {
println!("Starting download of ancillary snapshot");
let pb = Self::new_download_bytes_progress_bar(size);
self.progress_bar.add(pb.clone());
let mut ancillary_download_pb = self.ancillary_download_pb.write().await;
*ancillary_download_pb = Some(pb);
}
MithrilEvent::SnapshotAncillaryDownloadProgress {
download_id: _,
downloaded_bytes,
size: _,
} => {
let ancillary_download_pb = self.ancillary_download_pb.read().await;
if let Some(progress_bar) = ancillary_download_pb.as_ref() {
progress_bar.set_position(downloaded_bytes);
}
}
MithrilEvent::SnapshotAncillaryDownloadCompleted { download_id: _ } => {
let mut ancillary_download_pb = self.ancillary_download_pb.write().await;
if let Some(progress_bar) = ancillary_download_pb.as_ref() {
progress_bar.finish_with_message("Snapshot ancillary download completed");
}
*ancillary_download_pb = None;
}
MithrilEvent::CertificateChainValidationStarted {
certificate_chain_validation_id: _,
} => {
Expand Down
Loading
Loading