Skip to content

feat(logs): Improve logs with spans #194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 17, 2025

Conversation

iambriccardo
Copy link
Contributor

@iambriccardo iambriccardo commented Jul 16, 2025

This PR improves logging in several ways:

  • Span logging on close is no longer necessary since we now correctly log the request start and end.
  • The pipeline ID, publication name, and table ID are now added as metadata to spans within the replicator.
  • Spans are saved in the logs along with the full chain.

@iambriccardo iambriccardo force-pushed the riccardobusetti/feat/add-more-data-to-logs branch from f831319 to 241b81e Compare July 16, 2025 10:18
@iambriccardo iambriccardo changed the title riccardobusetti/feat/add more data to logs feat(logs): Improve logs with spans Jul 16, 2025
api/Cargo.toml Outdated
@@ -45,7 +45,7 @@ sqlx = { workspace = true, features = [
thiserror = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
tracing = { workspace = true, default-features = false }
tracing-actix-web = { workspace = true, features = ["emit_event_on_error"] }
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We emit errors on our own.

@iambriccardo iambriccardo marked this pull request as ready for review July 16, 2025 12:08
@iambriccardo iambriccardo requested a review from a team as a code owner July 16, 2025 12:08
Copy link
Contributor

@imor imor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a couple of comments.

let apply_worker_span = tracing::info_span!(
"apply_worker",
pipeline_id = self.pipeline_id,
publication_name = self.config.publication_name
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are pipeline_id and publication_name useful for every log line within the span? There's going to only one of each while the replicator is running. As opposed to something like a table_id which will be different.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea was that we want a reliable way to get a view of all logs of a certain pipeline for debugging a single customer.

@@ -145,7 +145,7 @@ async fn init_state_store(
Ok(PostgresStateStore::new(pipeline_id, pg_connection_config))
}

#[instrument(skip(pipeline))]
#[tracing::instrument(skip(pipeline), fields(pipeline_id = pipeline.id()))]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, is pipeline_id needed?

@iambriccardo iambriccardo merged commit 97cbe0a into main Jul 17, 2025
3 checks passed
@iambriccardo iambriccardo deleted the riccardobusetti/feat/add-more-data-to-logs branch July 17, 2025 07:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants