When using the new breadcrumbs handler with Messenger, the Messenger should either flush or clear *before* every new message

[dkarlovi]

Environment

sentry/sdk                             3.3.0                         This is a metapackage shipping sentry/sentry with a recommended HTTP client.
sentry/sentry                          3.11.0                        A PHP SDK for Sentry (http://sentry.io)
sentry/sentry-symfony                  4.4.0                         Symfony integration for Sentry (http://getsentry.com)

Steps to Reproduce

sentry:
    register_error_listener: false
    dsn: "%env(file:SENTRY_DSN_FILE)%"
    messenger:
        enabled: true
        capture_soft_fails: false
    tracing:
        enabled: false
    options:
        environment: '%env(APP_COHORT)%'
        integrations:
            - 'Sentry\Integration\IgnoreErrorsIntegration'

services:
    Sentry\Monolog\BreadcrumbHandler:
        arguments:
            - '@Sentry\State\HubInterface'
            - !php/const Monolog\Logger::INFO

monolog:
    handlers:
        sentry_breadcrumbs:
            type: service
            name: sentry_breadcrumbs
            id: Sentry\Monolog\BreadcrumbHandler
        sentry:
            type: sentry
            level: !php/const Monolog\Logger::ERROR
            hub_id: Sentry\State\HubInterface

Expected Result

The captured error contains breadcrumbs for the current message.

Actual Result

The captured error contains breadcrumbs for all the messages prior, most of which didn't trigger a Sentry capture.

[cleptric]

Just to clarify, you're talking about unrelated breadcrumbs during the current request? This sounds like it's expected to me, breadcrumbs should show a trail of events that happened before an event. Or do I miss something?

[dkarlovi]

The issue is it can show you many unrelated events, sometimes minutes before the current one. In the example I'm looking at, the first breadcrumb is from 10:48:07, the capture happened in 11:08:16, there were totally unrelated messages processed in the meantime, but they're still marked as "breadcrumbs".

From my POV, the breadcrumbs should be related to the current event, no? They're not meant to replace distributed tracing.

[cleptric]

Are you running in some sort of CLI context? For HTTP requests, yes, the breadcrumbs should only be related to the current request cycle and nothing more.

[dkarlovi]

Yes, this is Symfony Messenger specifically.

[dkarlovi]

But I think I understand what you mean, the state of the process is also part of the story. It having processed previous messages can also be something to influence the current issue. 👍 I guess it's working as intended, it just looks weird when you're not used to it. :)

[cleptric]

In our Laravel SDK, we do clear the breadcrumbs for each job.

https://github.com/getsentry/sentry-laravel/blob/8ce96842a9c3b047077fd59d66692ccf4eee5fc0/src/Sentry/Laravel/EventHandler.php#L573-L584

I can see how this would be helpful, not clearing them might be helpful as well. So it depends 🙂

[dkarlovi]

So, what do you think here? Do I reopen?

[cleptric]

Yes, let's take a look on how to improve this a bit.