Pytest plugin pytest-sentry not sending issue events because it is killed

[antonpirker]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

all

Steps to Reproduce

See this example project:
https://github.com/antonpirker/testing-sentry/tree/main/test-pytest-sentry

We run a flaky test multiples times using the pytest-sentry plugin for pytest (which can be found here: https://github.com/getsentry/pytest-sentry)

Expected Result

Issues representing the failed tests runs showing up in Sentry.

Actual Result

The envelopes never reach Sentry. It seems that pytest kills the process right in the middle of sending the envelope: https://github.com/getsentry/sentry-python/blob/2.23.1/sentry_sdk/transport.py#L349-L354

The line after self._request() is never reached.

[bukzor]

This may (or may not) be a workaround, below. The idea is to ensure that client.flush() is called before the main thread exits.

import pytest_sentry import hookwrapper

@hookwrapper(itemgetter=lambda session: session)
def pytest_sessionfinish(session):
    del session
    yield
    Scope.get_client().flush(timeout=2.0)

To do this in a generic way requires a bit more code. For an example, see
https://github.com/getsentry/sentry/blob/1785768/src/sentry/testutils/pytest/json_report_reruns.py

HOWEVER, it's a mystery why this isn't already handled by the AtExit integration. Its purpose is exactly this, and I double-checked that it's active and enabled, so why/how isn't it doing its job?

[szokeasaurusrex]

@antonpirker, why is this a bug in Sentry SDK and not in the pytest-sentry plugin?

[antonpirker]

Because no one ever looks at issues in pytest-sentry. This way we do not forget about this.