[v3-2-test] Dispose flask_sqlalchemy + airflow ORM engines in fab test fixtures (#65941)#66896
Closed
vatsrahul1001 wants to merge 1 commit into
Closed
[v3-2-test] Dispose flask_sqlalchemy + airflow ORM engines in fab test fixtures (#65941)#66896vatsrahul1001 wants to merge 1 commit into
vatsrahul1001 wants to merge 1 commit into
Conversation
…65941) Flake on a Pendulum2 providers run ended with 8 fab test errors — ``psycopg2.OperationalError: ... FATAL: sorry, too many clients already`` after 10031 tests had passed. Postgres' default ``max_connections=100`` was being exhausted before fab tests at the end of the bucket could acquire a connection. The accumulation came from fab test fixtures, not from per-test session leaks: - ``providers/fab/tests/unit/fab/www/views/conftest.py`` called ``settings.configure_orm()`` in an autouse module-scope fixture. ``configure_orm()`` reassigns the global engine **without** disposing the previous one, orphaning a pool of up to 15 pooled connections per call. ``settings.reconfigure_orm()`` is the helper that calls ``dispose_orm()`` first. - Several fab ``app`` fixtures call ``application.create_app()``, which does ``db = SQLAlchemy(flask_app)`` — flask_sqlalchemy eagerly creates a per-app engine in ``init_app()``. None of the fixtures disposed those engines on teardown, so each Flask app's pool survived the fixture and accumulated for the worker's lifetime. The fab bucket is one worker that runs many such fixtures back-to-back. Fixes: - ``views/conftest.py``: switch to ``reconfigure_orm()`` (no leak on re-entry); dispose the flask_sqlalchemy engines on the module-scope ``app`` fixture teardown. - ``www/test_auth.py``, ``views/test_views_custom_user_views.py``: add try/finally around the function-scope ``app`` fixture and dispose the flask_sqlalchemy engines per test. - ``auth_manager/test_security.py``: dispose engines on the module-scope ``app`` fixture teardown. - ``auth_manager/conftest.py``: convert the session-scope ``minimal_app_for_auth_api`` to ``yield`` and dispose engines on teardown. A follow-up branch that bumped ``max_connections`` to 300 has been dropped — with the leak fixed it's no longer load-bearing. Failing run: https://github.com/apache/airflow/actions/runs/24976790617/job/73131061767 (cherry picked from commit eee05dd)
vatsrahul1001
added
the
changelog:skip
vincbeck
previously approved these changes
May 14, 2026
vincbeck
dismissed
their stale review
This is a provider only change, no need to backport
Contributor
|
This is a provider change, no need to backport |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick of #65941
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 4.7) following the guidelines