Refactor PostgresHook and associated runtime tests#66893
Refactor PostgresHook and associated runtime tests#66893SameerMesiah97 wants to merge 2 commits into
Conversation
Dev-iL
left a comment
There was a problem hiding this comment.
Thank you - this is a much-needed refactor I never got around to doing. Mostly LGTM with two inquiries/concerns.
| conn_args["cursor_factory"] = self._get_cursor(raw_cursor) | ||
| if raw_cursor: | ||
| key, value = self._get_cursor_config(raw_cursor) | ||
| conn_args[key] = value |
There was a problem hiding this comment.
Looks like the PPG2 branch's handling of raw_cursor was kept:
raw_cursor = conn.extra_dejson.get("cursor") # PPG3, default None
raw_cursor = conn.extra_dejson.get("cursor", False) # PPG2, default FalseAlthough both are falsy - are you sure this is safe?
There was a problem hiding this comment.
I believe keeping the truthy check i.e. if raw_cursor should preserve existing behavior. Before the refactoring raw_cursor in the PPG2 path would default to False is no raw_cursor is found and the truthy check would prevent the cursor from being evaluated. The only difference now is that None is kept as None, which fails the truthy check. In other words, I see no need to change this.
There was a problem hiding this comment.
Hmm alright. I hope there was no good reason for that difference existing in the first place.
| return cast("CompatConnection", connection) | ||
|
|
||
| return cast("CompatConnection", ppg2_connect(**conn_args)) |
There was a problem hiding this comment.
Seeing how you're already refactoring this, is there a way to avoid the casts? Do you think we still need CompatConnection?
There was a problem hiding this comment.
I have removed the cast the from both.
…sycopg3. Consolidate duplicated insert/upsert and dialect tests into a shared base class while preserving version-specific behavior and lineage coverage.
6e2a98e to
2561091
Compare
|
I have addressed your feedback. |
Description
This change refactors psycopg-specific connection handling in
PostgresHookand consolidates duplicated runtime test coverage shared between psycopg2 and psycopg3 implementations.Connection setup responsibilities previously embedded directly inside
get_connhave been extracted into dedicated helper methods._get_cursor_confignow centralizes cursor configuration handling, while_create_connectionencapsulates driver-specific connection initialization, including adapter registration and notice handler setup.The accompanying test suite has also been reorganized to reduce duplicated behavioral coverage. Shared runtime tests have been moved into
_BasePostgresHookRuntimeTests, allowing psycopg2- and psycopg3-specific test classes to focus only on behavior that differs between implementations.Rationale
PostgresHook.get_connpreviously contained duplicated psycopg2- and psycopg3-specific connection setup logic. Maintaining separate inline code paths for cursor configuration, adapter registration, and notice handler initialization increases maintenance overhead and makes future changes harder to validate consistently across both drivers.Extracting
_get_cursor_configand_create_connectionsimplifies the control flow inget_conn, reduces branching complexity, and makes psycopg-specific behavior easier to reason about and extend independently.The test suite had a similar issue, with substantial overlap between psycopg2 and psycopg3 runtime behavioral tests. Consolidating identical coverage into
_BasePostgresHookRuntimeTestsimproves readability, reduces repeated CI execution for equivalent scenarios, and keeps implementation-specific tests scoped to behavior that actually differs.Tests
_BasePostgresHookRuntimeTests.execute_batch, and notice handling tests.copy_expertand notice handling coverage.Backwards Compatibility
This change only refactors test code and does not modify production behavior or public APIs.