Extract common extension utils into a separate package #123
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Move utility classes from typedb extension to a new top-level utils package that can be shared across multiple LocalStack extensions: - ProxiedDockerContainerExtension: base class for Docker-based extensions - ProxyResource: HTTP/1.1 request proxy resource - HTTP2/gRPC proxy utilities for forwarding binary traffic The new package is published as 'localstack-extensions-utils' on PyPI. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Change the module structure from localstack_extensions_utils to localstack.extensions.utils to follow LocalStack naming conventions. Co-Authored-By: Claude Opus 4.5 <[email protected]>
…nsions Rename from localstack.extensions.utils to localstack_extensions.utils to avoid conflicts with the main localstack package namespace. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Add Makefile with build, publish, format, and lint targets - Update README: rename Installation to Usage, show pyproject.toml dependency format Co-Authored-By: Claude Opus 4.5 <[email protected]>
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Merged
Contributor
|
I think here we should be moving tests for request handling to the new package, based on the typedb ones, but using a well-known and neutral dockerised service that has grpc + http support. Then tests in the specific extensions would just need to test basic connectivity or options for those. Not immediately sure what that other test service might be. (We might also be able to test these things without running the tests inside localstack itself?) |
- Add unit tests for HTTP/2 frame parsing and TcpForwarder (33 tests) - Add integration tests using grpcbin Docker container (19 tests) - Move test_get_frames_from_http2_stream from typedb to utils - Add test dependencies and pytest markers to pyproject.toml - Add test targets to Makefile (test, test-unit, test-integration) - Add proto files for grpcbin service definitions Co-Authored-By: Claude Opus 4.5 <[email protected]>
whummer
force-pushed
the
extract-utils-package
branch
from
d7cb435 to
75f2301
Compare
- Add .github/workflows/utils.yml with unit and integration test jobs - Use localstack.utils.net.wait_for_port_open instead of custom implementation Co-Authored-By: Claude Opus 4.5 <[email protected]>
Fixes missing jsonpatch transitive dependency when importing from localstack.utils.net in integration tests. Co-Authored-By: Claude Opus 4.5 <[email protected]>
jsonpatch is a transitive dependency of localstack that may not be installed automatically in all environments. Co-Authored-By: Claude Opus 4.5 <[email protected]>
The integration tests work at the raw HTTP/2 frame level and don't need generated gRPC stubs. This simplifies the test setup. Co-Authored-By: Claude Opus 4.5 <[email protected]>
The exports were accidentally removed, breaking imports for dependent packages like typedb. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Use localstack's DOCKER_CLIENT utility for container management in integration tests for better consistency and error handling. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Remove redundant 'assert True' statements (no exception = success) - Update docstrings to clarify these tests validate the utility functions (TcpForwarder, frame parsing), not the LocalStack proxy integration Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Remove test_tcp_forwarder.py (mocked unit tests not adding value) - Remove tests/conftest.py with custom marker logic - Remove marker definitions from pyproject.toml - Tests are now simply run by directory path Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Replace time.sleep() calls in tests with threading.Event for synchronization - Reduce fixture initialization sleep from 1.0s to 0.5s - Add parse_server_frames() helper for parsing HTTP/2 server responses - Fix first test to do proper HTTP/2 handshake instead of raw TCP connect Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While developing a couple of LS Extensions recently (e.g., TypeDB, Wiremock, ParadeDB), we saw some patterns occuring frequently - for example, creating an extension that wraps a running Docker container and proxies any HTTP/HTTP2/gRPC network traffic to/from it through the LS Gateway (the
ProxiedDockerContainerExtensioncan be used as a base class to abstract out the).This PR creates a new
localstack-extensions-utilspackage that can be used to externalize common extension utils and make them accessible to different extensions. The library will be pushed to pypi.org post-merge.Note 🤖 : This PR was co-created with Claude Code.