feat(webhooks): verify_and_parse_* API for compressed payloads (CHA-3071)#230
Open
nijeesh-stream wants to merge 10 commits into
Open
feat(webhooks): verify_and_parse_* API for compressed payloads (CHA-3071)#230nijeesh-stream wants to merge 10 commits into
nijeesh-stream wants to merge 10 commits into
Conversation
… (CHA-3071) Stream Chat backend can now compress outbound webhook payloads with gzip and, for SQS / SNS firehose delivery, base64-wrap the compressed bytes so they remain valid UTF-8 over the queue. Add two new client methods that let customers decompress + verify in a single call: - decompress_webhook_body(body, content_encoding=None, payload_encoding=None) primitive decode that handles gzip and/or base64 - verify_and_decode_webhook(body, x_signature, content_encoding=None, payload_encoding=None) decode + HMAC-SHA256 verify Both are exposed on the sync StreamChat and async StreamChatAsync clients through the shared StreamChatInterface base, mirroring the existing verify_webhook helper. The existing verify_webhook signature and behavior are unchanged for backward compatibility. A new WebhookSignatureError (extends StreamAPIException) is raised on signature mismatch, malformed gzip, or malformed base64. Unsupported encoding values raise ValueError with a message that points at the supported algorithm (gzip). The decoding logic lives in stream_chat/webhook.py so it can be tested without instantiating an HTTP client. The new tests cover the cross-SDK contract: passthrough, gzip round-trip, base64 round-trip, base64 + gzip (SQS / SNS shape), case-insensitive aliases, every unsupported content_encoding (br / brotli / zstd / deflate / compress / lz4), unsupported payload_encoding (hex / url / binary), invalid gzip / base64 input, and three signature-mismatch variants (wrong signature, signature over compressed bytes, signature over wrapped bytes). Docs: webhooks_overview.md gets a "Compressed webhook bodies" section with Django, Flask, and SQS / SNS usage examples. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Replaces the earlier verify_and_decode_webhook surface with the cross-SDK contract documented at https://getstream.io/chat/docs/node/webhooks_overview/. Module-level helpers in stream_chat.webhook: Primitives: ungzip_payload - gzip magic-byte detection + inflate decode_sqs_payload - base64 then ungzip-if-magic decode_sns_payload - alias for decode_sqs_payload verify_signature - constant-time HMAC-SHA256 comparison parse_event - JSON -> dict (typed event lands later) Composite (return parsed event dict): verify_and_parse_webhook verify_and_parse_sqs verify_and_parse_sns The composite functions auto-detect compression from body bytes, so the same handler stays correct whether or not Stream is currently compressing payloads, and behind middleware that auto-decompresses. Client instance methods (StreamChat / StreamChatAsync) mirror the three composite helpers with api_secret pulled from the client. The legacy verify_webhook(body, x_signature) -> bool boolean helper is unchanged for backward compatibility. Co-authored-by: Cursor <cursoragent@cursor.com>
nijeesh-stream
changed the title
binascii.Error is a subclass of ValueError, so listing both in the except clause triggers flake8-bugbear B014. Catching ValueError alone covers both cases. Co-authored-by: Cursor <cursoragent@cursor.com>
RFC 1952 defines the gzip magic number as the two-byte sequence 1F 8B; the third byte (CM) is informational and not part of the identifier. Trim the magic check from three bytes to two to match the spec and stay consistent with the reference implementations in the public docs. Co-authored-by: Cursor <cursoragent@cursor.com>
mogita
reviewed
May 11, 2026
Previously, passing a signature with non-ASCII bytes (e.g. b"\xff..."),
a non-ASCII unicode string, or a non-string type would raise
UnicodeDecodeError / TypeError from inside verify_signature, leaking
through verify_and_parse_webhook / _sqs / _sns and breaking the
documented contract that says malformed inputs must surface as
WebhookSignatureError.
The boolean primitive now returns False for those inputs (an
invalid-format signature can by definition never match), so the
composite helpers raise WebhookSignatureError("invalid webhook
signature") as expected. The constant-time HMAC comparison path is
unchanged for well-formed inputs.
Adds regression tests for non-ASCII bytes, non-ASCII str, and
non-string signature inputs at both the primitive and composite
layers.
Co-authored-by: Cursor <cursoragent@cursor.com>
The previous draft referenced helpers that were renamed during the refactor to the verify_and_parse_* contract (CHA-3071): - client.verify_and_decode_webhook(...) -> verify_and_parse_webhook - decompress_webhook_body(...) -> removed (no public form) - content_encoding / payload_encoding -> removed (magic-byte detect) Following the old snippets would hit AttributeError immediately. The section is rewritten to document the real surface area: - client.verify_and_parse_webhook(body, signature) - client.verify_and_parse_sqs(message_body, signature) - client.verify_and_parse_sns(message, signature) - module-level webhook.verify_and_parse_* helpers for stateless use - WebhookSignatureError as the single error class It also clarifies the return type (parsed dict, not raw bytes) and notes that the legacy verify_webhook bool helper stays unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>
decode_sns_payload now JSON-parses the SNS HTTP notification envelope
({"Type":"Notification","Message":"..."}) and extracts the inner
Message field before running the SQS pipeline. Falls through to the
pre-extracted Message string when the input is not a JSON envelope so
existing call sites keep working.
Test adds a realistic SNS HTTP notification body fixture and exercises
both the new envelope path and the existing pre-extracted Message path.
Docs updated to show the typical "pass the raw HTTP body" call site.
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…den fixtures (CHA-3071) Per Tommaso's suggestion, align the gzip helper with the GNU `gunzip` command name. The function was added in this PR and not yet released, so this is a straight rename with no back-compat alias. Adds Tommaso's reference fixtures to the test suite as named cases so future SDKs can sanity-check against the same payloads: aGVsbG93b3JsZA== -> helloworld (base64) H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld (base64+gzip) Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds first-class support for gzip-compressed webhook payloads (HTTP webhooks, SQS, SNS) and exposes a stable
verify_and_parse_*API that mirrors the cross-SDK contract published in Webhooks Overview.New public API (
stream_chat/webhook.py)Module-level primitives:
gunzip_payload(body) -> bytes— gzip-magic-byte detection, no-op when not compresseddecode_sqs_payload(body) -> bytes— base64-decode then gunzip-if-magicdecode_sns_payload(notification_body) -> bytes— JSON-parse the SNS HTTP notification envelope, extract the innerMessage, then run the SQS pipeline. Falls through to a pre-extractedMessagestring when the input is not a JSON envelopeverify_signature(body, signature, secret) -> bool— HMAC-SHA256 over the uncompressed body, with a constant-time comparison (matters for the HTTP webhook path where theX-Signatureheader is exposed publicly; SQS / SNS deliveries arrive over AWS-internal transports where timing-attack resistance is not strictly required)parse_event(payload) -> dict— JSON →dictModule-level composites (return
dict):verify_and_parse_webhook(body, signature, secret) -> dictverify_and_parse_sqs(body, signature, secret) -> dictverify_and_parse_sns(body, signature, secret) -> dictStreamChat/StreamChatAsyncinstances exposeverify_and_parse_webhook/verify_and_parse_sqs/verify_and_parse_snsthat use the configuredapi_secretautomatically.Backwards compatibility
StreamChat#verify_webhookis preserved for plain (uncompressed) bodies. The experimentaldecompress_webhook_bodyandverify_and_decode_webhooksurfaces are removed (they were never released).Tests
stream_chat/tests/test_webhook_compression.pycovers plain / gzip / base64 / base64+gzip payloads, signature mismatches, malformed bytes, JSON parsing intodict, and theStreamChat/StreamChatAsyncclient-bound paths. Linked Linear ticket: CHA-3071.Golden test fixtures (Tommaso)
Added shared reference fixtures to the test suite so future SDKs can sanity-check decoders against the same payloads:
Test plan
pytest stream_chat/tests/test_webhook_compression.py— 46 passedblack/isort— clean