feat(firestore): add BSON read deserialization support by ohmayr · Pull Request #18402 · googleapis/google-cloud-python

ohmayr · 2026-09-16T20:30:29Z

Adds opt-in BSON read deserialization support to the Google Cloud Firestore Python SDK.
When enabled via decode_bson=True, document fields containing BSON wire map structures returned by Firestore (such as {"__oid__": "507f191e810c19729de860ea"}) are automatically deserialized into their corresponding Python BSON container instances (BSONObjectId, BSONDecimal128, BSONTimestamp, BSONRegex, BSONBinary, BSONInt32, BSONMinKey, BSONMaxKey).

💻 Usage

Default Behavior (`decode_bson=False`)

Existing applications continue to receive raw map dictionaries by default to preserve 100% backward compatibility:

client = firestore.Client()
doc = client.collection("users").document("doc1").get()

Opt-in Behavior (`decode_bson=True`)

client = firestore.Client(decode_bson=True)
doc = client.collection("users").document("doc1").get()
# Returns deserialized BSON instance: {"user_id": BSONObjectId("507f191e810c19729de860ea")}
data = doc.to_dict()

🏛️ Design Decisions

Opt-In decode_bson=False Default (Enterprise Backward Safety): Defaulting to decode_bson=False ensures existing production code accessing raw dictionary keys (dict["user_id"]["oid"]) will not break upon upgrading the SDK.
Subtype 0 Binary Deserialization: Wire maps representing Subtype 0 BSON Binary (v[0] == 0) are deserialized into native Python bytes (b"..."), while non-zero subtypes ($1 \le v[0] \le 255$) deserialize into BSONBinary(data, subtype=v[0]) objects.
Explicit Non-None Fallback Control: Updated decode_dict() to explicitly check if decoded is not None: rather than relying on Python truthiness (or), preventing false fallback on empty byte payloads (b"") or falsy objects.
Recursive Nested Map & Array Support: Added _decode_bson_dict_recursive() to ensure BSON wire maps inside nested dictionaries and array elements are deserialized properly.

Fixes b/562164140 🦕

gemini-code-assist

Code Review

This pull request introduces several new BSON types (BSONInt32, BSONBinary, BSONTimestamp, BSONRegex, and BSONDecimal128) to the Firestore Python client, along with their respective decoders, integration tests, and unit tests. Feedback on these changes highlights a violation of Python's hash contract in BSONDecimal128 due to mixed-type equality with decimal.Decimal without matching hashes. Additionally, the reviewer recommended replacing the boolean or fallback logic in decode_dict with an explicit None check to prevent potential bugs with falsy decoded BSON objects.

daniel-sanche · 2026-09-18T20:38:05Z

-) -> Union[
-    None, bool, int, float, list, datetime.datetime, str, bytes, dict, GeoPoint, Vector
-]:
+def decode_value(value, client=None, decode_bson: Optional[bool] = None) -> Any:


Can we do this without changing this to Any? Can't we just add BSONType to the output list?

This would remove a lot of the value of the type annotations

daniel-sanche · 2026-09-18T20:39:14Z

-    Raises:
-        NotImplementedError: If the ``value_type`` is ``reference_value``.
-        ValueError: If the ``value_type`` is unknown.
+        Any: A native Python value converted from the ``value``.


Why did you drop the Raises section?

daniel-sanche · 2026-09-18T20:41:38Z

+def decode_dict(
+    value_fields,
+    client=None,
+    decode_bson: Optional[bool] = None,


Why does this need 3 states? Can't it just be a bool that defaults to False?

daniel-sanche · 2026-09-18T20:49:43Z

            Firestore protobuf to be decoded / parsed / converted.
        client (:class:`~google.cloud.firestore_v1.client.Client`):
            A client that has a document factory.
+        decode_bson (Optional[bool]): Whether to decode BSON extended types.


Can we have this opted in by default? It looks like other languages decode to BSON by default. And we do similar decoding to custom objects for GeoPoint, Vector, etc. Making this opt-in would really lower the value of for the feature

BSON is a new feature, so we wouldn't expect workarounds in existing code. But let me know if you have any specific breaking change concerns, and maybe we can find solutions

daniel-sanche · 2026-09-18T20:51:44Z

+    """Decode a single-key wire map dictionary if registered."""
+    if len(data) == 1:
+        key, val = next(iter(data.items()))
+        decoder = _BSON_DECODERS.get(key)


This seems like it should be implemented as a class method:

BSONType._from_dict(data)

or

BSONType._class_for_key(key)(value)

…ation - Perform automatic BSON deserialization in decode_dict and DocumentSnapshot.to_dict using _BSONType._from_dict. - Remove decode_bson configuration parameter across Client, AsyncClient, BaseClient, and DocumentSnapshot. - Preserve precise return type annotations in decode_dict and restore docstring Raises section. Towards #18402

…ecode_value - Restore full Union return type with _BSONType on decode_value. - Restore Returns and Raises docstring sections in decode_value matching base branch. - Remove unused _BSON_DECODERS import from _helpers.py. - Revert extraneous changes to pipeline_result.py. Towards #18402

… types with _BSONType - Annotate decode_dict with Union[dict, Vector, _BSONType]. - Update PipelineResult.data to return dict | Vector | _BSONType | None. - Import _BSONType under TYPE_CHECKING in pipeline_result.py. Towards #18402

…nsions for librarian - Make client a required positional parameter in decode_value and decode_dict. - Format comprehensions in _helpers.py as single lines to satisfy librarian generation check. Towards #18402

daniel-sanche

Looking better, but a few more comments

daniel-sanche · 2026-09-21T23:21:18Z

+        """Deserializes a BSON wire map dictionary into a BSON instance or bytes.
+
+        Args:
+            data (Any): Potential BSON wire map dictionary.


Can this really return Any type? I would assume BSONType | bytes | None

(Try to avoid using Any wherever possible)

daniel-sanche · 2026-09-21T23:23:49Z


-def decode_dict(value_fields, client) -> Union[dict, Vector]:
+def _decode_bson_dict_recursive(data: Any) -> Any:
+    """Recursively decodes BSON wire map dictionaries."""


IIUC, This method shouldn't be necessary. decode_dict is already recursive, and should hanle BSON on its own. But let me know if I'm missing something

daniel-sanche · 2026-09-21T23:26:26Z

            return None
-        return copy.deepcopy(self._data)
+        data = copy.deepcopy(self._data)
+        return _helpers._decode_bson_dict_recursive(data)


This shouldn't need to change, self._data should already be in a good format (i.e., it would have run through _decode_dict before being saved to _data)

daniel-sanche · 2026-09-21T23:31:09Z

+        decoder = _BSON_DECODERS.get(key)
+        if decoder is None:
+            return None
+        return decoder(val)


should we catch exceptions here, so we don't crash when reading data? Maybe fall back to None?

daniel-sanche · 2026-09-21T23:32:10Z

+    "__regex__": lambda v: BSONRegex(v["pattern"], v.get("options", ""))
+    if isinstance(v, dict) and "pattern" in v
+    else None,
+}


nit: If you wanted to be fancy, there's probably a way we could make these keys part of each class, and build this mapping dynamically. But I think keeping it static works fine too

daniel-sanche · 2026-09-21T23:37:28Z

+def decode_dict(
+    value_fields,
+    client,
+) -> Union[dict, Vector, _BSONType]:


daniel-sanche · 2026-09-21T23:40:15Z

-            str, bytes, dict, ~google.cloud.Firestore.GeoPoint]: A native
+            str, bytes, dict, ~google.cloud.Firestore.GeoPoint, \
+            ~google.cloud.firestore_v1.vector.Vector, \
+            ~google.cloud.firestore_v1.bson._BSONType]: A native \


IF _BSONType is being exposed in this way, it should actually be public

(Sorry, I think I suggested making it private at first. I didn't see the whole picture at the time, and making things private is my default)

ohmayr added this pull request to stack #18386 September 16, 2026 20:30

ohmayr force-pushed the bson-pr2-reads branch from 6518eca to 29e3585 Compare September 16, 2026 20:32

gemini-code-assist Bot reviewed Sep 16, 2026

View reviewed changes

Comment thread packages/google-cloud-firestore/google/cloud/firestore_v1/bson.py Outdated

Comment thread packages/google-cloud-firestore/google/cloud/firestore_v1/_helpers.py Outdated

ohmayr force-pushed the bson-pr2-reads branch from 29e3585 to b271751 Compare September 16, 2026 20:34

ohmayr force-pushed the bson-pr2-reads branch from b271751 to e598276 Compare September 16, 2026 20:44

ohmayr force-pushed the bson-pr2-reads branch from e598276 to be4c859 Compare September 16, 2026 20:58

ohmayr force-pushed the bson-pr2-reads branch 3 times, most recently from 6a402d4 to 2ba70f6 Compare September 16, 2026 21:24

ohmayr marked this pull request as ready for review September 16, 2026 21:24

ohmayr requested a review from a team as a code owner September 16, 2026 21:24

ohmayr force-pushed the bson-pr2-reads branch from 2ba70f6 to 3630dd6 Compare September 16, 2026 21:50

ohmayr force-pushed the bson-pr2-reads branch 3 times, most recently from 0ac138b to 38f629a Compare September 16, 2026 22:08

ohmayr force-pushed the bson-pr2-reads branch 2 times, most recently from 88f98a2 to 4cdbf85 Compare September 16, 2026 23:01

ohmayr force-pushed the bson-pr2-reads branch from 4cdbf85 to 5ec6976 Compare September 18, 2026 20:06

daniel-sanche reviewed Sep 18, 2026

View reviewed changes

ohmayr force-pushed the bson-pr2-reads branch from 5ec6976 to c8310e9 Compare September 18, 2026 22:11

ohmayr force-pushed the bson-pr2-reads branch from c8310e9 to 8fcabfc Compare September 18, 2026 22:52

ohmayr force-pushed the bson-pr2-reads branch from 8fcabfc to 6291c61 Compare September 19, 2026 12:05

ohmayr requested a review from a team as a code owner September 21, 2026 08:26

ohmayr added 2 commits September 21, 2026 08:42

feat(firestore): add BSON read deserialization support

eb43811

ohmayr force-pushed the bson-pr2-reads branch from 6291c61 to 9d5e3ea Compare September 21, 2026 09:06

ohmayr added 3 commits September 21, 2026 09:17

fix(firestore): restore required client parameter and format comprehe…

7caf76e

…nsions for librarian - Make client a required positional parameter in decode_value and decode_dict. - Format comprehensions in _helpers.py as single lines to satisfy librarian generation check. Towards #18402

daniel-sanche requested changes Sep 21, 2026

View reviewed changes

Conversation

ohmayr commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💻 Usage

Default Behavior (decode_bson=False)

Opt-in Behavior (decode_bson=True)

🏛️ Design Decisions

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniel-sanche Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniel-sanche left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ohmayr commented Sep 16, 2026 •

edited

Loading

Default Behavior (`decode_bson=False`)

Opt-in Behavior (`decode_bson=True`)

daniel-sanche Sep 18, 2026 •

edited

Loading