[firestore-emulator: datastore-mode] reserve_sequential breaks with negative keys

[artoale]

[REQUIRED] Environment info

firebase-tools: Using the gcloud cli version 511.0.0, firestore emulator: 1.19.9
Platform: macOS (arm), python client library

[REQUIRED] Test case

import os

from google.cloud import datastore

def main():
    PORT = 10901
    os.environ["DATASTORE_EMULATOR_HOST"] = "127.0.0.1:%s" % PORT
    os.environ["DATASTORE_PROJECT_ID"] = "test"

    def get_new_client():
        return datastore.Client(
            project=os.environ.get("GCLOUDC_PROJECT_ID", "test"),
            namespace=None,
            _http=None,
        )

    client = get_new_client()
    kind = "a_kind"
    key = client.key(kind, -1)

    client.reserve_ids_sequential(key, 1)


if __name__ == "__main__":
    main()

[REQUIRED] Steps to reproduce

1. Start the emulator with gcloud emulators firestore start --host-port=127.0.0.1:10901 --database-mode=datastore-mode
2. Install the python deps: pip install google-cloud-datastore==2.19.0
3. Run the above python script

[REQUIRED] Expected behavior

Runs successfully, without errors

[REQUIRED] Actual behavior

Command fails with

Traceback (most recent call last):
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 65, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/grpc/_channel.py", line 1181, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNKNOWN
        details = ""
        debug_error_string = "UNKNOWN:Error received from peer  {created_time:"2025-03-07T15:46:47.764377+01:00", grpc_status:2, grpc_message:""}"
>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/artoale/dev/gcloudc/example.py", line 25, in <module>
    main()
  File "/Users/artoale/dev/gcloudc/example.py", line 21, in main
    client.reserve_ids_sequential(key, 1)
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/cloud/datastore/client.py", line 976, in reserve_ids_sequential
    self._datastore_api.reserve_ids(
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/cloud/datastore_v1/services/datastore/client.py", line 1352, in reserve_ids
    response = rpc(
               ^^^^
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 113, in __call__
    return wrapped_func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/api_core/retry.py", line 349, in retry_wrapped_func
    return retry_target(
           ^^^^^^^^^^^^^
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/api_core/retry.py", line 191, in retry_target
    return target()
           ^^^^^^^^
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/api_core/timeout.py", line 120, in func_with_timeout
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/artoale/.pyenv/versions/3.11.9/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 67, in error_remapped_callable
    raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.Unknown: None 

Emulator logs

[firestore] WARNING: Operation failed: Cannot invoke "com.google.cloud.datastore.core.rep.IdAllocationPolicy.equals(Object)" because "idPolicy" is null
[firestore] java.lang.NullPointerException: Cannot invoke "com.google.cloud.datastore.core.rep.IdAllocationPolicy.equals(Object)" because "idPolicy" is null
[firestore]     at com.google.cloud.datastore.emulator.impl.firestore.FirestoreEmulatorHelper.filterIdsByScatteredPolicy(FirestoreEmulatorHelper.java:471)
[firestore]     at com.google.cloud.datastore.emulator.impl.firestore.FirestoreEmulatorHelper.reserveIds(FirestoreEmulatorHelper.java:454)
[firestore]     at com.google.cloud.datastore.emulator.impl.firestore.cloudv1.CloudDatastoreV1.reserveIds(CloudDatastoreV1.java:560)
[firestore]     at com.google.cloud.datastore.emulator.impl.firestore.cloudv1.CloudDatastoreV1Router.reserveIds(CloudDatastoreV1Router.java:132)
[firestore]     at com.google.cloud.datastore.emulator.firestore.cloudv1.CloudDatastoreV1GrpcAdapter$1.lambda$reserveIds$6(CloudDatastoreV1GrpcAdapter.java:98)
[firestore]     at com.google.cloud.datastore.emulator.firestore.cloudv1.CloudDatastoreV1GrpcAdapter.unary(CloudDatastoreV1GrpcAdapter.java:49)
[firestore]     at com.google.cloud.datastore.emulator.firestore.cloudv1.CloudDatastoreV1GrpcAdapter$1.reserveIds(CloudDatastoreV1GrpcAdapter.java:98)
[firestore]     at com.google.datastore.v1.DatastoreGrpc$MethodHandlers.invoke(DatastoreGrpc.java:719)
[firestore]     at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:182)
[firestore]     at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:356)
[firestore]     at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:861)
[firestore]     at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
[firestore]     at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
[firestore]     at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
[firestore]     at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
[firestore]     at java.base/java.lang.Thread.run(Thread.java:1575)
[firestore] 

I've tested the same script against a real firestore in datastore mode entity and it works

[aalej]

Hey @artoale, thanks for raising this and for filing a detailed report. I was able to reproduce the error, let me raise this to our engineering team.

Sharing the mcve here.