-
Notifications
You must be signed in to change notification settings - Fork 341
Description
Environment
- Operating System version: Reproducible in the Google Cloud Functions runtime environment (Linux-based). Not OS-dependent.
- Firebase SDK version: 6.9.0
- Firebase Product: Functions (
task_queue
) - Python version: 3.13
- Pip version: 23.2.1
The firebase_admin.functions.task_queue.enqueue()
method consistently fails on its first invocation within a "cold start" of a Python Cloud Function. The call raises a firebase_admin.exceptions.InvalidArgumentError
, which wraps an underlying 400 Client Error: Bad Request
from the Google Cloud Tasks API.
An immediate second call to enqueue()
with the exact same arguments and client instance succeeds. This behavior occurs regardless of whether the TaskQueue
object is pre-initialized globally (e.g., in an @init
function) or initialized directly within the function handler. This points to a race condition or a lazy-initialization issue with the authentication credentials used by the SDK's HTTP client, rather than an issue with the user's code.
This behavior forces developers to implement non-obvious workarounds, like a hardcoded try/except/retry block, to handle what appears to be a transient initialization problem within the SDK itself.
Steps to reproduce:
- Deploy the following Python 3.13 Cloud Function:
- Invoke the
trigger_task_enqueue
function on a new (cold) instance. - Observe the logs, which will show the first attempt failing and the immediate retry succeeding.
Relevant Code:
import os
import time
import firebase_admin
from firebase_admin.functions import task_queue
from firebase_functions import https_fn, options
from firebase_admin import exceptions
# Standard initialization
firebase_admin.initialize_app()
# The name of the function that will process the task
# Replace with your project details and target function name
TASK_HANDLER_FUNCTION_NAME = "process_task_handler"
QUEUE_NAME = f"locations/europe-west3/functions/{TASK_HANDLER_FUNCTION_NAME}"
@https_fn.on_call(region="europe-west3")
def trigger_task_enqueue(req: https_fn.CallableRequest) -> dict:
"""A simple function to trigger the enqueue process."""
# The bug occurs even when the queue is initialized directly in the function.
queue_client = task_queue(QUEUE_NAME)
task_payload = {"message": "hello world", "timestamp": time.time()}
# --- The Bug in Action ---
try:
# First attempt: This is expected to FAIL on a cold start
print("Attempt 1: Enqueuing task...")
queue_client.enqueue({'data': task_payload})
print("Success on first attempt (warm instance).")
return {"status": "SUCCESS_FIRST_TRY"}
except exceptions.InvalidArgumentError as e:
# Expected failure on cold start, immediately retry
print(f"Attempt 1 failed as expected with {type(e).__name__}. Retrying...")
try:
# Second attempt: This is expected to SUCCEED
queue_client.enqueue({'data': task_payload})
print("Success on second attempt.")
return {"status": "SUCCESS_SECOND_TRY"}
except Exception as e_retry:
# If this fails, it's a real, unexpected error
print(f"Unexpected error on second attempt: {e_retry}")
raise https_fn.HttpsError(https_fn.FunctionsErrorCode.INTERNAL, "Failed on retry.")
except Exception as e_initial:
print(f"The first attempt failed with an unexpected error: {e_initial}")
raise https_fn.HttpsError(https_fn.FunctionsErrorCode.INTERNAL, "Failed on initial attempt.")
@https_fn.on_task_dispatched(region="europe-west3")
def process_task_handler(req: https_fn.CallableRequest) -> None:
"""A dummy task handler to be the target of the queue."""
print(f"Task received with data: {req.data}")
Expected behavior
The first call to queue_client.enqueue()
should succeed without raising an InvalidArgumentError
. The SDK should handle its own internal authentication initialization seamlessly, retrying on initial auth-related HTTP errors before surfacing an exception to the user.
Actual behavior
The first call fails, and the immediate second call succeeds.
Logs from the first, failed attempt:
Attempt 1: Enqueuing task...
Attempt 1 failed as expected with InvalidArgumentError. Retrying...
(Plus the detailed InvalidArgumentError
and 400 Bad Request
stack traces, as previously documented)
Logs from the second, successful attempt:
Success on second attempt.
Analysis & Hypothesis
The root cause appears to be the lazy-loading of authentication tokens by the google.auth.transport.requests.AuthorizedSession
used within the SDK's _http_client.py
.
- On a cold start, the
AuthorizedSession
is initialized with credentials but does not yet have a fresh OAuth2 access token for thehttps://cloudtasks.googleapis.com/
scope. - The first
enqueue()
call attempts an API request with a missing or stale token. - The Cloud Tasks API correctly rejects this with a
400 Bad Request
. - The
firebase-admin
SDK catches this HTTP error but, instead of interpreting it as a trigger for an auth refresh and retrying, it immediately wraps it in a genericInvalidArgumentError
and raises it to the user. - The process of the failed request likely triggers the token refresh in the background.
- The immediate second
enqueue()
call uses the same client instance, which now possesses a valid token, so the request succeeds.
This behavior could be fixed by having the SDK's enqueue
method retry internally on HTTP 400/401/403 errors, which are strong indicators of a transient authentication issue, especially on the first request of a new client instance.