`task_queue.enqueue` fails on first call in Cloud Functions due to auth race condition

[jamanz]

Environment

• Operating System version: Reproducible in the Google Cloud Functions runtime environment (Linux-based). Not OS-dependent.
• Firebase SDK version: 6.9.0
• Firebase Product: Functions (task_queue)
• Python version: 3.13
• Pip version: 23.2.1

The firebase_admin.functions.task_queue.enqueue() method consistently fails on its first invocation within a "cold start" of a Python Cloud Function. The call raises a firebase_admin.exceptions.InvalidArgumentError, which wraps an underlying 400 Client Error: Bad Request from the Google Cloud Tasks API.

An immediate second call to enqueue() with the exact same arguments and client instance succeeds. This behavior occurs regardless of whether the TaskQueue object is pre-initialized globally (e.g., in an @init function) or initialized directly within the function handler. This points to a race condition or a lazy-initialization issue with the authentication credentials used by the SDK's HTTP client, rather than an issue with the user's code.

This behavior forces developers to implement non-obvious workarounds, like a hardcoded try/except/retry block, to handle what appears to be a transient initialization problem within the SDK itself.

Steps to reproduce:

1. Deploy the following Python 3.13 Cloud Function:
2. Invoke the trigger_task_enqueue function on a new (cold) instance.
3. Observe the logs, which will show the first attempt failing and the immediate retry succeeding.

Relevant Code:

import os
import time
import firebase_admin
from firebase_admin.functions import task_queue
from firebase_functions import https_fn, options
from firebase_admin import exceptions

# Standard initialization
firebase_admin.initialize_app()

# The name of the function that will process the task
# Replace with your project details and target function name
TASK_HANDLER_FUNCTION_NAME = "process_task_handler"
QUEUE_NAME = f"locations/europe-west3/functions/{TASK_HANDLER_FUNCTION_NAME}"

@https_fn.on_call(region="europe-west3")
def trigger_task_enqueue(req: https_fn.CallableRequest) -> dict:
    """A simple function to trigger the enqueue process."""
    
    # The bug occurs even when the queue is initialized directly in the function.
    queue_client = task_queue(QUEUE_NAME)
    
    task_payload = {"message": "hello world", "timestamp": time.time()}
    
    # --- The Bug in Action ---
    try:
        # First attempt: This is expected to FAIL on a cold start
        print("Attempt 1: Enqueuing task...")
        queue_client.enqueue({'data': task_payload})
        print("Success on first attempt (warm instance).")
        return {"status": "SUCCESS_FIRST_TRY"}
    except exceptions.InvalidArgumentError as e:
        # Expected failure on cold start, immediately retry
        print(f"Attempt 1 failed as expected with {type(e).__name__}. Retrying...")
        try:
            # Second attempt: This is expected to SUCCEED
            queue_client.enqueue({'data': task_payload})
            print("Success on second attempt.")
            return {"status": "SUCCESS_SECOND_TRY"}
        except Exception as e_retry:
            # If this fails, it's a real, unexpected error
            print(f"Unexpected error on second attempt: {e_retry}")
            raise https_fn.HttpsError(https_fn.FunctionsErrorCode.INTERNAL, "Failed on retry.")
    except Exception as e_initial:
        print(f"The first attempt failed with an unexpected error: {e_initial}")
        raise https_fn.HttpsError(https_fn.FunctionsErrorCode.INTERNAL, "Failed on initial attempt.")


@https_fn.on_task_dispatched(region="europe-west3")
def process_task_handler(req: https_fn.CallableRequest) -> None:
    """A dummy task handler to be the target of the queue."""
    print(f"Task received with data: {req.data}")

Expected behavior

The first call to queue_client.enqueue() should succeed without raising an InvalidArgumentError. The SDK should handle its own internal authentication initialization seamlessly, retrying on initial auth-related HTTP errors before surfacing an exception to the user.

Actual behavior

The first call fails, and the immediate second call succeeds.

Logs from the first, failed attempt:

Attempt 1: Enqueuing task...
Attempt 1 failed as expected with InvalidArgumentError. Retrying...

(Plus the detailed InvalidArgumentError and 400 Bad Request stack traces, as previously documented)

Logs from the second, successful attempt:

Success on second attempt.

Analysis & Hypothesis

The root cause appears to be the lazy-loading of authentication tokens by the google.auth.transport.requests.AuthorizedSession used within the SDK's _http_client.py.

1. On a cold start, the AuthorizedSession is initialized with credentials but does not yet have a fresh OAuth2 access token for the https://cloudtasks.googleapis.com/ scope.
2. The first enqueue() call attempts an API request with a missing or stale token.
3. The Cloud Tasks API correctly rejects this with a 400 Bad Request.
4. The firebase-admin SDK catches this HTTP error but, instead of interpreting it as a trigger for an auth refresh and retrying, it immediately wraps it in a generic InvalidArgumentError and raises it to the user.
5. The process of the failed request likely triggers the token refresh in the background.
6. The immediate second enqueue() call uses the same client instance, which now possesses a valid token, so the request succeeds.

This behavior could be fixed by having the SDK's enqueue method retry internally on HTTP 400/401/403 errors, which are strong indicators of a transient authentication issue, especially on the first request of a new client instance.

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.