azure-ai-inference Sample Issue (load_client method failed with (404) "Resource not found")

[jerryshia]

Section link1, link2, link3, link4,link5:

Reason:
ResourceNotFoundError: (404) Resource not found

@rohit-ganguly , @lmazuel , @achandmsft , @mayurid , @dargilco for notification.

[dargilco]

@jerryshia Not all endpoints support the /info route that the get_model_info() method uses. Which endpoint are you using? Free tier GitHub models endpoint? A model deployed via AI Foundry? Serverless API? Managed endpoint? Can you share the endpoint URL here? or IM me on Teams so we can share more info, including SDK logs as needed.

[mtcogco]

Hi @dargilco, not the OP here but having similar issues. I guessed that we might not have support to run get_model_info() but I similarly get a 404 when trying to use client.complete() as instructed on the docs here: https://learn.microsoft.com/en-us/azure/ai-studio/ai-services/concepts/endpoints?tabs=python

More specifically, I have different Meta Llama models deployed to an 'Azure AI Services' endpoint, and I specify the model name using the arg model when calling client.complete(). For example, I have tried to test with Llama-3.3-70B-Instruct as well as Meta-Llama-3.1-405B-Instruct. My endpoints page looks like this:

I've double checked for silly mistakes I can think of, e.g. incorrect endpoints (I have tried both with and without the /models appended to the end of the Azure AI model inference endpoint.

The docs also indicate that my API key should be 32 characters, but I can confirm that the API key provided by the Azure AI model inference endpoint is longer than this.

One additional note: If I send the request with the \models at the end of the URI, then I do not get a 404 but instead get a ClientAuthenticationError, which may be a separate issue that I need to post, but the call looks a little something like this:

MODEL_NAME = 'Llama-3.3-70B-Instruct'
client = ChatCompletionsClient(
    endpoint = ENDPOINT,
    credential = AzureKeyCredential(API_KEY)
)

response = client.complete(
    messages=[
        SystemMessage(content="You are a helpful assistant."),
        UserMessage(content="Explain Riemann's conjecture in 1 paragraph"),
    ],
    model = "Meta-Llama-3.1-405B-Instruct"
)

Which returns:

ClientAuthenticationError: (None) Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.
Code: None
Message: Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.

Appreciate that this might just be a daft user error, but I can't think of anything else to check and can't find any other docs or discussions about this system. Hopefully this is also helpful to OP, since it seems like our issues are relevant?

Happy to reach out on teams (I don't know your address?) but thought I'd post this anyway in the interest of sharing with others.

[mtcogco]

Additional info:

I can also confirm the following versions for these azure packages are installed in my venv:

Package	Version
azure-ai-inference	1.0.0b7
azure-ai-ml	1.24.0
azure-common	1.1.28
azure-core	1.32.0
azure-core-tracing-opentelemetry	1.0.0b11
azure-identity	1.19.0
azure-mgmt-core	1.5.0
azure-monitor-opentelemetry	1.6.4
azure-monitor-opentelemetry-exporter	1.0.0b33
azure-storage-blob	12.24.0
azure-storage-file-datalake	12.18.0
azure-storage-file-share	12.20.0

[dargilco]

@mtcogco please open a new GitHub issue for your issue, and I will try to help you there. It's unrelated to what @jerryshia wrote.

[dargilco]

I reached out directly to @jerryshia over IM, waiting for a response.

[dargilco]

@jerryshia closing since I have not heard back from you. Please reactivate if necessary.