[BUG] AzureOpenAIServerModel Sends Unsupported 'stop' Parameter for o1-mini

[karmatarap]

Describe the bug
When using the AzureOpenAIServerModel with the o1-mini deployment, the request being sent includes a "stop" parameter, which is not supported by the model. This results in a 400 error with the message:
“Unsupported parameter: 'stop' is not supported with this model.”

Code to reproduce the error

import os
from dotenv import load_dotenv
from smolagents import CodeAgent
from smolagents.models import AzureOpenAIServerModel

load_dotenv(override=True)

if __name__ == "__main__":
    model = AzureOpenAIServerModel(
        model_id="o1-mini",
        api_key=os.environ.get("AZURE_OPENAI_API_KEY"),
        api_version=os.environ.get("AZURE_OPENAI_API_VERSION"),
        azure_endpoint=os.environ.get("AZURE_OPENAI_API_BASE"),
        custom_role_conversions={"system": "assistant", "tool-call": "assistant", "tool-response": "user"}
    )
    agent = CodeAgent(tools=[], model=model, add_base_tools=True)
    agent.run("Could you give me the 118th number in the Fibonacci sequence?")

Error logs (if any)

Error in generating model output:
Error code: 400 - {'error': {'message': "Unsupported parameter: 'stop' is not supported with this model.", 'type': 'invalid_request_error', 'param': 'stop', 'code': 'unsupported_parameter'}}

Expected behavior
The request to Azure OpenAI should succeed without including a "stop" parameter when using the o1-mini model. The API call should complete successfully and return a valid response without triggering a 400 error.

Packages version:
smolagents==1.8.0

Additional context
The error appears to be caused by the internal method _prepare_completion_kwargs in the class AzureOpenAIServerModel class, which adds a "stop" parameter to the API request. A possible workaround is to override this method to remove the "stop" parameter if present. For example:

class PatchedAzureOpenAIServerModel(AzureOpenAIServerModel):
    def _prepare_completion_kwargs(self, *args, **kwargs):
        completion_kwargs = super()._prepare_completion_kwargs(*args, **kwargs)
        if 'stop' in completion_kwargs:
            del completion_kwargs['stop']
        return completion_kwargs

[imSanko]

We can implement a workaround by subclassing the AzureOpenAIServerModel and overriding the _prepare_completion_kwargs method to remove the stop parameter before sending the request.

Solution Code

import os
from dotenv import load_dotenv
from smolagents import CodeAgent
from smolagents.models import AzureOpenAIServerModel

load_dotenv(override=True)

class PatchedAzureOpenAIServerModel(AzureOpenAIServerModel):
    def _prepare_completion_kwargs(self, *args, **kwargs):
        completion_kwargs = super()._prepare_completion_kwargs(*args, **kwargs)
        
        # Remove the 'stop' parameter if it exists
        if 'stop' in completion_kwargs:
            del completion_kwargs['stop']
        
        return completion_kwargs

if __name__ == "__main__":
    model = PatchedAzureOpenAIServerModel(
        model_id="o1-mini",
        api_key=os.environ.get("AZURE_OPENAI_API_KEY"),
        api_version=os.environ.get("AZURE_OPENAI_API_VERSION"),
        azure_endpoint=os.environ.get("AZURE_OPENAI_API_BASE"),
        custom_role_conversions={"system": "assistant", "tool-call": "assistant", "tool-response": "user"}
    )
    agent = CodeAgent(tools=[], model=model, add_base_tools=True)
    agent.run("Could you give me the 118th number in the Fibonacci sequence?")

What are we doing?

1. Subclassing AzureOpenAIServerModel:

   • We create a new class PatchedAzureOpenAIServerModel that inherits from AzureOpenAIServerModel.
   • This allows us to override the _prepare_completion_kwargs method to modify the request parameters before they are sent to the API.

2. Overriding _prepare_completion_kwargs:

   • The overridden method first calls the parent class's _prepare_completion_kwargs method to get the default completion parameters.
   • It then checks if the stop parameter is present in the completion_kwargs dictionary. If it is, the parameter is removed using del.

3. Using the Patched Model:

   • In the if __name__ == "__main__": block, we instantiate the PatchedAzureOpenAIServerModel instead of the original AzureOpenAIServerModel.
   • This ensures that the stop parameter is not included in the API request, preventing the 400 error.

[vladiliescu]

Fyi, _prepare_completion_kwargs is defined in Model, and used explicitly by OpenAIServerModel on which AzureOpenAIServerModel depends. So this issue should reproduce with basic OpenAI as well.

Looking at the code, it seems like the model's __call__ method is invoked by the agent with stop_sequences != None, causing this error. I haven't had the time to understand why/when that happens, but a proper fix should take that into account.