Empty responses when running Qwen coder LLMs 0.5B and 1.5B

[pacama95]

Hi!

I am new to agents and I am playing with smolagents now. I really like it, it is easy to use and allow fast prototyping, but I am facing an issue now.

I am trying to run the LLM locally instead of using the HF inference API so I decided to use smaller LLMs which my computer can run (Qwen/Qwen2.5-Coder-0.5B-Instruct and Qwen/Qwen2.5-Coder-1.5B-Instruct). I managed to run them locally with transformers library getting an acceptable token per seconds rate, but when I try to load one of this models in a CodeAgent it takes much more time to start and the steps output are empty.

I am not sure if I am doing something wrong, or maybe it is not possible to use such small LLM with smolagents...

Here you can find a piece code of what I am doing:

from smolagents import CodeAgent, DuckDuckGoSearchTool, VisitWebpageTool, FinalAnswerTool, HfApiModel, TransformersModel

model = TransformersModel(model_id="Qwen/Qwen2.5-Coder-0.5B-Instruct", device_map='cuda')

documentation_analyst_agent = CodeAgent(
    model=model,
    tools=[
        DuckDuckGoSearchTool(),
        VisitWebpageTool(),
        FinalAnswerTool()
    ],
     additional_authorized_imports=[
        "requests",
        "json",
        "pandas",
        "numpy",
    ],
    planning_interval=3,
    name="documentation_analyst",
    description="Analyze the documentation from the provided source and creates a technical summary of it",
    verbosity_level=2,
    max_steps=5
)

# My actual prompt differs from this one, I am omitting it since it is quite longer, but the error is reproducible using this one
prompt = f"""You have to create a summary about how to use the API hosted in 'https:/an-api-hosted-somewhere.com'.
Extract the relevant data about how to use the API from this page {documentation_source}""" 

result = documentation_analyst_agent.run(prompt)

The result of that run is the following (all the steps are empty and the output is blank):

I would appreciate any feedback so we can try to find out the problem.