Skip to content

[Platform] Add tooling support for ollama to allow agent and toolbox usage #211

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

JoshuaBehrens
Copy link

Q A
Bug fix? no
New feature? yes?
Docs? no
License MIT

Running a toolbox agent with ollama was not yet supported. This ensures, that tool messages and responses work with a locally hosted ollama instance. It seems not to be a fix as it was not meant to work and it is not really a mentionable feature as one could expect it to work like the others so I am not really sure to work where to add more content for humans. I tested it with the given new example in examples/ollama/toolcall.php.

A current design decision had me struggling a bit. I was not yet able to solve in a good way the fact, that ollama itself cannot claim every model to have tool support. So someone who uses a model needs to know before its usage whether it has model support. This can be queried easily but the capabilities are part of the model, that you need to create before using the platform to query in the name of the model. So this is like hen-egg. If the model itself would not know its capabilities but there is a service, that has model and platform at hand, could evaluate it (and cache it). Is this something we want to establish?

Possible query:

curl http://localhost:11434/api/chat -d '{
  "model": "llama3:8b-instruct-q6_K",
  "tools": [{}],
  "stream": false
}'
{"error":"llama3:8b-instruct-q6_K does not support tools"}

@JoshuaBehrens JoshuaBehrens force-pushed the feature/ollama-tool-support branch from 18409bb to e89ead3 Compare July 26, 2025 18:22
@OskarStark OskarStark added the Platform Issues & PRs about the AI Platform component label Jul 26, 2025
@chr-hertel chr-hertel changed the title Add tooling support for ollama to allow agent and toolbox usage [Platform] Add tooling support for ollama to allow agent and toolbox usage Jul 27, 2025
Copy link
Contributor

@chr-hertel chr-hertel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I general this looks pretty clean and not like your first contribution here - thanks kicking this off - great work!

I made a few comments - can see where your thoughts are coming from - this is between those axis in Platform - working with two bridges.

And rolling out tool calling support to more bridges is def a new feature 👍

Copy link
Author

@JoshuaBehrens JoshuaBehrens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I general this looks pretty clean and not like your first contribution here - thanks kicking this off - great work!

ty :) likely not my last one. already looking into ollama embeddings

And rolling out tool calling support to more bridges is def a new feature 👍

anything more you need to have written out for a zero-ver? :D

@chr-hertel
Copy link
Contributor

Please rebase on main and adopt the changed test style, switching from #[Test] attribute to test method prefix, see #214.

@Kocal Kocal added Agent Issues & PRs about the AI Agent component and removed Agent Issues & PRs about the AI Agent component labels Jul 27, 2025
@JoshuaBehrens JoshuaBehrens force-pushed the feature/ollama-tool-support branch from e89ead3 to 22509a7 Compare July 27, 2025 19:31
@JoshuaBehrens
Copy link
Author

I rebased, changed the test and renamed the class. Anything more :) ?

@JoshuaBehrens JoshuaBehrens force-pushed the feature/ollama-tool-support branch from 22509a7 to 341184a Compare July 28, 2025 00:37
@JoshuaBehrens JoshuaBehrens force-pushed the feature/ollama-tool-support branch from 341184a to bbca652 Compare July 28, 2025 00:40
Copy link
Contributor

@chr-hertel chr-hertel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This! Looks! Great! :)

I believe your regex model work deserves some simple unit testing, but will give it a local test already

@chr-hertel
Copy link
Contributor

Tested with mistral, deepseek and llama - works 👍

@JoshuaBehrens
Copy link
Author

Great I will fix phpstan, rebase and we can get this merged. The new pull request for the model usage can follow after that :)

@chr-hertel
Copy link
Contributor

And that unit test for Symfony\AI\Platform\Bridge\Ollama\Ollama please :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Platform Issues & PRs about the AI Platform component Status: Needs Work
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants