-
Notifications
You must be signed in to change notification settings - Fork 789
Description
Currently, a client to interact with the Ollama API can be created using
from ollama import Client
client = Client(host='http://localhost:11434', ...)
However, there is no way to close the client and give back the resources it acquired. Eventually, Python's garbage collector will, but it would be nice to be able to do that as early as possible.
From my understanding, adding the following methods to ollama._client.Client
would address this:
class Client(BaseClient):
...
def close(self):
self._client.close()
def __enter__(self):
return self
def __exit__(self, exc_type, exc_value, traceback):
self.close()
return False
And then something similar for the AsyncClient
.
That way, the client could be used like:
with Client(host='http://localhost:11434', ...) as client:
client.chat(...)
If you want, I can look into this and prepare a PR.
Update: This should also fix warnings when using ollama-python with pytest:
.../.venv/lib/python3.12/site-packages/_pytest/unraisableexception.py:31: ResourceWarning: unclosed <socket.socket fd=15, family=2, type=1, proto=6, laddr=('127.0.0.1', 64484), raddr=('127.0.0.1', 11434)>
gc.collect()