Docker Proxy Sever

[PraveenAsh]

I liked LiteLLM, but it's a Python library, and I'm not a fan of its speed. Also, I don’t understand why it was written in Python.

A couple of things it doesn’t cover:

1. The library is great, but using their proxy server requires either extending their Docker or importing and writing code.
2. I agree with the Docker part, but the Req/Res transformers (not LLM transformers) need middleware, similar to authentication.
3. The core of the project, i.e., the proxy server, should be separated so that the user only needs to add the req/res transformations.
4. Middleware plugins: If users need usage tracking or freemium logic, it should be handled via middleware. Technically, even the req/res transformers are middleware.

[RPate97]

This issue is a bit unclear there are many proxy servers available right now including LiteLLM. Are you just looking for one implemented in TypeScript?