Custom OpenAI Base URL Support

[Tycoon-HJ]

I am in China and I am using a VPN. I can access chatgpt normally on my computer, but there is an error

[Tycoon-HJ]

please Add a custom API key with baseURL to facilitate my use of transit services😀

[RamiAwar]

Ouu sorry about that, yeah we're actually working on that now!

As a first step to supporting multiple models, we're going to refactor our backend to support an "LLM Provider" abstraction. Then we can make it configurable from the frontend. @anthony2261 is hard at work on that 😄

It might take a few weeks, so for the time being I can help you modify the source code if you need it 🙂 Let me know!

[RamiAwar]

Is the request to the openai api blocked in China? Is that what's causing the timeout for sure?

[Tycoon-HJ]

Is the request to the openai api blocked in China? Is that what's causing the timeout for sure?

yes

[Tycoon-HJ]

Ouu sorry about that, yeah we're actually working on that now!

As a first step to supporting multiple models, we're going to refactor our backend to support an "LLM Provider" abstraction. Then we can make it configurable from the frontend. @anthony2261 is hard at work on that 😄

It might take a few weeks, so for the time being I can help you modify the source code if you need it 🙂 Let me know!

Thank you, let me wait a little longer. I will experience dataline again when it is supported

[RamiAwar]

Will post an update here when we release this detailing how you can fix this!

Also sorry to hear about the block, it's really shitty that they did that. We'll be supporting more providers + local models soon after we do this LLM Provider refactor!

[JERRRR0318]

curious, is it solved yet? thank you

[RamiAwar]

not yet! in progress.

do you think we should build a temporary workaround maybe? if it's just an OpenAI custom URL should be easy to implement quickly @anthony2261

[JERRRR0318]

thanks for replying, yeah our forwarding api can't make direct requests to the official interface, so have to change the baseURL address configuration

[RamiAwar]

@anthony2261 wdyt of creating a hotfix for this before we do a full custom LLM provider implementation? maybe a settings input for openai base URL? ive seen this request several times now, it would unblock a lot of interested users. low effort high value imo

[anthony2261]

The PR for this is in review! Really close to solving this 😄 #287

[JERRRR0318]

That would be great😄! Thanks a lot, looking forward to the good news

[anthony2261]

We've just merged branch #287 which adds the ability to set a base_url from the app! You can find the new setting once the new release is out 🚀