Skip to content

Commit fede61d

Browse files
authored
chore: add page documenting use of alternative model providers (#193)
Signed-off-by: Taylor Price <[email protected]>
1 parent 5c6ff70 commit fede61d

File tree

3 files changed

+56
-0
lines changed

3 files changed

+56
-0
lines changed
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Alternative Model Providers
2+
3+
4+
## Usage
5+
6+
GPTScript can be used against alternative models that expose an OpenAI compatible API or have a provider shim available.
7+
8+
To use a model with an OpenAI compatible API:
9+
10+
```gptscript
11+
model: mistral-large-latest from https://api.mistral.ai/v1
12+
13+
Say hello world
14+
```
15+
16+
To use a model that requires a provider shim:
17+
18+
```gptscript
19+
model: claude-3-haiku-20240307 from github.com/gptscript-ai/anthropic-provider
20+
21+
Say hello world
22+
```
23+
24+
### Authentication
25+
26+
For OpenAI compatible providers, GPTScript will look for an API key to be configured with the prefix `GPTSCRIPT_PROVIDER_`, the base domain converted to environment variable format, and a suffix of `_API_KEY`.
27+
As an example if you are using `mistral-large-latest from https://api.mistral.ai/v1`, the environment variable would be `GPTSCRIPT_PROVIDER_API_MISTRAL_AI_API_KEY`
28+
29+
Each provider shim has different requirements for authentication. Please check the readme for the provider you are trying to use.
30+
31+
## Available Model Providers
32+
33+
The following shims are currently available:
34+
* [github.com/gptscript-ai/azure-provider](https://github.com/gptscript-ai/azure-provider)
35+
* [github.com/gptscript-ai/anthropic-provider](https://github.com/gptscript-ai/anthropic-provider)
36+
* [github.com/gptscript-ai/google-provider](https://github.com/gptscript-ai/google-provider)
37+
38+
## Listing available models
39+
40+
For any provider that supports listing models, you can use this command:
41+
42+
```bash
43+
# With a shim
44+
gptscript --list-models github.com/gptscript-ai/anthropic-provider
45+
46+
# To OpenAI compatible endpoint
47+
gptscript --list-models https://api.mistral.ai/v1
48+
```
49+
50+
51+
## Compatibility
52+
53+
While the shims provide support for using GPTScript with other models, the effectiveness of using a
54+
different model will depend on a combination of prompt engineering and the quality of the model. You may need to change
55+
wording or add more description if you are not getting the results you want. In some cases, the model might not be
56+
capable of intelligently handling the complex function calls.
File renamed without changes.

0 commit comments

Comments
 (0)