|
| 1 | +# Alternative Model Providers |
| 2 | + |
| 3 | + |
| 4 | +## Usage |
| 5 | + |
| 6 | +GPTScript can be used against alternative models that expose an OpenAI compatible API or have a provider shim available. |
| 7 | + |
| 8 | +To use a model with an OpenAI compatible API: |
| 9 | + |
| 10 | +```gptscript |
| 11 | +model: mistral-large-latest from https://api.mistral.ai/v1 |
| 12 | +
|
| 13 | +Say hello world |
| 14 | +``` |
| 15 | + |
| 16 | +To use a model that requires a provider shim: |
| 17 | + |
| 18 | +```gptscript |
| 19 | +model: claude-3-haiku-20240307 from github.com/gptscript-ai/anthropic-provider |
| 20 | +
|
| 21 | +Say hello world |
| 22 | +``` |
| 23 | + |
| 24 | +### Authentication |
| 25 | + |
| 26 | +For OpenAI compatible providers, GPTScript will look for an API key to be configured with the prefix `GPTSCRIPT_PROVIDER_`, the base domain converted to environment variable format, and a suffix of `_API_KEY`. |
| 27 | +As an example if you are using `mistral-large-latest from https://api.mistral.ai/v1`, the environment variable would be `GPTSCRIPT_PROVIDER_API_MISTRAL_AI_API_KEY` |
| 28 | + |
| 29 | +Each provider shim has different requirements for authentication. Please check the readme for the provider you are trying to use. |
| 30 | + |
| 31 | +## Available Model Providers |
| 32 | + |
| 33 | +The following shims are currently available: |
| 34 | +* [github.com/gptscript-ai/azure-provider](https://github.com/gptscript-ai/azure-provider) |
| 35 | +* [github.com/gptscript-ai/anthropic-provider](https://github.com/gptscript-ai/anthropic-provider) |
| 36 | +* [github.com/gptscript-ai/google-provider](https://github.com/gptscript-ai/google-provider) |
| 37 | + |
| 38 | +## Listing available models |
| 39 | + |
| 40 | +For any provider that supports listing models, you can use this command: |
| 41 | + |
| 42 | +```bash |
| 43 | +# With a shim |
| 44 | +gptscript --list-models github.com/gptscript-ai/anthropic-provider |
| 45 | + |
| 46 | +# To OpenAI compatible endpoint |
| 47 | +gptscript --list-models https://api.mistral.ai/v1 |
| 48 | +``` |
| 49 | + |
| 50 | + |
| 51 | +## Compatibility |
| 52 | + |
| 53 | +While the shims provide support for using GPTScript with other models, the effectiveness of using a |
| 54 | +different model will depend on a combination of prompt engineering and the quality of the model. You may need to change |
| 55 | +wording or add more description if you are not getting the results you want. In some cases, the model might not be |
| 56 | +capable of intelligently handling the complex function calls. |
0 commit comments