Skip to content

Commit

Permalink
Deploy preview for PR 665 🛫
Browse files Browse the repository at this point in the history
  • Loading branch information
JonasHelming committed Nov 29, 2024
1 parent ad466f1 commit 879f389
Show file tree
Hide file tree
Showing 3 changed files with 5 additions and 5 deletions.
6 changes: 3 additions & 3 deletions pr-previews/pr-665/docs/theia_ai/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -315,9 +315,9 @@ <h3 id="create-and-register-a-response-part-renderer" style="position:relative;"
<h2 id="custom-llm-provider" style="position:relative;"><a href="#custom-llm-provider" aria-label="custom llm provider permalink" class="anchor before"><svg aria-hidden="true" focusable="false" height="16" version="1.1" viewBox="0 0 16 16" width="16"><path fill-rule="evenodd" d="M4 9h1v1H4c-1.5 0-3-1.69-3-3.5S2.55 3 4 3h4c1.45 0 3 1.69 3 3.5 0 1.41-.91 2.72-2 3.25V8.59c.58-.45 1-1.27 1-2.09C10 5.22 8.98 4 8 4H4c-.98 0-2 1.22-2 2.5S3 9 4 9zm9-3h-1v1h1c1 0 2 1.22 2 2.5S13.98 12 13 12H9c-.98 0-2-1.22-2-2.5 0-.83.42-1.64 1-2.09V6.25c-1.09.53-2 1.84-2 3.25C6 11.31 7.55 13 9 13h4c1.45 0 3-1.69 3-3.5S14.5 6 13 6z"></path></svg></a>Custom LLM Provider</h2>
<p>Theia AI currently provides out-of-the-box support for any OpenAI-compatible LLM services (including the OpenAI models hosted by OpenAI or self-hosted compatible LLMs), Hugging Face, Ollama LLMs and Llamafile.</p>
<p><em>Note: Theia AI enables connections to various models. However, be aware that some models may require specific customizations and optimizations to perform well in all scenarios. If you encounter issues implementing those customizations, please <a href="https://github.com/eclipse-theia/theia/issues/new/choose" target="_blank" rel="nofollow noopener noreferrer">provide feedback</a>.</em></p>
<p>Please learn about the supported LLM providers at the example of the <a href="user_ai">Theia IDE</a>, which allows end users to configure the used LLM per agent. Of course, as a tool provider you can choose whether you provide this configurability to end users.</p>
<p>As a tool provider, you can also easily add support for other LLM APIs too. Please consider contributing any further LLM providers to the open source project, so that the community can benefit from increased compatibility and share the maintenance cost.</p>
<p>To support a specific LLM, you basically need to provide an implementation of the interface ‘LanguageModel’, which provides an abstraction between agents and the underlying LLM. This Language Model is then registered at the ‘LanguageModelRegistry’, to make it retrievable for agents, e.g. like this:</p>
<p>Learn more about which LLM providers are supported out of the box in at the example of the <a href="user_ai">Theia IDE</a>, which allows end users to configure the used LLM per agent. Of course, as a tool provider you can choose whether you provide this configurability to end users.</p>
<p>As a tool provider, you can easily add support for additional LLM APIs too. Please consider contributing your additional LLM providers to the open source project, so that the community can benefit from increased compatibility and share the maintenance cost.</p>
<p>To support a specific LLM API, you basically need to provide an implementation of the interface ‘LanguageModel’, which provides an abstraction between agents and the underlying LLM. This language model implementation then needs to be registered at the ‘LanguageModelRegistry’, to make it retrievable for agents as follows:</p>
<div class="gatsby-highlight" data-language="typescript"><pre class="language-typescript"><code class="language-typescript"><span class="token keyword">this</span><span class="token punctuation">.</span>languageModelRegistry<span class="token punctuation">.</span><span class="token function">addLanguageModels</span><span class="token punctuation">(</span><span class="token punctuation">[</span><span class="token keyword">new</span> <span class="token class-name">OllamaModel</span><span class="token punctuation">(</span><span class="token punctuation">)</span><span class="token punctuation">]</span><span class="token punctuation">)</span><span class="token punctuation">;</span></code></pre></div>
<p>If you want to allow the user to configure the LLM provider, e.g. change the URL or the available models, consider integrating your LLM provider with Theias preference system.</p>
<p>For further details, we recommend reviewing the available LLM provider in Theia AI:</p>
Expand Down
2 changes: 1 addition & 1 deletion pr-previews/pr-665/page-data/docs/theia_ai/page-data.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion pr-previews/pr-665/support/index.html

Large diffs are not rendered by default.

0 comments on commit 879f389

Please sign in to comment.