Skip to content
Merged

Dev #1109

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/features/ai-knowledge/skills.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ Follow the [Open Terminal setup guide](/features/extensibility/open-terminal#get
Add your Open Terminal instance as a Tool Server by following the [OpenAPI Tool Server Integration Guide](/features/extensibility/plugin/tools/openapi-servers/open-webui). You can connect it as:

- A **User Tool Server** (in **Settings → Tools**) — connects from your browser, ideal for personal or local instances
- A **Global Tool Server** (in **Admin Settings → Tools**) — connects from the backend, available to all users
- A **Global Tool Server** (in **Admin Settings → Integrations**) — connects from the backend, available to all users

Once connected, the Open Terminal tools (execute, file upload, file download) appear automatically in the chat interface.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ If you are running a multi-user or organizational deployment, **Jupyter is not r
- **Full shell access** — models can install packages, run scripts in any language, use system tools like ffmpeg, git, curl, etc.
- **Container isolation** — runs in its own Docker container, separate from Open WebUI and other services.
- **Rich pre-installed toolset** — the Docker image comes with Python 3.12, data science libraries, build tools, networking utilities, and more.
- **Built-in file browser** — browse, preview, create, delete, upload, and download files directly from the chat controls panel.

### Comparison

Expand All @@ -66,6 +67,6 @@ If you are running a multi-user or organizational deployment, **Jupyter is not r
| **Isolation** | ✅ Browser sandbox | ❌ Shared environment | ✅ Container-level (when using Docker) |
| **Multi-user safety** | ✅ Per-user by design | ⚠️ Not isolated | ⚠️ Single instance (per-user containers planned) |
| **File generation** | ❌ Very limited | ✅ Full support | ✅ Full support with upload/download |
| **Setup** | None (built-in) | Admin configures globally | Each user adds as a Tool Server |
| **Setup** | None (built-in) | Admin configures globally | Native integration via Settings → Integrations, or as a Tool Server |
| **Recommended for orgs** | ✅ Safe default | ❌ Not without isolation | ✅ Per-user by design |
| **Enterprise scalability** | ✅ Client-side, no server load | ❌ Single shared instance | ⚠️ Manual per-user instances |
9 changes: 9 additions & 0 deletions docs/features/extensibility/mcp.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,14 @@ Browser-based, multi-user deployments increase the surface area (CORS/CSRF, per-
* **Bearer**: Use this **only** if your MCP server requires a specific API token. You **must** populate the "Key" field.
* **OAuth 2.1**: For secured, enterprise deployments requiring Identity Provider flows.

:::warning OAuth 2.1 Tools Cannot Be Set as Default Tools
**Do not set OAuth 2.1 MCP tools as default/pre-enabled tools on a model.** The OAuth 2.1 authorization flow requires an interactive browser redirect (user consent, callback) that cannot happen transparently during a chat completion request.

If an OAuth 2.1 tool is set as a default and the user hasn't previously authenticated (or their refresh token has expired), the tool call will fail with **"Failed to connect to MCP server"** because the backend cannot initiate the browser-based auth flow mid-request.

**Workaround:** Users should manually enable OAuth 2.1 tools per-chat via the **➕** button in the chat input area. This triggers the auth flow before the tool is ever invoked. Token refresh works automatically once the initial authentication is complete.
:::

### Connection URLs

If you are running Open WebUI in **Docker** and your MCP server is on the **host machine**:
Expand All @@ -85,6 +93,7 @@ The chat shows "Failed to connect to MCP server" when using a tool, even if the
**Solutions**:
1. **Check Authentication**: Ensure you haven't selected `Bearer` without a key. Switch to `None` if no token is needed.
2. **Filter List Bug**: If the "Function Name Filter List" is empty, try adding a comma (`,`) to it.
3. **OAuth 2.1 Default Tool**: If the failing tool uses OAuth 2.1 and is set as a default tool on the model, this is a [known limitation](#oauth-21-tools-cannot-be-set-as-default-tools). Remove it from the model's default tools and have users enable it manually per-chat.

### Infinite loading screen after adding External Tool

Expand Down
345 changes: 303 additions & 42 deletions docs/features/extensibility/open-terminal/index.md

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion docs/features/extensibility/plugin/functions/action.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ class Valves(BaseModel):
priority: int = 0 # Lower = appears first in the button row
```

This uses the same priority mechanism as [filter functions](/features/extensibility/plugin/functions/filter), so the behavior is consistent across the plugin system. Without a `priority` valve, actions default to `0` and their order among equal priorities is non-deterministic.
This uses the same priority mechanism as [filter functions](/features/extensibility/plugin/functions/filter), so the behavior is consistent across the plugin system. Without a `priority` valve, actions default to `0` and their order among equal priorities is determined **alphabetically by function ID**.

## Advanced Capabilities

Expand Down
2 changes: 1 addition & 1 deletion docs/features/extensibility/plugin/functions/filter.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -527,7 +527,7 @@ class Filter:
| `2` | Runs after priority 1 |

:::tip Lower Priority = Earlier Execution
Filters are sorted in **ascending** order by priority. A filter with `priority=0` runs **before** a filter with `priority=1`, which runs before `priority=2`, and so forth.
Filters are sorted in **ascending** order by priority. A filter with `priority=0` runs **before** a filter with `priority=1`, which runs before `priority=2`, and so forth. When multiple filters share the same priority value, they are sorted **alphabetically by function ID** for deterministic ordering.
:::

---
Expand Down
78 changes: 68 additions & 10 deletions docs/features/extensibility/plugin/tools/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Because there are several ways to integrate "Tools" in Open WebUI, it's importan
| **Native MCP (HTTP)** | `Settings > Connections` | Standard MCP servers reachable via HTTP/SSE | External MCP Servers |
| **MCP via Proxy (MCPO)** | `Settings > Connections` | Local stdio-based MCP servers (e.g., Claude Desktop tools) | [MCPO Adapter](https://github.com/open-webui/mcpo) |
| **OpenAPI Servers** | `Settings > Connections` | Standard REST/OpenAPI web services | External Web APIs |
| **Open Terminal** | `Settings > Integrations` | Full shell access in an isolated Docker container (always-on) | [Open Terminal](https://github.com/open-webui/open-terminal) |

### 1. Native Features (Built-in)
These are deeply integrated into Open WebUI and generally don't require external scripts.
Expand Down Expand Up @@ -88,24 +89,34 @@ You can also let your LLM auto-select the right Tools using the [**AutoTool Filt

Open WebUI offers two distinct ways for models to interact with tools: a standard **Default Mode** and a high-performance **Native Mode (Agentic Mode)**. Choosing the right mode depends on your model's capabilities and your performance requirements.

### 🟡 Default Mode (Prompt-based)
### 🟡 Default Mode (Prompt-based) — Legacy

:::warning Legacy Mode
Default Mode is maintained purely for **backwards compatibility** with older or smaller models that lack native function-calling support. It is considered **legacy** and should not be used when your model supports native tool calling. New deployments should use **Native Mode** exclusively.
:::

In Default Mode, Open WebUI manages tool selection by injecting a specific prompt template that guides the model to output a tool request.
- **Compatibility**: Works with **practically any model**, including older or smaller local models that lack native function-calling support.
- **Flexibility**: Highly customizable via prompt templates.
- **Caveat**: Can be slower (requires extra tokens) and less reliable for complex, multi-step tool chaining.
- **Caveats**:
- Can be slower (requires extra tokens) and less reliable for complex, multi-step tool chaining.
- **Breaks KV cache**: The injected prompt changes every turn, preventing LLM engines from reusing cached key-value pairs. This increases latency and cost for every message in the conversation.
- Does not support built-in system tools (memory, notes, channels, etc.).

### 🟢 Native Mode (Agentic Mode / System Function Calling)
Native Mode (also called **Agentic Mode**) leverages the model's built-in capability to handle tool definitions and return structured tool calls (JSON). This is the **recommended mode** for high-performance agentic workflows.
### 🟢 Native Mode (Agentic Mode / System Function Calling) — Recommended
Native Mode (also called **Agentic Mode**) leverages the model's built-in capability to handle tool definitions and return structured tool calls (JSON). This is the **recommended mode** for all models that support it — which includes the vast majority of modern models (2024+).

:::warning Model Quality Matters
**Agentic tool calling requires high-quality models to work reliably.** While small local models may technically support function calling, they often struggle with the complex reasoning required for multi-step tool usage. For best results, use frontier models like **GPT-5**, **Claude 4.5 Sonnet**, **Gemini 3 Flash**, or **MiniMax M2.5**. Small local models may produce malformed JSON or fail to follow the strict state management required for agentic behavior.
:::

#### Why use Native Mode (Agentic Mode)?
- **Speed & Efficiency**: Lower latency as it avoids bulky prompt-based tool selection.
- **KV Cache Friendly**: Tool definitions are sent as structured parameters (not injected into the prompt), so they don't invalidate the KV cache between turns. This can significantly reduce latency and token costs.
- **Reliability**: Higher accuracy in following tool schemas (with quality models).
- **Multi-step Chaining**: Essential for **Agentic Research** and **Interleaved Thinking** where a model needs to call multiple tools in succession.
- **Autonomous Decision-Making**: Models can decide when to search, which tools to use, and how to combine results.
- **System Tools**: Only Native Mode unlocks the [built-in system tools](#built-in-system-tools-nativeagentic-mode) (memory, notes, knowledge, channels, etc.).

#### How to Enable Native Mode (Agentic Mode)
Native Mode can be enabled at two levels:
Expand Down Expand Up @@ -163,11 +174,14 @@ These models excel at multi-step reasoning, proper JSON formatting, and autonomo
**This is a DeepSeek model/API issue**, not an Open WebUI issue. Open WebUI correctly sends tools in standard OpenAI format — the malformed output originates from DeepSeek's non-standard internal format.
:::

| Feature | Default Mode | Native Mode |
| Feature | Default Mode (Legacy) | Native Mode (Recommended) |
|:---|:---|:---|
| **Status** | Legacy / backwards compat | ✅ Recommended |
| **Latency** | Medium/High | Low |
| **KV Cache** | ❌ Can break cache | ✅ Cache-friendly |
| **Model Compatibility** | Universal | Requires Tool-Calling Support |
| **Logic** | Prompt-based (Open WebUI) | Model-native (API/Ollama) |
| **System Tools** | ❌ Not available | ✅ Full access |
| **Complex Chaining** | ⚠️ Limited | ✅ Excellent |

### Built-in System Tools (Native/Agentic Mode)
Expand All @@ -179,12 +193,13 @@ These models excel at multi-step reasoning, proper JSON formatting, and autonomo
| **Search & Web** | *Requires `ENABLE_WEB_SEARCH` enabled AND per-chat "Web Search" toggle enabled.* |
| `search_web` | Search the public web for information. Best for current events, external references, or topics not covered in internal documents. |
| `fetch_url` | Visits a URL and extracts text content via the Web Loader. |
| **Knowledge Base** | *Requires per-model "Knowledge Base" category enabled (default: on).* |
| **Knowledge Base** | *Requires per-model "Knowledge Base" category enabled (default: on). Which tools are injected depends on whether the model has attached knowledge — see note below.* |
| `list_knowledge_bases` | List the user's accessible knowledge bases with file counts. **Use this first** to discover what knowledge is available. |
| `query_knowledge_bases` | Search KB *names and descriptions* by semantic similarity. Use to find which KB is relevant when you don't know which one to query. |
| `search_knowledge_bases` | Search knowledge bases by name/description (text match). |
| `query_knowledge_files` | Search *file contents* inside KBs using vector search. **This is your main tool for finding information.** When a KB is attached to the model, searches are automatically scoped to that KB. |
| `search_knowledge_files` | Search files across accessible knowledge bases by filename (not content). |
| `view_file` | Get the full content of any user-accessible file by its ID. Only injected when the model has attached knowledge files. |
| `view_knowledge_file` | Get the full content of a file from a knowledge base. |
| **Image Gen** | *Requires image generation enabled (per-tool) AND per-chat "Image Generation" toggle enabled.* |
| `generate_image` | Generates a new image based on a prompt. Requires `ENABLE_IMAGE_GENERATION`. |
Expand Down Expand Up @@ -229,6 +244,7 @@ These models excel at multi-step reasoning, proper JSON formatting, and autonomo
| `search_knowledge_bases` | `query` (required), `count` (default: 5), `skip` (default: 0) | Array of `{id, name, description, file_count}` |
| `query_knowledge_files` | `query` (required), `knowledge_ids` (optional), `count` (default: 5) | Array of `{id, filename, content_snippet, knowledge_id}` |
| `search_knowledge_files` | `query` (required), `knowledge_id` (optional), `count` (default: 5), `skip` (default: 0) | Array of `{id, filename, knowledge_id, knowledge_name}` |
| `view_file` | `file_id` (required) | `{id, filename, content, updated_at, created_at}` |
| `view_knowledge_file` | `file_id` (required) | `{id, filename, content}` |
| **Image Gen** | | |
| `generate_image` | `prompt` (required) | `{status, message, images}` — auto-displayed |
Expand Down Expand Up @@ -264,10 +280,52 @@ These models excel at multi-step reasoning, proper JSON formatting, and autonomo
Open WebUI automatically detects and stores your timezone when you log in. This allows time-related tools and features to provide accurate local times without any manual configuration. Your timezone is determined from your browser settings.
:::

:::info Recommended KB Tool Workflow
**When an attached KB is returning empty results:**
1. First call `list_knowledge_bases` to confirm the model can see the attached KB
2. Then use `query_knowledge_files` (without specifying `knowledge_ids` — it auto-scopes to attached KBs)
:::warning Knowledge Tools Change Based on Attached Knowledge
The set of knowledge tools injected into a model **changes depending on whether the model has knowledge attached** (via the Model Editor). These are **two mutually exclusive sets** — the model gets one or the other, never both.

**Model with attached knowledge** (files, collections, or notes):

| Tool | When Available |
|------|---------------|
| `query_knowledge_files` | Always (auto-scoped to attached KBs) |
| `view_file` | When attached knowledge includes files or collections |

The model **does not** get the browsing tools (`list_knowledge_bases`, `search_knowledge_bases`, etc.) because it doesn't need to discover KBs — the search is automatically scoped to its attachments.

**Model without attached knowledge** (general-purpose):

| Tool | Purpose |
|------|---------|
| `list_knowledge_bases` | Discover available KBs |
| `search_knowledge_bases` | Search KBs by name/description (text match) |
| `query_knowledge_bases` | Search KBs by name/description (semantic similarity) |
| `search_knowledge_files` | Search files by filename |
| `query_knowledge_files` | Search file contents (vector search) |
| `view_knowledge_file` | Read a full file from a KB |

This model has the full browsing set to autonomously discover and explore any KB the user has access to.
:::

:::caution Attached Knowledge Still Requires User Access
Attaching a knowledge base to a custom model does **not** bypass access control. When a user chats with the model, `query_knowledge_files` checks whether **that specific user** has permission to access each attached knowledge item. Items the user cannot access are silently excluded from search results.

**Access requirements by knowledge type:**

| Attached Type | User Needs |
|---------------|-----------|
| **Knowledge Base** (collection) | Owner, admin, or explicit read access grant |
| **Individual File** | Owner or admin only (no access grants) |
| **Note** | Owner, admin, or explicit read access grant |

**Example scenario**: An admin creates a private knowledge base and attaches it to a custom model shared with all users. Regular users chatting with this model will get **empty results** from `query_knowledge_files` because they don't have read access to the KB itself — even though they can use the model.

**Solution**: Make sure users who need access to the model's knowledge also have read access to the underlying knowledge base (via access grants or group permissions in the Knowledge settings).
:::

:::info Recommended KB Tool Workflow (No Attached Knowledge)
When using a model **without** attached knowledge:
1. First call `list_knowledge_bases` to discover what knowledge is available
2. Then use `query_knowledge_files` to search file contents within relevant KBs
3. If still empty, the files may not be embedded yet, or you may have **Full Context mode enabled** which bypasses the vector store

**Do NOT use Full Context mode with knowledge tools** — Full Context injects file content directly and doesn't store embeddings, so `query_knowledge_files` will return empty. Use Focused Retrieval (default) for tool-based access.
Expand Down
Loading