Skip to content

Epic: Complete Chutes API Integration (Umbrella PR) #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
echobt wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Epic: Complete Chutes API Integration (Umbrella PR) #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ graph TB
end

subgraph LLM["LLM Layer"]
Client["LiteLLM Client"]
Provider["Provider (Chutes/OpenRouter)"]
Client["LLM Client"]
Provider["Chutes API"]
end

subgraph Tools["Tool System"]
Expand Down Expand Up @@ -91,7 +91,7 @@ graph TB
- **Prompt Caching** - 90%+ cache hit rate for significant cost reduction
- **Context Management** - Intelligent pruning and compaction for long tasks
- **Self-Verification** - Automatic validation before task completion
- **Multi-Provider** - Supports Chutes AI, OpenRouter, and litellm-compatible providers
- **Chutes API** - Uses Chutes AI with OpenAI-compatible API format

---

Expand All @@ -105,7 +105,7 @@ baseagent/
│ │ ├── loop.py # Main agent loop
│ │ └── compaction.py # Context management
│ ├── llm/
│ │ └── client.py # LLM client (litellm)
│ │ └── client.py # LLM client (httpx)
│ ├── config/
│ │ └── defaults.py # Configuration
│ ├── tools/ # Tool implementations
Expand Down
16 changes: 8 additions & 8 deletions docs/architecture.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ graph TB
end

subgraph LLM["LLM Layer"]
client["client.py<br/>LiteLLM Client"]
client["client.py<br/>httpx Client"]
end

subgraph Config["Configuration"]
Expand Down Expand Up @@ -71,7 +71,7 @@ classDiagram
+log(msg)
}

class LiteLLMClient {
class LLMClient {
+model: str
+temperature: float
+max_tokens: int
Expand Down Expand Up @@ -105,8 +105,8 @@ classDiagram
+inject_content: Optional
}

AgentContext --> LiteLLMClient : uses
LiteLLMClient --> LLMResponse : returns
AgentContext --> LLMClient : uses
LLMClient --> LLMResponse : returns
LLMResponse --> FunctionCall : contains
AgentContext --> ToolRegistry : uses
ToolRegistry --> ToolResult : returns
Expand Down Expand Up @@ -171,13 +171,13 @@ sequenceDiagram
participant Loop as loop.py
participant Context as compaction.py
participant Cache as Prompt Cache
participant LLM as LiteLLM Client
participant LLM as httpx Client
participant Provider as API Provider
participant Tools as Tool Registry

User->>Entry: --instruction "Create hello.txt"
Entry->>Entry: Initialize AgentContext
Entry->>Entry: Initialize LiteLLMClient
Entry->>Entry: Initialize LLMClient
Entry->>Loop: run_agent_loop()

Loop->>Loop: Build messages [system, user, state]
Expand Down Expand Up @@ -261,8 +261,8 @@ Intelligent context management that:

### `src/llm/client.py` - LLM Client

LiteLLM-based client that:
- Supports multiple providers (Chutes, OpenRouter, etc.)
httpx-based client that:
- Uses Chutes API (OpenAI-compatible)
- Tracks token usage and costs
- Handles tool/function calling format
- Enforces cost limits
Expand Down
Loading