Skip to content

Commit c146475

Browse files
committed
Add blog post about TOON format support
- New blog post explaining TOON integration across Forge - SVG diagram showing the TOON format flow - Screenshot of TOON results in forge-ui - Updated blog index with new post
1 parent 6c22cab commit c146475

4 files changed

Lines changed: 275 additions & 0 deletions

File tree

blog/index.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,13 @@ Technical articles about building efficient AI agents with Agentic Forge.
66

77
<div class="blog-list">
88

9+
### [TOON Format: Cutting Tokens Without Cutting Information](/blog/toon-format-support)
10+
*January 2026*
11+
12+
How we implemented TOON format support across Agentic Forge to reduce tool result tokens by 30-60%, with a custom header workaround for MCP SDK limitations.
13+
14+
---
15+
916
### [Model Management: Beyond the Dropdown](/blog/model-management-ui)
1017
*January 2026*
1118

blog/toon-format-support.md

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
# TOON Format: Cutting Tokens Without Cutting Information
2+
3+
*January 2026*
4+
5+
When AI agents use tools, the results often come back as verbose JSON. A simple weather lookup might return 500 tokens of data when the actual information fits in 50. We added TOON format support to Agentic Forge to reduce token usage by 30-60% while preserving all the data.
6+
7+
## What is TOON?
8+
9+
[TOON (Token-Oriented Object Notation)](https://github.com/toon-format/toon) is a compact data format designed specifically for LLM consumption. It uses whitespace-delimited values with headers instead of JSON's verbose key-value syntax.
10+
11+
Here's the same weather data in both formats:
12+
13+
**JSON (typical MCP response):**
14+
```json
15+
{
16+
"location": "London",
17+
"current": {
18+
"temperature": 18,
19+
"humidity": 72,
20+
"conditions": "Partly cloudy",
21+
"wind_speed": 12
22+
},
23+
"forecast": [
24+
{"day": "Monday", "high": 20, "low": 14, "conditions": "Sunny"},
25+
{"day": "Tuesday", "high": 19, "low": 13, "conditions": "Cloudy"},
26+
{"day": "Wednesday", "high": 17, "low": 12, "conditions": "Rain"}
27+
]
28+
}
29+
```
30+
31+
**TOON (same data, fewer tokens):**
32+
```
33+
location: London
34+
current:
35+
temperature: 18
36+
humidity: 72
37+
conditions: Partly cloudy
38+
wind_speed: 12
39+
40+
day|high|low|conditions
41+
Monday|20|14|Sunny
42+
Tuesday|19|13|Cloudy
43+
Wednesday|17|12|Rain
44+
```
45+
46+
The TOON version uses pipe-delimited tables for uniform arrays, eliminating repeated keys. For the forecast data alone, this cuts tokens by roughly 50%.
47+
48+
## The Implementation Challenge
49+
50+
Adding TOON support seemed simple: set an `Accept: text/toon` header when calling Armory. But there was a problem.
51+
52+
**The MCP SDK overwrites the Accept header.**
53+
54+
The `mcp` Python library's `_prepare_headers()` method sets:
55+
```python
56+
"Accept": "application/json, text/event-stream"
57+
```
58+
59+
This happens after user headers are merged, meaning our `Accept: text/toon` gets overwritten before the request is sent. No amount of header configuration could fix this.
60+
61+
## The Solution: Custom Header
62+
63+
Instead of fighting the SDK, we introduced a custom header that the SDK doesn't touch:
64+
65+
```
66+
X-Prefer-Format: toon
67+
```
68+
69+
Armory checks this header first, falling back to the standard `Accept` header for non-MCP clients:
70+
71+
```python
72+
def get_preferred_format(request: Request) -> OutputFormat:
73+
# Check custom header first (for MCP clients)
74+
prefer_format = request.headers.get("x-prefer-format", "").lower()
75+
if prefer_format == "toon":
76+
return OutputFormat.TOON
77+
78+
# Fallback to Accept header (for direct HTTP clients)
79+
accept = request.headers.get("accept", "")
80+
if "text/toon" in accept:
81+
return OutputFormat.TOON
82+
83+
return OutputFormat.JSON
84+
```
85+
86+
## The Full Flow
87+
88+
Here's how TOON format flows through the system:
89+
90+
![TOON Format Flow](/diagrams/toon-format-flow.svg)
91+
92+
1. **forge-ui** — User enables TOON toggle, stored in conversation settings
93+
2. **forge-orchestrator** — Adds `X-Prefer-Format: toon` header to MCP requests
94+
3. **forge-armory** — Routes tool call to backend MCP server
95+
4. **MCP Server** — Returns JSON (servers don't need TOON support)
96+
5. **forge-armory** — Converts JSON response to TOON using `toon` library
97+
6. **Response flows back** — TOON string returns through orchestrator to UI
98+
99+
The key insight: **conversion happens in Armory, not in backends**. MCP servers continue returning JSON. Armory acts as a translation layer, converting to TOON when the client requests it.
100+
101+
## Smart Conversion
102+
103+
TOON isn't always smaller than JSON. For deeply nested objects or non-uniform arrays, JSON might actually be more compact. Armory's conversion logic handles this:
104+
105+
```python
106+
def to_json_or_toon(data: Any, format: OutputFormat) -> str | Any:
107+
if format == OutputFormat.TOON:
108+
toon_str = to_toon(data)
109+
json_str = json.dumps(data, default=str)
110+
111+
# Only use TOON if it's actually smaller
112+
if len(toon_str) < len(json_str):
113+
return toon_str
114+
115+
return data # Return as object for JSON serialization
116+
```
117+
118+
This ensures TOON is only used when it provides actual benefit.
119+
120+
## UI Display
121+
122+
Tool results now display differently based on format. JSON results get pretty-printed with indentation. TOON results display as plain text with proper line breaks.
123+
124+
![TOON in forge-ui](/screens/toon-format-ui.png)
125+
126+
The `ToolCallCard` component detects the format:
127+
128+
```typescript
129+
const isStringResult = computed(() => {
130+
return typeof props.toolCall.result === 'string'
131+
})
132+
133+
const formattedResult = computed(() => {
134+
if (isStringResult.value) {
135+
// TOON: display as-is with line breaks
136+
return props.toolCall.result as string
137+
} else {
138+
// JSON: pretty-print
139+
return JSON.stringify(props.toolCall.result, null, 2)
140+
}
141+
})
142+
```
143+
144+
## Results
145+
146+
With TOON enabled, we see 30-60% token reduction on tool results with tabular data (weather forecasts, search results, database queries). The format is especially effective for:
147+
148+
- **Uniform arrays** — Lists of similar objects
149+
- **Flat data** — Single-level key-value pairs
150+
- **Tabular results** — Database rows, API listings
151+
152+
It's less effective for deeply nested JSON or highly irregular structures, where the overhead of TOON headers may exceed JSON's verbosity.
153+
154+
## Try It
155+
156+
1. Start Agentic Forge with Docker Compose
157+
2. Open the chat interface
158+
3. Enable the TOON toggle in settings
159+
4. Ask about the weather or run a search
160+
5. Expand the tool call card to see the TOON result
161+
162+
## Source Code
163+
164+
- [forge-armory](https://github.com/agentic-forge/forge-armory) — TOON conversion and format negotiation
165+
- [forge-orchestrator](https://github.com/agentic-forge/forge-orchestrator) — X-Prefer-Format header handling
166+
- [forge-ui](https://github.com/agentic-forge/forge-ui) — TOON toggle and display
167+
- [toon-format/toon](https://github.com/toon-format/toon) — The TOON library
168+
169+
---
170+
171+
*This is part of a series on building [Agentic Forge](https://agentic-forge.github.io).*
Lines changed: 97 additions & 0 deletions
Loading

public/screens/toon-format-ui.png

161 KB
Loading

0 commit comments

Comments
 (0)