[BUG] hitting limits on big file includes

[5shekel]

Problem (one or two sentences)

i added an audio (opus) and json files to my initial prompt.
and roo-code included them "as is" into the context. saturating it immediately.

Context (who is affected and when)

happend wheni added an audio (opus) and json files to my initial prompt.
and roo-code included them "as is" into the context. saturating context immediately.

Reproduction steps

1. windows, vscode, openrouter opus4.5
2. start prompt referring to audio file (opus 20mb sized) and json file (2.7mb)

context>file read auto-truncate is at default.
contact me for source files if needed.

Expected result

prompt should be condensed

Actual result

context truncated above 1.8mil

Variations tried (optional)

No response

App Version

3.41.3

API Provider (optional)

OpenRouter

Model Used (optional)

opus 4.5

Roo Code Task Links (optional)

https://discord.com/channels/1332146336664915968/1462438061274038342

Relevant logs or errors (optional)

[roomote-v0]

I've analyzed this issue and here's my proposed implementation plan:

Root Cause

When files are mentioned using @/path/to/file in the initial prompt, the file content is read using extractTextFromFile() which only respects the maxReadFileLine setting (defaults to -1/unlimited). Unlike the read_file tool which has smart token-budget based truncation, file mentions have no size limits for text files.

This means:

• A 2.7MB JSON file (which is valid UTF-8 text, not binary) gets included in full
• The opus audio file should be detected as binary and omitted with "(Binary file xxx omitted)" - I'd like to confirm if this was the case or if it was also included

Proposed Implementation

I propose adding file size limits to the file mention processing in src/core/mentions/index.ts:

1. Add a configurable max file size limit for mentions - Add a new setting or use an existing context budget to limit how much content can be included from file mentions

2. Modify getFileOrFolderContent() function to:

   • Check file size before reading
   • For files exceeding a reasonable threshold (e.g., 100KB or configurable), truncate the content early
   • Add a notice indicating the file was truncated: "[File truncated: showing X of Y bytes. Use the read_file tool to examine specific sections.]"

3. Consider token-budget awareness - Similar to readFileWithTokenBudget() used by the read_file tool, we could implement token-aware truncation for file mentions

Questions

Before proceeding, I'd like to clarify:

1. Was the opus audio file also included in full, or was it correctly omitted as binary? (The isBinaryFile library should detect opus files as binary)

2. For the fix, would you prefer:

   • A) A simple file size limit (e.g., 100KB max per mentioned file)
   • B) Token-budget aware truncation (more sophisticated but aligns with how read_file works)
   • C) Leverage the existing maxReadFileLine setting more aggressively

Please mention @roomote in a comment with your answers/preference, and I'll proceed with the implementation.

[5shekel]

@roomote
i prefer B

[roomote-v0]

Thanks for your preference! I'll implement option B - token-budget aware truncation for file mentions. This will align the behavior with how the read_file tool works and provide more intelligent content truncation.

I'll explore the codebase now and open a PR to share my progress.

[roomote-v0]

Opened a PR with my attempt at implementing option B (token-budget aware truncation): #10830

The implementation limits each file mention to 10% of the model's context window. For example, with a 200k context window, each mentioned file is capped at ~20k tokens. When truncated, users will see a clear message indicating the file was truncated and suggesting to use the read_file tool to examine specific sections.

Feedback welcome!