Skip to content

refine response format #1368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iceljc merged 3 commits into from
Jun 24, 2026
Merged

refine response format #1368

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
7a72a0b
refine response format
Jun 24, 2026
37073e2
use state
Jun 24, 2026
9b90415
minor change
Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 13 additions & 13 deletions src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentLlmConfig.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ public class AgentLlmConfig
{
public AgentLlmConfig() { }

public AgentLlmConfig(AgentTemplateConfig templateConfig)
public AgentLlmConfig(AgentTemplateLlmConfig templateLlmConfig)
{
Provider = templateConfig.LlmConfig?.Provider;
Model = templateConfig.LlmConfig?.Model;
MaxOutputTokens = templateConfig.LlmConfig?.MaxOutputTokens;
ReasoningEffortLevel = templateConfig.LlmConfig?.ReasoningEffortLevel;
ResponseFormat = templateConfig.ResponseFormat;
Provider = templateLlmConfig?.Provider;
Model = templateLlmConfig?.Model;
MaxOutputTokens = templateLlmConfig?.MaxOutputTokens;
ReasoningEffortLevel = templateLlmConfig?.ReasoningEffortLevel;
ResponseFormat = templateLlmConfig?.ResponseFormat;
}

/// <summary>
Expand Down Expand Up @@ -53,6 +53,13 @@ public AgentLlmConfig(AgentTemplateConfig templateConfig)
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? ReasoningEffortLevel { get; set; }

/// <summary>
/// Response format: json, xml, markdown, yaml, etc.
/// </summary>
[JsonPropertyName("response_format")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? ResponseFormat { get; set; }

/// <summary>
/// Image composition config
/// </summary>
Expand All @@ -73,13 +80,6 @@ public AgentLlmConfig(AgentTemplateConfig templateConfig)
[JsonPropertyName("realtime")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public LlmRealtimeConfig? Realtime { get; set; }

/// <summary>
/// Response format: json, xml, markdown, yaml, etc.
/// </summary>
[JsonPropertyName("response_format")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? ResponseFormat { get; set; }
}

public class LlmImageCompositionConfig : LlmProviderModel
Expand Down
7 changes: 0 additions & 7 deletions src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentTemplate.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,6 @@ public class AgentTemplateConfig
{
public string Name { get; set; }

/// <summary>
/// Response format: json, xml, markdown, yaml, etc.
/// </summary>
[JsonPropertyName("response_format")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? ResponseFormat { get; set; }

[JsonPropertyName("llm_config")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public AgentTemplateLlmConfig? LlmConfig { get; set; }
Comment on lines 24 to 29

@qodo-code-review qodo-code-review Bot Jun 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Legacy response_format ignored 🐞 Bug ☼ Reliability

Existing stored templates that still use the old top-level response_format field will deserialize
without that value (field removed), so the runtime AgentLlmConfig.ResponseFormat will not be set
from legacy data. This silently changes LLM request behavior (e.g., OpenAI chat
options.ResponseFormat becomes null) after upgrade.
Agent Prompt 
### Issue description
`response_format` was removed from `AgentTemplateConfig` (and from Mongo template element), but repositories still load stored template configs/documents that may contain legacy top-level `response_format`. System.Text.Json and Mongo conventions will ignore unknown fields, so existing agents/templates silently lose this setting and runtime LLM calls won’t apply the intended response format.

### Issue Context
- Old schema: `[{ name, response_format, llm_config: {...} }]`
- New schema: `[{ name, llm_config: { ..., response_format } }]`
- Current load paths deserialize into `AgentTemplateConfig` and only apply `LlmConfig`, so legacy `response_format` is dropped.

### Fix Focus Areas
- src/Infrastructure/BotSharp.Abstraction/Agents/Models/AgentTemplate.cs[23-33]
- src/Infrastructure/BotSharp.Core/Repository/FileRepository/FileRepository.cs[511-522]
- src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentTemplateMongoElement.cs[5-30]

### Suggested approach
1. **File repository backward-compat**:
   - Re-introduce a deprecated/legacy `ResponseFormat` property on `AgentTemplateConfig` *only for deserialization* (e.g., `[JsonPropertyName("response_format")]` + `[Obsolete]`).
   - In `GetAgentTemplateConfigs`, after deserialization, migrate:
     - If `config.ResponseFormat` is not null/empty and `config.LlmConfig?.ResponseFormat` is null/empty, set `config.LlmConfig ??= new AgentTemplateLlmConfig(); config.LlmConfig.ResponseFormat = config.ResponseFormat;`
     - Optionally set `config.ResponseFormat = null` to avoid re-writing legacy field.
2. **Mongo backward-compat**:
   - Re-introduce a deprecated `ResponseFormat` field on `AgentTemplateMongoElement` to read existing documents.
   - In `ToDomainElement`, if `mongoTemplate.ResponseFormat` is set and `mongoTemplate.LlmConfig?.ResponseFormat` is null, migrate into the domain `LlmConfig.ResponseFormat`.
   - Ensure `ToMongoElement` does **not** populate the legacy field (keep writing only the new nested format).
3. Add a small regression test (or fixture) covering deserialization of old template config JSON / Mongo element with top-level `response_format` and verifying it ends up in `template.LlmConfig.ResponseFormat`.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Expand Down
7 changes: 7 additions & 0 deletions src/Infrastructure/BotSharp.Abstraction/Models/LlmConfigBase.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,13 @@ public class LlmConfigBase : LlmProviderModel
[JsonPropertyName("reasoning_effort_level")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? ReasoningEffortLevel { get; set; }

/// <summary>
/// Response format: json, xml, markdown, yaml, etc.
/// </summary>
[JsonPropertyName("response_format")]
[JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
public string? ResponseFormat { get; set; }
}

public class LlmProviderModel
Expand Down
1 change: 0 additions & 1 deletion src/Infrastructure/BotSharp.Core/Agents/Services/AgentService.CreateAgent.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,6 @@ private List<AgentTemplate> GetTemplatesFromFile(string fileDir)
var config = configs.FirstOrDefault(x => x.Name.IsEqualTo(name));
if (config != null)
{
template.ResponseFormat = config.ResponseFormat;
template.LlmConfig = config.LlmConfig;
}
templates.Add(template);
Expand Down
6 changes: 3 additions & 3 deletions src/Infrastructure/BotSharp.Core/Instructs/Services/InstructService.Execute.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -248,10 +248,10 @@ private async Task<InstructResult> RunLlm(
if (!string.IsNullOrEmpty(templateName))
{
prompt = agentService.RenderTemplate(agent, templateName);
var template = agent.Templates?.FirstOrDefault(x => x.Name.IsEqualTo(templateName));
if (template?.LlmConfig?.IsValid == true)
var templateLlmConfig = agent.Templates?.FirstOrDefault(x => x.Name.IsEqualTo(templateName))?.LlmConfig;
if (templateLlmConfig?.IsValid == true)
{
llmConfig = new AgentLlmConfig(template);
llmConfig = new AgentLlmConfig(templateLlmConfig);
}
}
else
Expand Down
5 changes: 3 additions & 2 deletions src/Infrastructure/BotSharp.Core/Instructs/Services/InstructService.Instruct.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,9 +68,10 @@ private async Task<Agent> BuildInnerAgent(InstructOptions? options)
{
var template = agent?.Templates?.FirstOrDefault(x => x.Name.IsEqualTo(options.TemplateName));
instruction = BuildInstruction(template?.Content ?? string.Empty, options?.Data ?? []);
if (template?.LlmConfig?.IsValid == true)
var templateLlmConfig = template?.LlmConfig;
if (templateLlmConfig?.IsValid == true)
{
llmConfig = new AgentLlmConfig(template);
llmConfig = new AgentLlmConfig(templateLlmConfig);
}
}
}
Expand Down
4 changes: 0 additions & 4 deletions src/Infrastructure/BotSharp.Core/Repository/FileRepository/FileRepository.Agent.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -440,7 +440,6 @@ private async Task UpdateAgentTemplates(string agentId, List<AgentTemplate> temp
var configs = templates.Select(t => new AgentTemplateConfig
{
Name = t.Name,
ResponseFormat = t.ResponseFormat,
LlmConfig = t.LlmConfig
}).ToList();

Expand Down Expand Up @@ -765,7 +764,6 @@ public async Task<string> GetAgentTemplate(string agentId, string templateName)
var found = configs?.FirstOrDefault(x => x.Name.IsEqualTo(templateName));
if (found != null)
{
template.ResponseFormat = found.ResponseFormat;
template.LlmConfig = found.LlmConfig;
}
}
Expand Down Expand Up @@ -807,15 +805,13 @@ public async Task<bool> PatchAgentTemplate(string agentId, AgentTemplate templat
var existingConfig = configs.FirstOrDefault(x => x.Name.IsEqualTo(template.Name));
if (existingConfig != null)
{
existingConfig.ResponseFormat = template.ResponseFormat;
existingConfig.LlmConfig = template.LlmConfig;
}
else
{
configs.Add(new AgentTemplateConfig
{
Name = template.Name,
ResponseFormat = template.ResponseFormat,
LlmConfig = template.LlmConfig
});
}
Expand Down
1 change: 0 additions & 1 deletion src/Infrastructure/BotSharp.Core/Repository/FileRepository/FileRepository.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -406,7 +406,6 @@ private List<AgentTemplate> FetchTemplates(string fileDir)
var config = configs.FirstOrDefault(x => x.Name.IsEqualTo(name));
if (config != null)
{
template.ResponseFormat = config.ResponseFormat;
template.LlmConfig = config.LlmConfig;
}
templates.Add(template);
Expand Down
7 changes: 5 additions & 2 deletions src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentTemplateLlmConfigMongoModel.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ public class AgentTemplateLlmConfigMongoModel
public string? Model { get; set; }
public int? MaxOutputTokens { get; set; }
public string? ReasoningEffortLevel { get; set; }
public string? ResponseFormat { get; set; }

public static AgentTemplateLlmConfigMongoModel? ToMongoModel(AgentTemplateLlmConfig? config)
{
Expand All @@ -21,7 +22,8 @@ public class AgentTemplateLlmConfigMongoModel
Provider = config.Provider,
Model = config.Model,
MaxOutputTokens = config.MaxOutputTokens,
ReasoningEffortLevel = config.ReasoningEffortLevel
ReasoningEffortLevel = config.ReasoningEffortLevel,
ResponseFormat = config.ResponseFormat
};
}

Expand All @@ -37,7 +39,8 @@ public class AgentTemplateLlmConfigMongoModel
Provider = config.Provider,
Model = config.Model,
MaxOutputTokens = config.MaxOutputTokens,
ReasoningEffortLevel = config.ReasoningEffortLevel
ReasoningEffortLevel = config.ReasoningEffortLevel,
ResponseFormat = config.ResponseFormat
};
}
}
3 changes: 0 additions & 3 deletions src/Plugins/BotSharp.Plugin.MongoStorage/Models/AgentTemplateMongoElement.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@ public class AgentTemplateMongoElement
{
public string Name { get; set; } = default!;
public string Content { get; set; } = string.Empty;
public string? ResponseFormat { get; set; }
public AgentTemplateLlmConfigMongoModel? LlmConfig { get; set; }

public static AgentTemplateMongoElement ToMongoElement(AgentTemplate template)
Expand All @@ -16,7 +15,6 @@ public static AgentTemplateMongoElement ToMongoElement(AgentTemplate template)
{
Name = template.Name,
Content = template.Content,
ResponseFormat = template.ResponseFormat,
LlmConfig = AgentTemplateLlmConfigMongoModel.ToMongoModel(template.LlmConfig)
};
}
Expand All @@ -27,7 +25,6 @@ public static AgentTemplate ToDomainElement(AgentTemplateMongoElement mongoTempl
{
Name = mongoTemplate.Name,
Content = mongoTemplate.Content,
ResponseFormat = mongoTemplate.ResponseFormat,
LlmConfig = AgentTemplateLlmConfigMongoModel.ToDomainModel(mongoTemplate.LlmConfig)
};
}
Expand Down
1 change: 0 additions & 1 deletion src/Plugins/BotSharp.Plugin.MongoStorage/Repository/MongoRepository.Agent.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -592,7 +592,6 @@ public async Task<bool> PatchAgentTemplate(string agentId, AgentTemplate templat
}

foundTemplate.Content = template.Content;
foundTemplate.ResponseFormat = template.ResponseFormat;
foundTemplate.LlmConfig = AgentTemplateLlmConfigMongoModel.ToMongoModel(template.LlmConfig);
var update = Builders<AgentDocument>.Update.Set(x => x.Templates, agent.Templates);
await _dc.Agents.UpdateOneAsync(filter, update);
Expand Down
3 changes: 2 additions & 1 deletion src/Plugins/BotSharp.Plugin.OpenAI/Providers/Chat/ChatCompletionProvider.Chat.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -628,7 +628,8 @@ private ChatCompletionOptions InitChatCompletionOption(Agent agent)

if (webSearchOptions == null)
{
options.ResponseFormat = GetChatResponseFormat(agent.LlmConfig?.ResponseFormat);
var format = _state.GetState("response_format").IfNullOrEmptyAs(agent.LlmConfig?.ResponseFormat);
options.ResponseFormat = GetChatResponseFormat(format);
}

return options;
Expand Down
20 changes: 12 additions & 8 deletions src/Plugins/BotSharp.Plugin.OpenAI/Providers/Chat/ChatCompletionProvider.Response.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -435,6 +435,7 @@ private async Task<RoleDialogModel> InnerCreateResponseStreamingAsync(Agent agen
MaxOutputTokenCount = maxTokens,
};

// Reasoning level
var reasoningEffortLevel = ParseResponseReasoning(settings?.Reasoning, agent);
if (reasoningEffortLevel.HasValue)
{
Expand All @@ -451,14 +452,7 @@ private async Task<RoleDialogModel> InnerCreateResponseStreamingAsync(Agent agen
}

// Response format
var textFormat = GetResponseTextFormat(agent.LlmConfig?.ResponseFormat);
if (textFormat != null)
{
options.TextOptions = new ResponseTextOptions
{
TextFormat = textFormat
};
}
SetResponseFormat(options, agent.LlmConfig);

// Prepare instruction and functions
var renderData = agentService.CollectRenderData(agent);
Expand Down Expand Up @@ -817,5 +811,15 @@ private void AddResponseToolChoice(CreateResponseOptions options)
options.ToolChoice = ResponseToolChoice.CreateRequiredChoice();
}
}

private void SetResponseFormat(CreateResponseOptions options, AgentLlmConfig? llmConfig)
{
var format = _state.GetState("response_format").IfNullOrEmptyAs(llmConfig?.ResponseFormat);
var responseFormat = GetResponseTextFormat(format);
options.TextOptions = responseFormat != null ? new ResponseTextOptions
{
TextFormat = responseFormat
} : null;
}
#endregion
}
Loading