Skip to content

Agent-chat HTML: render image DataContent as <img> pointing at hosted image endpoint #1219

Description

@JoshuaRowePhantom

Part of #1216

Summary

Change the agent-chat HTML transformer so that each image AIContent (DataContent with an image/* media type) renders as a small <img> preview pointing at the hosted image endpoint added in sub-item B, wrapped in an anchor so clicking opens the full-size image. Do NOT inline base64 into the HTML fragment stream. Must work for BOTH user-pasted images AND assistant/provider-generated images because both flow through identical AIContent / persistence / rendering paths.

Cross-references: #1202 / #1197 — recent agent-chat HTML pipeline work around RunningSubAgentsHtmlTransformer, ChatOutputHtmlRenderer, and chat-output-shell.html. This change lives in that same pipeline.

Dependencies

Root Cause / Current Behavior

The chat WebView loads embedded features/Phantom.Workspaces.Agent.Gui/Assets/chat-output-shell.html from AgentChatOutputControl.axaml.cs:189-199 and receives per-content HTML fragments through IChatOutputHtmlSink.UpdateContent(path, location, content) (AgentChatOutputControl.axaml.cs:128-130). The switch on AIContent type lives in features/Phantom.Workspaces.Agent.Gui/ViewModels/DocumentModels/ChatOutputHtmlRenderer.cs in RenderContent(...) around :439-508. Today the DataContent case (:497-500) renders only a text label of the media type:

case DataContent data:
    return IsImageMediaType(data.MediaType)
        ? TextBlock(contentId, "chat-meta",
            string.IsNullOrWhiteSpace(data.MediaType) ? "image" : data.MediaType,
            SerializeContentJson(data))
        : TextBlock(contentId, "chat-monospace",
            string.IsNullOrWhiteSpace(data.MediaType) ? "[data]" : $"[{data.MediaType}]",
            SerializeContentJson(data));

Content ids are already stable within a rendered session: ChatOutputHtmlRenderer.MessageId(historyIndex) => "history-{i}" and ContentId(messageId, subIndex) => "{messageId}-{subIndex}" (:60, :68). ChatMessageHtmlModel.Render (features/Phantom.Workspaces.Agent.Gui/ViewModels/DocumentModels/ChatOutputHtmlModels.cs:195-210) already knows the message index but does NOT know the AgentSessionId or the persistence Sequence — both of which the <img> URL needs.

Assistant-image coverage. ChatMessageHtmlModel.Render (ChatOutputHtmlModels.cs:201-210) uses Role only to pick styling flags (isDiagnostic, isHelp); every AIContent on every role runs through the same RenderContent switch, and MongoDbAgentPersistenceStore.StoreAsync serializes assistant ChatMessages the same way as user messages. Therefore provider-generated DataContent images land on the same branch with no additional code — but the test suite must assert this explicitly.

Affected Files

File Contribution
features/Phantom.Workspaces.Agent.Gui/ViewModels/DocumentModels/ChatOutputHtmlRenderer.cs Extend RenderContent signature to accept string agentSessionId, long messageSequence, int contentIndex. Replace the DataContent image branch (:497-500) with <a class="chat-image-link"><img class="chat-image-preview" src=".../image" ...></a>. Keep the non-image DataContent branch as-is.
features/Phantom.Workspaces.Agent.Gui/ViewModels/DocumentModels/ChatOutputHtmlModels.cs Thread agentSessionId + message Sequence from ChatMessageHtmlModel.Render (:195-210) into RenderContent. Pass loop index as contentIndex.
features/Phantom.Workspaces.Llm.Core/AgentChatHistoryItem.cs Add long? Sequence { get; init; } so the in-memory history item carries its persistence sequence (needed to build the URL). Populated at message-store time from the value returned by the persistence write.
features/Phantom.Workspaces.Agent.Gui/Assets/chat-output-shell.html Add .chat-image-preview / .chat-image-link CSS: max-width: 240px; max-height: 240px; object-fit: contain; border-radius: 4px; cursor: zoom-in; on the img; display: inline-block; on the link.

Design / Fix

Renderer change (sketch)

ChatOutputHtmlRenderer.RenderContent gains string agentSessionId, long messageSequence, int contentIndex:

case DataContent data when IsImageMediaType(data.MediaType):
{
    var mediaType = string.IsNullOrWhiteSpace(data.MediaType) ? "image" : data.MediaType!;
    var url = $"/agent/persistence/{Uri.EscapeDataString(agentSessionId)}"
            + $"/message/{messageSequence}/content/{contentIndex}/image";
    return $$"""
        <a class="chat-image-link" href="{{url}}" target="_blank" rel="noopener"
           data-details-target="{{contentId}}">
          <img class="chat-image-preview" src="{{url}}"
               alt="{{HtmlEncode(mediaType)}}" loading="lazy" />
        </a>
        """;
}
case DataContent data:
    return TextBlock(contentId, "chat-monospace",
        string.IsNullOrWhiteSpace(data.MediaType) ? "[data]" : $"[{data.MediaType}]",
        SerializeContentJson(data));

Click-to-full is handled by the existing anchor-click interceptor already exercised by AgentChatOutputControlTests.ChatOutputShellHtml_ContainsAnchorClickInterceptor (AgentChatOutputControlTests.cs:23), which routes anchor clicks through the WebView's URL-open path (raising UrlNavigationRequested). No new JavaScript required for the MVP; a follow-up may add an in-shell lightbox overlay.

CSS in chat-output-shell.html

.chat-image-preview {
    max-width: 240px;
    max-height: 240px;
    object-fit: contain;
    border-radius: 4px;
    cursor: zoom-in;
}
.chat-image-link { display: inline-block; }

Threading identity into the renderer

  • AgentChatHistoryItem gains long? Sequence { get; init; } (populated at message-store time).
  • ChatMessageHtmlModel.Render (ChatOutputHtmlModels.cs:195-210) passes agentSessionId (available from the owning session) and the item's Sequence into each RenderContent call along with the loop index subIndex as contentIndex.
  • If Sequence is null (should not happen post-persistence), fall back to the existing text-label branch to avoid emitting a broken URL.

Explicitly avoided

Do NOT emit src="data:image/png;base64,…". A single 4K-screenshot paste can exceed 5–10 MB; multiplying by every render/replace via IChatOutputHtmlSink.UpdateContent and by DOM parsing time in the WebView is prohibitive. Using an HTTP URL keeps the HTML fragment small, lets the WebView cache, and defers the payload to a background GET.

Coverage for generated (assistant) images

Because ChatMessageHtmlModel.Render (ChatOutputHtmlModels.cs:201-210) invokes RenderContent for every AIContent regardless of Role, and MongoDbAgentPersistenceStore.StoreAsync persists assistant messages the same way, provider-generated DataContent items land on this same branch with no additional code. Tests must assert this explicitly.

Expected Tests

Class names verified against existing suites (ChatOutputHtmlRendererTests, AgentChatOutputControlTests). Naming style Subject_Scenario_ExpectedOutcome.

Test Name Class What It Verifies
ChatHtml_WhenMessageHasImageContent_EmitsImgTagPointingAtHostEndpoint ChatOutputHtmlRendererTests Rendered fragment contains <img class="chat-image-preview" src="/agent/persistence/{sessionId}/message/{seq}/content/{idx}/image" with the correct sessionId / seq / idx values.
ChatHtml_WhenMessageHasImageContent_WrapsImgInAnchorForFullSize ChatOutputHtmlRendererTests Fragment includes surrounding <a class="chat-image-link" href="{same-url}" target="_blank" rel="noopener">.
ChatHtml_WhenMessageHasImageContent_DoesNotInlineBase64 ChatOutputHtmlRendererTests Emitted fragment MUST NOT contain data:image/ or the base64 payload.
ChatHtml_WhenAssistantGeneratesImage_RendersPreview ChatOutputHtmlRendererTests With ChatRole.Assistant + DataContent(image/png), the same <img> output is produced (role-agnostic).
ChatHtml_WhenDataContentIsNonImage_RendersMonospaceLabel ChatOutputHtmlRendererTests Non-image DataContent still uses the existing [mediaType] monospace branch — unchanged behavior.
ChatHtml_WhenMessageSequenceMissing_FallsBackToTextLabel ChatOutputHtmlRendererTests If AgentChatHistoryItem.Sequence is null (no persistence key yet), the renderer emits the pre-existing text label instead of a broken URL.
ChatHtml_EscapesAgentSessionIdInUrl ChatOutputHtmlRendererTests An agentSessionId containing URL-unsafe characters is percent-encoded via Uri.EscapeDataString in the emitted URL.
ChatOutputShellHtml_HasImagePreviewMaxSizeCss AgentChatOutputControlTests Shell HTML resource contains .chat-image-preview { max-width: 240px; max-height: 240px; ... }.
AgentChatOutputControl_ImagePreviewClick_RaisesUrlNavigationRequested AgentChatOutputControlTests Click on the <a class="chat-image-link"> triggers the existing anchor interceptor, delivering the endpoint URL through UrlNavigationRequested (companion to existing AgentChatOutputControl_OpenUrlMessage_RaisesUrlNavigationRequested).

Metadata

Metadata

Assignees

No one assigned

    Labels

    diagnosedRoot cause identifiedenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions