Support drag/drop and copy paste of images into CLI

[ahmetus]

Describe the feature or problem you'd like to solve

Sending with drag and drop or attaching images from the project directory to copilot

Proposed solution

Most of the agentic cli tools support attaching images in some ways: Copy-paste from the clipboard, referencing from the project directory, drag and drop image to copilot. This is a must have feature, because it is so useful for debugging or feature request from the model.

All the models copilot provides support image understanding. This is the biggest bottleneck for daily usage.

Example prompts or workflows

No response

Additional context

No response

[AmedeoPelliccia]

@ahmetus thanks. Agreed this is high-impact. Proposal below to make image attachments first-class for Copilot.

UTCS: A Template for Image Prompts

A compact schema to keep every image input consistent, traceable and reproducible by structuring around:

• U (UiX metadata)
• T (Templated content)
• C (Context & cache)
• S (Structure & Style sheet)

---

Supported Inputs

• Drag & drop
• Clipboard paste
• Project path reference (e.g. ./assets/snapshots/login.png)

---

Why UTCS?

• All Copilot models support vision
• Debugging often starts with a screenshot
• Stable metadata → better reproducibility & audit

---

UTCS v1 Example

{
  "U": {
    "templateID": "utcs.image.v1",
    "version": "1.0.0",
    "createdBy": "Amedeo",
    "createdAt": "2025-10-20T07:00:00Z"
  },
  "T": {
    "description": "",
    "contentType": "screenshot",
    "annotations": [],
    "MCP": {
      "model": "copilot-image-v1",
      "context": "",
      "purpose": ""
    }
  },
  "C": {
    "projectName": "MyAppFrontend",
    "modulePath": "ui/components/LoginForm",
    "sessionID": "sess-456",
    "memoryKey": "ui-bug-363",
    "memoryScope": "session"
  },
  "S": {
    "maxDimensions": "1920x1080",
    "thumbnailSize": "400x225",
    "borderStyle": "1px solid #ccc",
    "annotateFont": "Arial 12px",
    "annotateColor": "#e74c3c"
  }
}

---

Typical Flow

1. User drops error-login.png into Copilot.
2. UI auto-fills U and C; user adds T.description, MCP.context, MCP.purpose.
3. Thumbnail renders per S rules.
4. Model ingests the image + UTCS JSON for focused analysis.

---

Key Decisions

• Formats (phase 1): PNG, JPEG, WebP, SVG
• MemoryScope: session | project | user
• Annotations: inline for quick markups; file-backed for reuse
• Sanitization pipeline:
  1. Format probe
  2. Content sniff
  3. UTCS schema validate
  4. PII filter

---

Limits & Caching

• Max file size: 15 MB
• Max dimension: 8K on long side (auto-downscale >4K)
• Dedupe by SHA-256; cache under ~/.copilot/cache/images/

---

IDE & CLI Integration

• VS Code / JetBrains: drag-drop, paste, “Attach from project,” styled thumbnail
• CLI:

  copilot attach ./path.png \
    --desc "Login error" \
    --module ui/Login \
    --purpose debug

---

Feedback Requested

1. Are these JSON fields and defaults sufficient?
2. Does the memoryScope model align with your workflows?
3. Any objections to the size/format limits or downscaling approach?
4. Thoughts on CLI syntax vs. UI hooks?

Looking forward to your thoughts before we prototype!

[ahmetus]

@ahmetus thanks. Agreed this is high-impact. Proposal below to make image attachments first-class for Copilot.

UTCS: A Template for Image Prompts

A compact schema to keep every image input consistent, traceable and reproducible by structuring around:

• U (UiX metadata)
• T (Templated content)
• C (Context & cache)
• S (Structure & Style sheet)

Supported Inputs

• Drag & drop
• Clipboard paste
• Project path reference (e.g. ./assets/snapshots/login.png)

Why UTCS?

• All Copilot models support vision
• Debugging often starts with a screenshot
• Stable metadata → better reproducibility & audit

UTCS v1 Example

{
"U": {
"templateID": "utcs.image.v1",
"version": "1.0.0",
"createdBy": "Amedeo",
"createdAt": "2025-10-20T07:00:00Z"
},
"T": {
"description": "",
"contentType": "screenshot",
"annotations": [],
"MCP": {
"model": "copilot-image-v1",
"context": "",
"purpose": ""
}
},
"C": {
"projectName": "MyAppFrontend",
"modulePath": "ui/components/LoginForm",
"sessionID": "sess-456",
"memoryKey": "ui-bug-363",
"memoryScope": "session"
},
"S": {
"maxDimensions": "1920x1080",
"thumbnailSize": "400x225",
"borderStyle": "1px solid #ccc",
"annotateFont": "Arial 12px",
"annotateColor": "#e74c3c"
}
}

Typical Flow

1. User drops error-login.png into Copilot.
2. UI auto-fills U and C; user adds T.description, MCP.context, MCP.purpose.
3. Thumbnail renders per S rules.
4. Model ingests the image + UTCS JSON for focused analysis.

Key Decisions

• Formats (phase 1): PNG, JPEG, WebP, SVG

• MemoryScope: session | project | user

• Annotations: inline for quick markups; file-backed for reuse

• Sanitization pipeline:

  1. Format probe
  2. Content sniff
  3. UTCS schema validate
  4. PII filter

Limits & Caching

• Max file size: 15 MB
• Max dimension: 8K on long side (auto-downscale >4K)
• Dedupe by SHA-256; cache under ~/.copilot/cache/images/

IDE & CLI Integration

• VS Code / JetBrains: drag-drop, paste, “Attach from project,” styled thumbnail
• CLI:
  copilot attach ./path.png
  --desc "Login error"
  --module ui/Login
  --purpose debug

Feedback Requested

1. Are these JSON fields and defaults sufficient?
2. Does the memoryScope model align with your workflows?
3. Any objections to the size/format limits or downscaling approach?
4. Thoughts on CLI syntax vs. UI hooks?

Looking forward to your thoughts before we prototype!

Well, it all looks good, the important part is working implementation, sooner is better, so we can comfortably debug :) Thanks..

[EvanBoyle]

Hey folks, thanks for the report. The good news is that our API and most of the agent loop already supports this. We just need to wire up drag/drop and clipboard events to handle images. We will review this in planning and provide an update.

[MRayermannMSFT]

Hi @ahmetus , @AmedeoPelliccia , and any other followers of this issue. The requested feature should now be available in CLI 0.0.362.

All of the following now results in an image being attached to your prompt:

• Copy & paste image data
  • On Windows and Linux use alt+v
  • On macOS use ctrl+v
• Drag and drop image file
• Copy & paste (normal keyboard shortcuts) image file (does not currently work with Finder on macOS)
• Copy & paste (normal keyboard shortcuts) path to image file

Please leave a comment here if you have any problems with these features. Thanks!

[ahmetus]

@MRayermannMSFT Thanks, it works with Fedora 42 - Gnome 48 using standart gnome terminal. Good job, the only missing important feature left is now structured json output for non-interactive usage. It must be like copilot -p "Fix the bug in main.js" --allow-all-tools --output json So we can use for other tasks programmatically in a more comfortable way like othe agentic cli tools(most of them supports this format)

[MRayermannMSFT]

@ahmetus thanks for trying it out! Happy to hear it worked. Regarding the structured json output request, thanks for sharing that. Will pass that along to the rest of the team. 😃

[EvanBoyle]

@ahmetus would you mind opening a separate issue for the JSON output request?

[MRayermannMSFT]

Thanks all for the confirmations and feedback. If you have any other issues regarding image support, or anything else for the CLI, please open a new issue.