Skip to content

Upload files straight to storage through upload sessions - #20752

Draft
timkelty wants to merge 11 commits into
5.13from
direct-uploads-core
Draft

timkelty wants to merge 11 commits into
5.13from
direct-uploads-core

Conversation

@timkelty

@timkelty timkelty commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Description

An alternative to #20630, which only covers MCP. This PR covers web as well, which has been a long-standing request for both https://github.com/craftcms/aws-s3 and https://github.com/craftcms/cloud.

Upload sessions let files go straight from the browser or MCP client to storage, so they skip the request size limits (6 MB on Lambda). The design adheres to Craft 6's upload sessions wherever possible: the same names, parameters, response shapes and uploadsessions columns, so code written against it carries over.

sequenceDiagram
    participant Client as CP uploader / MCP client
    participant Craft
    participant Storage as Volume filesystem
    Client->>Craft: uploads/start {filename, size, folderId…}
    Craft->>Storage: temporaryUploadUrl("_upload-sessions/{id}.ext")
    Craft-->>Client: {id, transport: {type: put, options: {url, headers}}, urls}
    Client->>Storage: PUT file (or Craft's uploads/transfer if the URL is null)
    Client->>Craft: uploads/complete {id}
    Craft->>Storage: rename into the folder (or download + clean images/SVGs)
    Craft-->>Client: same result as assets/upload
Loading
 src/
+├── base/TemporaryUploadUrlInterface.php   # filesystems opt in by presigning a PUT URL
+├── services/Uploads.php                   # start / transfer / complete / claim / cancel, `uploadsessions` table (Craft 6's columns)
+├── uploads/AssetUploadHandler.php         # Craft 6's parameters, assets/upload's permissions and results
+├── events/AuthorizeUploadEvent.php        # guests are denied unless a handler allows them
+├── controllers/UploadsController.php      # guest `start` is rate-limited by IP
 ├── mcp/AssetUploads.php                   # assets_upload_url → Uploads::start(); replaces mcp/upload
 └── web/assets/
     ├── cp/src/js/
+    │   └── UploadSessionUploader.js       # createUploader() picks it for TemporaryUploadUrlInterface filesystems
+    └── uploads/
+        ├── UploadsAsset.php               # front-end bundle; sets the default url and csrfToken
+        ├── src/FileUpload.js              # one file's start → PUT → complete, as Craft 6's Craft.Uploads.FileUpload
+        └── src/uploads.js                 # <input data-upload-session> for plain front-end forms

Files are staged in the destination volume's own filesystem rather than a separate disk as in Craft 6, since Craft 5 can only copy between filesystems by downloading through PHP. Files are moved into place without being downloaded, unless Craft would clean them (front-end images, CP images while sanitizeCpImageUploads is on) or they're SVGs. Images moved into place get their dimensions, and JPEGs their alt text from XMP or IPTC metadata, from a bounded read of the start of the file, so they match what a local upload would save. Other volumes keep Craft.Uploader.

Text asset previews now read the file from a stream, rather than downloading a copy first.

Front-end forms and scripts can upload the same way by registering the craft\web\assets\uploads\UploadsAsset bundle:

{% do view.registerAssetBundle('craft\\web\\assets\\uploads\\UploadsAsset') %}

<form method="post">
  {{ actionInput('entries/save-entry') }}
  {{ csrfInput() }}
  <input type="file" name="fields[photos][]" multiple data-upload-session='{"fieldId": 12}'>
  <button>Save</button>
</form>
on change(input[data-upload-session])
  for each file, in the order chosen
    add <input type=hidden name="fields[photos][]">        # keeps the order
    new Craft.Uploads.FileUpload(file, {parameters}).upload()
      ok    → hidden.value = assetId
      error → remove hidden; input.setCustomValidity(message); dispatch 'uploaderror'
  drop the file input's name, so files are never posted

on submit(form)
  if uploads are pending → wait, then requestSubmit()

Craft.Uploads.FileUpload has Craft 6's constructor (file, {url, csrfToken, parameters, onProgress, onStateChange}), upload() and cancel(). The bundle sets the default url and csrfToken. Guests still need an Uploads::EVENT_AUTHORIZE_UPLOAD handler to allow them.

Evidence

Uploading an 8 MB file from the asset index, with PHP's built-in server limiting requests to 2 MB:

  • Before: Craft.Uploader (any volume today) posts the file to assets/upload, so it's refused: “The file “big-before.txt” could not be uploaded, because it exceeds the maximum upload size of 2.0 MB.”
    After: on a volume whose filesystem implements TemporaryUploadUrlInterface, Craft only sees two small requests, and the asset is saved:

    POST  craft    uploads/start          67 B
    PUT   storage  _upload-sessions/…txt  8,000,000 B
    POST  craft    uploads/complete       2 B
    → asset big-after.txt, 8,000,000 B; _upload-sessions/ empty
    
  • Front-end form, against a stub of uploads/start, a presigned storage URL and uploads/complete, choosing two files and clicking Save right away:

    • Before: the form posts the files themselves in a multipart body, through Craft's request size limit.
      After: each file is PUT straight to storage with its signed headers (the CSRF token isn't sent there), and the form waits for both uploads, then posts only asset IDs, in the order the files were chosen:

      POST  uploads/start  ×2   (X-CSRF-Token)
      PUT   storage      ×2   11 B, 13 B   (x-amz-meta-test: yes, no CSRF token)
      POST  uploads/complete ×2
      POST  /submit      fields[photos][]=102&fields[photos][]=103&title=Hello
      
    • A disallowed .exe shows Craft's message as the input's validation message (“exe” is not an allowed file extension.), and no asset ID is posted for it.

  • Tests (tests/unit/services/UploadsTest.php, plus the updated MCP tests):

    start → PUT → complete     moves the staged file into the folder, completing again returns the same result
    claim twice                second claim fails; a file re-sent to the URL is deleted when the session expires
    wrong size / not sent      complete fails and the staged file is deleted
    other user, guest          can't complete; guests are denied unless EVENT_AUTHORIZE_UPLOAD allows them
    transfer                   one file per URL, bounded to the declared size
    JPEG moved into place      dimensions and XMP alt text read past 120 KB of metadata, XMP preferred over an earlier IPTC caption
    

Related issues

timkelty and others added 2 commits November 17, 2026 06:54
Uploads start with a URL that sends the file straight to storage when the
staging filesystem implements TemporaryUploadUrlInterface, or to Craft's
uploads/transfer action otherwise. Asset uploads are staged in the
destination volume's filesystem and moved into place, so files Craft
doesn't need to clean are never downloaded. Sessions are stored in an
uploadsessions table that matches Craft 6's.

MCP's assets_upload_url now uses the service instead of its own records
and upload action.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-01a1e63f-d13c-70f3-bc1e-77416ea26a1e
Craft.UploadSessionUploader uploads files for asset indexes, Assets fields
and the Replace file action when the volume's filesystem implements
TemporaryUploadUrlInterface. Other volumes keep Craft.Uploader.

uploads/start now returns complete and cancel URLs, and replacing a file
returns the same data as assets/replace-file.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-4f59cd79-3305-42fc-9fa3-1c216fd7c31c

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Publicly served staging can enable stored XSS through unsanitized SVGs, and concurrent upload sessions can overwrite each other’s files.

5 open issues
What changed in this PR

Introduces shared upload sessions for control-panel and MCP uploads, allowing supported filesystems to receive files directly and bypass Craft’s request-size limits. The API and database structure follow Craft 6’s upload-session design.

Changes:

  • Adds upload-session storage, authorization, completion, cancellation, and expiry cleanup.
  • Integrates direct uploads into the control panel and MCP, with a fallback through Craft.
  • Adds filesystem extension points and tests for upload lifecycle, ownership, and asset handling.
File Description
tests/​unit/​services/​UploadsTest.php Tests session lifecycle, permissions, validation, and asset creation.
tests/​unit/​services/​uploads/​DirectUploadFs.php Provides a simulated direct-upload filesystem.
tests/​unit/​mcp/​AssetUploadsTest.php Updates mocks to use the uploads service.
tests/​unit/​mcp/​AssetFieldUploadTest.php Updates MCP transfer and cross-server tests.
tests/​functional/​McpServersCest.php Tests the replacement transfer endpoint.
src/​web/​assets/​uploads/​src/​FileUpload.js Implements the client-side start, transfer, and complete sequence.
src/​web/​assets/​cp/​src/​js/​UploadSessionUploader.js Adapts upload sessions to control-panel uploader events.
src/​web/​assets/​cp/​src/​js/​Craft.js Selects session uploads for compatible filesystems.
src/​web/​assets/​cp/​src/​Craft.js Imports the new control-panel uploader.
src/​web/​assets/​cp/​CpAsset.php Exposes compatible filesystem types and upload limits.
src/​uploads/​UploadHandlerInterface.php Defines upload authorization and completion contracts.
src/​uploads/​TemporaryUploadHandler.php Authorizes temporary uploads for later claiming.
src/​uploads/​StagedUpload.php Represents staged files and supports bounded local copying.
src/​uploads/​AssetUploadHandler.php Resolves destinations and creates or replaces assets.
src/​services/​Uploads.php Manages session lifecycle, transfers, ownership, and cleanup.
src/​services/​Gc.php Adds expired-upload cleanup to garbage collection.
src/​migrations/​m261117_000000_upload_sessions.php Creates the upload-session table and expiry index.
src/​migrations/​Install.php Creates upload-session storage on fresh installations.
src/​mcp/​capabilities/​Assets.php Adds optional upload size and signed-header guidance.
src/​mcp/​AssetUploads.php Delegates MCP upload reservation and claiming to sessions.
src/​events/​AuthorizeUploadEvent.php Exposes upload authorization decisions to event handlers.
src/​elements/​Asset.php Handles replacement failures without legacy uploader data.
src/​db/​Table.php Defines the upload-session table constant.
src/​controllers/​UploadsController.php Adds session endpoints and guest-start rate limiting.
src/​controllers/​McpController.php Removes the superseded MCP upload action.
src/​config/​app.php Registers the uploads service and advances the schema version.
src/​base/​TemporaryUploadUrlInterface.php Defines the filesystem contract for direct PUT uploads.
src/​base/​ApplicationTrait.php Exposes the uploads service through the application.
CHANGELOG-WIP.md Documents upload-session features and API changes.

Review effort: Balanced

Comment thread src/services/Uploads.php Resolved
Comment thread src/uploads/AssetUploadHandler.php Resolved
Comment thread src/services/Uploads.php Resolved
Comment thread src/uploads/AssetUploadHandler.php Resolved
timkelty and others added 5 commits November 17, 2026 07:14
Staged files have no extension, and SVGs are signed as binary data, so
storage doesn't serve unsanitized files as SVGs or pages before Craft
checks them. SVGs are always saved from a local copy, which also reads
their dimensions.

Canceled uploads are kept until they expire, since direct upload URLs
can't be revoked, so garbage collection deletes files sent afterward.
Moving files into a folder is locked per folder, so two uploads can't
pick the same filename. The client completes an upload when Craft's
transfer URL reports it already has the file, and sends the file again
when completion reports it missing.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-4f59cd79-3305-42fc-9fa3-1c216fd7c31c
- Upload actions allow guests per action, so control panel requests
  still need accessCp, and only transfer works while the system is off.
- Guests are denied before their destination is resolved when no
  EVENT_AUTHORIZE_UPLOAD handler is registered.
- Guests can start 60 uploads per minute per IP, as in Craft 6.
- Images, and files plugins handle with EVENT_BEFORE_HANDLE_FILE, are
  saved from a local copy, so their dimensions, alt text and MIME type
  are read as assets/upload reads them.
- Moves that silently fail, and saves that fail after a move, no longer
  leave files without assets, and a lost transferred file can be sent
  again.
- uploads/start rejects requests that send both folderId and fieldId,
  and filenames that aren't strings.
- The control panel uploader cancels uploads that fail.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-4f59cd79-3305-42fc-9fa3-1c216fd7c31c
Images are moved into place again unless Craft would clean them (every
front-end upload, and control panel uploads while sanitizeCpImageUploads
is on) or they're SVGs, so sites that turn cleaning off don't download
each image back from storage. Their dimensions are read from the start
of the file.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-4f59cd79-3305-42fc-9fa3-1c216fd7c31c
Images that upload sessions move into place within their filesystem now get alt
text from their XMP or IPTC metadata, read from the start of the file the way
their dimensions are. Only JPEGs are read, since other formats can store
metadata after the image data. Asset and the upload handler share the XMP and
IPTC parsing.

Amp-Thread-ID: https://ampcode.com/threads/T-101db22e-bfb1-424e-bb06-a7f398e8a8c1
Co-authored-by: Amp <amp@ampcode.com>
@timkelty
timkelty removed this pull request from stack #20756 November 17, 2026 17:29
timkelty and others added 4 commits November 17, 2026 17:37
The UploadsAsset bundle defines Craft.Uploads.FileUpload, which follows
Craft 6's client, and uploads files chosen in file inputs with a
data-upload-session attribute, replacing them with asset ID inputs. Forms
wait for uploads to finish before they submit.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-4f59cd79-3305-42fc-9fa3-1c216fd7c31c
…orms

Repeated submits while uploads are pending now submit once, generated
asset ID inputs keep the file input's form attribute, resetting the form
discards uploads and their asset IDs, a replaced selection's uploads no
longer affect the input, and required inputs are satisfied by their
uploads.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-4f59cd79-3305-42fc-9fa3-1c216fd7c31c
Uploads read the form's own CSRF input when it has one, since the
token written into the page is stale on statically cached pages. Each
input's files upload one at a time, as in the control panel, a new
selection replaces the previous one as a plain file input would, and
failed uploads are canceled so their files aren't left staged.

Co-authored-by: Amp <amp@ampcode.com>
Amp-Thread-ID: https://ampcode.com/threads/T-4f59cd79-3305-42fc-9fa3-1c216fd7c31c

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants