feat: add SFTP as a backup destination provider#3877
feat: add SFTP as a backup destination provider#3877toxfox69 wants to merge 2 commits intoDokploy:canaryfrom
Conversation
- Add sftpHost, sftpPort (default 22), sftpUser, sftpPassword, sftpPath columns to destination table schema - Add getSftpCredentials() building rclone sftp:// flags from destination - Add getRcloneCredentials() and getRcloneDestinationPath() helpers that dispatch to s3 or sftp based on destination.provider - Add migration 0148_sftp_destination.sql for new columns Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
| export const getRcloneCredentials = ( | ||
| destination: Destination, | ||
| ): { flags: string[]; backend: string } => { | ||
| if (destination.provider === "sftp") { | ||
| return { flags: getSftpCredentials(destination), backend: "sftp" }; | ||
| } | ||
| return { flags: getS3Credentials(destination), backend: "s3" }; | ||
| }; | ||
|
|
||
| export const getRcloneDestinationPath = ( | ||
| destination: Destination, | ||
| remotePath: string, | ||
| ): string => { | ||
| if (destination.provider === "sftp") { | ||
| const basePath = (destination.sftpPath ?? "").replace(/\/+$/, ""); | ||
| return `:sftp:${basePath}/${remotePath}`; | ||
| } | ||
| return `:s3:${destination.bucket}/${remotePath}`; | ||
| }; |
There was a problem hiding this comment.
The new getRcloneCredentials and getRcloneDestinationPath dispatcher functions (lines 91–109) are defined here but never imported or used anywhere in the codebase. Every backup file (postgres.ts, mysql.ts, mariadb.ts, mongo.ts, compose.ts, web-server.ts, index.ts) still imports and calls getS3Credentials directly and hardcodes :s3: paths.
For example, in postgres.ts (line 31–32):
const rcloneFlags = getS3Credentials(destination); // never routes to SFTP
const rcloneDestination = `:s3:${destination.bucket}/...`; // always S3All backup execution and cleanup call sites must be updated to use the new dispatcher functions for SFTP support to work end-to-end.
| export const getSftpCredentials = (destination: Destination) => { | ||
| const { sftpHost, sftpPort, sftpUser, sftpPassword } = destination; | ||
| return [ | ||
| `--sftp-host="${sftpHost}"`, | ||
| `--sftp-port="${sftpPort ?? 22}"`, | ||
| `--sftp-user="${sftpUser}"`, | ||
| `--sftp-pass="${sftpPassword}"`, | ||
| "--sftp-pass-is-base64=false", | ||
| ]; |
There was a problem hiding this comment.
--sftp-pass-is-base64=false is not a valid rclone SFTP flag. More critically, rclone's --sftp-pass expects a password that has been pre-obscured using rclone obscure <password>, not plaintext. Passing a raw plaintext password will result in rclone failing to authenticate.
The password must be obscured before being embedded in the flag—either store it already-obscured in the database, or obscure it at runtime via a subprocess call to rclone obscure.
| export const getSftpCredentials = (destination: Destination) => { | ||
| const { sftpHost, sftpPort, sftpUser, sftpPassword } = destination; | ||
| return [ | ||
| `--sftp-host="${sftpHost}"`, | ||
| `--sftp-port="${sftpPort ?? 22}"`, | ||
| `--sftp-user="${sftpUser}"`, | ||
| `--sftp-pass="${sftpPassword}"`, | ||
| "--sftp-pass-is-base64=false", | ||
| ]; |
There was a problem hiding this comment.
sftpHost, sftpUser, and sftpPassword are all nullable columns (no .notNull() constraint), so when null they are interpolated as the literal strings "null" in template literals. This produces invalid rclone flags like --sftp-host="null" and --sftp-user="null", causing silent rclone failures.
Add explicit null-checks before constructing flags:
if (!sftpHost || !sftpUser || !sftpPassword) {
throw new Error("SFTP destination is missing required credentials");
}Alternatively, add .notNull() constraints to these columns in the schema if they are required for SFTP destinations.
…ling - Replace direct getS3Credentials + hardcoded :s3: paths with getRcloneCredentials/getRcloneDestinationPath dispatchers in all 7 backup files (postgres, mysql, mariadb, mongo, compose, web-server, index/keepLatestNBackups) - Fix getSftpCredentials: use rclone obscure for password (rclone requires obscured passwords for --sftp-pass flag), remove invalid --sftp-pass-is-base64 flag - Add null checks: throw if sftpHost/sftpUser/sftpPassword are missing - Remove unused path import from index.ts Addresses all 3 issues from Greptile review.
Summary
Adds SFTP as a backup destination provider alongside existing S3 support.
Closes #416
Changes
packages/server/src/db/schema/destination.ts): AddedsftpHost,sftpPort(default 22),sftpUser,sftpPassword, andsftpPathcolumns to the destination table. Updated Zod validation schemas for create/update operations.packages/server/src/utils/backups/utils.ts): AddedgetSftpCredentials()to build rclone SFTP flags,getRcloneCredentials()dispatcher that routes to S3 or SFTP based ondestination.provider, andgetRcloneDestinationPath()for provider-aware remote path construction.apps/dokploy/drizzle/0148_sftp_destination.sql): ALTER TABLE statements adding the new SFTP columns.How it works
Uses rclone's built-in SFTP backend (
--sftp-host,--sftp-port,--sftp-user,--sftp-passflags), same approach as the existing S3 integration. Thessh2dependency is already in the project.Test plan
🤖 Generated with TIAMAT — autonomous AI agent by ENERGENAI LLC
Greptile Summary
This PR adds SFTP as a backup destination provider with schema, migration, and helper functions. However, the implementation is incomplete and contains critical issues that prevent SFTP backups from working.
Key issues:
Dead code — The new
getRcloneCredentialsandgetRcloneDestinationPathdispatcher functions are defined but never imported or called. Every backup file (postgres.ts,mysql.ts,mariadb.ts,mongo.ts,compose.ts,web-server.ts,index.ts) still hardcodes S3 credentials and:s3:paths. SFTP destinations will always fall through to S3.Broken password handling —
--sftp-pass-is-base64=falseis not a valid rclone flag. rclone's SFTP backend requires passwords to be pre-obscured usingrclone obscurebefore being passed as--sftp-pass. Plaintext passwords will cause authentication failure at runtime.Null credential fields —
sftpHost,sftpUser, andsftpPasswordare nullable. When null, they are interpolated as the literal strings"null"in shell flags (e.g.,--sftp-host="null"), causing silent rclone errors.The database schema and migration are structurally sound. Validation logic appropriately keeps S3 fields required and SFTP fields optional for provider-aware input handling.
Confidence Score: 1/5
Last reviewed commit: 1d87c7f
(2/5) Greptile learns from your feedback when you react with thumbs up/down!
Context used:
dashboard- AGENTS.md (source)