CLDSRV-899: Log replication events in server access logs by dvasilas · Pull Request #6164 · scality/cloudserver

dvasilas · 2026-05-04T13:53:26Z

Replication writes via the backbeat putData route are now visible in the destination bucket's server access logs.

Reads from the source bucket are already logged as REST.GET.OBJECT.

Study of AWS server access logging with replication: https://scality.atlassian.net/browse/S3C-11048?focusedCommentId=473763

bert-e · 2026-05-04T13:53:30Z

Hello dvasilas,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
`/after_pull_request`	Wait for the given pull request id to be merged before continuing with the current one.
`/bypass_author_approval`	Bypass the pull request author's approval	⭐
`/bypass_build_status`	Bypass the build and test status	⭐
`/bypass_commit_size`	Bypass the check on the size of the changeset `TBA`	⭐
`/bypass_incompatible_branch`	Bypass the check on the source branch prefix	⭐
`/bypass_jira_check`	Bypass the Jira issue check	⭐
`/bypass_peer_approval`	Bypass the pull request peers' approval	⭐
`/bypass_leader_approval`	Bypass the pull request leaders' approval	⭐
`/approve`	Instruct Bert-E that the author has approved the pull request.		✍️
`/create_pull_requests`	Allow the creation of integration pull requests.
`/create_integration_branches`	Allow the creation of integration branches.
`/no_octopus`	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
`/unanimity`	Change review acceptance criteria from `one reviewer at least` to `all reviewers`
`/wait`	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
`/help`	Print Bert-E's manual in the pull request.
`/status`	Print Bert-E's current status in the pull request `TBA`
`/clear`	Remove all comments from Bert-E from the history `TBA`
`/retry`	Re-start a fresh build `TBA`
`/build`	Re-start a fresh build `TBA`
`/force_reset`	Delete integration branches & pull requests, and restart merge process from the beginning.
`/reset`	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

bert-e · 2026-05-04T13:53:35Z

Incorrect fix version

The Fix Version/s in issue CLDSRV-899 contains:

None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

9.2.40
9.3.7
9.4.0

Please check the Fix Version/s of CLDSRV-899, or the target
branch of this pull request.

claude · 2026-05-04T13:55:00Z

LGTM

Clean, well-scoped change. The replication flag flows correctly from putData through getOperation and logServerAccess. The loggingEnabled guard correctly allows replication entries through alongside expiration. The synthesized requestURI matches the pattern used by getURI(). Tests cover the key scenarios including error paths. No issues found.

Review by Claude Code

codecov · 2026-05-04T13:55:16Z

Codecov Report

❌ Patch coverage is 80.55556% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 84.46%. Comparing base (22f67e0) to head (67a6c3d).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
lib/routes/routeBackbeat.js	41.66%	7 Missing ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
lib/utilities/serverAccessLogger.js	`84.87% <100.00%> (+1.47%)`	⬆️
lib/routes/routeBackbeat.js	`76.17% <41.66%> (-0.57%)`	⬇️

... and 1 file with indirect coverage changes

@@                 Coverage Diff                 @@
##           development/9.2    #6164      +/-   ##
===================================================
- Coverage            84.50%   84.46%   -0.05%     
===================================================
  Files                  204      204              
  Lines                13180    13216      +36     
===================================================
+ Hits                 11138    11163      +25     
- Misses                2042     2053      +11

Flag	Coverage Δ
file-ft-tests	`67.98% <13.88%> (-0.15%)`	⬇️
kmip-ft-tests	`28.42% <2.77%> (-0.08%)`	⬇️
mongo-v0-ft-tests	`69.25% <13.88%> (-0.12%)`	⬇️
mongo-v1-ft-tests	`69.26% <13.88%> (-0.16%)`	⬇️
multiple-backend	`35.69% <58.33%> (+0.06%)`	⬆️
sur-tests	`36.80% <2.77%> (-0.13%)`	⬇️
sur-tests-inflights	`37.81% <2.77%> (-0.10%)`	⬇️
unit	`70.30% <77.77%> (+0.02%)`	⬆️
utapi-v2-tests	`34.70% <2.77%> (-0.09%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

bert-e · 2026-05-07T11:16:30Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

claude · 2026-05-07T11:18:50Z

LGTM

Review by Claude Code

claude · 2026-05-07T15:46:04Z

LGTM

Review by Claude Code

bert-e · 2026-05-08T10:51:26Z

Incorrect fix version

The Fix Version/s in issue CLDSRV-899 contains:

9.2.40
9.3.7
9.4.0

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

9.2.40
9.3.8
9.4.0

Please check the Fix Version/s of CLDSRV-899, or the target
branch of this pull request.

claude · 2026-05-08T10:54:20Z

Typo in comment: "distinquish" → "distinguish" (routeBackbeat.js:540)
- One-click suggestion provided
Synthesized requestURI for replication uses URI-decoded objectKey, while normal S3 entries use raw HTTP-encoded URL — could cause inconsistent log parsing for keys with special characters (serverAccessLogger.js:660)
- Consider re-encoding the key to match non-replication entries

Review by Claude Code

claude · 2026-05-08T18:52:00Z

Typo in routeBackbeat.js:540 — "distinquish" should be "distinguish".
- Suggestion applied inline.
- Synthesized requestURI for replication logs uses decoded objectKey/bucketName, while normal requests use the raw URL-encoded request.url via getURI(). Object keys with special characters (spaces, unicode) will look different in replication vs. normal log entries.
- Consider wrapping the bucket and key with encodeURIComponent (or S3-style encoding) to match the non-replication format.

Review by Claude Code

claude · 2026-05-11T07:53:20Z

LGTM

Clean implementation. The replication-type detection in putMetadata correctly uses replicationInfo.status === 'REPLICA' to distinguish destination-side events. The loggingEnabled gate change is correct, and the URI synthesis with encodeURIComponent on the decoded objectKey produces proper S3-style log entries. Tests cover all four replication variants plus the error case.

Review by Claude Code

Replication writes via the backbeat putData route are now visible in bucket server access logs with operation = REST.PUT.OBJECT.

Destination-side delete-marker replication is now logged as REST.DELETE.OBJECT, same as AWS.

Destination-side PutObjectTagging and DeleteObjectTagging replication are now logged as REST.PUT.OBJECT_TAGGING (same as AWS). AWS replicates a DeleteObjectTagging on the destination as REST.PUT.OBJECT_TAGGING with an empty tag set URI: PUT /<bucket>/<key>?tagging&versionId=<vid>.

Destination-side PutObjectAcl replication is now logged as REST.PUT.ACL. AWS replicates a PutObjectAcl on the destination as REST.PUT.ACL with URI PUT /<bucket>/<key>?acl&versionId=<srcVersionId> and aclRequired = 'Yes'.

bert-e · 2026-05-11T08:48:46Z

Request integration branches

Waiting for integration branch creation to be requested by the user.

To request integration branches, please comment on this pull request with the following command:

/create_integration_branches

Alternatively, the /approve and /create_pull_requests commands will automatically
create the integration branches.

claude · 2026-05-11T08:49:08Z

LGTM

Review by Claude Code

claude · 2026-05-11T09:22:28Z

LGTM

Clean implementation with good test coverage. The replication sub-type detection in putMetadata correctly gates on REPLICA status to avoid false-flagging source-side updates. The URI synthesis in logServerAccess properly encodes object key segments and handles all four replication shapes (data, delete-marker, tagging, ACL). The loggingEnabled condition change correctly unblocks replication entries for log courier delivery while keeping other backbeat operations suppressed.

Review by Claude Code

claude · 2026-05-11T10:05:41Z

LGTM

Review by Claude Code

To reduce flakiness in mutliple-backend/Healthcheck response test.

claude · 2026-05-11T10:11:04Z

LGTM

Review by Claude Code

dvasilas force-pushed the improvement/CLDSRV-899-log-replication-events branch from b922c2f to 803285c Compare May 7, 2026 11:16

dvasilas force-pushed the improvement/CLDSRV-899-log-replication-events branch from 0c52210 to 812257c Compare May 8, 2026 10:51

claude Bot reviewed May 8, 2026

View reviewed changes

Comment thread lib/routes/routeBackbeat.js Outdated

claude Bot reviewed May 8, 2026

View reviewed changes

Comment thread lib/utilities/serverAccessLogger.js Outdated

dvasilas force-pushed the improvement/CLDSRV-899-log-replication-events branch from 812257c to 2c9ff6d Compare May 8, 2026 18:49

claude Bot reviewed May 8, 2026

View reviewed changes

Comment thread lib/routes/routeBackbeat.js Outdated

claude Bot reviewed May 8, 2026

View reviewed changes

Comment thread lib/utilities/serverAccessLogger.js Outdated

dvasilas force-pushed the improvement/CLDSRV-899-log-replication-events branch from 2c9ff6d to 7c20606 Compare May 11, 2026 07:49

dvasilas added 4 commits May 11, 2026 11:46

CLDSRV-899: Log replication events in server access logs

108d895

Replication writes via the backbeat putData route are now visible in bucket server access logs with operation = REST.PUT.OBJECT.

CLDSRV-899: Log delete-marker replication events

993dfe0

Destination-side delete-marker replication is now logged as REST.DELETE.OBJECT, same as AWS.

CLDSRV-899: Log put-acl replication events

4a52bdf

Destination-side PutObjectAcl replication is now logged as REST.PUT.ACL. AWS replicates a PutObjectAcl on the destination as REST.PUT.ACL with URI PUT /<bucket>/<key>?acl&versionId=<srcVersionId> and aclRequired = 'Yes'.

dvasilas force-pushed the improvement/CLDSRV-899-log-replication-events branch from 7c20606 to d40302d Compare May 11, 2026 08:46

dvasilas added 2 commits May 11, 2026 13:07

CLDSRV-899: Bump metadata-standalone image to 9.11.0

e2290c4

CLDSRV-899: Raise Healthcheck response test timeout to 60s

43f0304

To reduce flakiness in mutliple-backend/Healthcheck response test.

CLDSRV-899: Bump version to 9.2.40

67a6c3d

dvasilas force-pushed the improvement/CLDSRV-899-log-replication-events branch from fcf7e61 to 67a6c3d Compare May 11, 2026 10:08

BourgoisMickael approved these changes May 11, 2026

View reviewed changes

Conversation

dvasilas commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bert-e commented May 4, 2026

Hello dvasilas,

Uh oh!

bert-e commented May 4, 2026

Incorrect fix version

Uh oh!

claude Bot commented May 4, 2026

Uh oh!

codecov Bot commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

bert-e commented May 7, 2026

Request integration branches

Uh oh!

claude Bot commented May 7, 2026

Uh oh!

claude Bot commented May 7, 2026

Uh oh!

bert-e commented May 8, 2026

Incorrect fix version

Uh oh!

Uh oh!

Uh oh!

claude Bot commented May 8, 2026

Uh oh!

Uh oh!

Uh oh!

claude Bot commented May 8, 2026

Uh oh!

claude Bot commented May 11, 2026

Uh oh!

bert-e commented May 11, 2026

Request integration branches

Uh oh!

claude Bot commented May 11, 2026

Uh oh!

claude Bot commented May 11, 2026

Uh oh!

claude Bot commented May 11, 2026

Uh oh!

claude Bot commented May 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dvasilas commented May 4, 2026 •

edited

Loading

codecov Bot commented May 4, 2026 •

edited

Loading