Skip to content

Conversation

@filkeith
Copy link
Contributor

@filkeith filkeith commented Oct 19, 2025

  • added checksum check on upload for s3
  • added concurrent upload for s3
  • fixed single file backup

- some optimizations
- disabled checksum
- no checksum on restore
- returned back checskum as it doesn't affect performance
@filkeith filkeith force-pushed the FMWK-816-s3-optimizations branch from 94c806b to 0f4d896 Compare October 20, 2025 13:26
- starting from scratch
@filkeith filkeith force-pushed the FMWK-816-s3-optimizations branch from 0f4d896 to e6e45b1 Compare October 20, 2025 13:59
@codecov-commenter
Copy link

codecov-commenter commented Oct 21, 2025

Codecov Report

❌ Patch coverage is 65.62500% with 77 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.56%. Comparing base (67459ca) to head (5409a10).

Files with missing lines Patch % Lines
io/storage/aws/s3/writer.go 57.98% 42 Missing and 8 partials ⚠️
io/storage/common/retry_reader.go 38.88% 10 Missing and 1 partial ⚠️
io/storage/azure/blob/writer.go 37.50% 3 Missing and 2 partials ⚠️
io/storage/gcp/storage/writer.go 37.50% 3 Missing and 2 partials ⚠️
io/storage/common/writer.go 85.71% 2 Missing and 1 partial ⚠️
io/storage/local/writer.go 76.92% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #344      +/-   ##
==========================================
- Coverage   78.78%   78.56%   -0.22%     
==========================================
  Files         105      107       +2     
  Lines       10537    10681     +144     
==========================================
+ Hits         8302     8392      +90     
- Misses       1739     1786      +47     
- Partials      496      503       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- removed checksum validation on read
- fix single file backup
- test filepath for meta
- test single file backup with meta
- fix single file path
- fix single file backup
- fix range reader optimizations
@filkeith filkeith marked this pull request as ready for review October 21, 2025 14:15
@filkeith filkeith requested a review from reugn October 21, 2025 14:15
# Conflicts:
#	io/storage/aws/s3/writer.go
- set default value for s3 concurrency 1
@filkeith filkeith requested a review from reugn October 23, 2025 11:37
@reugn reugn changed the title FMWK-816 S3 optimizations FMWK-816 Optimize S3 operations Oct 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants