Skip to content

Use a more resilient s3 object name by default #606

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

Use a more resilient s3 object name by default #606

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3878f7c
Use a more resilient s3 object name by default
worr Oct 22, 2020
09fd13d
Merge branch 'master' into feature/fluentd-s3-efficient
ahma Nov 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions pkg/sdk/model/output/s3.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,8 +127,8 @@ type S3OutputConfig struct {
GrantFullControl string `json:"grant_full_control,omitempty"`
// The length of `%{hex_random}` placeholder(4-16)
HexRandomLength string `json:"hex_random_length,omitempty"`
// The format of S3 object keys (default: %{path}%{time_slice}_%{index}.%{file_extension})
S3ObjectKeyFormat string `json:"s3_object_key_format,omitempty"`
// The format of S3 object keys (default: %{path}%{time_slice}_%{uuid_hash}_%{index}.%{file_extension})
S3ObjectKeyFormat string `json:"s3_object_key_format,omitempty" plugin:"default:%{path}%{time_slice}_%{uuid_hash}_%{index}.%{file_extension}"`
// S3 bucket name
S3Bucket string `json:"s3_bucket"`
// Archive format on S3
Expand Down
1 change: 1 addition & 0 deletions pkg/sdk/model/output/s3_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ buffer:
@id test
path logs/${tag}/%Y/%m/%d/
s3_bucket logging-amazon-s3
s3_object_key_format %{path}%{time_slice}_%{uuid_hash}_%{index}.%{file_extension}
s3_region eu-central-1
<buffer tag,time>
@type file
Expand Down
3 changes: 3 additions & 0 deletions pkg/sdk/model/render/fluent_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -479,6 +479,7 @@ func TestRenderS3(t *testing.T) {
@id test
path /var/buffer
s3_bucket test_bucket
s3_object_key_format %{path}%{time_slice}_%{uuid_hash}_%{index}.%{file_extension}
<buffer tag,time>
@type file
path asd
Expand All @@ -502,6 +503,7 @@ func TestRenderS3(t *testing.T) {
@id test
path /var/buffer
s3_bucket test_bucket
s3_object_key_format %{path}%{time_slice}_%{uuid_hash}_%{index}.%{file_extension}
<instance_profile_credentials>
</instance_profile_credentials>`,
},
Expand All @@ -519,6 +521,7 @@ func TestRenderS3(t *testing.T) {
@id test
path /var/buffer
s3_bucket test_bucket
s3_object_key_format %{path}%{time_slice}_%{uuid_hash}_%{index}.%{file_extension}
<shared_credentials>
path e
profile_name f
Expand Down