Skip to content

Commit 714ccf8

Browse files
author
Andrew Gaul
committed
Allow non-AWS endpoints
This is useful for local Ceph S3 deployments. Fixes #10. Fixes #65.
1 parent f0365ed commit 714ccf8

File tree

3 files changed

+25
-1
lines changed

3 files changed

+25
-1
lines changed

CHANGELOG.md

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
1+
## 3.2.1
2+
- Support for non-AWS endpoints
3+
14
## 3.2.0
25
- Move to the new concurrency model `:single`
36
- use correct license identifier #99

CONTRIBUTORS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ Contributors:
1010
* Nick Ethier (nickethier)
1111
* Pier-Hugues Pellerin (ph)
1212
* Richard Pijnenburg (electrical)
13+
* Andrew Gaul (andrewgaul)
1314

1415
Note: If you've sent us patches, bug reports, or otherwise contributed to
1516
Logstash, and you aren't on the list above and want to be, please let us know

lib/logstash/outputs/s3.rb

Lines changed: 21 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
require "thread"
99
require "tmpdir"
1010
require "fileutils"
11+
require "uri"
1112

1213

1314
# INFORMATION:
@@ -59,9 +60,10 @@
5960
# This is an example of logstash config:
6061
# [source,ruby]
6162
# output {
62-
# s3{
63+
# s3 {
6364
# access_key_id => "crazy_key" (required)
6465
# secret_access_key => "monkey_access_key" (required)
66+
# endpoint => "http://127.0.0.1:8080" (optional, used for non-AWS endpoints, default = "")
6567
# region => "eu-west-1" (optional, default = "us-east-1")
6668
# bucket => "boss_please_open_your_bucket" (required)
6769
# size_file => 2048 (optional) - Bytes
@@ -84,6 +86,9 @@ class LogStash::Outputs::S3 < LogStash::Outputs::Base
8486
# S3 bucket
8587
config :bucket, :validate => :string
8688

89+
# endpoint
90+
config :endpoint, :validate => :string
91+
8792
# Set the size of file in bytes, this means that files on bucket when have dimension > file_size, they are stored in two or more file.
8893
# If you have tags then it will generate a specific size file for every tags
8994
##NOTE: define size of file is the better thing, because generate a local temporary file on disk and then put it in bucket.
@@ -147,6 +152,7 @@ def aws_s3_config
147152

148153
def full_options
149154
aws_options_hash.merge(signature_options)
155+
aws_options_hash.merge(endpoint_options)
150156
end
151157

152158
def signature_options
@@ -157,6 +163,20 @@ def signature_options
157163
end
158164
end
159165

166+
def endpoint_options
167+
if @endpoint
168+
uri = URI(@endpoint)
169+
{
170+
:s3_force_path_style => true,
171+
:s3_endpoint => uri.host,
172+
:s3_port => uri.port,
173+
:use_ssl => uri.scheme == "https",
174+
}
175+
else
176+
{}
177+
end
178+
end
179+
160180
def aws_service_endpoint(region)
161181
return {
162182
:s3_endpoint => region == 'us-east-1' ? 's3.amazonaws.com' : "s3-#{region}.amazonaws.com"

0 commit comments

Comments
 (0)