Merge pull request #34 from DataDog/gaetan.deputier/fluent-http

[Fluent]: v0.12.0

Author: gaetan-deputier

CHANGELOG.md

@@ -0,0 +1,5 @@
+## 0.12.0
+ - Migrate to Fluentd v1 APIs
+ - Enable HTTP forwarding by default
+ - Enable gzip compression by default for HTTP
+ - Add an option to disable SSL hostname verification

README.md

@@ -6,6 +6,11 @@ if you don't wan't to.
 
 ## Pre-requirements
 
+| fluent-plugin-datadog | Fluentd    | Ruby   |
+|:--------------------------|:-----------|:-------|
+| \>= 0.12.0               | \>= v1      | \>= 2.4 |
+| < 0.12.0                   | \>= v0.12.0 | \>= 2.1 |
+
 To add the plugin to your fluentd agent, use the following command:
 
     gem install fluent-plugin-datadog
@@ -38,6 +43,14 @@ TCP example:
   dd_source '<INTEGRATION_NAME>' 
   dd_tags '<KEY1:VALU1>,<KEY2:VALUE2>'
   dd_sourcecategory '<MY_SOURCE_CATEGORY>'
+  <buffer>
+          @type memory
+          flush_thread_count 4
+          flush_interval 3s
+          chunk_limit_size 5m
+          chunk_limit_records 500
+  </buffer>
+
 
 </match>
 ```
@@ -72,14 +85,19 @@ As fluent-plugin-datadog is an output_buffer, you can set all output_buffer prop
 | **tag_key** | Where to store the Fluentd tag. | "tag" |
 | **timestamp_key** | Name of the attribute which will contain timestamp of the log event. If nil, timestamp attribute is not added. | "@timestamp" |
 | **use_ssl** | If true, the agent initializes a secure connection to Datadog. In clear TCP otherwise. | true |
-| **ssl_port** | Port used to send logs over a SSL encripted connection to Datadog (use 443 for the EU region) | 10516 |
+| **no_ssl_validation** | Disable SSL validation (useful for proxy forwarding) | false |
+| **ssl_port** | Port used to send logs over a SSL encrypted connection to Datadog. If use_http is disabled, use 10516 for the US region and 443 for the EU region. | 443 |
 | **max_retries** | The number of retries before the output plugin stops. Set to -1 for unlimited retries | -1 |
+| **max_backoff** | The maximum time waited between each retry in seconds | 30 |
+| **use_http** | Enable HTTP forwarding. If you disable it, make sure to change the port to 10514 or ssl_port to 10516 | true |
+| **use_compression** | Enable log compression for HTTP | true |
+| **compression_level** | Set the log compression level for HTTP (1 to 9, 9 being the best ratio) | 6 |
 | **dd_source** | This tells Datadog what integration it is | nil |
 | **dd_sourcecategory** | Multiple value attribute. Can be used to refine the source attribute | nil |
 | **dd_tags** | Custom tags with the following format "key1:value1, key2:value2" | nil |
 | **dd_hostname** | Used by Datadog to identify the host submitting the logs. | `hostname -f` |
 | **service** | Used by Datadog to correlate between logs, traces and metrics. | nil |
-| **port** | Proxy port when logs are not directly forwarded to Datadog and ssl is not used | 10514 |
+| **port** | Proxy port when logs are not directly forwarded to Datadog and ssl is not used | 80 |
 | **host** | Proxy endpoint when logs are not directly forwarded to Datadog | intake.logs.datadoghq.com |
 
 ### Docker and Kubernetes tags

fluent-plugin-datadog.gemspec

@@ -9,7 +9,7 @@ $LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
 
 Gem::Specification.new do |spec|
   spec.name          = "fluent-plugin-datadog"
-  spec.version       = "0.11.1"
+  spec.version       = "0.12.0"
   spec.authors       = ["Datadog Solutions Team"]
   spec.email         = ["[email protected]"]
   spec.summary       = "Datadog output plugin for Fluent event collector"
@@ -22,6 +22,12 @@ Gem::Specification.new do |spec|
   spec.require_paths = ["lib"]
 
   spec.add_development_dependency "bundler", "~> 1.5"
+  spec.add_runtime_dependency "fluentd", [">= 1", "< 2"]
+  spec.add_runtime_dependency "net-http-persistent", '~> 2.9'
+
+  spec.add_development_dependency "test-unit", '~> 3.1'
   spec.add_development_dependency "rake"
   spec.add_development_dependency "yajl-ruby", "~> 1.2"
+
+
 end