CB-30433: update README, CHANGES.md -> CHANGELOG.md (with updates) (#178)

* CB-30433: update README, CHANGE.md -> CHANGELOG.md (with updates)

* CB-30433: fix typo

Co-authored-by: dseidel <dseidel@carbonblack.com>

Author: dseidel-b9

CHANGES.md CHANGELOG.mdCHANGES.md renamed to CHANGELOG.md

@@ -1,68 +1,108 @@
-# Changelog
+# CB Event Forwarder Changelog
 
-## cb-event-forwarer 3.5.0
-- kafka SASL support
-- OATH2 JWT optional support for http output
-- Support for sending EventText as bytearary httpoutput 
+## v3.6.2
 
-## cb-event-forwarder 3.1.2
+#### Features
+
+ * Event Forwarder can now be configured and operated from the CB EDR web console.
+ * There are no new features in Event Forwarder itself.
+
+#### Bug Fixes
+
+ * Fix signal handling for syslog and S3 output types
+ * Fix error handling for AMQP connections
+
+## v3.6.1
+
+### Features
+
+ * CentOS/RHEL 7.x compatibility with separate packages for el6 and el7.
+ * New metric support
+ * Threading for Kafka output,
+ * Ability to configure more options for kafka.
+
+### Bug Fixes
+
+ * Streamlined error reporting, removing superfluous and numerous
+`blocked_netconn` exceptions from the event forwarder stream.
+
+## v3.6.0
+
+ * Overhaul support for Kafka output
+ * Various fixes and support for compression in HTTP/S3 outputs.
+ * Use the new `[kafka.producer]` section to specify arbitrary Kafka producer
+ options based on the [Kafka producer API](https://docs.confluent.io/current/installation/configuration/producer-configs.html)
+ for details on the supported configuration options. This allows for supporting
+ Kafka producer TLS/SSL options, compression, and various others if desired.
+ Continue to specify `output_type=kafka` and
 
-The 3.1.2 release of cb-event-forwarder adds two features:
 
-* You can now send arbitrary messages for debugging/testing purposes through the forwarder to the output location.
+     [kafka]
+     brookers=comma-delimited-broker-list
+     
+in your configuration file to try things out.
+
+## v3.5.0
+ * Kafka SASL support
+ * OATH2 JWT optional support for http output
+ * Support for sending EventText as bytearary httpoutput 
+
+## v3.1.2
+
+ * You can now send arbitrary messages for debugging/testing purposes through the forwarder to the output location.
   This is only available when the cb-event-forwarder is started with the `-debug` command line switch. Messages
   sent via this mechanism are also logged for audit purposes.
-* S3: You can now explicitly specify the location of the AWS credential file to use for authentication in the
+ * S3: You can now explicitly specify the location of the AWS credential file to use for authentication in the
   `credential_profile` option in the `[s3]` section of the configuration file. To search for the credential profile
   `production` in the credentials stored in the file `/etc/cb/aws.creds`, set the `credential_profile` option to
   `/etc/cb/aws.creds:production`.
 
-## cb-event-forwarder 3.1.1
+## v3.1.1
 
 The 3.1.1 release of cb-event-forwarder fixes a critical bug when rolling over files. Previous versions of the
 cb-event-forwarder would stop rolling over files after the first of a new month. This release fixes that bug.
 
-## cb-event-forwarder 3.1.0
-
-The 3.1.0 release of cb-event-forwarder adds the following features over 3.0.0:
+## v3.1.0
 
-* "Deep links" into the Cb server UI are now optionally available in the output
-  * These links allow you to directly access the relevant sensor, binary, or process context for each event output
+ * "Deep links" into the CB server UI are now optionally available in the output
+   * These links allow you to directly access the relevant sensor, binary, or process context for each event output
     by the cb-event-forwarder.
-  * The new variable `cb_server_url` has been added to the configuration file to support this new feature. Set this
+   * The new variable `cb_server_url` has been added to the configuration file to support this new feature. Set this
     variable to the base URL of the Carbon Black web UI. **If this variable is not set, then no links are generated.**
-  * The new links are available in the `link_process`, `link_child` (in child process events), `link_md5` and 
+   * The new links are available in the `link_process`, `link_child` (in child process events), `link_md5` and 
     `link_sensor` keys of the JSON or LEEF output.
-  * Note that links to processes and binaries may result in 404 errors until the process and binary data is committed
+   * Note that links to processes and binaries may result in 404 errors until the process and binary data is committed
     to disk on the Carbon Black server. Process events received via the event-forwarder may take up to 15 minutes or 
     longer before they're visible on the Carbon Black web UI.
 * All Carbon Black 5.1 event types are now supported
-  * Microsoft EMET
-  * Carbon Black Tamper events
-  * Cross-process (process open/thread create) events
-  * Carbon Black process/network blocking events
+   * Microsoft EMET
+   * Carbon Black Tamper events
+   * Cross-process (process open/thread create) events
+   * Carbon Black process/network blocking events
 * Network events now include the local IP and port number of the network connection (available on Carbon Black 5.1 
   servers and sensors)
-  * The IP four-tuple is now available as (`local_ip`, `local_port`, `remote_ip`, and `remote_port`) in the JSON/LEEF
+   * The IP four-tuple is now available as (`local_ip`, `local_port`, `remote_ip`, and `remote_port`) in the JSON/LEEF
     output
-* Provide a human-readable status page for statistics
-  * By default, these statistics are available via HTTP on port 33706 of the system running the cb-event-forwarder.
-* Fix regressions on output from cb-event-forwarder 2.x on some JSON message types
-  * cb-event-forwarder 3.0.0 was missing the `computer_name` field from some JSON messages
+* Provided a human-readable status page for statistics
+   * By default, these statistics are available via HTTP on port 33706 of the system running the cb-event-forwarder.
+* Fixed regressions on output from cb-event-forwarder 2.x on some JSON message types
+   * cb-event-forwarder 3.0.0 was missing the `computer_name` field from some JSON messages
 * New Amazon S3 options; see the `[s3]` section of the configuration file
-  * Specify whether the files uploaded to S3 should be encrypted with server-side encryption (see `server_side_encryption`)
-  * Define an ACL policy to apply to files uploaded to S3 (see `acl_policy`)
-  * Specify the credential profile used when connecting to S3 (see `credential_profile`)
+   * Specify whether the files uploaded to S3 should be encrypted with server-side encryption (see `server_side_encryption`)
+   * Define an ACL policy to apply to files uploaded to S3 (see `acl_policy`)
+   * Specify the credential profile used when connecting to S3 (see `credential_profile`)
+
+---
 
-# Changes from the cb-event-forwarder 2.x to 3.x
+# Changes from v2.x to v3.x
 
 In general, the new cb-event-forwarder 3.0 is designed to be a drop-in replacement for previous versions of the
 event forwarder. There are a few bug fixes, configuration changes and enhancements of note. The most important change
 is that the service is now managed by the "upstart" system in CentOS 6. The `service` command is no longer used to
 control the service; instead use `start cb-event-forwarder` and `stop cb-event-forwarder` to manually start and stop
 the service.
 
-## Configuration
+### Configuration
 
 The configuration file location still defaults to `/etc/cb/integrations/event-forwarder/cb-event-forwarder.conf` and
 most existing configuration files will work unchanged with this new version. 
@@ -92,15 +132,15 @@ The following changes have been made to the configuration file in version 3.0:
 
 * The `stdout` output option has been removed.
 
-## Output format
+### Output format
 
 * The `tcp` output now places a newline (`\r\n`) between each event in the output stream
 
 * Bugfix: the output from the `childproc` event type now contains the correct `process_guid` value
 
 * Bugfix: the output from the `procend` event type now contains the MD5 from the process that exited in the `md5` value
 
-## Operations
+### Operations
 
 * The daemon is now managed by the "upstart" system in CentOS 6.
   * Use the `start` and `stop` commands to control the daemon: `start cb-event-forwarder`.

README.md

@@ -1,35 +1,38 @@
-# Cb Response Event Forwarder
+# CB Event Forwarder
 
 ## Overview
 
-The Cb Response Event Forwarder is a standalone service that will listen on the Cb Response enterprise bus and export
+The CB EDR Event Forwarder is a standalone service that will listen on the Cb EDR enterprise bus and export
 events (both watchlist/feed hits as well as raw endpoint events, if configured) in a normalized JSON or LEEF format.
 The events can be saved to a file, delivered to a network service or archived automatically to an Amazon AWS S3 bucket.
 These events can be consumed by any external system that accepts JSON or LEEF, including Splunk and IBM QRadar.
 
-The list of events to collect is configurable.
-By default all feed and watchlist hits, alerts, binary notifications, and raw sensor events are exported into JSON.  The
-configuration file for the connector is stored in `/etc/cb/integrations/event-forwarder/cb-event-forwarder.conf`.
+The list of events to collect is configurable. By default all feed and watchlist hits, alerts, binary notifications, and
+raw sensor events are exported into JSON. The configuration file for the connector is stored in
+`/etc/cb/integrations/event-forwarder/cb-event-forwarder.conf`.
+
+Starting with version 7.1.0 of Carbon Black EDR, you can use the EDR web console to configure and control Event Forwarder,
+as long as you follow the installation and configuration steps detailed below.
 
 ## Support
 
 The pre-built RPM is supported via our [User eXchange (Jive)](https://community.carbonblack.com/community/developer-relations) 
-and via email to dev-support@carbonblack.com.  
+and via email to dev-support@carbonblack.com. 
 
 ## Raw Sensor Events 
 
-We have seen a performance impact when exporting all raw sensor events onto the enterprise bus.  We do not recommend
-exporting all the events.  The performance impacts are seen when the events are broadcast on the bus, by enabling the
-"DatastoreBroadcastEventTypes".  We recommend that at most, only process and netconn events be broadcast on the event
+We have seen a performance impact when exporting all raw sensor events onto the enterprise bus. We do not recommend
+exporting all the events. The performance impacts are seen when the events are broadcast on the bus, by enabling the
+"DatastoreBroadcastEventTypes". We recommend that at most, only process and netconn events be broadcast on the event
 bus. 
 
 ## Quickstart Guide
 
 The cb-event-forwarder can be installed on any 64-bit Linux machine running CentOS 6.x. 
-It can be installed on the same machine as the Cb Response server, or another machine. 
+It can be installed on the same machine as the Cb EDR server, or another machine. 
 If you are forwarding a large volume of events to QRadar (for example, all file modifications and/or registry 
-modifications), or are forwarding events from a Cb Response cluster, then installing it on a separate machine is recommended. 
-Otherwise, it is acceptable to install the cb-event-forwarder on the Cb Response server itself.
+modifications), or are forwarding events from a Cb EDR cluster, then installing it on a separate machine is recommended. 
+Otherwise, it is acceptable to install the cb-event-forwarder on the Cb EDR server itself.
 
 ### Installation
 
@@ -48,26 +51,37 @@ CB EDR is installed (in the case of a cluster installer, this means the master n
    ```
    yum install cb-event-forwarder
    ```
-3. If you will be using the CB EDR console to configure and operate the event forwarder, run the following script to set
-the appropriate permissions needed by EDR:
+3. If you are using CB EDR 7.1.0 or greater and wish to use the CB EDR console to configure and operate the Event
+Forwarder, run the following script to set the appropriate permissions needed by EDR:
 
    ```
    /usr/share/cb/integrations/event-forwarder/cb-edr-fix-permissions.sh
    ```
 
 ### Configure the cb-event-forwarder
 
-1. If installing on a machine *other than* the Cb Response server, copy the RabbitMQ username and password into the 
+1. If installing on a machine *other than* the Cb EDR server, copy the RabbitMQ username and password into the 
 `rabbit_mq_username` and `rabbit_mq_password` variables in `/etc/cb/integrations/event-forwarder/cb-event-forwarder.conf` 
-file. Also fill out the `cb_server_hostname` with the hostname or IP address where the Cb Response server can be reached.
-If the cb-event-forwarder is forwarding events from a Cb Response cluster, the `cb_server_hostname` should be set
-to the hostname or IP address of the Cb Response master node.
+file. Also fill out the `cb_server_hostname` with the hostname or IP address where the Cb EDR server can be reached.
+If the cb-event-forwarder is forwarding events from a Cb EDR cluster, the `cb_server_hostname` should be set
+to the hostname or IP address of the Cb EDR master node.
    
 2. Ensure that the configuration is valid by running the cb-event-forwarder in Check mode: 
 `/usr/share/cb/integrations/event-forwarder/cb-event-forwarder -check` as root. If everything is OK, you will see a 
 message starting with "Initialized output”. If there are any errors, those errors will be printed to your screen.
 
-### Configure Cb Response
+### Configure Cb EDR
+
+#### Console Support
+
+If you are using CB EDR 7.1.0 or greater and wish to use the CB EDR console to configure and operate the Event
+Forwarder, you will need to add the following setting to `/etc/cb/cb.conf` (on the master node, if this is a cluster):
+
+    EventForwarderEnabled=True
+ 
+ after which you must restart services (or restart the cluster).
+
+#### Event Publishing
 
 By default, Cb publishes the `feed.*` and `watchlist.*` events over the bus (see the [Events documentation](EVENTS.md)
 for more information). 
@@ -79,10 +93,10 @@ If you want to capture raw sensor events or the `binaryinfo.*` notifications, yo
 * If you are capturing binary observed events you also need to edit the `EnableSolrBinaryInfoNotifications` option in 
 `/etc/cb/cb.conf` and set it to `True`.
 
-Cb Response needs to be restarted if any variables were changed in `/etc/cb/cb.conf` by executing
+Cb EDR needs to be restarted if any variables were changed in `/etc/cb/cb.conf` by executing
 `service cb-enterprise restart`. 
 
-If you are configuring the cb-event-forwarder on a Cb Response cluster, the `DatastoreBroadcastEventTypes` and/or
+If you are configuring the cb-event-forwarder on a Cb EDR cluster, the `DatastoreBroadcastEventTypes` and/or
 `EnableSolrBinaryInfoNotifications` settings
 must be distributed to the `/etc/cb/cb.conf` configuration file on all minion nodes and the cluster stopped and started using
 the `/usr/share/cb/cbcluster stop && /usr/share/cb/cbcluster start` command.
@@ -103,15 +117,15 @@ Once the service is installed, it is configured to start automatically on system
 
 ## Splunk
 
-The Cb Response event forwarder can be used to export Cb Response events in a way easily configured for Splunk.  You'll
-need to install and configure the Splunk TA to consume the Cb Response event data.   It is recommended that the event
-bridge use a file based output with Splunk universal forwarder configured to monitor the file.   
+The Cb EDR Event Forwarder can be used to export Cb EDR events in a way easily configured for Splunk. You'll
+need to install and configure the Splunk TA to consume the Cb EDR event data. We recommend that the event
+bridge use a file-based output with the Splunk universal forwarder configured to monitor the file. 
 
 More information about configuring the Splunk TA can be found [here](http://docs.splunk.com/Documentation/AddOns/latest/Bit9CarbonBlack/About)
 
 ## QRadar
 
-The Cb Response event forwarder can forward Cb Response events in the LEEF format to QRadar. To forward Cb Response
+The Cb EDR Event Forwarder can forward Cb EDR events in the LEEF format to QRadar. To forward Cb EDR
 events to a QRadar server:
 
 1. Modify `/etc/cb/integrations/event-forwarder/cb-event-forwarder.conf` to include 
@@ -231,6 +245,4 @@ To build an RPM package, use `make rpm`. By default, the result will be located
 
 ## Changelog
 
-This connector has been completely rewritten for version 3.0.0 for greatly enhanced reliability and performance. 
-See the [releases page](https://github.com/carbonblack/cb-event-forwarder/releases) .
-for more information on new features introduced with each new version and upgrading from cb-event-forwarder 2.x.
+See CHANGELOG.md.