This PoC shows some (basic) search-functionality provided by Splunk, which is discussed by Analyzing Log Analysis: An Empirical Study of User Log Mining For this demo searches have been prepared to showcase how certain analysis-tasks are replicated using Splunk. The following analysis-tasks have been prepared (source can be found in savedsearches.conf):
- filtering of events
- filter_by_fields
- filter_columns
- adding a column/field to each event using the
eval-statement - grouping events by shared fields and calucation metrics upon each group
- aggregation_count_by_method
- aggregation_count_by_multiple_fields
- aggregation_methods_by_field
- renaming fields
- rename_column
The queries can be found in the savedsearches.conf or in Splunk: Search & Reporting > Reports
In this PoC the Universalforwarder is used to monitor the /usr/share/data/accesss.log-file on the local filesystem and output the data to Splunk.
To run the PoC simply execute the run.sh script. It will start all the docker-container and apply runtime configuration, aswell as output log messages and cleanup after you exit.
Connectivity-configuration is handled in the docker-compose.yml. Splunk configuration is applied using apps (see apps-directory). Universalforwarder configuration is handled via the Forwarder-Management: Configuration is stored in apps in the deployment-apps-directory of splunk and is then automatically distributed to forwarder.