Elastic Stack (ELK) was originally developed to store, search, and visualize large amounts of data. Organizations initially used it to monitor application performance and perform searches across large datasets.
Beats are lightweight data shippers installed on endpoints or servers.
Examples:
- Filebeat – Collects log files
- Winlogbeat – Collects Windows Event Logs
- Packetbeat – Collects network traffic
- Metricbeat – Collects system and service metrics
- Auditbeat – Collects audit data
Logstash is the data processing engine of the Elastic Stack.
It:
- Receives data from Beats, network ports, APIs, or files.
- Parses and filters data.
- Normalizes logs into structured field-value pairs.
- Sends processed data to Elasticsearch.
Elasticsearch is the search and analytics engine.
It:
- Stores structured log data.
- Indexes documents for fast searching.
- Performs real-time searches and analytics.
- Serves as the central database of the Elastic Stack.
Kibana is the visualization layer.
It allows users to:
- Search logs
- Build dashboards
- Create charts and graphs
- Visualize trends
- Investigate security incidents
- Monitor infrastructure
- Beats collect data from systems and applications.
- Logstash receives and processes the collected data.
- Elasticsearch stores and indexes the processed data.
- Kibana queries Elasticsearch and presents the data through interactive dashboards and visualizations.