Skip to content

Commit 5e7c048

Browse files
mitchelbaker-cisatylmorr-snlaarz-snlmreeve-snlcausand22
authored
Quality of life improvement for documentation (#604)
* Quality of life improvement for documentation (#602) * Readme qualify of life improvements * Update README.md * Add upgrade and move ansible install location * add note to note change passwords manually * Update README.md * Update README.md * Update elastic-agent-mangement.md * Update elastic-agent-mangement.md * change hardcoded url to relative path * formatting improvements for Kibana URL * formatting improvements for Kibana URL * Update duplicate title in 'Retrieving Passwords' * update relative path anchor reference * Adding in Kendall's updates to community engagement + some small fixes * Adding note on scaling to these docs * Add small updates: 1. add a discussion around LME users and passwords to refer to elastic's docs 2. add changes to add in 24.04 notes around initial testing. 3. small fixes * Adding link and removing link * Update and rename elastic-agent-mangement.md to elastic-agent-management.md update url for passwords on readme, update url to remove 5601 port, and rename file * PUshing updates to Readme --------- Co-authored-by: Andrew Arz <[email protected]> Co-authored-by: mitchelbaker-cisa <[email protected]> Co-authored-by: Michael Reeves <[email protected]> * revise changes to match develop * add missing docs * Added video links and made small grammar corrections * Updated video link * grammar changes * fixing clunky wording --------- Co-authored-by: Tyler Morris <[email protected]> Co-authored-by: Andrew Arz <[email protected]> Co-authored-by: Michael Reeves <[email protected]> Co-authored-by: Connor Aubry <[email protected]>
1 parent e11d045 commit 5e7c048

File tree

9 files changed

+536
-371
lines changed

9 files changed

+536
-371
lines changed

README.md

Lines changed: 309 additions & 355 deletions
Large diffs are not rendered by default.

docs/markdown/agents/elastic-agent-mangement.md renamed to docs/markdown/agents/elastic-agent-management.md

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,9 @@ This guide will walk you through the process of enrolling an Elastic agent.
55
## Steps to Enroll an Agent
66

77
1. **Access the Fleet Menu**
8-
- Open the LME dashboard
8+
- Open the LME dashboard: `https://{SERVER_IP}`
9+
- Password information can be found in the [README](/README.md#retrieving-passwords).
10+
- Open the "hamburger" menu icon in the top left (three horizontal lines)
911
- Scroll down and select "Fleet" from the menu
1012

1113
2. **Add a New Agent**
@@ -40,7 +42,8 @@ This guide will walk you through the process of enrolling an Elastic agent.
4042
![example-screenshot](/docs/imgs/insecure-powershell.png)
4143
4244
8. **Execute the Command**
43-
- Recommend running each line individually so you can see a clear picture of the status of each command ran. The entire process will download an agent, unzip it, and install it.
45+
- Run the command on the desired host.
46+
- It is recommended to run each line individually so you can see the status of each command. The entire process will download an agent, unzip it, and install it.
4447
4548
From Fleet you should see the agent enrolled now.
4649
@@ -51,7 +54,9 @@ This guide will walk you through the process of adding a Windows integration to
5154
## Steps to Add Windows Integration
5255
5356
1. **Access Fleet and Agent Policies**
54-
- Open the LME dashboard
57+
- Open the LME dashboard: `https://{SERVER_IP}`
58+
- Password information can be found in the [Readme](/README.md#retrieving-passwords).
59+
- Open the "hamburger" menu icon in the top left (three horizontal lines)
5560
- Select "Fleet" from the menu
5661
- Click on "Agent policies"
5762
@@ -64,6 +69,7 @@ This guide will walk you through the process of adding a Windows integration to
6469
6570
4. **Choose Windows Integration**
6671
- From the list of available integrations, select "Windows"
72+
- Select "add Windows"
6773
6874
5. **Configure Windows Integration**
6975
- Scroll down to review the options available
@@ -100,5 +106,3 @@ The Elastic agent has multiple debugging commands that can be run to troubleshoo
100106
In addition, you can use this [link](https://www.elastic.co/guide/en/fleet/current/installation-layout.html) to navigate/find the directories for where Elastic agent is installed on the operating system.
101107
102108
If there are issues with running the command involving a pipe file, the elastic endpoint service (a windows service started by the agent) is in a failed state, and retarting the machine will most likely fix it, check out this [link](https://discuss.elastic.co/t/windows-pipe-elastic-agent-system-access-is-denied/316344) However, this isn't required if the agent is showing as healthy, only if you want to run other cli agent debugging commands.
103-
104-

docs/markdown/agents/wazuh-agent-mangement.md renamed to docs/markdown/agents/wazuh-agent-management.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ You can confirm the version is accurate with a list from wazuh's versions [HERE]
4545
1. **Download the Wazuh Agent**
4646
- Download the Wazuh agent MSI installer from the following URL:
4747
```
48-
https://packages.wazuh.com/4.x/windows/wazuh-agent-{WAZUH_AGENT_VERSION}.msi
48+
https://packages.wazuh.com/4.x/windows/wazuh-agent-{WAZUH_AGENT_VERSION}-1.msi
4949
```
5050
- Replace `{WAZUH_AGENT_VERSION}` with the appropriate version number.
5151
- You can also use the below powershell command:
@@ -153,4 +153,4 @@ Replace `[agent_id]` with the ID of the agent you want to check. This will provi
153153

154154
This command gives you a quick overview of how many agents are active, disconnected, or never connected.
155155

156-
See official Wazuh documentation for more steps on [agent_control](https://documentation.wazuh.com/current/user-manual/reference/tools/agent-control.html)
156+
See official Wazuh documentation for more steps on [agent_control](https://documentation.wazuh.com/current/user-manual/reference/tools/agent-control.html)

docs/markdown/maintenance/certificates.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,21 @@
11
# Certificates
2-
#
2+
33
The LME installation makes use of a number of TLS certificates to protect communications between the server components and agents, and also secures the connections between Elasticsearch and Kibana.
44
By default the installation will create certificates and this documentation describes how to modify and update the cert store.
55

66
## Regenerating Self-Signed Certificates
77
The easiest way to do this is to delete the `lme_certs` volume, and restart lme.service:
88

9-
This is destructive and not recommended,but there could be cases.
9+
This is destructive and not recommended, but there could be cases.
1010
```bash
1111
sudo -i podman volume rm lme_certs
1212
sudo systemctl restart lme.service
1313
```
1414

1515
## Using Your Own Certificates
16-
You can certificates signed by an existing root CA as part of the LME install by generating certificates manually with the correct settings and placing these within the required directory inside the LME folder. **NOTE: The default supported method of LME installation is to use the automatically created self-signed certificates, and we will be unable to support any problems that arise from generating the certificates manually incorrectly.**
16+
You can use certificates signed by an existing root CA as part of the LME install by generating certificates manually with the correct settings and placing these within the required directory inside the LME folder.
1717

18+
**NOTE: The default supported method of LME installation is to use the automatically created self-signed certificates, and we will be unable to support any problems that arise from generating the certificates manually incorrectly.**
1819

1920
### Certificate Creation
2021
If you create certificates ensure their subject alt names allow for the ips/dns entries listed below, as well as the ips/domains you'll be connecting to the service as:
@@ -52,7 +53,7 @@ instances:
5253
- "127.0.0.1"
5354
```
5455

55-
For example, the new kibana cert would need to support the above alternative names... you can also ensure its setup properly by viewing the current cert (assuming you've already mounted the `lme_certs` podman volume.
56+
For example, the new kibana cert would need to support the above alternative names. You can also ensure its setup properly by viewing the current cert (assuming you've already mounted the `lme_certs` podman volume).
5657
```bash
5758
root@ubuntu:~$ cat /var/lib/containers/storage/volumes/lme_certs/_data/kibana/kibana.crt | openssl x509 -text | grep -i Alternative -A 1
5859
X509v3 Subject Alternative Name:

docs/markdown/prerequisites.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,8 @@ Users must pay for hosting, bandwidth and time; for an estimate of server specs
5454
To keep LME simple, our guide only covers single server setups. Considering the differences across environments and scaling needs, we cannot provide an estimate of server resources beyond single server setups.
5555
It’s possible to scale the solution to multiple event collectors and ELK nodes, but that will require more experience with the technologies involved. However, we plan to publish documentation for scaling LME in the future.
5656

57+
Please see the above blogpost from elastic for discussion on how to scale an elastic stack cluster.
58+
5759
## Required infrastructure
5860

5961
To begin installing LME, you will need access to the following servers or you will need to create them:
@@ -98,6 +100,4 @@ Servers can be either on premise, in a public cloud, or in a private cloud. It i
98100
## What firewall rules are needed?
99101
Please see our cloud documentation for a discussion on firewalls [LME in the Cloud](/docs/markdown/loggging-guidance/cloud.md).
100102

101-
You must ensure that the client machine you want to monitor can reach the main LME ports as described in the ReadMe [Required Ports section](/README.md#required-ports).
102-
103-
103+
You must ensure that the client machine you want to monitor can reach the main LME ports as described in the ReadMe [Required Ports section](/README.md#required-ports).
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Architecture:
2+
LME runs on Ubuntu 22.04 and leverages Podman containers for security, performance, and scalability. We’ve integrated Wazuh, Elastic, and ElastAlert open source tools to provide log management, endpoint security monitoring, alerting, and data visualization capabilities. This modular, flexible architecture supports efficient log storage, search, and threat detection, and enables you to scale as your logging needs evolve.
3+
4+
### Diagram:
5+
6+
![diagram](/docs/imgs/lme-architecture-v2.png)
7+
8+
### Containers:
9+
Containerization allows each component of LME to run independently, increasing system security, improving performance, and making troubleshooting easier.
10+
11+
LME uses Podman as its container engine because it is more secure (by default) against container escape attacks than other engines like Docker. It's far more debug and programmer friendly. We’re making use of Podman’s unique features, such as Quadlets (Podman's systemd integration) and User Namespacing, to increase system security and operational efficiency.
12+
13+
LME uses these containers:
14+
15+
- **Setup**: Runs `/config/setup/init-setup.sh` based on the configuration of DNS defined in `/config/setup/instances.yml`. The script will create a certificate authority (CA), underlying certificates for each service, and initialize the admin accounts for Elasticsearch(user:`elastic`) and Kibana(user:`kibana_system`).
16+
- **Elasticsearch**: Runs LME's database and indexes all logs.
17+
- **Kibana**: The front end for querying logs, visualizing data, and managing fleet agents.
18+
- **Elastic Fleet-Server**: Executes an [elastic agent ](https://github.com/elastic/elastic-agent) in fleet-server mode. Coordinates elastic agents to gather client logs and status. Configuration is inspired by the [elastic-container](https://github.com/peasead/elastic-container) project.
19+
- **Wazuh-Manager**: Allows LME to deploy and manage Wazuh agents.
20+
- Wazuh (open source) gives EDR (Endpoint Detection Response) with security dashboards to cover the security of all of the machines.
21+
- **LME-Frontend** (*coming in a future release*): Will host an API and GUI that unifies the architecture behind one interface.
22+
23+
### Required Ports:
24+
Ports required are as follows:
25+
- Elasticsearch: *9200*
26+
- Kibana: *443,5601*
27+
- Wazuh: *1514,1515,1516,55000,514*
28+
- Agent: *8220*
29+
30+
**Note**: For Kibana, 5601 is the default port. We've also set kibana to listen on 443 as well.
31+
32+
### Agents and Agent Management:
33+
LME leverages both Wazuh and Elastic agents providing more comprehensive logging and security monitoring across various log sources. The agents gather critical data from endpoints and send it back to the LME server for analysis, offering organizations deeper visibility into their security posture. We also make use of the Wazuh Manager and Elastic Fleet for agent orchestration and management.
34+
35+
- **Wazuh Agents**: Enables Endpoint Detection and Response (EDR) on client systems, providing advanced security features like intrusion detection and anomaly detection. For more information, see [Wazuh's agent documentation](https://github.com/wazuh/wazuh-agent).
36+
- **Wazuh Manager**: Responsible for managing Wazuh Agents across endpoints, and overseeing agent registration, configuration, and data collection, providing centralized control for monitoring security events and analyzing data.
37+
- **Elastic Agents**: Enhance log collection and management, allowing for greater control and customization in how data is collected and analyzed. Agents also feature a vast collection of integrations for many log types/applications. For more information, see [Elastic's agent documentation](https://github.com/elastic/elastic-agent).
38+
- **Elastic Fleet**: Manages Elastic Agents across your infrastructure, providing centralized control over agent deployment, configuration, and monitoring. It simplifies the process of adding and managing agents on various endpoints. ElasticFleet also supports centralized updates and policy management.
39+
40+
41+
### Alerting:
42+
LME has setup [ElastAlert](https://elastalert2.readthedocs.io/en/latest/index.html), an open-source alerting framework, to automate alerting based on data stored in Elasticsearch. It monitors Elasticsearch for specific patterns, thresholds, or anomalies, and generates alerts when predefined conditions are met. This provides proactive detection of potential security incidents, enabling faster response and investigation. ElastAlert’s flexible rule system allows for custom alerts tailored to your organization’s security monitoring needs, making it a critical component of the LME alerting framework.
43+
44+
### Log Storage and Search:
45+
46+
As the core component for log search and storage, [Elasticsearch](https://www.elastic.co/elasticsearch) indexes and stores logs and detections collected from Elastic and Wazuh Agents, allowing for fast, real-time querying of security events. Elasticsearch enables users to search and filter large datasets efficiently, providing a powerful backend for data analysis and visualization in Kibana. Its scalability and flexibility make it essential for handling the high-volume log data generated across different endpoints within LME's architecture.
47+
48+
### Data Visualization and Querying:
49+
[Kibana](https://www.elastic.co/kibana) is the visualization and analytics interface in LME, providing users with tools to visualize and monitor log data stored in Elasticsearch. It enables the creation of custom dashboards and visualizations, allowing users to easily track security events, detect anomalies, and analyze trends. Kibana's intuitive interface supports real-time insights into the security posture of an organization, making it an essential tool for data-driven decision-making in LME’s centralized logging and security monitoring framework.
Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# General notes on custom configuration for LME
2+
The configuration files are located in /config/. These steps will guide you through setting up LME.
3+
4+
## certificates and user passwords:
5+
- instances.yml defines the certificates to be created.
6+
- Shell scripts will initialize accounts and generate certificates. They run from the quadlet definitions lme-setup-accts and lme-setup-certs.
7+
8+
## Podman Quadlet Configuration
9+
- Quadlet configuration for containers is located in /quadlet/. These map to the root systemd unit files but execute as non-privileged users.

docs/markdown/reference/passwords.md

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Password Encryption:
2+
Ansible-vault is used to enable password encryption, securely storing all LME user and service user passwords at rest
3+
We do submit a hash of the password to Have I Been Pwned to check to see if it is compromised: [READ MORE HERE](https://haveibeenpwned.com/FAQs), but since they're all randomly generated this should be rare.
4+
5+
### Where Are Passwords Stored?:
6+
```bash
7+
# Define user-specific paths
8+
USER_VAULT_DIR="/etc/lme/vault"
9+
PASSWORD_FILE="/etc/lme/pass.sh"
10+
```
11+
12+
### Grabbing Passwords:
13+
To view the appropriate service user password run the following commands:
14+
```bash
15+
#script:
16+
$CLONE_DIRECTORY/scripts/extract_secrets.sh -p #to print
17+
18+
#add them as variables to your current shell
19+
source $CLONE_DIRECTORY/scripts/extract_secrets.sh #without printing values
20+
source $CLONE_DIRECTORY/scripts/extract_secrets.sh -q #with no output
21+
```
22+
23+
### Manually Setting Up Passwords and Accessing Passwords **Unsupported**:
24+
**These steps are not fully supported by CISA and are left if others would like to support this in their environment**
25+
26+
Run the password_management.sh script:
27+
```bash
28+
lme-user@ubuntu:~/LME-TEST$ sudo -i ${PWD}/scripts/password_management.sh -h
29+
-i: Initialize all password environment variables and settings
30+
-s: set_user: Set user password
31+
-p: Manage Podman secret
32+
-l: List Podman secrets
33+
-h: print this list
34+
```
35+
36+
A cli one liner to grab passwords (this also demonstrates how we're using Ansible-vault in extract_secrets.sh):
37+
```bash
38+
#where wazuh_api is the service user whose password you want:
39+
USER_NAME=wazuh_api
40+
sudo -i ansible-vault view /etc/lme/vault/$(sudo -i podman secret ls | grep $USER_NAME | awk '{print $1}')
41+
```

0 commit comments

Comments
 (0)