Skip to content

Commit 7e3e266

Browse files
committed
Large README update
1 parent a5c45d7 commit 7e3e266

File tree

1 file changed

+120
-52
lines changed

1 file changed

+120
-52
lines changed

README.md

Lines changed: 120 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -1,73 +1,140 @@
1-
# Ansible Playbook for setting up a proxmox cluster with some debian and (LXC) container VMs, some tasmota IOT devices, and some Openwrt Access Points and Routers with VLAN separation of the SSIDs on each of the radios
1+
# Ansible Playbook for Proxmox, Debian (LXC) Containers, VMs, Physical nodes, Tasmota IOT, OpenWRT Routers and Access Points
22

3-
A couple of [Ansible](http://docs.ansible.com/) playbooks which runs a series of configuration steps to set up an SOE based on Debian, in order to provide a solid foundation for subsequent actions.
3+
## Introduction
4+
5+
### What are we?
6+
7+
We're a couple of [Ansible](http://docs.ansible.com/) playbooks which
8+
runs a series of configuration steps to set up a SOE based on Debian,
9+
in order to provide a solid foundation for subsequent actions outside
10+
the control of these playbooks. We can provide optional ***(partial)
11+
configuration of a Proxmox cluster, manage Debian (LXC) containers,
12+
VMs, and physical hardware (including some hardware management),
13+
configuration of Tasmota IOT devices, and some OpenWRT Access Points
14+
and Routers configurable with VLAN separation of the SSIDs on each of
15+
the radios***.
416

517
It borrows heavily from the work of: [Luke Harvey](https://github.com/lukeharvey), [Bryan Kennedy](https://plusbryan.com/my-first-5-minutes-on-a-server-or-essential-security-for-linux-servers), [Ryan Eschinger](http://ryaneschinger.com/blog/securing-a-server-with-ansible/), [Ashley Rich](https://github.com/A5hleyRich/wordpress-ansible), and [Digital Ocean](https://www.digitalocean.com/community/tutorials/initial-server-setup-with-ubuntu-14-04)
618

719
It will perform the following:
8-
* Create a new ansible "super user" with with root privileges and public key authentication on your proxmox and debian machines as well as the containers
20+
* Setup (not initial deploy)/manage Debian 10 (Buster), 11 (Bullseye), 12 (Bookworm) (and Proxmox 6,7,8) physical machines, LXC and VM (Proxmox) instances
21+
* Create a new Ansible "super user" with with root privileges and public key authentication on your Proxmox and Debian machines as well as the containers
922
* Set root user credentials, and a primary user credentials, ssh keys, sudoers to root
1023
* Implement several SSH hardening techniques
1124
* Configure the timezone and enable time synchronization
1225
* Modify the hostname and hosts file
1326
* Install the admin's account with some nice .bashrc settings
1427
* Install a package baseline appropriate for laptops, desktops, virtualisation hosts, containers, as appropriate
15-
* Override some of debian's more annoying defaults
16-
* Manage tasmota settings on your IOT devices
17-
* Manage openwrt settings on your openwrt devices, including installing VLANs 10,30,40,70 and configuring radios on them
18-
19-
# What can we automate, what point-and-drool do we still need to do?
20-
For physical boxes, VMs and LXC containers, I take the vanilla debian
28+
* Override some of Debian's more annoying defaults per my opinionated opinion
29+
* Manage Tasmota settings on your IOT devices
30+
* Manage OpenWRT settings on your OpenWRT devices, including installing VLANs 10,30,40,70 and configuring radios on them
31+
32+
## What aren't we?
33+
34+
This playbook does not deploy Proxmox from scratch. It also doesn't
35+
require Proxmox - but I find it very helpful. It can convert an
36+
existing Proxmox cluster into being managed like any of our other
37+
Debian SOE's (as far as our SOE interaction is concerned - user
38+
accounts, keys, SOE configuration, backups, parts of networking;
39+
however our playbooks don't do Proxmox management on the Proxmox
40+
cluster). Likewise, we don't rely on any aspect of that (I do it,
41+
because I want my helpful tools to also be used on my Proxmox
42+
cluster). It doesn't deploy new VMs/CTs via Proxmox yet - you still
43+
need to deploy them some other means, or manually. We also don't
44+
preclude you doing management outside these playbooks. And of course,
45+
not practicing what I preach, I don't have a properly thought out dev
46+
environment nor have any automated testing infrastructure, because I
47+
ain't doing anything important here. It doesn't manage every aspect
48+
of machines - plenty of my machines can be redeployed from scratch and
49+
will come up identically because their OS settings are managed via
50+
these plays, and I have otherwise bind mounted their data into their
51+
LXC containers, but someone had to populate that data and application
52+
configuration in the first place. And the Proxmox cluster, Ceph
53+
installation and configuration get managed the Proxmox way. But Ceph
54+
secrets distribution and mounting on clients? We can manage that here.
55+
56+
While it can manage OpenWRT installations, it similarly isn't going to
57+
deploy OpenWRT from scratch (although we can now apply a sysconfig
58+
update and keep your firmware versions encoded into our config!), and
59+
you'll want to set up initial networking, switching, wifi radio, etc
60+
at initial installation time, but we come along and install our
61+
packages and configs then tweak those UCI settings per our config.
62+
63+
Similarly story for Tasmota - very few of my devices have had their
64+
GPIOs programmed through these playbooks, for example. But every time
65+
I shift an AP around, I update the relevant primary/failover SSID of
66+
all my Tasmota devices. It wouldn't be much work to arrange firmware
67+
to be patched like in the OpenWRT playbook either (makes me more
68+
nervous, but is it any harder to open up a bricked 240v smart plug
69+
with triangle security screws vs a bricked TP-Link Archer D7 v1 held
70+
together with fragile plastic clips?).
71+
72+
We are not DRY - many configs have to be specified all over the place.
73+
Sorry.
74+
75+
## What can we automate, what point-and-drool do we still need to do?
76+
77+
For physical boxes, VMs and LXC containers (both provided through
78+
Proxmox in my case), I rely on the machine being deployed the
79+
traditional manual way (haven't yet automated talking to the Proxmox
80+
API, even though the Proxmox boxes themselves are managed in this
81+
playbook like any other Debian box). We take this vanilla Debian
2182
install where I've just clicked through the install ISO steps in a
22-
fairly braindead fashion (our playbooks goes and fixes a bunch of
23-
things asked in install anyway), apply bootstrap.yml first, then
24-
initial_server_config.yml to bring the configuration up to my SOE and
25-
upon any further update to my config or after the machine is patched.
26-
I don't automate the entire playbook being played because I still have
27-
far too many pets and manual changes that need checking, and the
28-
playbook is quite slow to run all the way through, so I'm frequently
29-
using `--diff`, `--check` and `--tags` to limit the scope of changes
30-
to what I'm currently concerned with. It would be easy to email
31-
myself an output of `--check --diff initial_server_config.yml` every
32-
night, but I get enough email as-is.
33-
34-
For OpenWRT APs and tasmota configurations however, we still rely on
83+
fairly braindead fashion (I haven't setup Debian fully-automated, nor
84+
preseed or anything like that; our playbooks goes and then fixes a
85+
bunch of things asked in install anyway), apply bootstrap.yml first,
86+
then initial_server_config.yml to bring the configuration up to my SOE
87+
and upon any further update to my config or after the machine is
88+
patched. I don't automate the entire playbook being regularly played
89+
or setup automated drift-detection, because I still have far too many
90+
pets and manual changes that need checking, and the playbook is quite
91+
slow to run all the way through, so I'm frequently using `--diff`,
92+
`--check` and `--tags` to limit the scope of changes to what I'm
93+
currently concerned with. It would be easy to email myself an output
94+
of `--check --diff initial_server_config.yml` every night, but I get
95+
enough email as-is.
96+
97+
For OpenWRT APs and Tasmota configurations however, we still rely on
3598
some manual configuration for a new device from scratch before
3699
applying these playbooks - but it does make it easier to wholesale
37100
change all of your SSIDs for example. For upgrades however, we can
38101
automate all of the tasks needed to bring your device back into
39102
compliance with your config without any manual configuration. I apply
40-
it routinely after any minor upgrade of the openwrt device, perhaps
103+
it routinely after any minor upgrade of the OpenWRT device, perhaps
41104
checking with `--diff` `--check` first if I'm a little nervous. I do
42105
test on my VMs first (sometimes I'm even sensible enough to test on
43106
one of my virtualised APs before I test it on my internet gateway),
44107
and take a snapshot manually beforehand. For major upgrades, it
45108
hasn't broken majorly for me yet, but my planning tends to be a little
46-
more careful around these events. For setting up a new openwrt AP, my
109+
more careful around these events. For setting up a new OpenWRT AP, my
47110
manual configuration tends to be limited to setting switch vlan
48111
tagging information in /etc/config/network, and assigning radio0 and
49112
radio1 consistently with my other devices, before letting
50113
uci_config.yml loose on VLAN and network definitions and what SSIDs
51114
we've assigned to which radios. This sometimes involves swapping
52115
radio0 and radio1 pci/hardware devices in /etc/config/wireless.
53116

54-
## Requirements
117+
# Requirements
55118

56-
* [Ansible](http://docs.ansible.com/ansible/intro_installation.html) installed locally on your machine
57-
* You probably want to install `ansible-mitogen` (and `python3-mitogen`) on your ansible server too, for my ansible.cfg sets `strategy = mitogen_linear` to greatly accelerate the playbook (it works with that setting disabled if you can't install migoten, but mitogen has never created any detectable problems for me). I have only tested this from a debian machine (debian 11,12).
119+
* [Ansible](http://docs.ansible.com/ansible/intro_installation.html) installed locally on your machine to run the playbooks. Tested on Debian's Ansible from Debian 10 (Buster) to Debian 12 (Bookworm)
120+
* You probably want to install `ansible-mitogen` (and `python3-mitogen`) on your Ansible server too, for my ansible.cfg sets `strategy = mitogen_linear` to greatly accelerate the playbook (it works with that setting disabled if you can't install migoten, but mitogen has never created any detectable problems for me). I have only tested this from a Debian machine (Debian 11,12).
58121
* Ideally, you'd create a gpg encrypted file in misc/vault-password.gpg, and verify it can be read with: misc/get-vault-pass.sh
59-
* Openwrt plays rely on [ansible-openwrt](https://github.com/gekmihesg/ansible-openwrt), which is published as a [galaxy collection](https://galaxy.ansible.com/ui/repo/published/nn708/openwrt/).
60-
* Tasmota plays rely on [ansible-tasmota](https://github.com/tobias-richter/ansible-tasmota), which is available through [ansible galaxy](https://galaxy.ansible.com/ui/standalone/roles/tobias_richter/tasmota/), but which [I've modified](https://github.com/spacelama/ansible-tasmota) to allow for and transparently recovers from the tasmota device spontaneously rebooting after certain configurations are applied.
122+
* OpenWRT plays rely on [ansible-openwrt](https://github.com/gekmihesg/ansible-openwrt), which is published as a [galaxy collection](https://galaxy.ansible.com/ui/repo/published/nn708/openwrt/).
123+
* Tasmota plays rely on [ansible-tasmota](https://github.com/tobias-richter/ansible-tasmota), which is available through [ansible galaxy](https://galaxy.ansible.com/ui/standalone/roles/tobias_richter/tasmota/), but which [I've modified](https://github.com/spacelama/ansible-tasmota) to allow for and transparently recovers from the Tasmota device spontaneously rebooting after certain configurations are applied.
61124

62-
## Configuration
125+
# Initial playbook setup and Configuration
63126

64127
Clone the repo
65128

66129
```
67130
$ git clone --recurse-submodules https://github.com/spacelama/ansible-initial-server-setup.git
68131
```
69132

70-
# Debian Linux LXC and VMs, desktops, servers, Proxmox etc
133+
## Configure Debian Linux LXC and VMs, desktops, servers, Proxmox etc
134+
135+
Look at all the .gitignore files scattered around for an indication as
136+
to what secrets you may need. Populate those files with secrets
137+
properly encoded with the vault.
71138

72139
Modify the variables in **_vars/main.yml_** according to your needs:
73140

@@ -85,7 +152,7 @@ Modify the variables in **_vars/main.yml_** according to your needs:
85152

86153
**ssh_port:** your chosen SSH port
87154

88-
Modify **_hosts.yml_** with your various host settings
155+
Modify **_hosts.yml_** with your various host settings (and public keys etc).
89156

90157
I put a bunch of vault encoded per-host secrets in
91158
`host_vars/<hostname>.yml` too, encrypted via `ansible-vault
@@ -96,9 +163,9 @@ further protect their contents (likewise for `files/main.*.password`
96163
and `files/ap.*` which should have been vault entries in the first
97164
place). Host settings I want to track in git are in `hosts.yml`.
98165

99-
## Bootstrapping
166+
## Bootstrapping your new LXC/VM/Physical node to be Ansible managed
100167

101-
Install the ansible_adm account and the sudo permissions for this account to escalate to root with:
168+
Install the Ansible_adm account and the sudo permissions for this account to escalate to root with:
102169

103170
`$ ansible-playbook bootstrap.yml -u root -k --extra-vars "target=dirac-new" --ask-vault-pass # always run with --check when first starting out!`
104171

@@ -116,33 +183,33 @@ Fix up an lxc container:
116183

117184
## Testing
118185

119-
It's not foolproof, but try `--check` prior to each real ansible.
186+
It's not foolproof, but try `--check` prior to each real ansible invocation.
120187

121188
`--diff` is extremely handy, but not foolproof when also running --check.
122189

123190
I frequently `--limit` to hosts or away from hosts.
124191

125192
`$ ansible-playbook --ask-vault-pass initial_server_setup.yml --diff --check --limit='!dirac-new,!fs-new,!hass-debian,!mail'`
126193

127-
--limit also useful when you get a new openwrt AP or tasmota device:
194+
--limit also useful when you get a new OpenWRT AP or Tasmota device:
128195

129196
`$ ansible-playbook -v openwrt_maintenance.yml --diff --check`
130197

131198
`$ ansible-playbook tasmota_maintenance.yml --diff --check --limit patiofluro-power,loungefrontlight-power --extra-vars "setpsk=true" --extra-vars "setsyslog=true"`
132199

133-
## Production
200+
# Production
134201

135202
Then run the playbooks:
136203

137204
`$ ansible-playbook --ask-vault-pass initial_server_setup.yml --diff --limit='!dirac-new,!fs-new,!hass-debian,!mail'`
138205

139-
Likewise for tasmota and openwrt:
206+
Likewise for Tasmota and OpenWRT:
140207

141208
`$ ansible-playbook -v openwrt_maintenance.yml --diff`
142209

143210
`$ ansible-playbook tasmota_maintenance.yml --diff --limit patiofluro-power,loungefrontlight-power --extra-vars "setpsk=true" --extra-vars "setsyslog=true"`
144211

145-
## Using tags to limit the scope of changes
212+
# Using tags to limit the scope of changes
146213

147214
initial_server_setup.yml uses tags on each role.
148215

@@ -154,19 +221,19 @@ or to stop the webserver role from running while still running everything else:
154221

155222
`$ ansible-playbook --diff initial_server_setup.yml --limit met,webserver,iot --skip-tags webserver --check`
156223

157-
## Ad hoc commands:
224+
# Ad hoc commands:
158225

159226
Using [ansbak](https://github.com/spacelama/ansbak), you can get an
160227
analogous result to using `pdsh ... | dshbak -c`, using your same
161-
ansible inventory as usual (such that this has become my main way of
228+
Ansible inventory as usual (such that this has become my main way of
162229
interacting with my fleet instead of pdsh or cssh):
163230

164231
`$ ANSIBLE_FORCE_COLOR=true ansible laptops,desktops,servers,containers -i ~/Ansible/hosts.yml -m shell -a "sudo env bash -c 'dpkg --get-selections | grep munin'" | ansbak.py`
165232

166233

167-
# Openwrt routers, wireless APs, VLANs
234+
# OpenWRT routers, wireless APs, VLANs
168235

169-
`vars/openwrt.yml` contains some settings for all your openwrt devices
236+
`vars/openwrt.yml` contains some settings for all your OpenWRT devices
170237
(routers, APs etc), and sets up a bunch of VLANs for your IOT devices,
171238
windows devices etc, assigned per MAC address (VLAN decided by which
172239
SSID your device joins - my IOT devices from China only know about my
@@ -187,9 +254,9 @@ firewalls, banip etc), but I've been using this to configure fresh APs
187254
from scratch. Have a good backup of your APs before you run this for
188255
the first time though if you've already set them up in any way. The
189256
radio stuff is expected to be quite fragile, and has only received
190-
most testing on current openwrt 22.03. and 23.05.*
257+
most testing on current OpenWRT 22.03.*, 23.05.*, 24.10.*.
191258

192-
My inventory is in hosts.yml, and tells us whether the openwrt device
259+
My inventory is in hosts.yml, and tells us whether the OpenWRT device
193260
uses DSA switch config or the old definition, via
194261
`openwrt_dsa_switch_config`. IP address are decided by
195262
`inet_addr_suffix` in your inventory to assign
@@ -201,7 +268,7 @@ offload the biggest of non-essential packages from even my smallest
201268
wavlink wn575a3 with 8MB of flash). `type` is 'ap' or 'router' and
202269
decided which packages to install and how to set up DHCP.
203270

204-
Run the playbook to configure all openwrt devices configured in
271+
Run the playbook to configure all OpenWRT devices configured in
205272
hosts.yml:
206273

207274
`$ ansible-playbook openwrt_maintenance.yml --diff # --check to verify changes first`
@@ -218,7 +285,8 @@ the package installation and file sync with:
218285
`$ ansible-playbook openwrt_maintenance.yml --diff --extra-vars "run_uci_config=yes" --extra-vars "run_packages_config=no" --tags openwrt_config`
219286

220287
We can also issue firmware patches, and then run the configuration
221-
update step again (which assumes run_uci_config=yes):
288+
update step again (which assumes run_uci_config=yes, which of course
289+
is more fragile over major version updates):
222290

223291
`$ ansible-playbook openwrt_upgrade.yml --diff`
224292

@@ -231,7 +299,7 @@ them, so long as they have firmware_image set. If unset for a
231299
particular device, that device never gets patched by this playbook.
232300

233301
Keep in mind this playbook is much more likely to fail than the
234-
openwrt_maintenance one, because who knows what assumptions we make or
302+
OpenWRT_maintenance one, because who knows what assumptions we make or
235303
your configuration makes that will break in each update (be careful of
236304
migrations to DSA switches, as always), so keep firm backups (or be
237305
prepared to run the playbook multiple times until you get it right).
@@ -262,10 +330,10 @@ This will set SSID1 and you might choose fallback SSID2 per host in
262330
your hosts.yml inventory - here, we set the first one to be your
263331
primary SSID that you mesh or roam between throughout your host, and
264332
SSID2 might be the *second* closest AP to where your device normally
265-
sits. That way, tasmota will lock onto (and roam via 80211.r) your
333+
sits. That way, Tasmota will lock onto (and roam via 80211.r) your
266334
closest AP on the primary SSID by default, but if that AP continues to
267335
serve valid wifi connections but loses connectivity to the network
268-
itself, tasmota's watchdog will notice this loss of packet
336+
itself, Tasmota's watchdog will notice this loss of packet
269337
connectivity, and will failover to the second closest AP that is
270338
hopefully still on a working network.
271339

@@ -277,12 +345,12 @@ I set `disable_default_reset_on_power_reset7=1` on devices that
277345
frequently lose their power, so they don't accidentally get firmware
278346
reset.
279347

280-
The ansible tasmota provider thinks PSK and syslog change every time
348+
The Ansible Tasmota provider thinks PSK and syslog change every time
281349
you try to adjust them, even if actually unchanged, so by default, I
282350
don't set them. They only get attempted to be set when you supply
283351
setpsk=true and setsyslog=true. But since writing that, I've set a
284352
lot more parameters, some of which unconditionally overwrite the
285-
setting even when unchanged, and some of these result in the tasmota
353+
setting even when unchanged, and some of these result in the Tasmota
286354
device rebooting every time. I've had to put a [workaround in
287355
ansible-tasmota](https://github.com/spacelama/ansible-tasmota) to
288356
allow the device to recover and continue setting subsequent

0 commit comments

Comments
 (0)