Skip to content

Commit

Permalink
Cutting 2021-09-22.0 Release.
Browse files Browse the repository at this point in the history
Remove Saltstack; add Ansible.

Added:
* Add Movie Studio encoding templates.
* Add fix for failed to run vncproxy on pve.
* Add note about fc-cache now generating .uuid files.
* Add GPG agent forwarding for WSL2 and Linux machines.
* Add CLI static DHCP configuration.
* Add apt auto selection to docs.
* Add 7 days to die administrative commands link.
* Add git commands for creating repository tracked hooks.
* Add disable for Asus Armoury Crate.
* Add additional git merge instructions for backing out and generating log.
* Add firefly baremetal setup instructions.
* Add git stash notes.
* Add crashplan LXC/KVM/Baremetal instructions.
* Add Instructions for GPU passthru to LXC containers.
* Add pve subscription removal service.
* Add gitea troubleshooting information for timeouts and duplicate keys.
* Add dropbear service.
* Add ZFS sync send/recv commands with automation.
* Add installing older game versions on Steam.
* Add SSH blocked through wireguard network resolution.
* Add wireguard-initramfs instructions.
* Add PFX RSA public/private, certificate extraction instructions.
* Add wireguard kernel debugging configuration.
* Add ansible notes.
* Add ansible auto-decrypt vault with security key scripts.
* Add Movie Studio encoding templates.
* Add fix for failed to run vncproxy on pve.
* Add note about fc-cache now generating .uuid files.
* Add GPG agent forwarding for WSL2 and Linux machines.
* Add CLI static DHCP configuration.
* Add apt auto selection to docs.

Changed:
* Correct links and formatting for gpg/ansible docs as well.
* Update ZFS manaul disk replacement instructions.
* Update for automatic partitioning, manual swap, locating devices/ZFS GUID.
* Update PFX RSA cert extraction to single commands.
* Update ZFS instructions with Encryption and dataset usage.
* Update proxmox instructions for version 7.
* Update PVE with GPU passthru instructions.

Removed:
* Remove saltstack configuration notes, add ansible configuration notes.

Fixed:
* Update pygments to 2.7.4 addressing CVE-2021-27291
* Update jinja, urllibs based on security advisories.
  • Loading branch information
r-pufky committed Sep 22, 2021
1 parent 165706a commit ecf34f5
Show file tree
Hide file tree
Showing 337 changed files with 161,939 additions and 83,633 deletions.
52 changes: 52 additions & 0 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,57 @@
# [Changelog][3g]

## 2021-09-22.0
Remove Saltstack; add Ansible.

Added:
* Add Movie Studio encoding templates.
* Add fix for failed to run vncproxy on pve.
* Add note about fc-cache now generating .uuid files.
* Add GPG agent forwarding for WSL2 and Linux machines.
* Add CLI static DHCP configuration.
* Add apt auto selection to docs.
* Add 7 days to die administrative commands link.
* Add git commands for creating repository tracked hooks.
* Add disable for Asus Armoury Crate.
* Add additional git merge instructions for backing out and generating log.
* Add firefly baremetal setup instructions.
* Add git stash notes.
* Add crashplan LXC/KVM/Baremetal instructions.
* Add Instructions for GPU passthru to LXC containers.
* Add pve subscription removal service.
* Add gitea troubleshooting information for timeouts and duplicate keys.
* Add dropbear service.
* Add ZFS sync send/recv commands with automation.
* Add installing older game versions on Steam.
* Add SSH blocked through wireguard network resolution.
* Add wireguard-initramfs instructions.
* Add PFX RSA public/private, certificate extraction instructions.
* Add wireguard kernel debugging configuration.
* Add ansible notes.
* Add ansible auto-decrypt vault with security key scripts.
* Add Movie Studio encoding templates.
* Add fix for failed to run vncproxy on pve.
* Add note about fc-cache now generating .uuid files.
* Add GPG agent forwarding for WSL2 and Linux machines.
* Add CLI static DHCP configuration.
* Add apt auto selection to docs.

Changed:
* Correct links and formatting for gpg/ansible docs as well.
* Update ZFS manaul disk replacement instructions.
* Update for automatic partitioning, manual swap, locating devices/ZFS GUID.
* Update PFX RSA cert extraction to single commands.
* Update ZFS instructions with Encryption and dataset usage.
* Update proxmox instructions for version 7.
* Update PVE with GPU passthru instructions.

Removed:
* Remove saltstack configuration notes, add ansible configuration notes.

Fixed:
* Update pygments to 2.7.4 addressing CVE-2021-27291
* Update jinja, urllibs based on security advisories.

## 2021-02-28.0
Finish sphinx CT directive migration.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,139 @@
#!/usr/bin/env bash
#
# Incremental ZFS send/recv backup script
# Original: https://github.com/bahamas10/zincrsend
# This Version: https://github.com/r-pufky/zincrsend
#
# Exit codes:
# 0: success.
# 1: local snapshot creation failed.
# 2: latest remote snapshot does not exist locally (manual intervention
# required).
# 3: ZFS send/recv failed.


################################################################################
# Configuration options
################################################################################
# Recursive datasets to send. (-R) will remove snapshots that have been deleted
# locally on the remote end as well. Dataset does *NOT* need to have children.
datasets=(
tank/example
)

# Remote server connection settings.
remote_server='172.31.255.254'
remote_user='example_user'
remote_port='22'
remote_pool='backup_tank'
remote_command_prefix='sudo'
remote_ssh_opts=(-i example_user.key)

# prefix to use for snapshots created by this script
snapshot_prefix=''
# Number of snapshots to retain after successful sync. 0 disables.
snapshot_retention=2
# snapshot options: https://openzfs.github.io/openzfs-docs/man/8/zfs-snapshot.8.html
snapshot_opts=(-r)
# send options: https://openzfs.github.io/openzfs-docs/man/8/zfs-send.8.html
send_opts=(-R -w)
################################################################################

SSH() {
echo "ssh ${remote_ssh_opts[*]} ${remote_server} ${remote_command_prefix} $*"
ssh \
"${remote_ssh_opts[@]}" \
-l "${remote_user}" \
-p "${remote_port}" \
"${remote_server}" \
"${remote_command_prefix}" \
"${@}"
}

process() {
local ds=${1}

echo ''
echo "processing dataset: ${ds}"
echo ''

# Step 1 - snapshot locally
local now=$(date +%s)
local snap=${ds}@${snapshot_prefix}${now}
echo "creating snapshot locally: ${snap}"
if ! sudo /usr/sbin/zfs snapshot "${snapshot_opts[@]}" "${snap}"; then
echo "[ERROR] failed to snapshot ${ds}" >&2
exit 1
fi

# Step 2 - find the latest remote snapshot
local rds=$remote_pool/${ds#*/}
local inc_snap=
local inc_opts=()
echo "fetching latest remote snapshot for dataset: ${rds}"
local rsnap=$(SSH /usr/sbin/zfs list -H -o name,creation -p -t snapshot -r "${rds}" | \
grep "^${rds}@" | \
sort -n -k 2 | \
tail -1 | \
awk '{ print $1 }')

if [[ -n ${rsnap} ]]; then
echo "latest remote snapshot: ${rsnap}"
inc_snap=${rsnap#*@}
# assert that ${inc_snap} exists locally
if ! sudo /usr/sbin/zfs list -t snapshot "${ds}@${inc_snap}" &>/dev/null; then
echo "[ERROR] could not find ${rsnap} locally (${ds}@${inc_snap} not found)" >&2
exit 2
fi
inc_opts+=(-I "@${inc_snap}")
else
echo "no snapshot found for ${ds} - doing full send/recv"
fi

# Step 3: send from latest remote to newly created or do a full send
if [[ -n ${inc_snap} ]]; then
echo "zfs sending (incremental) @${inc_snap} -> ${snap} to ${rds}"
else
echo "zfs sending ${snap} to ${rds}"
fi
# Receive options: Always use snapshot as base (remote changes on after
# snapshot will cause recieve to fail otherwise); recieving pool receieves
# filesystem unmounted to prevent mount collisions.
if ! sudo /usr/sbin/zfs send "${send_opts[@]}" "${inc_opts[@]}" "${snap}" | SSH /usr/sbin/zfs recv -Fuv "${rds}"; then
echo "[ERROR] failed to send $snap to ${remote_server} ${rds}" >&2
exit 3
fi

# Step 4: After successful sync, trim the last X snapshots (sync'ed on next run).
if [[ ${snapshot_retention} -gt 0 ]]; then
echo "retainng the last ${snapshot_retention} snapshots for ${ds}"
# Identify the latest X snapshots for a given dataset (creation, newest to oldest)
zfs_latest=`/usr/sbin/zfs list -H -t snapshot -o name -S creation | grep ^${ds}@ | head -${snapshot_retention}`
# Identify ALL snapshots for a given dataset (creation, newest to oldest)
zfs_delete=`/usr/sbin/zfs list -H -t snapshot -o name -S creation | grep ^${ds}@`

echo "all snapshots: $(echo ${zfs_delete[@]})"
echo "retained snapshots: $(echo ${zfs_latest[@]})"
# Remove latest snapshots from all set.
for keep_snap in ${zfs_latest[@]}; do
zfs_delete=( "${zfs_delete[@]/${keep_snap}}" );
done

echo "snapshots to remove: $(echo ${zfs_delete[@]})"
# Destroy old snapshots
for snap in ${zfs_delete[@]}; do
/usr/sbin/zfs destroy ${snap}
done
else
echo "zfs snapshot rentention management disabled"
fi
}

echo "starting on $(date)"

code=0
for ds in "${datasets[@]}"; do
process "${ds}"
done
echo
echo "script ran for ~$((SECONDS / 60)) minutes (${SECONDS} seconds)"
13 changes: 13 additions & 0 deletions docs/_downloads/56d718151ad63cb015401870f925b21b/gpg-agent.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# https://github.com/drduh/config/blob/master/gpg-agent.conf
# https://www.gnupg.org/documentation/manuals/gnupg/Agent-Options.html
enable-ssh-support
ttyname $GPG_TTY
default-cache-ttl 60
max-cache-ttl 120
pinentry-program /usr/bin/pinentry-curses
#pinentry-program /usr/bin/pinentry-tty
#pinentry-program /usr/bin/pinentry-gtk-2
#pinentry-program /usr/bin/pinentry-x11
#pinentry-program /usr/bin/pinentry-gnome3
#pinentry-program /usr/local/bin/pinentry-curses
#pinentry-program /usr/local/bin/pinentry-mac
61 changes: 61 additions & 0 deletions docs/_downloads/b6776082685e0b7b36237247aad145c3/gpg.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# https://github.com/drduh/config/blob/master/gpg.conf
# https://www.gnupg.org/documentation/manuals/gnupg/GPG-Configuration-Options.html
# https://www.gnupg.org/documentation/manuals/gnupg/GPG-Esoteric-Options.html
# Use AES256, 192, or 128 as cipher
personal-cipher-preferences AES256 AES192 AES
# Use SHA512, 384, or 256 as digest
personal-digest-preferences SHA512 SHA384 SHA256
# Use ZLIB, BZIP2, ZIP, or no compression
personal-compress-preferences ZLIB BZIP2 ZIP Uncompressed
# Default preferences for new keys
default-preference-list SHA512 SHA384 SHA256 AES256 AES192 AES ZLIB BZIP2 ZIP Uncompressed
# SHA512 as digest to sign keys
cert-digest-algo SHA512
# SHA512 as digest for symmetric ops
s2k-digest-algo SHA512
# AES256 as cipher for symmetric ops
s2k-cipher-algo AES256
# UTF-8 support for compatibility
charset utf-8
# Show Unix timestamps
fixed-list-mode
# No comments in signature
no-comments
# No version in output
no-emit-version
# Disable banner
no-greeting
# Long hexidecimal key format
keyid-format 0xlong
# Display UID validity
list-options show-uid-validity
verify-options show-uid-validity
# Display all keys and their fingerprints
with-fingerprint
# Display key origins and updates
#with-key-origin
# Cross-certify subkeys are present and valid
require-cross-certification
# Disable caching of passphrase for symmetrical ops
no-symkey-cache
# Enable smartcard
use-agent
# Disable recipient key ID in messages
throw-keyids
# Default/trusted key ID to use (helpful with throw-keyids)
#default-key 0xFF3E7D88647EBCDB
#trusted-key 0xFF3E7D88647EBCDB
# Group recipient keys (preferred ID last)
#group keygroup = 0xFF00000000000001 0xFF00000000000002 0xFF3E7D88647EBCDB
# Keyserver URL
#keyserver hkps://keys.openpgp.org
#keyserver hkps://keyserver.ubuntu.com:443
#keyserver hkps://hkps.pool.sks-keyservers.net
#keyserver hkps://pgp.ocf.berkeley.edu
# Proxy to use for keyservers
#keyserver-options http-proxy=http://127.0.0.1:8118
#keyserver-options http-proxy=socks5-hostname://127.0.0.1:9050
# Verbose output
#verbose
# Show expired subkeys
#list-options show-unusable-subkeys
Loading

0 comments on commit ecf34f5

Please sign in to comment.