Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tools): create prototype for infra CLI #69

Draft
wants to merge 27 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
627c5e9
feat(tools): draft golang CLI
nicklasfrahm Dec 9, 2023
af02f18
docs(cli): draft user experience for enrollment
nicklasfrahm Dec 12, 2023
bf280f0
fix(docs): draft proposal for bootstrap process
nicklasfrahm Dec 15, 2023
6420f02
fix(cli): add flags to usage info
nicklasfrahm Dec 15, 2023
ec0b25d
Merge branch 'main' into feature/cli
nicklasfrahm Dec 16, 2023
b3f2294
fix(cli): create pseudo-code for zone bootstrap
nicklasfrahm Dec 17, 2023
5507835
fix(cli): rename bootstrap command to up
nicklasfrahm Dec 17, 2023
45cb134
Merge branch 'main' into feature/cli
nicklasfrahm Dec 18, 2023
3f27718
feat(cli): define command signature
nicklasfrahm Dec 19, 2023
06cd170
Merge remote-tracking branch 'origin/main' into feature/cli
nicklasfrahm Dec 19, 2023
df14bee
fix(cli): move router configuration into own type
nicklasfrahm Dec 19, 2023
ca04509
fix(tools): allow installation via Makefile
nicklasfrahm Dec 24, 2023
2c76e4f
Merge remote-tracking branch 'origin/main' into feature/cli
nicklasfrahm Dec 25, 2023
1fb4465
fix(config): use new interface names
nicklasfrahm Dec 25, 2023
f0bf7a9
fix(config): use new interface names
nicklasfrahm Dec 25, 2023
39191c1
fix(config): align config/ dir with go standard project layout
nicklasfrahm Dec 28, 2023
9ed8ba2
fix(cli): add shorthand flag
nicklasfrahm Dec 29, 2023
1fc7dbb
fix(cli): create reusable function
nicklasfrahm Dec 29, 2023
947fe9d
fix(config): create data structure for zone config
nicklasfrahm Dec 29, 2023
07711d4
Merge remote-tracking branch 'origin/main' into feature/cli
nicklasfrahm Dec 29, 2023
89738e3
chore(deps): update go modules
nicklasfrahm Dec 29, 2023
58d5bb4
fix(cli): implement prototype for preflight check
nicklasfrahm Dec 29, 2023
8b69ae7
fix(cli): improve port probe
nicklasfrahm Dec 29, 2023
570ca20
fix(cli): create SSH client
nicklasfrahm Dec 29, 2023
6a7fb4a
fix(config): fix search domain
nicklasfrahm Jan 4, 2024
7b477af
fix(docs): document identity management
nicklasfrahm Jan 23, 2024
356e285
Merge remote-tracking branch 'origin/main' into feature/cli
nicklasfrahm Jan 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 21 additions & 13 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,26 +1,34 @@
REGISTRY := ghcr.io
REPO := nicklasfrahm/infrastructure
TARGET ?= metal
SOURCES := $(shell find . -name "*.go")
PLATFORM ?= $(shell go version | cut -d " " -f 4)
GOOS := $(shell echo $(PLATFORM) | cut -d "/" -f 1)
GOARCH := $(shell echo $(PLATFORM) | cut -d "/" -f 2)
SUFFIX := $(GOOS)-$(GOARCH)
VERSION ?= $(shell git describe --always --tags --dirty)
TARGET ?= ic
PLATFORM ?= $(shell go version | cut -d " " -f 4)
VERSION ?= $(shell git describe --always --tags --dirty)
REPO := $(shell git remote get-url origin | sed 's|https://github.com||g' | grep -oE '[a-z-]+/[a-z-]+')
SOURCES := $(shell find . -name "*.go")
GOOS := $(shell echo $(PLATFORM) | cut -d "/" -f 1)
GOARCH := $(shell echo $(PLATFORM) | cut -d "/" -f 2)
SUFFIX := $(GOOS)-$(GOARCH)
BUILD_FLAGS := -ldflags="-s -w -X main.version=$(VERSION)"
BIN_DIR := /usr/local/bin
REGISTRY := ghcr.io

ifeq ($(GOOS),windows)
SUFFIX = $(GOOS)-$(GOARCH).exe
endif

BINARY ?= bin/$(TARGET)-$(SUFFIX)
BINARY ?= bin/$(TARGET)-$(SUFFIX)

build: bin/$(TARGET)-$(SUFFIX)
build: $(BINARY)

bin/$(TARGET)-$(SUFFIX): $(SOURCES)
$(BINARY): $(SOURCES)
@mkdir -p $(@D)
CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build $(BUILD_FLAGS) -o $(BINARY) cmd/$(TARGET)/*.go
CGO_ENABLED=0 GOOS=$(GOOS) GOARCH=$(GOARCH) go build $(BUILD_FLAGS) -o $@ cmd/$(TARGET)/*.go

install: $(BIN_DIR)/$(TARGET)

$(BIN_DIR)/$(TARGET): $(BINARY)
sudo install -m 0755 $^ $@

uninstall:
sudo rm -f $(BIN_DIR)/$(TARGET)

.PHONY: docker
docker:
Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,17 @@ Since 2018 I started to make significant changes by automating the configuration

Now, I am in the process of upgrading my infrastructure to become cloud-native and fully automated. The goal is to provide a RESTful API to provision baremetal machines from scratch, configure networking, and bootstrap and autoscale Kubernetes clusters.

## Identity 🔑

**Status:** 🚧 Implementation

The primary method of authentication will be Open ID Connect (OIDC) and OIDC federation. In this setup [accounts.google.com][google-accounts] will serve as the root of trust to support the following scenarios:

- **Single-sign-on (SSO) as a user**
In a browser and using CLIs, I can use my Google account, which is protected via two physical security keys, to authenticate myself to my services.
- **A service running in Kubernetes**
Each service running in Kubernetes will have a corresponding service account in Google Cloud. The service account will then use OIDC federation to authenticate itself to any other services and across clusters.

## Networking 🔌

Below, I describe the network setup of my Homelab. Some parts of it are already implemented, while others are still being conceptualized.
Expand Down Expand Up @@ -157,3 +168,4 @@ This project is licensed under the terms of the [MIT license][file-license].
[docs-netplan-examples]: https://netplan.io/examples/
[docs-netplan]: https://netplan.io
[website-argo-installation]: https://www.arthurkoziel.com/setting-up-argocd-with-helm/
[google-accounts]: https://accounts.google.com
50 changes: 50 additions & 0 deletions cmd/ic/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
package main

import (
"os"

"github.com/spf13/cobra"

"github.com/nicklasfrahm/infrastructure/cmd/ic/zone"
)

var version = "dev"
var help bool

var rootCmd = &cobra.Command{
Use: "ic",
Short: "A CLI to manage infrastructure",
Long: ` _
(_) ___
| |/ __|
| | (__
|_|\___|

ic is a CLI to manage infrastructure. It provides
a variety of commands to manage different stages
of the infrastructure lifecycle.`,
PersistentPreRun: func(cmd *cobra.Command, args []string) {
if help {
cmd.Help()
os.Exit(0)
}
},
RunE: func(cmd *cobra.Command, args []string) error {
cmd.Help()
os.Exit(1)
return nil
},
Version: version,
SilenceUsage: true,
}

func init() {
rootCmd.PersistentFlags().BoolVarP(&help, "help", "h", false, "Print this help")
rootCmd.AddCommand(zone.Command)
}

func main() {
if err := rootCmd.Execute(); err != nil {
os.Exit(1)
}
}
23 changes: 23 additions & 0 deletions cmd/ic/zone/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
package zone

import (
"os"

"github.com/spf13/cobra"
)

var Command = &cobra.Command{
Use: "zone",
Short: `Manage availability zones`,
Long: `Bootstrap new availability zones and manage the
lifecycle of existing ones.`,
RunE: func(cmd *cobra.Command, args []string) error {
cmd.Help()
os.Exit(1)
return nil
},
}

func init() {
Command.AddCommand(upCmd)
}
59 changes: 59 additions & 0 deletions cmd/ic/zone/up.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
package zone

import (
"fmt"

_ "github.com/joho/godotenv/autoload"
"github.com/spf13/cobra"

"github.com/nicklasfrahm/infrastructure/pkg/kraut/zone"
)

var (
zoneConfig = zone.Zone{
Router: &zone.ZoneRouter{},
}
configFile string
)

var upCmd = &cobra.Command{
Use: "up <host>",
Short: "Bootstrap a new availability zone",
Long: `This command will bootstrap a new zone by connecting
to the specified IP and setting up a k3s cluster on
the host that will then set up the required services
for managing the lifecycle of the zone.

To manage a zone, the CLI needs credentials for the
DNS provider that is used to manage the DNS records
for the zone. These credentials can only be provided
via the environment variable DNS_PROVIDER_CREDENTIAL
and DNS_PROVIDER or via a ".env" file in the current
working directory.`,
Args: cobra.ExactArgs(1),
ArgAliases: []string{"host"},
ValidArgs: []string{"host"},
RunE: func(cmd *cobra.Command, args []string) error {
// This should be safe because of the ExactArgs(1) constraint,
// but we still need to check it to avoid panics.
if len(args) != 1 {
return fmt.Errorf("accepts 1 arg(s), received %d", len(args))
}
host := args[0]

if err := zone.Up(host, &zoneConfig); err != nil {
return err
}

return nil
},
}

func init() {
upCmd.Flags().StringVarP(&zoneConfig.Name, "name", "n", "", "name of the zone")
upCmd.Flags().StringVarP(&zoneConfig.Domain, "domain", "d", "", "domain that will contain the DNS records for the zone")
upCmd.Flags().StringVarP(&zoneConfig.Router.Hostname, "hostname", "H", "", "hostname of the router serving the zone")
upCmd.Flags().IPVarP(&zoneConfig.Router.ID, "router-id", "r", nil, "IPv4 address of the router serving the zone")
upCmd.Flags().Uint32VarP(&zoneConfig.Router.ASN, "asn", "a", 0, "autonomous system number of the zone")
upCmd.Flags().StringVarP(&configFile, "config", "c", "", "path to the configuration file")
}
2 changes: 1 addition & 1 deletion cmd/metal/hardwares.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ func Hardware(useRemote bool) *fiber.App {
var bytes []byte
var err error

path := fmt.Sprintf("configs/hardwares/%s.yaml", name)
path := fmt.Sprintf("config/hardwares/%s.yaml", name)
if useRemote {
// Use the GitHub API to fetch the file.
path = fmt.Sprintf(fileURLTemplate, repository, path)
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
7 changes: 7 additions & 0 deletions config/kraut/aar1.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# TODO: Create a manifest with a structure similar to ones Kubernetes.
domain: nicklasfrahm.dev
name: aar1
router:
hostname: alfa
id: 172.31.255.0
asn: 65000
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ network:
- 1.1.1.1
- 1.0.0.1
search:
- nicklasfrahm.xyz
- srv.nicklasfrahm.dev
- nicklasfrahm.dev
enp3s0:
dhcp4: false
dhcp6: false
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,7 @@ network:
name: lo
addresses:
- 172.31.255.1/32
- fdff:ffff:ffff:ffff::1:0/128
eth0:
# Spoof MAC address to avoid same IP on multiple sites.
match:
macaddress: ae:99:7d:fb:4f:13
wan:
macaddress: ae:99:7d:fb:4f:11
dhcp4: true
dhcp4-overrides:
Expand All @@ -25,19 +21,19 @@ network:
search:
- srv.nicklasfrahm.dev
- nicklasfrahm.dev
eth1:
lan1:
dhcp4: false
dhcp6: false
optional: true
eth2:
lan2:
dhcp4: false
dhcp6: false
optional: true
bridges:
br0:
interfaces:
- eth1
- eth2
- lan1
- lan2
dhcp4: false
dhcp6: false
addresses:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
flush ruleset

# Declare variables.
define iface_wan = "eth0"
define iface_wan = "wan"

# Create a table for firewalling.
table inet firewall {
Expand Down
File renamed without changes.
File renamed without changes.
35 changes: 35 additions & 0 deletions docs/appliance-lifecycle.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Appliance lifycycle

An appliance is a piece of hardware that comes with a pre-installed operating system. This document describes how the lifecycle of appliances is managed.

## Provisioning 🪄

For different hardware the appliance provisioning process is different. Please see the different sections below for more information.

### Nano Pi R5S 📡

1. Build a fresh appliance image:

```bash
BOARD=nanopi-r5s make build-appliance
```

2. Use [Etcher][etcher] to flash the image to an SD card.
3. Log in to the appliance using the default credentials:

- Username: `nicklasfrahm`
- Password: `nicklasfrahm`

4. Fetch the IP address of the appliance:

```bash
ip addr show wan | grep 'inet ' | awk '{print $2}'
```

5. On a different machine, run the bootstrap command:

```bash
ic metal bootstrap <ip>
```

[etcher]: https://etcher.balena.io/
Loading