Skip to content

Commit 361fd35

Browse files
gilcrestclaude
gilcrest
and
claude
committed
Refactor config to multi-target CUE pattern and update dependencies
Consolidate local.cue and staging.cue into a single config.cue with multi-target support using snake_case naming. Restructure CUE schema accordingly and move Taskfile out of dbt/ subdirectory. Add top-level config.json for ff JSON config parser. Refactor cmd package to extract flags validation and add printFlags helper. Comment out Genesis and GCP deploy code pending rework. Make SQL migration constraints idempotent. Update Go to 1.24.11 and upgrade all dependencies. Document config values as example/placeholder only. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 7e55a4c commit 361fd35

19 files changed

Lines changed: 401 additions & 632 deletions

File tree

‎README.md‎

Lines changed: 33 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -82,31 +82,29 @@ The `movieAdmin` role is set up to grant access to all resources. It's a demo...
8282
8383
--------
8484

85-
### Step 3 - Prepare Environment (2 options)
85+
### Step 2 - Prepare Environment (2 options)
8686

87-
All Mage programs in this project which take an environment (env) parameter (e.g., `func DBUp(env string)`), must have certain environment variables set. These environment variables can be set independently [option 1](#option-1---set-your-environment-independently) or based on a configuration file [option 2](#option-2---set-your-environment-through-a-config-file). Depending on which environment method you choose, the values to pass to the env parameter when running Mage programs in this project are as follows:
88-
89-
| env string | File Path | Description |
90-
|------------|-----------------------|----------------------------------------------------------------------------------------------------------------|
91-
| current | N/A | Uses the current session environment. Environment will not be overriden from a config file |
92-
| local | ./config/local.json | Uses the `local.json` config file to set the environment |
93-
| staging | ./config/staging.json | Uses the `staging.json` config file to set the environment in [Google Cloud Run](https://cloud.google.com/run) |
94-
95-
The base environment variables to be set are:
87+
Set environment variables for the database:
9688

9789
| Environment Variable | Description |
9890
|----------------------|----------------------------------------------------------------------------------------------------|
99-
| PORT | Port the server will listen on |
100-
| LOG_LEVEL | zerolog logging level (debug, info, etc.) |
101-
| LOG_LEVEL_MIN | sets the minimum accepted logging level |
102-
| LOG_ERROR_STACK | If true, log error stacktrace using github.com/pkg/errors, else just log error (includes op stack) |
10391
| DB_HOST | The host name of the database server. |
10492
| DB_PORT | The port number the database server is listening on. |
10593
| DB_NAME | The database name. |
10694
| DB_USER | PostgreSQL™ user name to connect as. |
10795
| DB_PASSWORD | Password to be used if the server demands password authentication. |
10896
| DB_SEARCH_PATH | Schema Search Path |
109-
| ENCRYPT_KEY | Encryption Key |
97+
98+
These environment variables can be set independently [option 1](#option-1---set-your-environment-independently) or based on a configuration file [option 2](#option-2---set-your-environment-through-a-config-file). Depending on which environment method you choose, the values to pass to the env parameter when running Mage programs in this project are as follows:
99+
100+
101+
| env string | File Path | Description |
102+
|------------|-----------------------|----------------------------------------------------------------------------------------------------------------|
103+
| current | N/A | Uses the current session environment. Environment will not be overriden from a config file |
104+
| local | ./config/local.json | Uses the `local.json` config file to set the environment |
105+
| staging | ./config/staging.json | Uses the `staging.json` config file to set the environment in [Google Cloud Run](https://cloud.google.com/run) |
106+
107+
The base environment variables to be set are:
110108

111109
> The same environment variables are used when running the web server, but are not mandatory. When running the web server, if you prefer, you can bypass environment variables and instead send command line flags (more about that later).
112110
@@ -505,6 +503,26 @@ The above image is a high-level view of an example request that is processed by
505503

506504
> `diygoapi` package layout is based on several projects, but the primary source of inspiration is the [WTF Dial app repo](https://github.com/benbjohnson/wtf) and [accompanying blog](https://www.gobeyond.dev/) from [Ben Johnson](https://github.com/benbjohnson). It's really a wonderful resource and I encourage everyone to read it.
507505
506+
### Environment Variables
507+
508+
The following environment variables are used to configure the web server:
509+
510+
| Environment Variable | Description |
511+
|----------------------|----------------------------------------------------------------------------------------------------|
512+
| PORT | Port the server will listen on |
513+
| LOG_LEVEL | zerolog logging level (debug, info, etc.) |
514+
| LOG_LEVEL_MIN | sets the minimum accepted logging level |
515+
| LOG_ERROR_STACK | If true, log error stacktrace using github.com/pkg/errors, else just log error (includes op stack) |
516+
| DB_HOST | The host name of the database server. |
517+
| DB_PORT | The port number the database server is listening on. |
518+
| DB_NAME | The database name. |
519+
| DB_USER | PostgreSQL™ user name to connect as. |
520+
| DB_PASSWORD | Password to be used if the server demands password authentication. |
521+
| DB_SEARCH_PATH | Schema Search Path |
522+
| ENCRYPT_KEY | Encryption Key |
523+
524+
> Note: If an environment variable is not set, the default is used
525+
508526
### Errors
509527

510528
Handling errors is really important in Go. Errors are first class citizens and there are many different approaches for handling them. I have based my error handling on a [blog post from Rob Pike](https://commandcenter.blogspot.com/2017/12/error-handling-in-upspin.html) and have modified it to meet my needs. The post is many years old now, but I find the lessons there still hold true at least for my requirements.

‎cmd/cmd.go‎

Lines changed: 32 additions & 114 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,11 @@ package cmd
22

33
import (
44
"context"
5-
"flag"
65
"fmt"
76
"net/http"
87
"os"
98

109
"github.com/jackc/pgx/v5/pgxpool"
11-
"github.com/peterbourgon/ff/v3"
1210
"github.com/rs/zerolog"
1311
"golang.org/x/text/language"
1412

@@ -21,104 +19,6 @@ import (
2119
"github.com/gilcrest/diygoapi/sqldb"
2220
)
2321

24-
const (
25-
// log level environment variable name
26-
loglevelEnv string = "LOG_LEVEL"
27-
// minimum accepted log level environment variable name
28-
logLevelMinEnv string = "LOG_LEVEL_MIN"
29-
// log error stack environment variable name
30-
logErrorStackEnv string = "LOG_ERROR_STACK"
31-
// server port environment variable name
32-
portEnv string = "PORT"
33-
// encryption key environment variable name
34-
encryptKeyEnv string = "ENCRYPT_KEY"
35-
)
36-
37-
type flags struct {
38-
// log-level flag allows for setting logging level, e.g. to run the server
39-
// with level set to debug, it'd be: ./server -log-level=debug
40-
// If not set, defaults to error
41-
loglvl string
42-
43-
// log-level-min flag sets the minimum accepted logging level
44-
// - e.g. in production, you may have a policy to never allow logs at
45-
// trace level. You could set the minimum log level to Debug. Even
46-
// if the Global log level is set to Trace, only logs at Debug
47-
// and above would be logged. Default level is trace.
48-
logLvlMin string
49-
50-
// logErrorStack flag determines whether or not a full error stack
51-
// should be logged. If true, error stacks are logged, if false,
52-
// just the error is logged
53-
logErrorStack bool
54-
55-
// port flag is what http.ListenAndServe will listen on. default is 8080 if not set
56-
port int
57-
58-
// dbhost is the database host
59-
dbhost string
60-
61-
// dbport is the database port
62-
dbport int
63-
64-
// dbname is the database name
65-
dbname string
66-
67-
// dbuser is the database user
68-
dbuser string
69-
70-
// dbpassword is the database user's password
71-
dbpassword string
72-
73-
// dbsearchpath is the database search path
74-
dbsearchpath string
75-
76-
// encryptkey is the encryption key
77-
encryptkey string
78-
}
79-
80-
// newFlags parses the command line flags using ff and returns
81-
// a flags struct or an error
82-
func newFlags(args []string) (flags, error) {
83-
const op errs.Op = "cmd/newFlags"
84-
// create new FlagSet using the program name being executed (args[0])
85-
// as the name of the FlagSet
86-
fs := flag.NewFlagSet(args[0], flag.ContinueOnError)
87-
var (
88-
logLvlMin = fs.String("log-level-min", "trace", fmt.Sprintf("sets minimum log level (trace, debug, info, warn, error, fatal, panic, disabled), (also via %s)", logLevelMinEnv))
89-
loglvl = fs.String("log-level", "info", fmt.Sprintf("sets log level (trace, debug, info, warn, error, fatal, panic, disabled), (also via %s)", loglevelEnv))
90-
logErrorStack = fs.Bool("log-error-stack", false, fmt.Sprintf("if true, log full error stacktrace using github.com/pkg/errors, else just log error, (also via %s)", logErrorStackEnv))
91-
port = fs.Int("port", 8080, fmt.Sprintf("listen port for server (also via %s)", portEnv))
92-
dbhost = fs.String("db-host", "", fmt.Sprintf("postgresql database host (also via %s)", sqldb.DBHostEnv))
93-
dbport = fs.Int("db-port", 5432, fmt.Sprintf("postgresql database port (also via %s)", sqldb.DBPortEnv))
94-
dbname = fs.String("db-name", "", fmt.Sprintf("postgresql database name (also via %s)", sqldb.DBNameEnv))
95-
dbuser = fs.String("db-user", "", fmt.Sprintf("postgresql database user (also via %s)", sqldb.DBUserEnv))
96-
dbpassword = fs.String("db-password", "", fmt.Sprintf("postgresql database password (also via %s)", sqldb.DBPasswordEnv))
97-
dbsearchpath = fs.String("db-search-path", "", fmt.Sprintf("postgresql database search path (also via %s)", sqldb.DBSearchPathEnv))
98-
encryptkey = fs.String("encrypt-key", "", fmt.Sprintf("encryption key (also via %s)", encryptKeyEnv))
99-
)
100-
101-
// Parse the command line flags from above
102-
err := ff.Parse(fs, args[1:], ff.WithEnvVars())
103-
if err != nil {
104-
return flags{}, errs.E(op, err)
105-
}
106-
107-
return flags{
108-
loglvl: *loglvl,
109-
logLvlMin: *logLvlMin,
110-
logErrorStack: *logErrorStack,
111-
port: *port,
112-
dbhost: *dbhost,
113-
dbport: *dbport,
114-
dbname: *dbname,
115-
dbuser: *dbuser,
116-
dbpassword: *dbpassword,
117-
dbsearchpath: *dbsearchpath,
118-
encryptkey: *encryptkey,
119-
}, nil
120-
}
121-
12222
// Run parses command line flags and starts the server
12323
func Run(args []string) (err error) {
12424
const op errs.Op = "cmd/Run"
@@ -165,21 +65,9 @@ func Run(args []string) (err error) {
16565
logger.LogErrorStackViaPkgErrors(flgs.logErrorStack)
16666
lgr.Info().Msgf("log error stack via github.com/pkg/errors set to %t", flgs.logErrorStack)
16767

168-
// validate port in acceptable range
169-
err = portRange(flgs.port)
68+
err = flgs.Validate()
17069
if err != nil {
171-
lgr.Fatal().Err(err).Msg("portRange() error")
172-
}
173-
174-
// initialize Server enfolding a http.Server with default timeouts,
175-
// a mux router and a zerolog.Logger
176-
s := server.New(http.NewServeMux(), server.NewDriver(), lgr)
177-
178-
// set listener address
179-
s.Addr = fmt.Sprintf(":%d", flgs.port)
180-
181-
if flgs.encryptkey == "" {
182-
lgr.Fatal().Msg("no encryption key found")
70+
lgr.Fatal().Err(err).Msg("flags.Validate() error")
18371
}
18472

18573
// decode and retrieve encryption key
@@ -189,6 +77,13 @@ func Run(args []string) (err error) {
18977
lgr.Fatal().Err(err).Msg("secure.ParseEncryptionKey() error")
19078
}
19179

80+
// initialize Server enfolding a http.Server with default timeouts,
81+
// a mux router and a zerolog.Logger
82+
s := server.New(http.NewServeMux(), server.NewDriver(), lgr)
83+
84+
// set Server listener address
85+
s.Addr = fmt.Sprintf(":%d", flgs.port)
86+
19287
ctx := context.Background()
19388

19489
// initialize PostgreSQL database
@@ -270,3 +165,26 @@ func portRange(port int) error {
270165
}
271166
return nil
272167
}
168+
169+
// printFlags prints the flags to stdout
170+
func printFlags(flgs flags) {
171+
fmt.Printf("Log Level: %s\n", flgs.loglvl)
172+
fmt.Printf("Log Level Min: %s\n", flgs.logLvlMin)
173+
fmt.Printf("Log Error Stack: %t\n", flgs.logErrorStack)
174+
fmt.Printf("Port: %d\n", flgs.port)
175+
fmt.Printf("DB Host: %s\n", flgs.dbhost)
176+
fmt.Printf("DB Port: %d\n", flgs.dbport)
177+
fmt.Printf("DB Name: %s\n", flgs.dbname)
178+
fmt.Printf("DB Search Path: %s\n", flgs.dbsearchpath)
179+
fmt.Printf("DB User: %s\n", flgs.dbuser)
180+
if flgs.dbpassword == "" {
181+
fmt.Println("DB Password: <empty>")
182+
} else {
183+
fmt.Println("DB Password: <not empty>")
184+
}
185+
if flgs.encryptkey == "" {
186+
fmt.Println("Encryption Key: <empty>")
187+
} else {
188+
fmt.Println("Encryption Key: <not empty>")
189+
}
190+
}

‎cmd/ddl.go‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -110,13 +110,16 @@ func PSQLArgs(up bool) ([]string, error) {
110110
}
111111

112112
// newFlags will retrieve the database info from the environment using ff
113-
flgs, err := newFlags([]string{"server"})
113+
var f flags
114+
f, err = newFlags([]string{"server"})
114115
if err != nil {
115116
return nil, errs.E(op, err)
116117
}
117118

119+
printFlags(f)
120+
118121
// command line args for psql are constructed
119-
args := []string{"-w", "-d", newPostgreSQLDSN(flgs).ConnectionURI(), "-c", "select current_database(), current_user, version()"}
122+
args := []string{"-w", "-d", newPostgreSQLDSN(f).ConnectionURI(), "-q", "-c", "select current_database(), current_user, version()"}
120123

121124
for _, file := range ddlFiles {
122125
args = append(args, "-f")

‎cmd/gcp.go‎

Lines changed: 52 additions & 60 deletions
Original file line numberDiff line numberDiff line change
@@ -1,62 +1,54 @@
11
package cmd
22

3-
import (
4-
"fmt"
5-
"strconv"
6-
"strings"
7-
8-
"github.com/gilcrest/diygoapi/sqldb"
9-
)
10-
11-
// GCPCloudRunDeployImage builds arguments for running a service on
12-
// Cloud Run given an Artifact Registry image.
13-
func GCPCloudRunDeployImage(f ConfigFile, image GCPArtifactRegistryContainerImage) []string {
14-
15-
var (
16-
// Google Cloud Run Service Name
17-
serviceName = f.Config.GCP.CloudRun.ServiceName
18-
// Google Cloud SQL Instance Name
19-
gcpCloudSQLInstanceConnectionName = f.Config.GCP.CloudSQL.InstanceConnectionName
20-
// postgresql database name
21-
)
22-
23-
args := []string{"run", "deploy", serviceName, "--image", image.String(), "--platform", "managed", "--no-allow-unauthenticated"}
24-
25-
args = append(args, "--add-cloudsql-instances", gcpCloudSQLInstanceConnectionName)
26-
27-
icn := fmt.Sprintf(`INSTANCE-CONNECTION-NAME=%s`, gcpCloudSQLInstanceConnectionName)
28-
dbName := fmt.Sprintf(`%s=%s`, sqldb.DBNameEnv, f.Config.Database.Name)
29-
dbUser := fmt.Sprintf(`%s=%s`, sqldb.DBUserEnv, f.Config.Database.User)
30-
dbPassword := fmt.Sprintf(`%s=%s`, sqldb.DBPasswordEnv, f.Config.Database.Password)
31-
dbHost := fmt.Sprintf(`%s=%s`, sqldb.DBHostEnv, f.Config.Database.Host)
32-
dbPort := fmt.Sprintf(`%s=%s`, sqldb.DBPortEnv, strconv.Itoa(f.Config.Database.Port))
33-
dbSearchPath := fmt.Sprintf(`%s=%s`, sqldb.DBSearchPathEnv, f.Config.Database.SearchPath)
34-
encryptKey := fmt.Sprintf(`%s=%s`, encryptKeyEnv, f.Config.EncryptionKey)
35-
36-
envVars := []string{icn, dbName, dbUser, dbPassword, dbHost, dbPort, dbSearchPath, encryptKey}
37-
38-
args = append(args, "--set-env-vars", strings.Join(envVars, ","))
39-
40-
return args
41-
}
42-
43-
// GCPArtifactRegistryContainerImage defines a GCP Artifact Registry
44-
// build image according to https://cloud.google.com/artifact-registry/docs/docker/names
45-
// The String method prints the build string needed to build to
46-
// Artifact Registry using gcloud as well as deploy it to Cloud Run.
47-
type GCPArtifactRegistryContainerImage struct {
48-
ProjectID string
49-
RepositoryLocation string
50-
RepositoryName string
51-
ImageName string
52-
ImageTag string
53-
}
54-
55-
// String outputs the Google Artifact Registry image name.
56-
// LOCATION-docker.pkg.dev/PROJECT-ID/REPOSITORY/IMAGE:TAG
57-
func (i GCPArtifactRegistryContainerImage) String() string {
58-
if i.ImageTag != "" {
59-
return fmt.Sprintf("%s-docker.pkg.dev/%s/%s/%s:%s", i.RepositoryLocation, i.ProjectID, i.RepositoryName, i.ImageName, i.ImageTag)
60-
}
61-
return fmt.Sprintf("%s-docker.pkg.dev/%s/%s/%s", i.RepositoryLocation, i.ProjectID, i.RepositoryName, i.ImageName)
62-
}
3+
//// GCPCloudRunDeployImage builds arguments for running a service on
4+
//// Cloud Run given an Artifact Registry image.
5+
//func GCPCloudRunDeployImage(f ConfigFile, image GCPArtifactRegistryContainerImage) []string {
6+
//
7+
// var (
8+
// // Google Cloud Run Service Name
9+
// serviceName = f.Config.GCP.CloudRun.ServiceName
10+
// // Google Cloud SQL Instance Name
11+
// gcpCloudSQLInstanceConnectionName = f.Config.GCP.CloudSQL.InstanceConnectionName
12+
// // postgresql database name
13+
// )
14+
//
15+
// args := []string{"run", "deploy", serviceName, "--image", image.String(), "--platform", "managed", "--no-allow-unauthenticated"}
16+
//
17+
// args = append(args, "--add-cloudsql-instances", gcpCloudSQLInstanceConnectionName)
18+
//
19+
// icn := fmt.Sprintf(`INSTANCE-CONNECTION-NAME=%s`, gcpCloudSQLInstanceConnectionName)
20+
// dbName := fmt.Sprintf(`%s=%s`, sqldb.DBNameEnv, f.Config.Database.Name)
21+
// dbUser := fmt.Sprintf(`%s=%s`, sqldb.DBUserEnv, f.Config.Database.User)
22+
// dbPassword := fmt.Sprintf(`%s=%s`, sqldb.DBPasswordEnv, f.Config.Database.Password)
23+
// dbHost := fmt.Sprintf(`%s=%s`, sqldb.DBHostEnv, f.Config.Database.Host)
24+
// dbPort := fmt.Sprintf(`%s=%s`, sqldb.DBPortEnv, strconv.Itoa(f.Config.Database.Port))
25+
// dbSearchPath := fmt.Sprintf(`%s=%s`, sqldb.DBSearchPathEnv, f.Config.Database.SearchPath)
26+
// encryptKey := fmt.Sprintf(`%s=%s`, encryptKeyEnv, f.Config.EncryptionKey)
27+
//
28+
// envVars := []string{icn, dbName, dbUser, dbPassword, dbHost, dbPort, dbSearchPath, encryptKey}
29+
//
30+
// args = append(args, "--set-env-vars", strings.Join(envVars, ","))
31+
//
32+
// return args
33+
//}
34+
//
35+
//// GCPArtifactRegistryContainerImage defines a GCP Artifact Registry
36+
//// build image according to https://cloud.google.com/artifact-registry/docs/docker/names
37+
//// The String method prints the build string needed to build to
38+
//// Artifact Registry using gcloud as well as deploy it to Cloud Run.
39+
//type GCPArtifactRegistryContainerImage struct {
40+
// ProjectID string
41+
// RepositoryLocation string
42+
// RepositoryName string
43+
// ImageName string
44+
// ImageTag string
45+
//}
46+
//
47+
//// String outputs the Google Artifact Registry image name.
48+
//// LOCATION-docker.pkg.dev/PROJECT-ID/REPOSITORY/IMAGE:TAG
49+
//func (i GCPArtifactRegistryContainerImage) String() string {
50+
// if i.ImageTag != "" {
51+
// return fmt.Sprintf("%s-docker.pkg.dev/%s/%s/%s:%s", i.RepositoryLocation, i.ProjectID, i.RepositoryName, i.ImageName, i.ImageTag)
52+
// }
53+
// return fmt.Sprintf("%s-docker.pkg.dev/%s/%s/%s", i.RepositoryLocation, i.ProjectID, i.RepositoryName, i.ImageName)
54+
//}

0 commit comments

Comments
 (0)