You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Route kiln test through docker-virtual registry and refactor test-script generation (#643)
## Summary
- Route `kiln test` image pulls through the internal Broadcom
Artifactory docker-virtual registry, eliminating the need for a separate
`docker login` before running tile tests.
- Rebuild the test-script generation layer (`testPlan` / `suiteStep`) so
each suite runs in its own subshell with an independent exit code,
colored pass/fail summary, and optional per-suite timestamps.
- Add `--verbose` / `--silent` / `--stability` flags; quiet npm by
default; set `GOMAXPROCS` inside the container.
- Refactor for maintainability: decompose `runTest`, fix naming
inconsistencies, and clean up small code smells.
## What changed
### Dockerfile (`internal/test/Dockerfile`)
- All `FROM` lines now use
`tas-rel-eng-docker-virtual.usw1.packages.broadcom.com` (the same
registry as `AuthConfigs`) so `kiln test` can pull base images without a
prior `docker login`.
- Stable tool layers (Go runtime, `jq`, `nodejs`, `npm`) are ordered
before the `ARG ARTIFACTORY_USERNAME` declaration so they survive
credential rotation.
- `ginkgo@v1.16.5` is pinned; `GOPROXY` credentials are scoped to a
single `RUN` step (not exported as `ENV`) to keep that layer stable.
- `ops-manifest` gem install credentials are exported to `ENV` for
runtime use.
### `kiln test` behaviour
- New flags: `--stability` (run only stability tests), `--silent`
(suppress kiln info lines), `--verbose` (print container ID and
per-suite timestamps).
- `--verbose` defaults to `false`; npm runs `--silent` / `ci --silent`
by default and uses the louder form only when `--verbose` is passed.
- `GOMAXPROCS` is set to the host's CPU count inside the container.
- Credentials may be supplied via `-e ARTIFACTORY_USERNAME=... -e
ARTIFACTORY_PASSWORD=...` or exported in the shell; `kiln test` exits
with an error before contacting Docker if either is missing.
### `internal/test/container.go` refactors
- **`RunMetadata` → `RunStability`**: field name now matches the
`--stability` flag and "Stability Tests" suite label.
- **Decompose `runTest`**: extracted `buildTestImage` (image build +
registry auth) and `startAndWaitContainer` (container lifecycle + log
drain + signal handling) so `runTest` is a 30-line orchestrator.
- **Single credential resolution**:
`requiredArtifactoryCredentialsFromMap` reuses the already-decoded
`environmentVars` map, eliminating a redundant `decodeEnvironment` call.
- **`verbose` as a render parameter**: removed `verbose` from the
`testPlan` struct; `testPlan` is now pure data and `script(verbose
bool)` makes the rendering concern explicit.
- **`GinkgoFlags` typo fixed**: `TileTest.Options.GingkoFlags` →
`GinkgoFlags`.
- **`errors.New`**: replaced `fmt.Errorf("%s", detail)` in
`checkImageBuildResponse`.
- **`filepath` consistency**: removed the `"path"` import;
`path.Dir/Base` replaced with `filepath.Dir/Base` for host filesystem
operations.
### README
Updated `kiln test` docs to reflect the new flags (`--stability`,
`--ginkgo-flags`, `--manifest`, `--migrations`), the credential flow
through `docker-virtual`, and the `DockerVirtualRegistryHost` alignment
requirement.
### `kiln test` output excerpt
#### `--verbose` flag
```
SUCCESS! -- 520 Passed | 0 Failed | 0 Pending | 4 Skipped
Ginkgo ran 1 suite in 8m46.345003494s
Test Suite Passed
[21:13:42] Completed: Manifest Tests
[21:03:23] ✓ Migration Tests Passed
[21:04:55] ✓ Stability Tests Passed
[21:13:42] ✓ Manifest Tests Passed
```
#### without `--verbose` flag
```
SUCCESS! -- 520 Passed | 0 Failed | 0 Pending | 4 Skipped
Ginkgo ran 1 suite in 8m10.665858116s
Test Suite Passed
✓ Migration Tests Passed
✓ Stability Tests Passed
✓ Manifest Tests Passed
```
Copy file name to clipboardExpand all lines: README.md
+26-17Lines changed: 26 additions & 17 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -520,56 +520,65 @@ Any variables that Kilnfile needs for the kiln re-bake command should be set in
520
520
521
521
### `test`
522
522
523
-
The `test` command exercises to ginkgo tests under the `/<tile>/test/manifest` and `/<tile>/migrations` paths of the `pivotal/tas` repos (where `<tile>` is tas, ist, or tasw).
523
+
The `test` command exercises the Ginkgo tests under the `/<tile>/test/manifest` and `/<tile>/migrations` paths of the `pivotal/tas` repos (where `<tile>` is tas, ist, or tasw).
524
524
525
-
Running these tests requires a docker daemon. It also requires the user to
526
-
provide Artifactory credentials via the ARTIFACTORY_USERNAME and
527
-
ARTIFACTORY_PASSWORD environment variables to allow the ops-manifest gem to
528
-
be installed. The credentials must have access to the `tas-rel-eng-gem-dev-local`
529
-
repository within Broadcom's Artifactory.
525
+
Running these tests requires a Docker daemon (or Podman API-compatible socket). You must provide **ARTIFACTORY_USERNAME** and **ARTIFACTORY_PASSWORD** using **`-e`** and/or **exported** environment variables. They are used for the **ops-manifest** gem (`tas-rel-eng-gem-dev-local`), for **Go module** downloads during **`go install ginkgo`** (via **`GOPROXY`** / **`GOSUMDB=off`** in the embedded Dockerfile), and Kiln sends the same credentials to the daemon as **registry auth** so base images can be pulled from **docker-virtual** (`tas-rel-eng-docker-virtual.usw1.packages.broadcom.com`) **without a separate `docker login`** for `kiln test`.
526
+
527
+
The embedded Dockerfile pins **`FROM`** paths on that registry. The registry hostname in Kiln’s **`AuthConfigs`** must stay aligned with those **`FROM`** lines (see **`DockerVirtualRegistryHost`** in `internal/test/container.go`). Passwords with characters that are special in URLs may not behave the same as URL-encoded credentials when interpolated into **`GOPROXY`** inside the Dockerfile.
528
+
529
+
If either credential is missing, `kiln test` exits with an error before talking to Docker.
530
530
531
531
If you run into this docker error `could not execute "test": failed to connect to Docker daemon: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running`,
532
532
then create a symlink `sudo ln -s ~/.docker/run/docker.sock /var/run/docker.sock`
533
533
534
-
Here are command line examples:
534
+
Examples:
535
535
536
536
```
537
537
$ cd ~/workspace/tas/ist
538
538
$ kiln test -e ARTIFACTORY_USERNAME=myuser -e ARTIFACTORY_PASSWORD=secretpassword
The `--ginkgo-flags` flag can be used to pass through Ginkgo test flags. The defaults being passed through are `-r -p -slowSpecThreshold 15`. Pass `help` as a flag to retrieve the available options for the embedded version of ginkgo.
550
553
551
-
The`--ginkgo-manifest-flags` flag can be used to pass through Ginkgo test flags. The defaults being passed through are `-r -p -slowSpecThreshold 15`. Pass `help` as a flag to retrieve the available options for the embeded version of ginkgo.
554
+
####`--manifest`
552
555
553
-
####`--manifest-only`
556
+
The`--manifest` flag can be used to run only Manifest tests.
554
557
555
-
The`--manifest-only` flag can be used to run only Manifest tests. If not passed, `kiln test` will run both Manifest and Migration tests by default.
558
+
####`--migrations`
556
559
557
-
####`--migrations-only`
560
+
The`--migrations` flag can be used to run only Migration tests.
558
561
559
-
The `--migrations-only` flag can be used to run only Migration tests. If not passed, `kiln test` will run both Manifest and Migration tests by default.
562
+
#### `--stability`
563
+
564
+
The `--stability` flag can be used to run only Stability tests.
560
565
561
566
##### `--tile-path`
562
567
563
-
The `--tile-path` (`-tp`) flag can be set the path the directory you wish to test. It defaults to the current working directory. For example
568
+
The `--tile-path` (`-tp`) flag can be set to the directory you wish to test. It defaults to the current working directory. For example:
564
569
565
570
```
566
-
$ kiln test -tp ~/workspace/tas/ist
571
+
$ kiln test -e ARTIFACTORY_USERNAME=myuser -e ARTIFACTORY_PASSWORD=secret -tp ~/workspace/tas/ist
567
572
```
568
573
569
574
##### `--verbose`
570
575
571
576
The `--verbose` (`-v`) flag will log additional debugging info.
572
577
578
+
##### `--silent`
579
+
580
+
The `--silent` (`-s`) flag hides Kiln info lines (not Ginkgo output).
Copy file name to clipboardExpand all lines: internal/commands/test_tile.go
+14-10Lines changed: 14 additions & 10 deletions
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,6 @@ package commands
2
2
3
3
import (
4
4
"context"
5
-
_ "embed"
6
5
"fmt"
7
6
"io"
8
7
"os"
@@ -17,15 +16,15 @@ type TileTestFunction func(ctx context.Context, w io.Writer, configuration test.
17
16
18
17
typeTileTeststruct {
19
18
Optionsstruct {
20
-
TilePathstring` long:"tile-path" default:"." description:"Path to the Tile directory (e.g., ~/workspace/tas/ist)."`
21
-
Verbosebool`short:"v" long:"verbose" default:"true" description:"Print info lines. This doesn't affect Ginkgo output."`
19
+
TilePathstring` long:"tile-path" default:"." description:"Path to the Tile directory (e.g. ~/workspace/tas/ist)."`
20
+
Verbosebool`short:"v" long:"verbose" default:"false" description:"Print container ID, per-suite start/end timestamps, and timestamped pass/fail summary. Also enables npm install output. Does not affect Ginkgo or npm test output."`
22
21
Silentbool`short:"s" long:"silent" default:"false" description:"Hide info lines. This doesn't affect Ginkgo output."`
23
22
Manifestbool` long:"manifest" default:"false" description:"Focus the Manifest tests."`
24
23
Migrationsbool` long:"migrations" default:"false" description:"Focus the Migration tests."`
25
24
Stabilitybool` long:"stability" default:"false" description:"Focus the Stability tests."`
26
25
27
-
EnvironmentVars []string`short:"e" long:"environment-variable" description:"Pass environment variable to the test suites. For example --stability -e 'PRODUCT=srt'."`
28
-
GingkoFlagsstring` long:"ginkgo-flags" default:"-r -p -slowSpecThreshold 15" description:"Flags to pass to the Ginkgo Manifest and Stability test suites."`
26
+
EnvironmentVars []string`short:"e" long:"environment-variable" description:"Pass environment variables to the test suites (e.g. -e 'PRODUCT=srt'). Include -e ARTIFACTORY_USERNAME=... and -e ARTIFACTORY_PASSWORD=... unless they are exported."`
27
+
GinkgoFlagsstring` long:"ginkgo-flags" default:"-r -p -slowSpecThreshold 15" description:"Flags to pass to the Ginkgo Manifest and Stability test suites."`
Description: "Run the Manifest, Migrations, and Stability tests for a Tile in a Docker container. Requires a Docker daemon to be running and Artifactory credentials to be provided via the ARTIFACTORY_USERNAME and ARTIFACTORY_PASSWORD environment variables to install the ops-manifest gem.",
86
+
Description: "Run the Manifest, Migrations, and Stability tests for a Tile in a Docker container. Requires a Docker daemon. Requires ARTIFACTORY_USERNAME and ARTIFACTORY_PASSWORD (via -e or your environment) for the test image build and ops-manifest gem. Kiln passes the same credentials to the Docker daemon for pulling base images from docker-virtual (no separate docker login needed for kiln test).",
0 commit comments