Skip to content

fix: resolve GTNH server packs from versions.json #4170

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
WowSuchRicky wants to merge 2 commits into from
Closed

fix: resolve GTNH server packs from versions.json #4170

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
871136f
docs: add GTNH versions downloader implementation plan
WowSuchRicky Jul 15, 2026
6582414
fix: resolve GTNH packs from versions metadata
WowSuchRicky Jul 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 231 additions & 0 deletions .hermes/plans/2026-07-15_210000-gtnh-versions-json-fix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,231 @@
# GTNH versions.json Downloader Fix Implementation Plan

> **For Hermes:** Use the checked-in task list to coordinate parallel implementation and review agents. Do not merge agent changes without running the validation gates below.

**Goal:** Make the GTNH deployment path reliably resolve, validate, download, and install stable, beta, RC, and explicitly pinned GTNH server packs from `https://downloads.gtnewhorizons.com/versions.json`.

**Architecture:** Keep the feature in the existing Bash deployment flow at `scripts/start-deployGTNH`. Normalize the selected release to one JSON shape, select the server URL using the container Java runtime, and preserve the existing `mc-image-helper` download/cache/extract/update pipeline. Add focused shell-level tests using local fixtures and stubbed commands so selection logic is deterministic and does not depend on the live GTNH service.

**Tech Stack:** Bash, `jq`, `curl`, `mc-image-helper`, existing Docker image test harness, Markdown documentation.

---

## Current findings and constraints

- The fork is `WowSuchRicky/docker-minecraft-server`, based on upstream `master`.
- The current upstream branch already contains a partial `versions.json` migration in `scripts/start-deployGTNH` (`32b026d0 Fix GTNH versions.json parsing (#4146)`).
- The current script has inconsistent JSON shapes: `latest`/`latest-dev` retain `{key,value}`, while explicit versions use only the value object, yet later code reads `.value.server...`. This must be made uniform.
- `latest-dev` currently filters only `title == "Beta release"`; the intended behavior must include beta and RC releases.
- The live GTNH service currently exposes beta `2.9.0-beta-2` with a Java 17–25 server URL. The implementation must not hard-code that version.
- Do not regress stable `latest`, explicit version selection, Java 8 selection, Java 17+ selection, existing `.gtnh-version` caching, config backups, or `SKIP_GTNH_UPDATE_CHECK`.
- Do not put live external pack archives into the repository.

## Coordination model

Work in separate branches/worktrees and integrate in this order:

1. **Discovery/research agent:** inspect upstream history and identify the smallest compatible change surface.
2. **Core implementation agent:** implement release parsing/selection and download URL validation.
3. **Test agent:** add deterministic fixtures/stubs and regression tests for selection behavior.
4. **Documentation/integration agent:** update GTNH documentation and examples, and review compatibility.
5. **Independent review agent:** inspect the combined diff for correctness, shell safety, upgrade behavior, and regressions.

Agents must report exact files changed, commands run, and test output. No agent may rewrite another agent's unrelated changes.

---

## Task 1: Baseline and implementation contract

**Objective:** Record the current behavior and define the observable contract before editing production code.

**Files:**
- Read: `scripts/start-deployGTNH`
- Read: `docs/types-and-platforms/mod-platforms/gtnh.md`
- Read: `examples/gtnh/docker-compose-type-gtnh.yaml`
- Create: `tests/gtnh-versions-json/README.md` if the test harness needs a dedicated fixture area.

**Required contract:**

- `GTNH_PACK_VERSION=latest` selects the newest stable release.
- `GTNH_PACK_VERSION=latest-dev` selects the newest beta or RC release.
- An exact version selects that version only.
- Selection is based on release metadata, not a directory listing.
- Java 8 selects `.server.java8Url`.
- Java 17+ selects `.server.java17_2XUrl` and rejects versions whose `maxJavaVersion` is lower than the runtime.
- Unsupported Java versions fail with a clear error.
- Missing release, missing server URL, malformed metadata, failed HTTP retrieval, and invalid selected URL all fail before attempting extraction.
- The selected version and URL are logged without exposing credentials or unrelated environment values.

**Verification:** Capture baseline test command(s) and current failure/success behavior before implementation.

---

## Task 2: Normalize versions.json parsing

**Objective:** Make every selection path return the same release object shape.

**Files:**
- Modify: `scripts/start-deployGTNH`
- Test: `tests/gtnh-versions-json/` or the repository's existing GTNH test location.

**Implementation requirements:**

- Fetch `versions.json` once per startup path and fail closed if `curl -fsSL` fails.
- Support the observed top-level forms `{"versions": {...}}` and a direct version map (`(.versions // .)`).
- For `latest`, filter `title == "Stable release"`.
- For `latest-dev`, filter `title == "Beta release"` or `title == "RC release"` (also tolerate a clearly documented case-insensitive equivalent if the source uses one).
- For an explicit version, match the object key exactly after the existing input normalization rules.
- Return a normalized object containing at least `version`, `releaseDate`, `title`, `maxJavaVersion`, and `server`.
- Use `jq -e`/explicit null checks so an empty selection cannot silently become the JSON literal `null`.
- Use a deterministic ordering: parse ISO-like release dates chronologically, then use the version key as a stable tie-breaker.
- Preserve whitespace-safe handling of `GTNH_PACK_VERSION`.

**Verification:** Unit-level shell tests must exercise all three selection modes and assert the normalized version and server URL.

---

## Task 3: Java compatibility and URL validation

**Objective:** Select the correct server archive and reject unusable metadata before download.

**Files:**
- Modify: `scripts/start-deployGTNH`
- Test: `tests/gtnh-versions-json/`

**Implementation requirements:**

- Read `mc-image-helper java-release` once.
- Java 8: require a non-empty `.server.java8Url`.
- Java >=17: require a numeric `maxJavaVersion`, reject runtimes above it, and require `.server.java17_2XUrl`.
- Java 9–16 and unsupported values: fail with the existing clear unsupported-runtime style.
- Reject `null`, empty, non-HTTPS, or malformed selected URLs.
- Keep the URL in a variable used by the existing `mc-image-helper get` call; do not interpolate untrusted metadata into shell syntax.
- Do not perform a full multi-hundred-megabyte pack download during unit tests. Stub `mc-image-helper`, `curl`, or the metadata-fetch helper.

**Verification:** Tests cover Java 8, Java 17, Java 25, Java too new, missing Java 8 URL, missing Java 17+ URL, and invalid URL metadata.

---

## Task 4: Download and cache safety

**Objective:** Ensure a selected release cannot corrupt an existing installation when the archive is unavailable or invalid.

**Files:**
- Modify: `scripts/start-deployGTNH` only if required; otherwise document that existing `mc-image-helper get` semantics are retained.
- Test: Existing GTNH test fixtures plus focused regression tests.

**Implementation requirements:**

- Preserve `/data/.gtnh-version` comparison behavior.
- Preserve `--skip-up-to-date` behavior from `mc-image-helper get`.
- Ensure a failed metadata lookup or download exits before deleting current server resources.
- Ensure failed extraction does not update `.gtnh-version` or remove a known-good installation.
- Verify the selected archive basename remains the cache/version identity used by the current update path.

**Verification:** Add or run a failure-path test proving an existing installation remains present after metadata/download failure.

---

## Task 5: Documentation and examples

**Objective:** Align user-facing GTNH documentation with the corrected behavior.

**Files:**
- Modify: `docs/types-and-platforms/mod-platforms/gtnh.md`
- Modify: `examples/gtnh/docker-compose-type-gtnh.yaml` only if comments or values are stale.
- Do not modify homelab deployment files in this repository.

**Documentation requirements:**

- Explain `latest` versus `latest-dev`.
- State that `latest-dev` includes beta and RC releases.
- Explain Java 8 versus Java 17+ server archive selection.
- Explain exact version pinning and `SKIP_GTNH_UPDATE_CHECK`.
- Document clear failure behavior when GTNH metadata or compatible server URLs are unavailable.
- Keep the documented Java argument behavior accurate for Java 17+ and Java 8.

**Verification:** Build or run the repository documentation checks used by CI.

---

## Task 6: Regression and integration validation

**Objective:** Prove the fix works without relying solely on a live external service.

**Files:**
- Test files created/modified by Tasks 2–4.
- CI configuration only if a focused test job is required and consistent with repository conventions.

**Required checks:**

```bash
bash -n scripts/start-deployGTNH
shellcheck scripts/start-deployGTNH
# Run the focused GTNH/version tests
# Run the repository test harness used by CI
bash tests/test.sh
```

Also verify:

- Stable `latest` selection.
- Beta/RC `latest-dev` selection.
- Exact `2.9.0-beta-2` selection from a fixture.
- Java 8 and Java 17+ URL selection.
- Explicit unsupported-version and malformed-metadata failures.
- No changes to unrelated mod-platform deployment scripts.

If Docker-based integration tests are available and practical, run a test using a tiny fixture archive or a mocked metadata/download endpoint. Do not make CI depend on the live GTNH archive service for every test.

---

## Task 7: Independent review and hardening

**Objective:** Review the combined change before publishing it from the fork.

**Review focus:**

- Bash quoting, `set -e` behavior, pipeline failure propagation, and `jq` exit status.
- JSON shape consistency across all selection modes.
- Beta/RC semantics and semantic/date ordering.
- Java compatibility behavior.
- URL validation and SSRF/security implications.
- Preservation of existing installation/update/backup behavior.
- Test determinism and absence of large binary fixtures.
- Documentation accuracy.

**Required output:** A structured review listing blocking issues, non-blocking suggestions, and a pass/fail verdict. Fix blocking issues, rerun all validation, and obtain a second review if the implementation changes materially.

---

## Task 8: Fork branch, PR, and downstream rollout

**Objective:** Publish the corrected implementation from the fork and prepare the homelab to consume it.

**Files:**
- Repository source and tests from Tasks 2–7.
- Downstream homelab repository only in a separate follow-up MR after an image is built and verified.

**Steps:**

1. Commit focused changes with conventional messages.
2. Push `feat/gtnh-versions-json` to `WowSuchRicky/docker-minecraft-server`.
3. Run GitHub Actions and inspect failures; do not claim success from a queued workflow.
4. Open a PR from the fork to `itzg/docker-minecraft-server` with reproduction, root cause, tests, and compatibility notes.
5. Build a fork image only if the project workflow supports it; record the immutable digest.
6. Create a separate homelab MR that switches only GTNH beta from the temporary custom-pack workaround back to native `TYPE=GTNH` plus `GTNH_PACK_VERSION=latest-dev` using the fork image digest.
7. Merge/reconcile in GitOps order and verify the beta pod, restart count, selected pack version, and server readiness.

**Do not:** push to upstream `master`, publish mutable-only image references for production testing, or remove the homelab workaround before the replacement image has passed a real startup test.

---

## Acceptance criteria

- Native `TYPE=GTNH` with `GTNH_PACK_VERSION=latest-dev` selects a beta or RC server pack from `versions.json` and no longer exits with “Server files not found”.
- Stable and exact-version behavior remains compatible.
- Java 8 and Java 17+ archive selection is correct and guarded by compatibility metadata.
- Focused tests and the repository test suite pass.
- Documentation explains the supported version-selection behavior.
- A GitHub PR is opened from the fork with reproducible evidence.
- The homelab beta workload is not switched back until an immutable fixed image has been verified.
6 changes: 4 additions & 2 deletions docs/types-and-platforms/mod-platforms/gtnh.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,11 @@ Configuration options with defaults:

## Set Modpack version

As GTNH is a Minecraft 1.7.10 modpack, when using it your minecraft version is set to 1.7.10 by default. The [modpack version](https://www.gtnewhorizons.com/downloads/) can be selected by setting `GTNH_PACK_VERSION` to `latest`, `latest-dev` or any specific version number. `latest` will automatically select the latest full release version available and deploy the server with it (Note: this will also automatically update the server on startup). `latest-dev` does the same but selects the latest version marked as beta or RC (it won't select a full release version even if a newer exist). The third (and recommended) option is setting the server to a specific version like `2.8.1` to manage updates manually.
As GTNH is a Minecraft 1.7.10 modpack, when using it your minecraft version is set to 1.7.10 by default. The [modpack version](https://www.gtnewhorizons.com/downloads/) can be selected by setting `GTNH_PACK_VERSION` to `latest`, `latest-dev` or any specific version number. `latest` selects the newest release marked `Stable release`; `latest-dev` selects the newest beta or release-candidate build and will not select a stable release. Both moving selectors automatically update the server on startup when a newer matching pack is available. The third (and recommended) option is setting the server to a specific version like `2.8.1` to manage updates manually.

> To actively prevent an update from happening you can set the environment variable `SKIP_GTNH_UPDATE_CHECK` to true this will prevent any update check from running, but will also prevent the server install from running, so just set it after the initial setup.
The image resolves versions from the GTNH [`versions.json`](https://downloads.gtnewhorizons.com/versions.json) metadata. Java 8 uses the selected release's `server.java8Url`; Java 17 and newer use `server.java17_2XUrl` and must not exceed the release's `maxJavaVersion`. Startup fails with an actionable error when metadata, a compatible server URL, or the selected version is unavailable.

> To actively prevent an update from happening you can set the environment variable `SKIP_GTNH_UPDATE_CHECK` to true. This skips both update checks and initial installation, so set it only after the server has completed its first setup.

## Resource requirements

Expand Down
67 changes: 28 additions & 39 deletions scripts/start-deployGTNH
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,70 +7,59 @@
function getGTNHdownloadPath(){
gtnh_download_path=""
local release_object=""
local versions_json=""
local current_java_version=""
local versions_url="https://downloads.gtnewhorizons.com/versions.json"
current_java_version=$(mc-image-helper java-release)

# Select release JSON object
if ! versions_json=$(curl -fsSL "$versions_url"); then
logError "Failed to retrieve releases from $versions_url"
exit 1
fi

# Normalize every selection mode to {version, releaseDate, title, maxJavaVersion, server}.
local release_filter=''
if [ "$GTNH_PACK_VERSION" == "latest-dev" ]; then
if ! release_object="$(
curl -fsSL "https://downloads.gtnewhorizons.com/versions.json" \
| jq -r '(.versions // .)|to_entries|sort_by(.value.releaseDate)|map(select(.value.title=="Beta release"))|.[-1]'
)"; then logError "Failed to retrieve release from https://downloads.gtnewhorizons.com/versions.json"
exit 1
fi
log "Selected $(jq '.key' <<< "$release_object") as latest dev version for download."

release_filter='map(select((.value.title // "") | test("^(beta|rc) release$"; "i"))) | sort_by(.value.releaseDate, .key) | .[-1]'
elif [ "$GTNH_PACK_VERSION" == "latest" ]; then
if ! release_object="$(
curl -fsSL "https://downloads.gtnewhorizons.com/versions.json" \
| jq -r '(.versions // .)|to_entries|sort_by(.value.releaseDate)|map(select(.value.title=="Stable release"))|.[-1]'
)"; then logError "Failed to retrieve release from https://downloads.gtnewhorizons.com/versions.json"
exit 1
fi
log "Selected $(jq '.key' <<< "$release_object") as latest version for download."

release_filter='map(select((.value.title // "") == "Stable release")) | sort_by(.value.releaseDate, .key) | .[-1]'
else
if ! release_object="$(
curl -fsSL "https://downloads.gtnewhorizons.com/versions.json" \
| jq -r --arg USRIN "$GTNH_PACK_VERSION" '(.versions // .)|to_entries|sort_by(.value.releaseDate)|map(select(.key==$USRIN))|.[]'
)"; then logError "Failed to retrieve release from https://downloads.gtnewhorizons.com/versions.json"
exit 1
fi
log "Selected $(jq -r '.key' <<< "$release_object") as matching version for download."

# shellcheck disable=SC2016
release_filter='map(select(.key == $requested_version)) | .[0]'
fi

if [[ -z $release_object ]]; then
if ! release_object=$(jq -e -c --arg requested_version "$GTNH_PACK_VERSION" \
"(.versions // .) | to_entries | $release_filter | select(.) | {version: .key, releaseDate: .value.releaseDate, title: .value.title, maxJavaVersion: .value.maxJavaVersion, server: .value.server}" \
<<< "$versions_json"); then
logError "No release found for GTNH_PACK_VERSION=$GTNH_PACK_VERSION. Please check if the version is valid."
exit 1
fi

# Select compatible server files for java version

log "Selected $(jq -r '.version' <<< "$release_object") ($(jq -r '.title' <<< "$release_object")) for download."

# Select compatible server files for the Java version.
if (( current_java_version == 8 )); then
gtnh_download_path=$(jq -r '.value.server.java8Url' <<< "$release_object")
gtnh_download_path=$(jq -e -r '.server.java8Url // empty' <<< "$release_object")
log "Use GTNH Server Java 8 release."

elif (( current_java_version >= 17 )); then
if (( current_java_version > $(jq '.value.maxJavaVersion' <<< "$release_object") )); then
local max_java_version=""
max_java_version=$(jq -e -r '.maxJavaVersion // empty' <<< "$release_object")
if [[ -z "$max_java_version" ]] || (( current_java_version > max_java_version )); then
logError "Container Java version $current_java_version is not supported by GTNH. Try an older release."
exit 1
fi

gtnh_download_path=$(jq -r '.value.server.java17_2XUrl' <<< "$release_object")
gtnh_download_path=$(jq -e -r '.server.java17_2XUrl // empty' <<< "$release_object")
log "Use GTNH Server Java 17+ release."

else
logError "Container Java version $current_java_version is not supported by GTNH."
exit 1

fi
if [[ -z $gtnh_download_path ]]; then
logError "Server files not found! Please check if the selected GTNH_PACK_VERSION=$GTNH_PACK_VERSION is valid."

if [[ -z "$gtnh_download_path" || "$gtnh_download_path" != https://* || "$gtnh_download_path" == *[[:space:]]* ]]; then
logError "Server files not found or URL is invalid for GTNH_PACK_VERSION=$GTNH_PACK_VERSION."
exit 1
fi
debug "Download source URL: $gtnh_download_path"

}

function deleteGTNHbackup(){
Expand Down
Loading