golang: split src into separate packages

[GeorgeSapkin]

📦 Package Details

Maintainer: me

Description:

Split misc sources and test data into separate packages. Reduces target src package size by a third.

Move architecture-dependent generated source from src to the main package.

Mark doc, misc, src and tests packages with PKGARCH:=all as they don't have any architecture-specific files.

Installing into share and symlinking to lib breaks tests:

--- FAIL: TestAllDependencies (0.01s)

    moddeps_test.go:49: findGorootModules didn't find the well-known module "std"

--- FAIL: TestDependencyVersionsConsistent (0.00s)

    moddeps_test.go:356: findGorootModules didn't find the well-known module "std"

Install into lib directly instead.

Fix stripping and strip compiler only.

Add libraries used in test data to extra_provides to pass dependency checks in package-pack.

Remove unnecessary dependencies.

Fix setting architecture-specific defaults in zbootstrap.go.

Don't set GO_LDSO as it's determined automatically at link time.

Replace Makefile variables in comments to avoid expanding them unnecessarily.

Fixes: c137c38 ("golang: new packages")
Fixes: b211946 ("golang: Update to 1.24.0")
Fixes: #27633
Link: golang/go@cce90c1
Link: golang/go#54197

---

🧪 Run Testing Details

• OpenWrt Version: HEAD, 25.12-rc3
• OpenWrt Target/Subtarget: x86/64
• OpenWrt Device: QEMU

Tested:

• go tool dist test on target with tests installed. TestGroupCleanupUserNamespace fails.
• Building and running AdGuard Home.
• Target compiler.

---

✅ Formalities

• I have reviewed the CONTRIBUTING.md file for detailed contributing guidelines.

[jefferyto]

What happens if you run go tool dist test on device without testdata present?

Are you sure the src package is the same across all architectures?

[GeorgeSapkin]

What happens if you run go tool dist test on device without testdata present?

It returns [no test files] for every test:

##### Testing packages.
# internal/abi
internal/abi/abi_test.s:23:1: [amd64] FuncPCTestFn: function FuncPCTestFn missing Go declaration
?   	archive/tar	[no test files]
?   	archive/zip	[no test files]
?   	bufio	[no test files]
?   	bytes	[no test files]
?   	cmp	[no test files]
?   	compress/bzip2	[no test files]
?   	compress/flate	[no test files]
?   	compress/gzip	[no test files]
?   	compress/lzw	[no test files]
?   	compress/zlib	[no test files]

Same as on my Fedora without tests installed. Do you think it makes sense to package tests at all? That is have a separate test package as other distros do. I'm somewhat skeptical if people are actually using target Go to begin with, let alone the test part.

Are you sure the src package is the same across all architectures?

I'm trying to model this after bigger distributions where src is noarch, like Fedora:

Name            : golang-src
Epoch           : 0
Version         : 1.25.6
Release         : 1.fc43
Architecture    : noarch
Installed size  : 81.4 MiB
Source          : golang-1.25.6-1.fc43.src.rpm
From repository : updates
Summary         : Golang compiler source tree
URL             : https://go.dev
License         : BSD-3-Clause AND LicenseRef-Fedora-Public-Domain
Description     : Golang compiler source tree
Vendor          : Fedora Project

If it's not fully noarch now, then it still needs work. I looked at the packaged files and I couldn't immediately spot anything, but source, scripts, and some docs. And of course it's beside the point, as AFAICT all packages are built and pulled per arch despite being marked as noarch, i.e. there's no all/noarch target.

[jefferyto]

Do you think it makes sense to package tests at all? That is have a separate test package as other distros do.

Yes I do - On normal desktop systems, if the user doesn't like how Go is packaged by their distro, it is fairly easy for them to compile their own Go from source. On OpenWrt devices with constrained CPU/RAM, it would be very difficult to compile Go from source. That's why my intention was to keep the target package as stock as possible, including having access to tests and test data.

If it's not fully noarch now, then it still needs work.

It's not - if you diff the package contents from different architectures you will find that usr/share/go-1.XX/src/internal/buildcfg/zbootstrap.go is architecture (actually target) specific.

This file specifies the default values for various GOXXX variables and is generated by Go during compilation. The values are fine when doing a "normal" build (compile for use on the same machine), but AFAIK when cross-compiling the values may be incorrect for the target machine. That's why there is a list of mods:

packages/lang/golang/golang-version.mk

Lines 174 to 181 in b19c784

	PKG_GO_ZBOOTSTRAP_MODS?= \
	s/defaultGO386 = `[^`]*`/defaultGO386 = `$(or $(GO_386),sse2)`/; \
	s/defaultGOAMD64 = `[^`]*`/defaultGOAMD64 = `$(or $(GO_AMD64),v1)`/; \
	s/defaultGOARM = `[^`]*`/defaultGOARM = `$(or $(GO_ARM),7)`/; \
	s/defaultGOARM64 = `[^`]*`/defaultGOARM64 = `$(or $(GO_ARM64),v8.0)`/; \
	s/defaultGOMIPS = `[^`]*`/defaultGOMIPS = `$(or $(GO_MIPS),hardfloat)`/; \
	s/defaultGOMIPS64 = `[^`]*`/defaultGOMIPS64 = `$(or $(GO_MIPS64),hardfloat)`/; \
	s/defaultGOPPC64 = `[^`]*`/defaultGOPPC64 = `$(or $(GO_PPC64),power8)`/;

and the file updated during build:

packages/lang/golang/golang-version.mk

Lines 229 to 237 in b19c784

	$(SED) '$(PKG_GO_ZBOOTSTRAP_MODS)' "$(PKG_GO_ZBOOTSTRAP_PATH)"
	if echo 'int main() { return 0; }' | $(TARGET_CC) -o $(PKG_BUILD_DIR)/test-ldso -x c - > /dev/null 2>&1; then \
	LDSO=$$$$( \
	readelf -l $(PKG_BUILD_DIR)/test-ldso | \
	sed -n -e 's/^.*interpreter: \(.*\)[]]/\1/p' \
	) ; \
	fi ; \
	$(SED) "s,defaultGO_LDSO = \`[^\`]*\`,defaultGO_LDSO = \`$$$$LDSO\`," "$(PKG_GO_ZBOOTSTRAP_PATH)"

You could move that file from the src package to the main binary package (like in Debian/Ubuntu), then the src package can be marked as architecture-independent (there are more generated files and Debian includes all of them in the binary package, but I believe the rest are identical across architectures). Or you could combine the binary and src packages into one package (like Arch Linux), since the source files are required anyway. I think I kept hoping the src package can be made optional one day, but more likely is I didn't think it was worth the effort to make more changes.

Some things I found while looking into this:

• The various defaultGOXXX values in zbootstrap.go don't appear to be correct in snapshot/25.12. I checked the arm_cortex-a7 packages for 24.10, 25.12 and snapshot. The 24.10 package has the correct value for defaultGOARM (5) but the 25.12 and snapshot packages have the Go default instead (7).

  This means on an arm_cortex-a7 device running snapshot or 25.12, if the user compiles a Go program without setting the GOARM environment variable (letting Go use its built-in default), then the Go program would be compiled for GOARM=7 and would crash/fail to run on the device.

  I think this is because in Go 1.23 (on 24.10) the variables start with a lowercase "d" (e.g. defaultGO386) whereas in Go 1.25 the variables start with an uppercase "D" (DefaultGO386). This change appears to have occurred in Go 1.24 (golang/go@cce90c1). The bootstrap mods (sed commands) will need to be adjusted.

• I think the default LDSO value no longer needs to be set. This was originally added for Go 1.17 when make.bash detected GO_LDSO for musl. I believe this is not necessary since Go 1.20 (cmd/link: determine GO_LDSO on Linux at link time instead of baking in make.bash golang/go#54197).

[GeorgeSapkin]

Thanks for the detailed answer, as usual. I split tests into a separate package. Turns out it's a bit more complicated than it seems as there are more files that need to be packaged. And it took me a minute to figure out that some tests kept crashing due to OOM in the VM. I fixed the issues that you brought up as well (even though they are OOS for this PR). I also fixed stripping so now main binaries are stripped, but not the other packages. With tests installed, the target test suite produces the same output as on my desktop. Edit: actually some OS-related tests like TestGroupCleanupUserNamespace seems to fail, as they expect a different command output.

There are more version-related changes that have been overlooked in the past, e.g. OS/arch combos and checks, that I'm leaving for a separate PR.

[GeorgeSapkin]

I changed the install path for both host and target go to be under lib instead of a symlink from share to lib, again, same as Fedora. Otherwise several tests failed:

--- FAIL: TestAllDependencies (0.01s)

    moddeps_test.go:49: findGorootModules didn't find the well-known module "std"

--- FAIL: TestDependencyVersionsConsistent (0.00s)

    moddeps_test.go:356: findGorootModules didn't find the well-known module "std"

Looks like these tests can't deal with symlinks.

The only remaining failing test is syscall-related:

--- FAIL: TestGroupCleanupUserNamespace (0.00s)
    exec_linux_test.go:210: id: uid=0(root) gid=0(root)
    exec_linux_test.go:216: expected prefix: "uid=0(root) gid=0(root) groups=0(root)"