Hugo attempts to download module from GitHub even after it has been downloaded to _vendors folder

[gcushen]

Currently, Hugo can attempt to download a module from GitHub even if it has been downloaded to the _vendors folder.

The expected behavior is that Hugo checks if the module in go.mod has been downloaded to the _vendors folder before attempting to access the internet.

This can be a common use case where you have a developer creating a modular Hugo project for a business stakeholder or client, but only the developer should have access to the private Hugo module in the repo, so the module is vendored for the stakeholder/client.

As a workaround, if the private Hugo module is deleted from go.mod then Hugo will use the vendored module. Ideally, it should not require the workaround of the stakeholder/client deleting the module from go.mod as then their go.mod is out of sync with the developer's copy.

What version of Hugo are you using (hugo version)?

$ hugo version 0.126.2

Does this issue reproduce with the latest release?

Yes

[bep]

Do you have a simple example that shows the behaviour you describe?

[jmooring]

I am unable to reproduce the problem as described.

git clone --single-branch -b hugo-github-issue-12555 https://github.com/jmooring/hugo-testing hugo-github-issue-12555
cd hugo-github-issue-12555

Terminate all network connections, then...

hugo server

You should see 3 articles listed on the home page.

[bep]

The current logic is rather simple (taken from memory): If the main project has a _vendor dir and that directory resolves all the modules defined in the site config, then all is fine and dandy. All other cases will be handed to go which may or may not ask the internet for advice.

[gcushen]

To reproduce:

1. hugo new site test
2. cd test
3. echo -e "module test\ngo 1.19\nrequire github.com/gcushen/private-go-module v0.0.0-20240506024847-96f2f37c71bb" >> go.mod
4. echo -e "\n\n[[module.imports]]\npath = 'github.com/gcushen/private-go-module'" >> hugo.toml
5. mkdir -p '_vendor/github.com/gcushen/private-go-module'
6. echo -e "# github.com/gcushen/private-go-module v0.0.0-20240506024847-96f2f37c71bb" >> ./_vendor/modules.txt
7. Add dummy content: echo 'title: Test' >> '_vendor/github.com/gcushen/private-go-module/config.yaml'
8. hugo server

This results in the following error and inability to view the site even though the module is vendored:

Error: command error: failed to load modules: failed to download modules: failed to execute 'go [mod download -modcacherw]': failed to execute binary "go" with args [mod download -modcacherw]: go: github.com/gcushen/private-go-module@v0.0.0-20240506024847-96f2f37c71bb: invalid version: git ls-remote -q origin in /tmp/hugo_cache/modules/filecache/modules/pkg/mod/cache/vcs/9c89562b4e8d021544f76a9645661b038d05b9e1ad5a17389373f2b4ff435feb: exit status 128:
        remote: Invalid username or password.
        fatal: Authentication failed for 'https://github.com/gcushen/private-go-module/'
 *errors.errorString

This can be a common use case where you have a developer creating a modular Hugo project for a business stakeholder or client, but only the developer should have access to the private Hugo module in the repo, so the module is vendored for the stakeholder/client.

[bep]

@gcushen that's not a "simple example".

[gcushen]

@gcushen that's not a "simple example".

It's a fully reproducible example of importing an arbitrary private Hugo module in only 8 steps, based on the minimally reproducible hugo new site command. How would you suggest to make it simpler?

[bep]

@gcushen

How would you suggest to make it simpler?

OK, maybe "not simple" wasn't the correct way to describe your example. But manually constructing the Go module/vendoring setup (example line below) is not the real thing and filled with many sources of error.

echo -e "# github.com/gcushen/private-go-module v0.0.0-20240506024847-96f2f37c71bb" >> ./_vendor/modules.txt

That said, I tested your example, and it runs fine on my MacBook. I also tried to delete the _vendor folder, a new build triggers a "Please make sure you have the correct access rights and the repository exists", which makes sense as this is a private repo.

Can you

1. Check your environment for any Hugo module related env vars that would somehow disable vendoring (e.g. env | grep HUGO_), e.g. HUGO_IGNOREVENDORPATHS.
2. Print the full output of hugo version.

Also, is this something only you experience or is this something that's reported from your end users?

[bep]

If you're able to insert some debug prints in Hugo code, you can check if this condition triggers or not (and possibly also why):

hugo/modules/collect.go

Line 194 in c8dac67

if c.ccfg.IgnoreVendor == nil && c.isVendored(c.ccfg.WorkingDir) {