`--file` treats query params in HTTP URLs as path assignments

[mamachanko]

What steps did you take:

ytt the bootstrapper resources I am given by Tanzu Platform in form of a HTTP URL which contains query params

What happened:

ytt -f "https://example.com/?id=123&source=hello"

ytt: Error: Expected file 'https://example.com/?id=123&source=hello' to only have single '=' sign to for relative path assignment

What did you expect:

ytt fetches the file and templates it.

Anything else you would like to add:

I can workaround the issue with, say

http "https://example.com/?id=123&source=hello" | ytt -f-

Environment:

• ytt version (use ytt --version): 0.50.0
• OS (e.g. from /etc/os-release): macOS

---

Vote on this request

This is an invitation to the community to vote on issues, to help us prioritize our backlog. Use the "smiley face" up to the right of this comment to vote.

👍 "I would like to see this addressed as soon as possible"
👎 "There are other more important things to focus on right now"

We are also happy to receive and review Pull Requests if you want to help working on this issue.

[100mik]

Nice catch! We should prioritise this fix sooner rather than later. But it is handy that we have a workaround.
I will mark this as a good first issue in case someone is looking to lend a hand 🙌🏼

[ernstvonoelsen]

@100mik @renuy

here's how I tested my suggested solution:

$ git branch
  develop
* issue-928
$
$
$
$ ./hack/build.sh
++ git describe --tags
++ grep -Eo '[0-9]+\.[0-9]+\.[0-9]+'
+ LATEST_GIT_TAG=0.51.0
+ VERSION=0.51.0
+ export CGO_ENABLED=0
+ CGO_ENABLED=0
+ LDFLAGS='-X carvel.dev/ytt/pkg/version.Version=0.51.0'
+ rm -f website/generated.go
++ go list ./...
++ grep -v yaml.v2
+ go fmt carvel.dev/ytt/cmd/ytt carvel.dev/ytt/cmd/ytt-lambda-website carvel.dev/ytt/pkg carvel.dev/ytt/pkg/cmd carvel.dev/ytt/pkg/cmd/template carvel.dev/ytt/pkg/cmd/ui carvel.dev/ytt/pkg/experiments carvel.dev/ytt/pkg/filepos carvel.dev/ytt/pkg/files carvel.dev/ytt/pkg/orderedmap carvel.dev/ytt/pkg/schema carvel.dev/ytt/pkg/spell carvel.dev/ytt/pkg/template carvel.dev/ytt/pkg/template/core carvel.dev/ytt/pkg/texttemplate carvel.dev/ytt/pkg/validations carvel.dev/ytt/pkg/version carvel.dev/ytt/pkg/website carvel.dev/ytt/pkg/workspace carvel.dev/ytt/pkg/workspace/datavalues carvel.dev/ytt/pkg/workspace/ref carvel.dev/ytt/pkg/yamlfmt carvel.dev/ytt/pkg/yamlmeta carvel.dev/ytt/pkg/yamltemplate carvel.dev/ytt/pkg/yttlibrary carvel.dev/ytt/pkg/yttlibrary/overlay carvel.dev/ytt/pkg/yttlibraryext carvel.dev/ytt/pkg/yttlibraryext/toml carvel.dev/ytt/test/e2e carvel.dev/ytt/test/filetests
pkg/website/generated.go
+ go mod vendor
+ go mod tidy
+ ./hack/generate-website-assets.sh
+ rm -f pkg/website/generated.go
+ cd pkg/website
+ go run ./../../cmd/ytt -f . -f ../../examples/playground/basics -f ../../examples/playground/overlays -f ../../examples/playground/getting-started --file-mark 'alt-example**/*:type=data' --file-mark 'example**/*:type=data' --file-mark generated.go.txt:exclusive-for-output=true --dangerous-emptied-output-directory ../../tmp/
creating: ../../tmp/generated.go.txt
+ mv tmp/generated.go.txt pkg/website/generated.go
+ go build '-ldflags=-X carvel.dev/ytt/pkg/version.Version=0.51.0' -trimpath -o ytt ./cmd/ytt/...
+ ./ytt version
ytt version 0.51.0
+ export GOOS=linux GOARCH=amd64
+ GOOS=linux
+ GOARCH=amd64
+ go build '-ldflags=-X carvel.dev/ytt/pkg/version.Version=0.51.0' -trimpath -o ./tmp/ytt ./cmd/ytt/...
+ go build '-ldflags=-X carvel.dev/ytt/pkg/version.Version=0.51.0' -trimpath -o ./tmp/main ./cmd/ytt-lambda-website/...
+ cd tmp
+ chmod +x main ytt
+ rm -f ytt-lambda-website.zip
+ zip ytt-lambda-website.zip main ytt
  adding: main (deflated 47%)
  adding: ytt (deflated 47%)
+ echo SUCCESS
SUCCESS
$
$
$
$ python3 -m http.server --directory yaml_resources/ &
[1] 4309
$
$ ./ytt --file http://localhost:8000/test.yaml?hello=world
127.0.0.1 - - [05/Dec/2024 05:42:06] "GET /test.yaml?hello=world HTTP/1.1" 200 -
hello: world
foo:
- bar1
- bar2
$

where

$ cat yaml_resources/test.yaml 
# comment
hello: world
foo:
  - bar1
  - bar2

is the content of the fetched template file which indeed is processed correctly.

So from the above protocol, one can deduce that the server receives the full request, including the query params. Calculating an HTTPSource.relPath with explicitly cropped query params does hence not affect the source caching/fetching.