AttachProviderBinary takes no effect

[t0yv0]

What happened?

AttachProviderBinary takes no effect and ignores the intent.

Example

package main

import (
	pt "github.com/pulumi/providertest/pulumitest"
	"github.com/pulumi/providertest/pulumitest/opttest"
	"github.com/stretchr/testify/require"
	"os"
	"os/exec"
	"path/filepath"
	"testing"
)

func Test_Random(t *testing.T) {

	wd, err := os.Getwd()
	require.NoError(t, err)

	wda, err := filepath.Abs(wd)
	require.NoError(t, err)

	wd = wda

	cmd := exec.Command("make", "-B", "provider")
	cmd.Dir = filepath.Join(wd, "..", "pulumi-random")
	err = cmd.Run()
	require.NoError(t, err)

	randomProviderBin := filepath.Join(wd, "..", "pulumi-random", "bin")
	randomProvider := filepath.Join(wd, "..", "pulumi-random", "bin", "pulumi-resource-random")
	ptest := pt.NewPulumiTest(t,
		"test-proj",
		opttest.SkipInstall(),
		opttest.AttachProviderBinary("random", randomProvider),
		opttest.Env("PATH", os.Getenv("PATH")+string(os.PathListSeparator)+randomProviderBin),
	)

	upResult := ptest.Up(t)

	t.Logf("%s", upResult.StdOut)
	t.Logf("%s", upResult.StdErr)

}

I had to add the "PATH" override for this to use the binary I want. Without it Pulumi would install non-local random provider release.

anton@anton-mbp-m3> pulumi plugin ls                                                                                                                                                                   ~/code/refresh-by-default/tests
NAME    KIND      VERSION  SIZE   INSTALLED      LAST USED
random  resource  4.18.1   44 MB  6 seconds ago  6 seconds ago

Output of pulumi about

github.com/pulumi/providertest v0.3.1

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[blampe]

@t0yv0 I'm confused by this line, but you say it resolved things?

opttest.Env("PATH", os.Getenv("PATH")+string(os.PathListSeparator)+randomProviderBin),

Doesn't that point to the binary itself, not the path containing it? This seems like it shouldn't have any effect at all? Or maybe I'm out of my shell depth.

[t0yv0]

My intent as a user is to make sure that the test uses a locally built provider binary instead of testing some auto-downloaded unspecified version. I thought this does it:

randomProvider := filepath.Join(wd, "..", "pulumi-random", "bin", "pulumi-resource-random")
opttest.AttachProviderBinary("random", randomProvider),

But it did not seem to do it. Perhaps there's a bug or perhaps I am using it wrong. For example, maybe pulumi-resource-random does not exist at specified location, or else it expected a folder path and I gave it a binary path; in these cases I would love it to fail and tell me instead of proceeding and testing the wrong provider.

Once I failed to work this out I fell back on:

opttest.Env("PATH", os.Getenv("PATH")+string(os.PathListSeparator)+randomProviderBin),

This does work as I want it to work, but is a little annoying. In addition in complex scenarios where the provider needs to change from V1 to V2/WIP this would not work as PATH will override both.

Not sure if this answers your question, if not I can try again if you rephrase.

[blampe]

My intent as a user is to make sure that the test uses a locally built provider binary instead of testing some auto-downloaded unspecified version.

Of course :)

I was remarking that the PATH you set doesn't seem to be valid. I think the issue is that the test is inheriting the parent process' PATH, so it might not actually matter what you set it to. Does it work if you set it to gibberish? The right fix might be to not inherit the parent PATH at all.

[t0yv0]

Setting PATH does work though, it is not pretty but it brings the provider WIP into the test, which I can verify by injecting panics and such. Anyway it is not relevant to the issue :) It is a workaround I can keep using, but would be nice to fix the Attach issue if there is one. I might have a peek once I am spinning fewer plates.