Update pulumi/pulumi to v3.158.0

[guineveresaenger]

This skips one version of pulumi.

Also updates pulumi-yaml and pulumi-java to their respective latest.

Small adjustments to GetPluginPath and ResolvePlugin.

Fixes #2958.

[codecov]

Codecov Report

Attention: Patch coverage is 71.42857% with 4 lines in your changes missing coverage. Please review.

Project coverage is 68.49%. Comparing base (fa1d237) to head (f13d0e5).
Report is 9 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/tf2pulumi/il/plugin_info.go	50.00%	2 Missing and 1 partial ⚠️
pkg/pf/internal/plugin/provider_server.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2971      +/-   ##
==========================================
+ Coverage   68.30%   68.49%   +0.18%     
==========================================
  Files         330      330              
  Lines       42082    42105      +23     
==========================================
+ Hits        28746    28839      +93     
+ Misses      11757    11689      -68     
+ Partials     1579     1577       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[t0yv0]

Why is this necessary, curious? Adds to the scope of the change. Perhaps separate PR?

[guineveresaenger]

Local test runs do not enforce this constraint unless you run make, which is difficult to do for an individual test. A lot of the pain here comes from different environments doing different things. :/

[t0yv0]

Good point on enforcement, let us fix this. This kind of change needs to be a separate PR otherwise it is hard to see which change comes from what IMO.

[guineveresaenger]

Yeah, completely agreed.

[t0yv0]

This can needlessly panic. A bit dangerous.

[t0yv0]

Why change from simple? Actually the delta on the "simple" provider is interesting to investigate.

[guineveresaenger]

It is because of a change in the converter that doesn't allow for "fake" providers. Since this code path calls out to the command line, the converter attempts to download a provider called "simple", and it results in breaking Java codegen in this test.

The delta is not really all that interesting, because it is caused by the fact that this provider doesn't exist in real life.

[guineveresaenger]

@lunaris and I did investigate for quite a while on what happened here on Friday.

[lunaris]

The challenge here now is that the converter will call out to Terraform's registry in order to try and resolve versions for providers that it thinks are outside the Pulumiverse. It's always done this, but this check is now earlier in the call chain in order to facilitate converting "any TF provider". We do this by dynamically bridging providers not in the Pulumiverse to e.g. facilitate generating mappings. This test now fails because "simple" is not in the Pulumiverse, but it is also not a "real" TF provider, so version resolution (and consequently producing a hint for mapping, which powers part of this test case) fails. There are several options here, none of them (on the surface) ideal, but enumerating in case people have thoughts/better ideas than I:

• Don't fail mapping hint generation for TF providers we can't resolve. I think this is fine, but it does make it trickier for convert to produce errors when e.g. the user has genuinely mentioned a provider that (as far as we're concerned) doesn't exist. That said, I think fixing that gets us further in this test (though it may fail later on due to missing/incompletely-mocked schema, as discussed elsewhere e.g. in the case of Java imports)
• More completely mock out the processes that power this test -- we already mock out mappings (passing a JSON file on the CLI to the converter, so that it can just use those mappings as-is without remote calls), but we don't mock out the TF registry. We could look at doing that somehow here (likely not pretty, but possible)?
• Use a "real" provider so that the TF registry lookups succeed. This is the approach currently in this PR I believe. Again, not ideal, but I think (?) the simplest change?

[t0yv0]

Reasonable but maybe a different PR? Too many moving parts at once. Also it needs to pin a version of openstack such as in here or why not use one that's already there perhaps.

pulumi-terraform-bridge/Makefile

Line 8 in 4c35e78

install_plugins::

[t0yv0]

I'm rebuilding this change #2973 as requested through ops handoff FWIW.

[t0yv0]

Landed #2973 but there might be some interesting residual things in this PR that are worth merging up. There is in particular an upgrade to the TF converter version we use in tests. Note that it is not currently coupled with the converter version deployed in the bridged providers (pinned in .ci-mgmt.yaml) but both could use updating I guess.