Use `tofu show -json -module=DIR` to extract module metadata

[apparentlymart]

Currently the module metadata extraction uses an unreleased fork of OpenTofu CLI that has an extra command tofu metadata dump, written only for this purpose as a temporary solution:

registry-ui/backend/internal/moduleindex/moduleschema/tofu.go

Line 93 in c4ea85a

cmd := exec.Command(command, "metadata", "dump", "-json")

In opentofu/opentofu#3003 and opentofu/opentofu#3013 we added an actually-documented alternative in the form of tofu show -json -module=DIR, which returns a subset of the JSON configuration representation which should include enough information to replace our use of tofu metadata dump.

At some point after OpenTofu v1.11.0 is released (thereby making these new features available in a stable release) we should switch to using the documented API instead of making a local build from a branch, and possibly consider making the automatic running of this automatically use the latest available non-prerelease version as described by our get.opentofu.org API, so it'll automatically adopt any improvements we release in future (though that's a tradeoff; fixing to a specific release until we intentionally change it means the registry UI is less likely to become broken if a future OpenTofu release has a bug.)

[apparentlymart]

For now this is blocked until the OpenTofu v1.11.0 release, since we're not in any huge rush for this and so not worth the effort to implement a temporary step such as building OpenTofu from the main branch.

[cam72cam]

I'd argue that we should switch this over before the release to ensure that it's working correctly. We already build from source off of a very hacky branch.

[apparentlymart]

What kind of "not working correctly" are you concerned about? (I'm asking because for some kinds of misbehavior there are lower-effort ways we can improve our confidence, of course.)

We certainly could do something sooner, of course. That would effectively split this work across both the v1.11 and v1.12 milestones (the latter making it use real releases rather than a build from main), whereas I'd been assuming we'd just do it all in one go in the v1.12 development period.

I don't have any strong objection, but this work seems pretty unimportant relative to other stuff in our v1.11 milestone -- it only changes an implementation detail, so doesn't directly benefit end-users -- so I imagine we'd treat it as lower priority than the other work in there. Ultimately your call, though.