Completion of required_providers `source` and `version` from Terraform Registry

[radeksimko]

Background

Terraform has a rich ecosystem of providers published in the Terraform Registry.

In order to use a provider in Terraform 0.12+, the user has to declare it explicitly, e.g.

terraform {
  required_providers {
    hcs = {
      source = "hashicorp/hcs"
      version = "0.3.0"
    }
  }
}

We provide completion for most of the above, except for the values of source and version. As a result, unless the user is copy-pasting the provider address and/or version they can make a mistake which would be preventable with completion.

Expected User Experience

source

terraform {
  required_providers {
    hcs = {
      source = "# HERE #"
    }
  }
}

Upon typing >= 2 characters, the user will be provided with a list of providers from the Registry matching the typed name. e.g. if they type ad then the following list of 14 providers would come up:

https://registry.terraform.io/search/providers?q=ad

version

terraform {
  required_providers {
    hcs = {
      source = "hashicorp/aws"
      version = "# HERE #"
    }
  }
}

Upon completion, the user will be provided with a list of published versions of a given provider, e.g. in case of hashicorp/aws they will see the following:

• 3.46.0
• 3.45.0
• 3.44.0
• etc.

Proposal

hashicorp/hcl-lang

#868

hashicorp/terraform-schema

• Set CompletionHooks for module source to CompleteRegistryProviderSources & CompleteLocalProviderSources
• Update earlydecoder to obtain and track each requirement's *hcl.Range within module.Meta so that the completion hook for version has enough context
• Set CompletionHooks for module version to CompleteRegistryProviderVersions

hashicorp/terraform-ls

• Store updated module.Meta as above in state

  terraform-ls/internal/state/module.go

  Lines 18 to 26 in 687a81b

  	type ModuleMetadata struct {
  	CoreRequirements version.Constraints
  	Backend *tfmod.Backend
  	ProviderReferences map[tfmod.ProviderRef]tfaddr.Provider
  	ProviderRequirements tfmod.ProviderRequirements
  	Variables map[string]tfmod.Variable
  	Outputs map[string]tfmod.Output
  	Filenames []string
  	}

• Expose ranges of module calls via Meta of PathContext to allow CompleteRegistryProviderVersions to establish which module source it should provide versions for
• Introduce CompleteRegistryProviderSources completion hook to return addresses of modules from the Registry/Algolia
• Introduce CompleteLocalProviderSources completion hook to return paths of all indexed modules (relative to where they're being called from)
• Introduce CompleteRegistryProviderVersions completion hook to return versions of a given Registry module from the Registry
• Register CompleteRegistryProviderSources, CompleteLocalProviderSources and CompleteRegistryProviderVersions to the DecoderContext

  terraform-ls/internal/decoder/decoder.go

  Lines 82 to 85 in f49cd99

  	dCtx := decoder.DecoderContext{
  	UtmSource: "terraform-ls",
  	UseUtmContent: true,
  	}

• Introduce ResolveRegistryProviderSource resolve hook for module source description
• Introduce ResolveRegistryProviderVersion resolve hook for module version description

Implementation Notes

• Registry Search can be done via Algolia
  • Algolia Go SDK: https://github.com/algolia/algoliasearch-client-go
  • https://www.algolia.com/doc/api-reference/api-methods/search/#examples
  • Example code snippet: https://gist.github.com/radeksimko/f0446c813bb13265de1d5390ff717b70
  • relevant index is tf-registry:prod:providers, relevant App ID & API key is in team 1Password although realistically we'll end up just baking it into the source code and none of it is secret per se
• Lookup of module versions can be done via Registry API
  • https://registry.terraform.io/v2/providers/{PROVIDER_ID}/provider-versions, e.g. https://registry.terraform.io/v2/providers/323/provider-versions