From e6d96ab6ed5ec316a43705e2885bb0d6f16e50e7 Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Tue, 6 May 2025 12:50:31 -0700 Subject: [PATCH 01/13] add build a provider page --- .../extending-pulumi/build-a-provider.md | 756 ++++++++++++++++++ 1 file changed, 756 insertions(+) create mode 100644 content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md new file mode 100644 index 000000000000..f957a0efaae0 --- /dev/null +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -0,0 +1,756 @@ +--- +title_tag: "Build a Provider" +meta_desc: "Learn the process for building a Pulumi Provider that can be packaged and published in the Pulumi Registry." +title: Build a Provider +h1: Build a Provider +meta_image: /images/docs/meta-images/docs-meta.png +menu: + iac: + name: Build a Provider + parent: iac-extending-pulumi + weight: 4 +--- + +## When to use a provider + +A Pulumi Provider allows you to define new resource types, enabling integration with virtually any service or tool. Pulumi providers are ideal when you need to manage resources that are not yet supported by existing Pulumi providers or when you require custom behavior for managing external systems or APIs. Providers are a powerful extension point, but before building a full provider consider if your use case can be covered by [building a component](/docs/iac/using-pulumi/extending-pulumi/build-a-component/) or using [dynamic provider functions](/docs/iac/using-pulumi/extending-pulumi/build-a-dynamic-provider/). + +## What’s needed to implement a provider? + +### The provider interface + +The Pulumi [Provider](https://pkg.go.dev/github.com/pulumi/pulumi-go-provider#Provider) interface implments the following core methods which form the foundation of Pulumi's resource lifecycle management: + +- **Create** – provisions a new resource +- **Read** – fetches the resource +- **Update** – updates an existing resource +- **Delete** – removes a resource +- **Diff** – computes the differences between the current resource and desired updates to the resource +- **Check** – validates the input parameters +- **Configure** – initializes provider-wide settings (e.g. credentials) + +### How a provider runs + +Pulumi providers are [gRPC](https://grpc.io/) servers that respond to commands from the Pulumi engine. When a Pulumi program runs (e.g. `pulumi up`), the engine connects to the provider process and sends instructions to create, update, or delete resources via calls to the provider interface implementation. + +### Configuration, secrets, outputs, and state + +Beyond the core functions involved with managing resources, there are a number of other aspects to a Pulumi provider. It is necessary to configure the provider, pass secrets, return output values, and store the state of resources. The Pulumi provider interface has built-in facilities for all of those concerns: + +- **Configuration and Secrets**: Set via [Pulumi ESC](/docs/esc/) [environments](/docs/esc/environments/) and/or `pulumi config`. Encrypted secrets and configuration values are passed to the provider at runtime. +- **Outputs**: Providers return outputs from resources, which can be referenced by other resources. +- **State**: Pulumi maintains resource state to track dependencies and detect changes. + +### Handling errors and failures + +Providers should report meaningful error messages. It’s important to handle transient failures and make operations [idempotent](https://en.wikipedia.org/wiki/Idempotence) to avoid inconsistent states. + +### The provider schema + +A provider schema defines the resources, their input and output properties, data sources, and configuration options. This schema enables Pulumi to generate SDKs for multiple languages and ensures consistency across them, as well as providing documentation. + +{{% notes type="info" %}} + +Historically it was necessary to hand-author and maintain the `schema.json` file that accompanied your provider implementation, however, now most of this is handled on-the-fly by the [Pulumi Provider SDK](/docs/iac/using-pulumi/extending-pulumi/pulumi-provider-sdk/) and the file is no longer necessary. + +{{% /notes %}} + +## Language support and the Pulumi Provider SDK + +Pulumi providers can be written in Go, TypeScript, .NET, and Java, and used in any Pulumi program, in any supported language. The [Pulumi Provider SDK](/docs/iac/using-pulumi/extending-pulumi/pulumi-provider-sdk/) is a framework for building providers in Go. We strongly recommend using the SDK, as it is the most full-featured and streamlined way to create a new provider. + +Some advantages of using the Pulumi Provider SDK: + +- **Minimal Code Required**: You define your resource types and implementation using Go structs and methods, and the SDK handles the rest (RPC, auto-generated schema for multi-language support, etc). +- **Includes a Testing Framework**: Testing custom providers is made much easier with the SDK's built-in testing framework. +- **Middleware Support**: Enhances providers with layers like token dispatch, schema generation, and cancellation propagation. + +## Example: Build a custom `file` provider + +Let's walk through the implementation of an example provider using the Pulumi Provider SDK. The [`file` provider](https://github.com/pulumi/pulumi-go-provider/blob/main/examples/file/main.go) will demonstrate how to manage local files as resources within Pulumi. It is a minimal but powerful illustration of the provider development process. + +### Features of the `file` provider + +The `file` provider can be used to create and modify a local file. Here's an example of how to use it in a Pulumi program: + +```yaml +resources: + managedFile: + type: file:File + properties: + path: ${pulumi.cwd}/managed.txt + content: | + An important piece of information +``` + +In this example, we create a new resource called `managedFile` of type `file:File`. We can specify the path to write it to, and the contents that should be in it. During an update, Pulumi will use the `file` provider to ensure the file exists and has the specified contents. + +Now let's create the `file` provider that implements this. + +### Set up the project + +Since the provider is a separate code project from your Pulumi programs, the first step is to create a new directory for the provider code. + +```sh +$ mkdir file-provider +$ cd file-provider +``` + +{{< chooser language "go" >}} + +{{% choosable language go %}} + +#### Create `go.mod` file + +The `go.mod` file defined the Go project. It sets up the name of the Go module, the runtime requirements, and the module dependencies. We need the Pulumi Provider SDK (aka `github.com/pulumi/pulumi-go-provider`) as well as the standard Pulumi SDK (aka `github.com/pulumi/pulumi/sdk/v3`). + +```go +module example.com/file-provider + +go 1.24 + +toolchain go1.24.0 + +require ( + github.com/pulumi/pulumi-go-provider v1.0.0 + github.com/pulumi/pulumi/sdk/v3 v3.167.0 +) +``` + +Once that file is created, go ahead and download the dependencies using `go get`: + +```sh +$ go get example.com/file-provider +``` + +{{% /choosable %}} + +{{< /chooser >}} + +#### Create `PulumiPlugin.yaml` file + +The only other file you'll need is the `PulumiPlugin.yaml` file. This tells Pulumi two things: that this code can be loaded as a provider plugin, and what runtime environment to use for that. + +{{< chooser language "go" >}} + +{{% choosable language go %}} + +```yaml +runtime: go +``` + +{{% /choosable %}} + +{{< /chooser >}} + +### Implement the interface + +{{< chooser language "go" >}} + +{{% choosable language go %}} + +To implment the provider, first create a file called `main.go` with the following contents: + +```go +package main + +import ( + "context" + "fmt" + "os" + + p "github.com/pulumi/pulumi-go-provider" + "github.com/pulumi/pulumi-go-provider/infer" + "github.com/pulumi/pulumi/sdk/v3/go/common/tokens" + "github.com/pulumi/pulumi/sdk/v3/go/property" +) + +func main() { + err := p.RunProvider("file", "0.1.0", provider()) + if err != nil { + fmt.Fprintf(os.Stderr, "Error: %s", err.Error()) + os.Exit(1) + } +} + +func provider() p.Provider { + return infer.Provider(infer.Options{ + Resources: []infer.InferredResource{infer.Resource[*File]()}, + ModuleMap: map[tokens.ModuleName]tokens.ModuleName{ + "file": "index", + }, + }) +} + +type File struct{} + +var _ = (infer.CustomDelete[FileState])((*File)(nil)) +var _ = (infer.CustomCheck[FileArgs])((*File)(nil)) +var _ = (infer.CustomUpdate[FileArgs, FileState])((*File)(nil)) +var _ = (infer.CustomDiff[FileArgs, FileState])((*File)(nil)) +var _ = (infer.CustomRead[FileArgs, FileState])((*File)(nil)) +var _ = (infer.ExplicitDependencies[FileArgs, FileState])((*File)(nil)) +var _ = (infer.Annotated)((*File)(nil)) +var _ = (infer.Annotated)((*FileArgs)(nil)) +var _ = (infer.Annotated)((*FileState)(nil)) + +func (f *File) Annotate(a infer.Annotator) { + a.Describe(&f, "A file projected into a pulumi resource") +} + +type FileArgs struct { + Path string `pulumi:"path,optional"` + Force bool `pulumi:"force,optional"` + Content string `pulumi:"content"` +} + +func (f *FileArgs) Annotate(a infer.Annotator) { + a.Describe(&f.Content, "The content of the file.") + a.Describe(&f.Force, "If an already existing file should be deleted if it exists.") + a.Describe(&f.Path, "The path of the file. This defaults to the name of the pulumi resource.") +} + +type FileState struct { + Path string `pulumi:"path"` + Force bool `pulumi:"force"` + Content string `pulumi:"content"` +} + +func (f *FileState) Annotate(a infer.Annotator) { + a.Describe(&f.Content, "The content of the file.") + a.Describe(&f.Force, "If an already existing file should be deleted if it exists.") + a.Describe(&f.Path, "The path of the file.") +} + +func (*File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (resp infer.CreateResponse[FileState], err error) { + if !req.Inputs.Force { + _, err := os.Stat(req.Inputs.Path) + if !os.IsNotExist(err) { + return resp, fmt.Errorf("file already exists; pass force=true to override") + } + } + + if req.Preview { // Don't do the actual creating if in preview + return infer.CreateResponse[FileState]{ID: req.Inputs.Path}, nil + } + + f, err := os.Create(req.Inputs.Path) + if err != nil { + return resp, err + } + defer f.Close() + n, err := f.WriteString(req.Inputs.Content) + if err != nil { + return resp, err + } + if n != len(req.Inputs.Content) { + return resp, fmt.Errorf("only wrote %d/%d bytes", n, len(req.Inputs.Content)) + } + return infer.CreateResponse[FileState]{ + ID: req.Inputs.Path, + Output: FileState{ + Path: req.Inputs.Path, + Force: req.Inputs.Force, + Content: req.Inputs.Content, + }, + }, nil +} + +func (*File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (infer.DeleteResponse, error) { + err := os.Remove(req.State.Path) + if os.IsNotExist(err) { + p.GetLogger(ctx).Warningf("file %q already deleted", req.State.Path) + err = nil + } + return infer.DeleteResponse{}, err +} + +func (*File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResponse[FileArgs], error) { + if _, ok := req.NewInputs.GetOk("path"); !ok { + req.NewInputs = req.NewInputs.Set("path", property.New(req.Name)) + } + args, f, err := infer.DefaultCheck[FileArgs](ctx, req.NewInputs) + + return infer.CheckResponse[FileArgs]{ + Inputs: args, + Failures: f, + }, err +} + +func (*File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileState]) (infer.UpdateResponse[FileState], error) { + if !req.Preview && req.State.Content != req.Inputs.Content { + f, err := os.Create(req.State.Path) + if err != nil { + return infer.UpdateResponse[FileState]{}, err + } + defer f.Close() + n, err := f.WriteString(req.Inputs.Content) + if err != nil { + return infer.UpdateResponse[FileState]{}, err + } + if n != len(req.Inputs.Content) { + return infer.UpdateResponse[FileState]{}, fmt.Errorf("only wrote %d/%d bytes", n, len(req.Inputs.Content)) + } + } + + return infer.UpdateResponse[FileState]{ + Output: FileState{ + Path: req.Inputs.Path, + Force: req.Inputs.Force, + Content: req.Inputs.Content, + }, + }, nil + +} + +func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState]) (infer.DiffResponse, error) { + diff := map[string]p.PropertyDiff{} + if req.Inputs.Content != req.State.Content { + diff["content"] = p.PropertyDiff{Kind: p.Update} + } + if req.Inputs.Force != req.State.Force { + diff["force"] = p.PropertyDiff{Kind: p.Update} + } + if req.Inputs.Path != req.State.Path { + diff["path"] = p.PropertyDiff{Kind: p.UpdateReplace} + } + return infer.DiffResponse{ + DeleteBeforeReplace: true, + HasChanges: len(diff) > 0, + DetailedDiff: diff, + }, nil +} + +func (*File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState]) (infer.ReadResponse[FileArgs, FileState], error) { + path := req.ID + byteContent, err := os.ReadFile(path) + if err != nil { + return infer.ReadResponse[FileArgs, FileState]{}, err + } + content := string(byteContent) + return infer.ReadResponse[FileArgs, FileState]{ + ID: path, + Inputs: FileArgs{ + Path: path, + Force: req.Inputs.Force && req.State.Force, + Content: content, + }, + State: FileState{ + Path: path, + Force: req.Inputs.Force && req.State.Force, + Content: content, + }, + }, nil +} + +func (*File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *FileState) { + f.OutputField(&state.Content).DependsOn(f.InputField(&args.Content)) + f.OutputField(&state.Force).DependsOn(f.InputField(&args.Force)) + f.OutputField(&state.Path).DependsOn(f.InputField(&args.Path)) +} +``` + +We'll go through this code in detail in a bit, but for now, let's give it a try in a Pulumi program. + +{{% /choosable %}} + +{{< /chooser >}} + +### Use the provider in a Pulumi program + +Let's create a minimal Pulumi program that uses our new provider: + +```sh +$ cd .. +$ mkdir use-file-provider +$ cd use-file-provider +$ pulumi new yaml +``` + +This will initiatize a minimal YAML program. + +Edit the `Pulumi.yaml` file to have the following contents: + +```yaml +name: use-file-provider +runtime: yaml + +plugins: + providers: + - name: file + path: ../file-provider + +resources: + managedFile: + type: file:File + properties: + path: ${pulumi.cwd}/managed.txt + content: | + An important piece of information +``` + +Save that and then run `pulumi up`: + +```sh +$ pulumi up +... + + Type Name Status + + pulumi:pulumi:Stack use-file-provider-dev created (2s) + + └─ file:index:File managedFile created (0.06s) + +Resources: + + 2 created +``` + +If all went well, you should see a new file created called `managed.txt` with the contents `An important piece of information`. + +You can verify this using `cat`: + +```sh +$ cat managed.txt +An important piece of information +``` + +### Detailed breakdown of provider implementation + +{{< chooser language "go" >}} + +{{% choosable language go %}} + +#### Preample and dependencies + +Like any other Go langauge module, you start with a `package` declaration and and `import` block. Here we are adding a few important packages from the base library (`context`, `fmt`, and `os`) which will help us with file operations, and a selection of imports from the Pulumi Provider SDK. + +```go +package main + +import ( + "context" + "fmt" + "os" + + p "github.com/pulumi/pulumi-go-provider" + "github.com/pulumi/pulumi-go-provider/infer" + "github.com/pulumi/pulumi/sdk/v3/go/common/tokens" + "github.com/pulumi/pulumi/sdk/v3/go/property" +) +``` + +#### Provider entrypoint and indentity definitions + +The next section involves defining the `main()` entrypoint function and the `provider()` constructor. + +The entry point uses `p.RunProvider(...)` to launch the provider process. It takes as arguments the name of the provider, version, and the provider instance itself. The provider instance is returned from the `provider()` constructor. Note that the name and version here will be what your end-user sees in the Pulumi program and should be unique to avoid confusion. + +The constructor uses `infer.Provider(...)` with an options struct that lists the types of the new Pulumi resources that this provider will export. The `ModuleMap` defines the namespace for these resource types. Note that the `index` portion of this definition isn't used when the user declares a resources. In this example, we're exporting only one resource type: `file:File`. + +```go +func main() { + err := p.RunProvider("file", "0.1.0", provider()) + if err != nil { + fmt.Fprintf(os.Stderr, "Error: %s", err.Error()) + os.Exit(1) + } +} + +func provider() p.Provider { + return infer.Provider(infer.Options{ + Resources: []infer.InferredResource{infer.Resource[*File]()}, + ModuleMap: map[tokens.ModuleName]tokens.ModuleName{ + "file": "index", + }, + }) +} +``` + +#### Define the `File` resource and implement the provider interface + +Next, lets define the `File` resource. Start with a simple empty struct to define the type. Then add the `infer` declarations that will allow the provider SDK to work with the resource. + +Here we are letting the SDK know that we have implemented `Check`, `Read`, `Update`, `Delete`, and `Diff`. The `ExplicitDependencies` call lets the SDK know we have defined our own `WireDependencies`. Finally, we add a description to the resource using `Annotate`. + +```go +type File struct{} + +var _ = (infer.CustomDelete[FileState])((*File)(nil)) +var _ = (infer.CustomCheck[FileArgs])((*File)(nil)) +var _ = (infer.CustomUpdate[FileArgs, FileState])((*File)(nil)) +var _ = (infer.CustomDiff[FileArgs, FileState])((*File)(nil)) +var _ = (infer.CustomRead[FileArgs, FileState])((*File)(nil)) +var _ = (infer.ExplicitDependencies[FileArgs, FileState])((*File)(nil)) +var _ = (infer.Annotated)((*File)(nil)) +var _ = (infer.Annotated)((*FileArgs)(nil)) +var _ = (infer.Annotated)((*FileState)(nil)) + +func (f *File) Annotate(a infer.Annotator) { + a.Describe(&f, "A file projected into a pulumi resource") +} +``` + +#### Define the resource arguments + +Every resource needs to define the arguments used to configure it. These are done in a separate arguments type. Here we define the file path, contents, and if an existing file should be overwritten or not. + +In the struct we use tags prefixed with `pulumi:` to define the language-neutral argument names. This is an important part of the Pulumi developer experience for the users of your provider. We advise using [camelCase](https://en.wikipedia.org/wiki/Camel_case) naming to provide a consistent experience across all of Pulumi's authoring languages. This is also where you can indicate if an argument is optional. + +After defining the arguments, describe them using the `Annotator` from the `infer` library. This will provide context-sensitive information to developers when authoring in Pulumi with your provider. + +```go +type FileArgs struct { + Path string `pulumi:"path,optional"` + Force bool `pulumi:"force,optional"` + Content string `pulumi:"content"` +} + +func (f *FileArgs) Annotate(a infer.Annotator) { + a.Describe(&f.Content, "The content of the file.") + a.Describe(&f.Force, "If an already existing file should be deleted if it exists.") + a.Describe(&f.Path, "The path of the file. This defaults to the name of the pulumi resource.") +} +``` + +#### Define the resource state + +A resource needs to declare and manage its own state within Pulumi, so that Pulumi knows when to perform the create or update operations. Here we define a `FileState` type that indicates the necessary fields to manage. In this case, the state is very similar to the creation arguments, but this may not always be the case. + +As before, the tags in the struct specify the language-neutral property name to store, and we provide descriptions via annotations. + +```go +type FileState struct { + Path string `pulumi:"path"` + Force bool `pulumi:"force"` + Content string `pulumi:"content"` +} + +func (f *FileState) Annotate(a infer.Annotator) { + a.Describe(&f.Content, "The content of the file.") + a.Describe(&f.Force, "If an already existing file should be deleted if it exists.") + a.Describe(&f.Path, "The path of the file.") +} +``` + +#### Implement the resource CRUD operations + +Here's where the business logic of the resource operations happens. In this example, we are going to implement the full interface, with custom implementations for `Create`, `Read`, `Update`, `Delete`, `Check`, and `Diff`. However, the Pulumi Provider SDK provides default implementations for all of these functions other than `Create`, so in many cases, you may only need to implement one or two of these functions to meet your business goals. + +##### The `Create` operation + +The `Create` operation handles the logic of determining if the resource exists or not, and if not, it creates it using the provided argument context. In many providers this is where you would interact with external cloud APIs, databases, and other systems. In this example, we're going to interact with our local filesystem using calls to Go's `os` library. + +First, we check to see if the user configured the `force` option. We can access that through the `req.Inputs` collection, which is will be an instance of `FileArgs`. If `force` is true, we don't need to check to see if the file exists, otherwise, use `os.Stat` and `os.IsNotExist` to see if the specified directory and filename already exist. If it does, we exist early with an error. This will let Pulumi know that the create operation failed, and it will be propogated up to the end user via the console/log output. + +To return an error, we return a null `CreateResponse` instance with an error string. The Provider SDK functions use a request and response pattern for each operation, parameterized by the argument and state types. The next piece of logic checks `req.Preview` to see if we are in a *preview* mode or an *update* mode. If we are in preview mode, don't take any actions that would mutate the state and just return early. + +Now we can implement the core logic of writing to the filesystem using the base library functions. + +Finally, the last thing to do is construct the response object for the Create operation, setting the unique ID for the resource, and the new resource state values as outputs. Returning that response object without errors lets the Pulumi engine know that this operation was successful. Recording the state will be handled by the Pulumi engine. + +```go +func (*File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (resp infer.CreateResponse[FileState], err error) { + if !req.Inputs.Force { + _, err := os.Stat(req.Inputs.Path) + if !os.IsNotExist(err) { + return resp, fmt.Errorf("file already exists; pass force=true to override") + } + } + + if req.Preview { // Don't do the actual creating if in preview + return infer.CreateResponse[FileState]{ID: req.Inputs.Path}, nil + } + + f, err := os.Create(req.Inputs.Path) + if err != nil { + return resp, err + } + defer f.Close() + n, err := f.WriteString(req.Inputs.Content) + if err != nil { + return resp, err + } + if n != len(req.Inputs.Content) { + return resp, fmt.Errorf("only wrote %d/%d bytes", n, len(req.Inputs.Content)) + } + return infer.CreateResponse[FileState]{ + ID: req.Inputs.Path, + Output: FileState{ + Path: req.Inputs.Path, + Force: req.Inputs.Force, + Content: req.Inputs.Content, + }, + }, nil +} +``` + +##### The `Delete` operation + +The `Delete` operation removes a resource safely. This operation follows a similar pattern to `Create`, but instead of a `CreateRequest`/`CreateResponse` we have `DeleteRequest`/`DeleteResponse`. + +One new concept inroduced in this example function is the use of the Pulumi logger. Calling `p.GetLogger(ctx)` gets you a logger with a familiar interface. This is how you might pass informational messages and warnings to the user without throwing an error. + +```go +func (*File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (infer.DeleteResponse, error) { + err := os.Remove(req.State.Path) + if os.IsNotExist(err) { + p.GetLogger(ctx).Warningf("file %q already deleted", req.State.Path) + err = nil + } + return infer.DeleteResponse{}, err +} +``` + +##### The `Check` operation + +The `Check` operation is used to validate the inputs to a resource, including logic for setting sensible defaults or otherwise mutating the inputs before they are passed to the other functions. + +```go +func (*File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResponse[FileArgs], error) { + if _, ok := req.NewInputs.GetOk("path"); !ok { + req.NewInputs = req.NewInputs.Set("path", property.New(req.Name)) + } + args, f, err := infer.DefaultCheck[FileArgs](ctx, req.NewInputs) + + return infer.CheckResponse[FileArgs]{ + Inputs: args, + Failures: f, + }, err +} +``` + +##### The `Update` operation + +The `Update` operation modifies the resource with new values. After checking to see if we are in a preview mode of not, and that the input contents are different than the recorded state of the content, we overwrite the file with the new contents. + +```go +func (*File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileState]) (infer.UpdateResponse[FileState], error) { + if !req.Preview && req.State.Content != req.Inputs.Content { + f, err := os.Create(req.State.Path) + if err != nil { + return infer.UpdateResponse[FileState]{}, err + } + defer f.Close() + n, err := f.WriteString(req.Inputs.Content) + if err != nil { + return infer.UpdateResponse[FileState]{}, err + } + if n != len(req.Inputs.Content) { + return infer.UpdateResponse[FileState]{}, fmt.Errorf("only wrote %d/%d bytes", n, len(req.Inputs.Content)) + } + } + + return infer.UpdateResponse[FileState]{ + Output: FileState{ + Path: req.Inputs.Path, + Force: req.Inputs.Force, + Content: req.Inputs.Content, + }, + }, nil + +} +``` + +##### The `Diff` operation + +The `Diff` operation compares a resource's recorded state (if any) to the current input values for the resource, to determine what kind of changes need to be made. The `diff` object is a map of property names to the kind of diff operation (if any) that a property needs to have made. The logic of this function is straightforward: compare `req.Inputs.` to `req.State.` and if they aren't equal, add to the diff that this property needs to be updated using `p.PropertyDiff{Kind: p.Update}`. Only include the properties that need to change in the `diff` map. Finally return a `DiffResponse` containing the `diff` map, and set a few other options to configure the update behavior. + +```go +func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState]) (infer.DiffResponse, error) { + diff := map[string]p.PropertyDiff{} + if req.Inputs.Content != req.State.Content { + diff["content"] = p.PropertyDiff{Kind: p.Update} + } + if req.Inputs.Force != req.State.Force { + diff["force"] = p.PropertyDiff{Kind: p.Update} + } + if req.Inputs.Path != req.State.Path { + diff["path"] = p.PropertyDiff{Kind: p.UpdateReplace} + } + return infer.DiffResponse{ + DeleteBeforeReplace: true, + HasChanges: len(diff) > 0, + DetailedDiff: diff, + }, nil +} +``` + +#### The `Read` function + +The `Read` operation fetches the resource. The `ReadRequest` has a `ID` property that can be used, in this case, to determine the path to the file, and the base library functions can be used to read the file from disk, populating the `Content` field. + +```go +func (*File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState]) (infer.ReadResponse[FileArgs, FileState], error) { + path := req.ID + byteContent, err := os.ReadFile(path) + if err != nil { + return infer.ReadResponse[FileArgs, FileState]{}, err + } + content := string(byteContent) + return infer.ReadResponse[FileArgs, FileState]{ + ID: path, + Inputs: FileArgs{ + Path: path, + Force: req.Inputs.Force && req.State.Force, + Content: content, + }, + State: FileState{ + Path: path, + Force: req.Inputs.Force && req.State.Force, + Content: content, + }, + }, nil +} +``` + +#### Managing resource output fields + +Finally, `WireDependencies` defines the outputs that are made available on the resource, logically connecting the inputs and stored state values. + +```go +func (*File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *FileState) { + f.OutputField(&state.Content).DependsOn(f.InputField(&args.Content)) + f.OutputField(&state.Force).DependsOn(f.InputField(&args.Force)) + f.OutputField(&state.Path).DependsOn(f.InputField(&args.Path)) +} +``` + +{{% /choosable %}} + +{{< /chooser >}} + + + +### Multi-language support + +In our above example, we created a provider in Go and used it in YAML. This "just works" by default. However, if you would like to suuse your provider from the other Pulumi authoring langauges (e.g. TypeScript, Python, Java, Go, C#) it will be necessary to generate SDKs for each target language. + +That is a very streamlined process with the Pulumi Provider SDK. The following command will generate language SDKs for all supported languages: + +```sh +pulumi package gen-sdk +``` + +See [`pulumi package gen-sdk --help`](/docs/iac/cli/commands/pulumi_package_gen-sdk/) for more options. + +{{% notes type="info" %}} + +Historically, Pulumi providers required a `schema.json` file. This is now generated on the fly by the Pulumi Provider SDK and so you don't need to have this file on disk to use the provider. If you would like to do so, e.g., for debugging purposes, you can use [`pulumi package get-schema `](/docs/iac/cli/commands/pulumi_package_get-schema/). + +{{% /notes %}} + + + +## Packaging and Publishing + +Using a provider from another directory on your local filesystem is the easiest way to develop a new custom provider. However, once you're ready to share with others at your company, or with the world, you'll need to explore how to publish and package your provider for consumption. There are many ways to accomplish this, from hosting either publicly or privately in GitHub and GitLab, using a private registry within Pulumi Cloud, or publishing to the public Pulumi registry. + +See the [Pulumi package authoring guide](/docs/iac/using-pulumi/pulumi-packages/authoring/) for full details. + + From 7786555c1803cb1f75e47bd0404941f015ba98b1 Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Tue, 6 May 2025 15:03:46 -0700 Subject: [PATCH 02/13] edit typos --- .../extending-pulumi/build-a-provider.md | 52 +++++-------------- 1 file changed, 13 insertions(+), 39 deletions(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index f957a0efaae0..fc363a53bf56 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -96,13 +96,11 @@ $ mkdir file-provider $ cd file-provider ``` -{{< chooser language "go" >}} - -{{% choosable language go %}} - #### Create `go.mod` file -The `go.mod` file defined the Go project. It sets up the name of the Go module, the runtime requirements, and the module dependencies. We need the Pulumi Provider SDK (aka `github.com/pulumi/pulumi-go-provider`) as well as the standard Pulumi SDK (aka `github.com/pulumi/pulumi/sdk/v3`). +The `go.mod` file defines the Go project. It sets up the name of the Go module, the runtime requirements, and the module dependencies. We need the Pulumi Provider SDK (aka `github.com/pulumi/pulumi-go-provider`) as well as the standard Pulumi SDK (aka `github.com/pulumi/pulumi/sdk/v3`). + +***Example**: `go.mod`* ```go module example.com/file-provider @@ -117,39 +115,27 @@ require ( ) ``` -Once that file is created, go ahead and download the dependencies using `go get`: +Once that file is created, download the dependencies using `go get`: ```sh $ go get example.com/file-provider ``` -{{% /choosable %}} - -{{< /chooser >}} - #### Create `PulumiPlugin.yaml` file The only other file you'll need is the `PulumiPlugin.yaml` file. This tells Pulumi two things: that this code can be loaded as a provider plugin, and what runtime environment to use for that. -{{< chooser language "go" >}} - -{{% choosable language go %}} +***Example:** `PulumiPlugin.yaml`* ```yaml runtime: go ``` -{{% /choosable %}} - -{{< /chooser >}} - ### Implement the interface -{{< chooser language "go" >}} +To implement the provider, first create a file called `main.go` with the following contents: -{{% choosable language go %}} - -To implment the provider, first create a file called `main.go` with the following contents: +***Example:** `main.go`* ```go package main @@ -350,15 +336,11 @@ func (*File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *File } ``` -We'll go through this code in detail in a bit, but for now, let's give it a try in a Pulumi program. - -{{% /choosable %}} - -{{< /chooser >}} +We'll go through this code in detail in a moment, but for now, let's give it a try in a Pulumi program. ### Use the provider in a Pulumi program -Let's create a minimal Pulumi program that uses our new provider: +First, create a Pulumi program that uses our new provider: ```sh $ cd .. @@ -367,9 +349,9 @@ $ cd use-file-provider $ pulumi new yaml ``` -This will initiatize a minimal YAML program. +This will initiatize a minimal YAML program. Let's modify the default YAML file: -Edit the `Pulumi.yaml` file to have the following contents: +***Example:** The `Pulumi.yaml` file* ```yaml name: use-file-provider @@ -414,10 +396,6 @@ An important piece of information ### Detailed breakdown of provider implementation -{{< chooser language "go" >}} - -{{% choosable language go %}} - #### Preample and dependencies Like any other Go langauge module, you start with a `package` declaration and and `import` block. Here we are adding a few important packages from the base library (`context`, `fmt`, and `os`) which will help us with file operations, and a selection of imports from the Pulumi Provider SDK. @@ -673,7 +651,7 @@ func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState } ``` -#### The `Read` function +#### The `Read` operation The `Read` operation fetches the resource. The `ReadRequest` has a `ID` property that can be used, in this case, to determine the path to the file, and the base library functions can be used to read the file from disk, populating the `Content` field. @@ -713,10 +691,6 @@ func (*File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *File } ``` -{{% /choosable %}} - -{{< /chooser >}} - -## Packaging and Publishing +## Packaging and publishing Using a provider from another directory on your local filesystem is the easiest way to develop a new custom provider. However, once you're ready to share with others at your company, or with the world, you'll need to explore how to publish and package your provider for consumption. There are many ways to accomplish this, from hosting either publicly or privately in GitHub and GitLab, using a private registry within Pulumi Cloud, or publishing to the public Pulumi registry. From 6388159ffd2a6edf61c0639ec71c596bf37fee4c Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Tue, 6 May 2025 17:05:32 -0500 Subject: [PATCH 03/13] Update content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md Co-authored-by: Eron Wright --- .../docs/iac/using-pulumi/extending-pulumi/build-a-provider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index f957a0efaae0..a32bd7f4a83a 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -725,7 +725,7 @@ func (*File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *File ### Multi-language support -In our above example, we created a provider in Go and used it in YAML. This "just works" by default. However, if you would like to suuse your provider from the other Pulumi authoring langauges (e.g. TypeScript, Python, Java, Go, C#) it will be necessary to generate SDKs for each target language. +In our above example, we created a provider in Go and used it in YAML. This "just works" by default. However, if you would like to use your provider from the other Pulumi authoring langauges (e.g. TypeScript, Python, Java, Go, C#) it will be necessary to generate SDKs for each target language. That is a very streamlined process with the Pulumi Provider SDK. The following command will generate language SDKs for all supported languages: From fa7d68aa071539d022788ff41b1e0bd3a1d95a2c Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Tue, 6 May 2025 18:18:41 -0500 Subject: [PATCH 04/13] Update content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md Co-authored-by: Eron Wright --- .../docs/iac/using-pulumi/extending-pulumi/build-a-provider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index 7aa1502e015a..d45dc77fe093 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -19,7 +19,7 @@ A Pulumi Provider allows you to define new resource types, enabling integration ### The provider interface -The Pulumi [Provider](https://pkg.go.dev/github.com/pulumi/pulumi-go-provider#Provider) interface implments the following core methods which form the foundation of Pulumi's resource lifecycle management: +The Pulumi [Provider](https://pkg.go.dev/github.com/pulumi/pulumi-go-provider#Provider) interface implements the following core methods which form the foundation of Pulumi's resource lifecycle management: - **Create** – provisions a new resource - **Read** – fetches the resource From 378dc5398e990938834ae55e3f52f5c835e997e9 Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Fri, 9 May 2025 12:21:47 -0700 Subject: [PATCH 05/13] update code sample --- .../extending-pulumi/build-a-provider.md | 128 ++++++++---------- 1 file changed, 60 insertions(+), 68 deletions(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index 7aa1502e015a..a391d364468d 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -147,39 +147,27 @@ import ( p "github.com/pulumi/pulumi-go-provider" "github.com/pulumi/pulumi-go-provider/infer" - "github.com/pulumi/pulumi/sdk/v3/go/common/tokens" "github.com/pulumi/pulumi/sdk/v3/go/property" ) func main() { - err := p.RunProvider("file", "0.1.0", provider()) + provider, err := infer.NewProviderBuilder(). + WithResources( + infer.Resource[File](), + ). + WithNamespace("example"). + Build() + if err != nil { fmt.Fprintf(os.Stderr, "Error: %s", err.Error()) os.Exit(1) } -} -func provider() p.Provider { - return infer.Provider(infer.Options{ - Resources: []infer.InferredResource{infer.Resource[*File]()}, - ModuleMap: map[tokens.ModuleName]tokens.ModuleName{ - "file": "index", - }, - }) + provider.Run(context.Background(), "file", "0.1.0") } type File struct{} -var _ = (infer.CustomDelete[FileState])((*File)(nil)) -var _ = (infer.CustomCheck[FileArgs])((*File)(nil)) -var _ = (infer.CustomUpdate[FileArgs, FileState])((*File)(nil)) -var _ = (infer.CustomDiff[FileArgs, FileState])((*File)(nil)) -var _ = (infer.CustomRead[FileArgs, FileState])((*File)(nil)) -var _ = (infer.ExplicitDependencies[FileArgs, FileState])((*File)(nil)) -var _ = (infer.Annotated)((*File)(nil)) -var _ = (infer.Annotated)((*FileArgs)(nil)) -var _ = (infer.Annotated)((*FileState)(nil)) - func (f *File) Annotate(a infer.Annotator) { a.Describe(&f, "A file projected into a pulumi resource") } @@ -208,7 +196,7 @@ func (f *FileState) Annotate(a infer.Annotator) { a.Describe(&f.Path, "The path of the file.") } -func (*File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (resp infer.CreateResponse[FileState], err error) { +func (File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (resp infer.CreateResponse[FileState], err error) { if !req.Inputs.Force { _, err := os.Stat(req.Inputs.Path) if !os.IsNotExist(err) { @@ -216,7 +204,7 @@ func (*File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (res } } - if req.Preview { // Don't do the actual creating if in preview + if req.DryRun { // Don't do the actual creating if in preview return infer.CreateResponse[FileState]{ID: req.Inputs.Path}, nil } @@ -242,7 +230,7 @@ func (*File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (res }, nil } -func (*File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (infer.DeleteResponse, error) { +func (File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (infer.DeleteResponse, error) { err := os.Remove(req.State.Path) if os.IsNotExist(err) { p.GetLogger(ctx).Warningf("file %q already deleted", req.State.Path) @@ -251,7 +239,7 @@ func (*File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (in return infer.DeleteResponse{}, err } -func (*File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResponse[FileArgs], error) { +func (File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResponse[FileArgs], error) { if _, ok := req.NewInputs.GetOk("path"); !ok { req.NewInputs = req.NewInputs.Set("path", property.New(req.Name)) } @@ -263,8 +251,13 @@ func (*File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResp }, err } -func (*File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileState]) (infer.UpdateResponse[FileState], error) { - if !req.Preview && req.State.Content != req.Inputs.Content { +func (File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileState]) (infer.UpdateResponse[FileState], error) { + if req.DryRun { // Don't do the update if in preview + return infer.UpdateResponse[FileState]{}, nil + } + + _, err := os.Stat(req.Inputs.Path) + if req.State.Content != req.Inputs.Content || os.IsNotExist(err) { f, err := os.Create(req.State.Path) if err != nil { return infer.UpdateResponse[FileState]{}, err @@ -286,10 +279,9 @@ func (*File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileS Content: req.Inputs.Content, }, }, nil - } -func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState]) (infer.DiffResponse, error) { +func (File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState]) (infer.DiffResponse, error) { diff := map[string]p.PropertyDiff{} if req.Inputs.Content != req.State.Content { diff["content"] = p.PropertyDiff{Kind: p.Update} @@ -299,6 +291,11 @@ func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState } if req.Inputs.Path != req.State.Path { diff["path"] = p.PropertyDiff{Kind: p.UpdateReplace} + } else { + _, err := os.Stat(req.Inputs.Path) + if os.IsNotExist(err) { + diff["path"] = p.PropertyDiff{Kind: p.Add} + } } return infer.DiffResponse{ DeleteBeforeReplace: true, @@ -307,7 +304,7 @@ func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState }, nil } -func (*File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState]) (infer.ReadResponse[FileArgs, FileState], error) { +func (File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState]) (infer.ReadResponse[FileArgs, FileState], error) { path := req.ID byteContent, err := os.ReadFile(path) if err != nil { @@ -329,7 +326,7 @@ func (*File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState }, nil } -func (*File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *FileState) { +func (File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *FileState) { f.OutputField(&state.Content).DependsOn(f.InputField(&args.Content)) f.OutputField(&state.Force).DependsOn(f.InputField(&args.Force)) f.OutputField(&state.Path).DependsOn(f.InputField(&args.Path)) @@ -396,7 +393,7 @@ An important piece of information ### Detailed breakdown of provider implementation -#### Preample and dependencies +#### Preamble and dependencies Like any other Go langauge module, you start with a `package` declaration and and `import` block. Here we are adding a few important packages from the base library (`context`, `fmt`, and `os`) which will help us with file operations, and a selection of imports from the Pulumi Provider SDK. @@ -410,57 +407,43 @@ import ( p "github.com/pulumi/pulumi-go-provider" "github.com/pulumi/pulumi-go-provider/infer" - "github.com/pulumi/pulumi/sdk/v3/go/common/tokens" "github.com/pulumi/pulumi/sdk/v3/go/property" ) ``` #### Provider entrypoint and indentity definitions -The next section involves defining the `main()` entrypoint function and the `provider()` constructor. +The next section involves defining the `main()` entrypoint function and using `infer.NewProviderBuilder()` to construct the provider instance. -The entry point uses `p.RunProvider(...)` to launch the provider process. It takes as arguments the name of the provider, version, and the provider instance itself. The provider instance is returned from the `provider()` constructor. Note that the name and version here will be what your end-user sees in the Pulumi program and should be unique to avoid confusion. +The entry point uses `provider.Run(...)` to launch the provider process. It takes as arguments the name and version of the provider. Note that the name and version here will be what your end-user sees in the Pulumi program and should be unique to avoid confusion. -The constructor uses `infer.Provider(...)` with an options struct that lists the types of the new Pulumi resources that this provider will export. The `ModuleMap` defines the namespace for these resource types. Note that the `index` portion of this definition isn't used when the user declares a resources. In this example, we're exporting only one resource type: `file:File`. +Building the provider instance with `infer.NewProviderBuilder(...)` uses a fluent-programming style, setting various configuration options via a chain of methods starting with the word `.With...`. In this example, we use `.WithResource` to export the `File` resource, and `.WithNamespace` to set the namespace that the generated language-specific SDKs will use. Finally, calling `.Build()` will return the provider instance. ```go func main() { - err := p.RunProvider("file", "0.1.0", provider()) + provider, err := infer.NewProviderBuilder(). + WithResources( + infer.Resource[File](), + ). + WithNamespace("example"). + Build() + if err != nil { fmt.Fprintf(os.Stderr, "Error: %s", err.Error()) os.Exit(1) } -} -func provider() p.Provider { - return infer.Provider(infer.Options{ - Resources: []infer.InferredResource{infer.Resource[*File]()}, - ModuleMap: map[tokens.ModuleName]tokens.ModuleName{ - "file": "index", - }, - }) + provider.Run(context.Background(), "file", "0.1.0") } ``` #### Define the `File` resource and implement the provider interface -Next, lets define the `File` resource. Start with a simple empty struct to define the type. Then add the `infer` declarations that will allow the provider SDK to work with the resource. - -Here we are letting the SDK know that we have implemented `Check`, `Read`, `Update`, `Delete`, and `Diff`. The `ExplicitDependencies` call lets the SDK know we have defined our own `WireDependencies`. Finally, we add a description to the resource using `Annotate`. +Next, lets define the `File` resource. Start with a simple empty struct to define the type. Then add a description to the resource using `Annotate`. ```go type File struct{} -var _ = (infer.CustomDelete[FileState])((*File)(nil)) -var _ = (infer.CustomCheck[FileArgs])((*File)(nil)) -var _ = (infer.CustomUpdate[FileArgs, FileState])((*File)(nil)) -var _ = (infer.CustomDiff[FileArgs, FileState])((*File)(nil)) -var _ = (infer.CustomRead[FileArgs, FileState])((*File)(nil)) -var _ = (infer.ExplicitDependencies[FileArgs, FileState])((*File)(nil)) -var _ = (infer.Annotated)((*File)(nil)) -var _ = (infer.Annotated)((*FileArgs)(nil)) -var _ = (infer.Annotated)((*FileState)(nil)) - func (f *File) Annotate(a infer.Annotator) { a.Describe(&f, "A file projected into a pulumi resource") } @@ -518,14 +501,14 @@ The `Create` operation handles the logic of determining if the resource exists o First, we check to see if the user configured the `force` option. We can access that through the `req.Inputs` collection, which is will be an instance of `FileArgs`. If `force` is true, we don't need to check to see if the file exists, otherwise, use `os.Stat` and `os.IsNotExist` to see if the specified directory and filename already exist. If it does, we exist early with an error. This will let Pulumi know that the create operation failed, and it will be propogated up to the end user via the console/log output. -To return an error, we return a null `CreateResponse` instance with an error string. The Provider SDK functions use a request and response pattern for each operation, parameterized by the argument and state types. The next piece of logic checks `req.Preview` to see if we are in a *preview* mode or an *update* mode. If we are in preview mode, don't take any actions that would mutate the state and just return early. +To return an error, we return a null `CreateResponse` instance with an error string. The Provider SDK functions use a request and response pattern for each operation, parameterized by the argument and state types. The next piece of logic checks `req.DryRun` to see if we are in a *preview* mode or an *update* mode. If we are in preview mode, don't take any actions that would mutate the state and just return early. Now we can implement the core logic of writing to the filesystem using the base library functions. Finally, the last thing to do is construct the response object for the Create operation, setting the unique ID for the resource, and the new resource state values as outputs. Returning that response object without errors lets the Pulumi engine know that this operation was successful. Recording the state will be handled by the Pulumi engine. ```go -func (*File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (resp infer.CreateResponse[FileState], err error) { +func (File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (resp infer.CreateResponse[FileState], err error) { if !req.Inputs.Force { _, err := os.Stat(req.Inputs.Path) if !os.IsNotExist(err) { @@ -533,7 +516,7 @@ func (*File) Create(ctx context.Context, req infer.CreateRequest[FileArgs]) (res } } - if req.Preview { // Don't do the actual creating if in preview + if req.DryRun { // Don't do the actual creating if in preview return infer.CreateResponse[FileState]{ID: req.Inputs.Path}, nil } @@ -567,7 +550,7 @@ The `Delete` operation removes a resource safely. This operation follows a simil One new concept inroduced in this example function is the use of the Pulumi logger. Calling `p.GetLogger(ctx)` gets you a logger with a familiar interface. This is how you might pass informational messages and warnings to the user without throwing an error. ```go -func (*File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (infer.DeleteResponse, error) { +func (File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (infer.DeleteResponse, error) { err := os.Remove(req.State.Path) if os.IsNotExist(err) { p.GetLogger(ctx).Warningf("file %q already deleted", req.State.Path) @@ -582,7 +565,7 @@ func (*File) Delete(ctx context.Context, req infer.DeleteRequest[FileState]) (in The `Check` operation is used to validate the inputs to a resource, including logic for setting sensible defaults or otherwise mutating the inputs before they are passed to the other functions. ```go -func (*File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResponse[FileArgs], error) { +func (File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResponse[FileArgs], error) { if _, ok := req.NewInputs.GetOk("path"); !ok { req.NewInputs = req.NewInputs.Set("path", property.New(req.Name)) } @@ -600,8 +583,13 @@ func (*File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckResp The `Update` operation modifies the resource with new values. After checking to see if we are in a preview mode of not, and that the input contents are different than the recorded state of the content, we overwrite the file with the new contents. ```go -func (*File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileState]) (infer.UpdateResponse[FileState], error) { - if !req.Preview && req.State.Content != req.Inputs.Content { +func (File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileState]) (infer.UpdateResponse[FileState], error) { + if req.DryRun { // Don't do the update if in preview + return infer.UpdateResponse[FileState]{}, nil + } + + _, err := os.Stat(req.Inputs.Path) + if req.State.Content != req.Inputs.Content || os.IsNotExist(err) { f, err := os.Create(req.State.Path) if err != nil { return infer.UpdateResponse[FileState]{}, err @@ -623,7 +611,6 @@ func (*File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileS Content: req.Inputs.Content, }, }, nil - } ``` @@ -632,7 +619,7 @@ func (*File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileS The `Diff` operation compares a resource's recorded state (if any) to the current input values for the resource, to determine what kind of changes need to be made. The `diff` object is a map of property names to the kind of diff operation (if any) that a property needs to have made. The logic of this function is straightforward: compare `req.Inputs.` to `req.State.` and if they aren't equal, add to the diff that this property needs to be updated using `p.PropertyDiff{Kind: p.Update}`. Only include the properties that need to change in the `diff` map. Finally return a `DiffResponse` containing the `diff` map, and set a few other options to configure the update behavior. ```go -func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState]) (infer.DiffResponse, error) { +func (File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState]) (infer.DiffResponse, error) { diff := map[string]p.PropertyDiff{} if req.Inputs.Content != req.State.Content { diff["content"] = p.PropertyDiff{Kind: p.Update} @@ -642,6 +629,11 @@ func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState } if req.Inputs.Path != req.State.Path { diff["path"] = p.PropertyDiff{Kind: p.UpdateReplace} + } else { + _, err := os.Stat(req.Inputs.Path) + if os.IsNotExist(err) { + diff["path"] = p.PropertyDiff{Kind: p.Add} + } } return infer.DiffResponse{ DeleteBeforeReplace: true, @@ -656,7 +648,7 @@ func (*File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState The `Read` operation fetches the resource. The `ReadRequest` has a `ID` property that can be used, in this case, to determine the path to the file, and the base library functions can be used to read the file from disk, populating the `Content` field. ```go -func (*File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState]) (infer.ReadResponse[FileArgs, FileState], error) { +func (File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState]) (infer.ReadResponse[FileArgs, FileState], error) { path := req.ID byteContent, err := os.ReadFile(path) if err != nil { @@ -684,7 +676,7 @@ func (*File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState Finally, `WireDependencies` defines the outputs that are made available on the resource, logically connecting the inputs and stored state values. ```go -func (*File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *FileState) { +func (File) WireDependencies(f infer.FieldSelector, args *FileArgs, state *FileState) { f.OutputField(&state.Content).DependsOn(f.InputField(&args.Content)) f.OutputField(&state.Force).DependsOn(f.InputField(&args.Force)) f.OutputField(&state.Path).DependsOn(f.InputField(&args.Path)) From 36b16393ba5b88976afff7f17c2ccfe8b7e4e50d Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Fri, 9 May 2025 12:23:17 -0700 Subject: [PATCH 06/13] change dispatch language --- .../docs/iac/using-pulumi/extending-pulumi/build-a-provider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index a391d364468d..d05d92ce97cd 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -63,7 +63,7 @@ Some advantages of using the Pulumi Provider SDK: - **Minimal Code Required**: You define your resource types and implementation using Go structs and methods, and the SDK handles the rest (RPC, auto-generated schema for multi-language support, etc). - **Includes a Testing Framework**: Testing custom providers is made much easier with the SDK's built-in testing framework. -- **Middleware Support**: Enhances providers with layers like token dispatch, schema generation, and cancellation propagation. +- **Middleware Support**: Enhances providers with layers like dispatch logic, schema generation, and cancellation propagation. ## Example: Build a custom `file` provider From 099ca0780b0b30db815f19862cb1cfe78b53cc2e Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Wed, 14 May 2025 13:11:08 -0500 Subject: [PATCH 07/13] Update content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md Co-authored-by: Eron Wright --- .../docs/iac/using-pulumi/extending-pulumi/build-a-provider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index 902311aaa177..8cc96bfe195d 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -411,7 +411,7 @@ import ( ) ``` -#### Provider entrypoint and indentity definitions +#### Provider entrypoint and identity definitions The next section involves defining the `main()` entrypoint function and using `infer.NewProviderBuilder()` to construct the provider instance. From b8b6160f3984d51ca2210fd4e3a450dc4974852f Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Wed, 14 May 2025 13:11:19 -0500 Subject: [PATCH 08/13] Update content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md Co-authored-by: Eron Wright --- .../docs/iac/using-pulumi/extending-pulumi/build-a-provider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index 8cc96bfe195d..e3900ce57ca8 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -51,7 +51,7 @@ A provider schema defines the resources, their input and output properties, data {{% notes type="info" %}} -Historically it was necessary to hand-author and maintain the `schema.json` file that accompanied your provider implementation, however, now most of this is handled on-the-fly by the [Pulumi Provider SDK](/docs/iac/using-pulumi/extending-pulumi/pulumi-provider-sdk/) and the file is no longer necessary. +Historically it was necessary to hand-author and maintain the `schema.json` file that accompanied your provider implementation, however, now most of this is generated automatically by the [Pulumi Provider SDK](/docs/iac/using-pulumi/extending-pulumi/pulumi-provider-sdk/) and the file is no longer necessary. {{% /notes %}} From 12a913c13fd214a5d547cc6583180bb4f3b3f478 Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Wed, 14 May 2025 13:11:35 -0500 Subject: [PATCH 09/13] Update content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md Co-authored-by: Eron Wright --- .../docs/iac/using-pulumi/extending-pulumi/build-a-provider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index e3900ce57ca8..4040f6822ae9 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -645,7 +645,7 @@ func (File) Diff(ctx context.Context, req infer.DiffRequest[FileArgs, FileState] #### The `Read` operation -The `Read` operation fetches the resource. The `ReadRequest` has a `ID` property that can be used, in this case, to determine the path to the file, and the base library functions can be used to read the file from disk, populating the `Content` field. +The `Read` operation fetches the resource, e.g. to refresh the live state. The `ReadRequest` has a `ID` property that can be used, in this case, to determine the path to the file, and the base library functions can be used to read the file from disk, populating the `Content` field. ```go func (File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState]) (infer.ReadResponse[FileArgs, FileState], error) { From aa694f584ed129bdef49b8c379c3819c87520b7c Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Wed, 14 May 2025 11:27:37 -0700 Subject: [PATCH 10/13] update per review --- .../extending-pulumi/build-a-provider.md | 35 +++++++++---------- 1 file changed, 16 insertions(+), 19 deletions(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index 4040f6822ae9..3c41768c8b67 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -256,20 +256,17 @@ func (File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileSt return infer.UpdateResponse[FileState]{}, nil } - _, err := os.Stat(req.Inputs.Path) - if req.State.Content != req.Inputs.Content || os.IsNotExist(err) { - f, err := os.Create(req.State.Path) - if err != nil { - return infer.UpdateResponse[FileState]{}, err - } - defer f.Close() - n, err := f.WriteString(req.Inputs.Content) - if err != nil { - return infer.UpdateResponse[FileState]{}, err - } - if n != len(req.Inputs.Content) { - return infer.UpdateResponse[FileState]{}, fmt.Errorf("only wrote %d/%d bytes", n, len(req.Inputs.Content)) - } + f, err := os.Create(req.State.Path) + if err != nil { + return infer.UpdateResponse[FileState]{}, err + } + defer f.Close() + n, err := f.WriteString(req.Inputs.Content) + if err != nil { + return infer.UpdateResponse[FileState]{}, err + } + if n != len(req.Inputs.Content) { + return infer.UpdateResponse[FileState]{}, fmt.Errorf("only wrote %d/%d bytes", n, len(req.Inputs.Content)) } return infer.UpdateResponse[FileState]{ @@ -315,12 +312,12 @@ func (File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState] ID: path, Inputs: FileArgs{ Path: path, - Force: req.Inputs.Force && req.State.Force, + Force: req.State.Force, Content: content, }, State: FileState{ Path: path, - Force: req.Inputs.Force && req.State.Force, + Force: req.State.Force, Content: content, }, }, nil @@ -580,7 +577,7 @@ func (File) Check(ctx context.Context, req infer.CheckRequest) (infer.CheckRespo ##### The `Update` operation -The `Update` operation modifies the resource with new values. After checking to see if we are in a preview mode of not, and that the input contents are different than the recorded state of the content, we overwrite the file with the new contents. +The `Update` operation modifies the resource with new values. After checking to see if we are in a preview mode of not, we overwrite the file with the new contents. Note that it's not necessary to check if the input contents are different than current state of the content, as this logic is handled by the `Diff` operation. ```go func (File) Update(ctx context.Context, req infer.UpdateRequest[FileArgs, FileState]) (infer.UpdateResponse[FileState], error) { @@ -659,12 +656,12 @@ func (File) Read(ctx context.Context, req infer.ReadRequest[FileArgs, FileState] ID: path, Inputs: FileArgs{ Path: path, - Force: req.Inputs.Force && req.State.Force, + Force: req.State.Force, Content: content, }, State: FileState{ Path: path, - Force: req.Inputs.Force && req.State.Force, + Force: req.State.Force, Content: content, }, }, nil From 5bb3369a2e090fe64e660534cc87018369269275 Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Wed, 14 May 2025 11:37:23 -0700 Subject: [PATCH 11/13] add run error handling and notes about WithNamespace --- .../extending-pulumi/build-a-provider.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index 3c41768c8b67..d4382d124b5a 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -163,7 +163,12 @@ func main() { os.Exit(1) } - provider.Run(context.Background(), "file", "0.1.0") + err := provider.Run(context.Background(), "file", "0.1.0") + + if err != nil { + fmt.Fprintf(os.Stderr, "Error: %s", err.Error()) + os.Exit(1) + } } type File struct{} @@ -414,7 +419,9 @@ The next section involves defining the `main()` entrypoint function and using `i The entry point uses `provider.Run(...)` to launch the provider process. It takes as arguments the name and version of the provider. Note that the name and version here will be what your end-user sees in the Pulumi program and should be unique to avoid confusion. -Building the provider instance with `infer.NewProviderBuilder(...)` uses a fluent-programming style, setting various configuration options via a chain of methods starting with the word `.With...`. In this example, we use `.WithResource` to export the `File` resource, and `.WithNamespace` to set the namespace that the generated language-specific SDKs will use. Finally, calling `.Build()` will return the provider instance. +Building the provider instance with `infer.NewProviderBuilder(...)` uses a fluent-programming style, setting various configuration options via a chain of methods starting with the word `.With...`. In this example, we use `.WithResource` to export the `File` resource, and `.WithNamespace` to set the namespace that the generated language-specific SDKs will use. The namespace is used as a grouping of your organization's packages. We suggest using something like your GitHub organization name. If not provided, default value will be `pulumi`, which is intended for our first-party packages. + +Finally, calling `.Build()` will return the provider instance. ```go func main() { @@ -430,7 +437,12 @@ func main() { os.Exit(1) } - provider.Run(context.Background(), "file", "0.1.0") + err := provider.Run(context.Background(), "file", "0.1.0") + + if err != nil { + fmt.Fprintf(os.Stderr, "Error: %s", err.Error()) + os.Exit(1) + } } ``` From 1d2602fd6d3e223ccd0fe6fc11b1ffda60caa889 Mon Sep 17 00:00:00 2001 From: Troy Howard Date: Wed, 14 May 2025 12:09:13 -0700 Subject: [PATCH 12/13] update language re: schema --- .../docs/iac/using-pulumi/extending-pulumi/build-a-provider.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index d4382d124b5a..e21d3d0d4870 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -47,7 +47,7 @@ Providers should report meaningful error messages. It’s important to handle tr ### The provider schema -A provider schema defines the resources, their input and output properties, data sources, and configuration options. This schema enables Pulumi to generate SDKs for multiple languages and ensures consistency across them, as well as providing documentation. +A provider's [package schema](/docs/iac/using-pulumi/extending-pulumi/schema/) defines the resources, their input and output properties, descriptions, and configuration options. This schema enables Pulumi to generate SDKs for multiple languages and ensures consistency across them, as well as providing documentation. {{% notes type="info" %}} From ae0c5cda0b1af3296eaaab05781acd19df58e737 Mon Sep 17 00:00:00 2001 From: Bryce Lampe Date: Wed, 14 May 2025 13:02:12 -0700 Subject: [PATCH 13/13] update for new infer.Resource syntax --- .../iac/using-pulumi/extending-pulumi/build-a-provider.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md index e21d3d0d4870..ab81a7da69d2 100644 --- a/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md +++ b/content/docs/iac/using-pulumi/extending-pulumi/build-a-provider.md @@ -153,7 +153,7 @@ import ( func main() { provider, err := infer.NewProviderBuilder(). WithResources( - infer.Resource[File](), + infer.Resource(File{}), ). WithNamespace("example"). Build() @@ -427,7 +427,7 @@ Finally, calling `.Build()` will return the provider instance. func main() { provider, err := infer.NewProviderBuilder(). WithResources( - infer.Resource[File](), + infer.Resource(File{}), ). WithNamespace("example"). Build()