Update Lambda and GCF docs for runtime inference (#21266)

huonw · web-flow · commit fa583d6b36df · 2024-08-08T08:44:14.000+10:00
This updates the Lambda and GCF docs (very belatedly) for the "new" behaviours related to choosing platform specific code, like inference of the `runtime` field (#19314), and associated complete platforms (#19253). This includes stripping out the `runtime` field from the "main" flow, and adding an separate subsection for how to specify the runtime explicitly.
diff --git a/docs/docs/python/integrations/aws-lambda.mdx b/docs/docs/python/integrations/aws-lambda.mdx
@@ -33,7 +33,7 @@ First, add your lambda function in a Python file like you would [normally do wit
 
 Then, in your BUILD file, make sure that you have a `python_source` or `python_sources` target with the handler file included in the `sources` field. You can use [`pants tailor ::`](../../getting-started/initial-configuration.mdx#5-generate-build-files) to automate this.
 
-Add a `python_aws_lambda_function` target and define the `runtime` and `handler` fields. The `runtime` should be one of the values from [https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html](https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html). The `handler` has the form `handler_file.py:handler_func`, which Pants will convert into a well-formed entry point. Alternatively, you can set `handler` to the format `path.to.module:handler_func`.
+Add a `python_aws_lambda_function` target and define the `handler` fields. The `handler` has the form `handler_file.py:handler_func`, which Pants will convert into a well-formed entry point. Alternatively, you can set `handler` to the format `path.to.module:handler_func`.
 
 For example:
 
@@ -43,7 +43,6 @@ python_sources(name="lib")
 
 python_aws_lambda_function(
     name="lambda",
-    runtime="python3.8",
     # Pants will convert this to `project.lambda_example:example_handler`.
     handler="lambda_example.py:example_handler",
 )
@@ -66,6 +65,49 @@ Use [layout](../../../reference/targets/python_aws_lambda_function.mdx#layout) t
 `file` / `files` targets will not be included in the built AWS Lambda artifacts because filesystem APIs like `open()` would not load them as expected. Instead, use the `resource` and `resources` target. See [Assets and archives](../../using-pants/assets-and-archives.mdx) for further explanation.
 :::
 
+### Specifying a runtime explicitly
+
+When building an Lambda artifact, Pants and the underlying Pex tool need to know details about target runtime to be able to choose appropriate artifacts for third-party dependencies that have native code. These details can be inferred or provided in three ways, from highest precedence to lowest precedence:
+
+1. An explicit value for [the `complete_platforms` field](../../../reference/targets/python_aws_lambda_function.mdx#complete_platforms). The "complete platforms" are the underlying source of truth.
+    ```python title="BUILD"
+    file(name="lambda-platform", source="lambda-platform.json")
+
+    python_aws_lambda_function(
+        name="lambda",
+        handler="lambda_example.py:example_handler",
+        # Explicit complete platforms:
+        complete_platforms=[":lambda-platform"],
+    )
+    ```
+2. An explicit value for [the `runtime` field](../../../reference/targets/python_aws_lambda_function.mdx#runtime) and, optionally, [the `architecture` field](../../../reference/targets/python_aws_lambda_function.mdx#architecture): Pants uses these to pick an appropriate "complete platforms" value, from options that Pants has pre-packaged. These are static exports from docker images provided by AWS, relying on the environment being relatively stable. (If Pants doesn't have an appropriate "complete platforms" default built-in, you will be prompted to use option 1 above.)
+    ```python title="BUILD"
+    python_aws_lambda_function(
+        name="lambda",
+        handler="lambda_example.py:example_handler",
+        # Explicit runtime, `complete_platforms` taken from Pants' built-in defaults:
+        runtime="python3.12",
+        # Override the default x86_64 architecture:
+        architecture="arm64",
+    )
+    ```
+3. Inferred from [the relevant interpreter constraints](../overview/interpreter-compatibility.mdx): the interpreter constraints may unambiguously imply a value for the `runtime` and thus `complete_platforms` fields. For example, `interpreter_constaints = ["==3.12.*"]` implies `runtime="python3.12"`. This only works with interpreter constraints that cover all patch versions of a given minor release series: `>=3.11,<3.13` is too wide (it covers both 3.11 and 3.12), while `==3.12.0` is too specific (AWS's `python3.12` runtime may not use that exact patch version). As with option 2, the architecture is `x86_64` by default, but can changed using the `architecture` field.
+
+    ```toml tab={"label":"pants.toml"}
+    [python]
+    interpreter_constraints = ["==3.12.*"]
+    ```
+    ```python tab={"label":"project/BUILD"}
+    python_aws_lambda_function(
+        name="lambda",
+        handler="lambda_example.py:example_handler",
+        # `runtime` inferred and `complete_platforms` from built-in defaults,
+        # `architecture` defaults to x86_64, but can be overridden
+    )
+    ```
+
+This guide is written using the last option, with the default `x86_64` architecture, but you can add `runtime`, `complete_platforms` and/or `architecture` to any examples using the `python_aws_lambda_function` or `python_aws_lambda_layer` targets.
+
 ## Step 3: Run `package`
 
 Now run `pants package` on your `python_aws_lambda_function` target to create a zipped file.
@@ -118,7 +160,6 @@ python_sources()
 
 python_aws_lambda_function(
     name="lambda",
-    runtime="python3.8",
     handler="main.py:lambda_handler"
 )
 
@@ -148,15 +189,13 @@ python_sources(name="lib")
 
 python_aws_lambda_function(
     name="function",
-    runtime="python3.8",
     handler="lambda_example.py:example_handler",
     # only include the sources, the boto3 requirement is packaged in `:layer`
     include_requirements=False,
 )
 
 python_aws_lambda_layer(
     name="layer",
-    runtime="python3.8"
     # specify the handler file, and pants will automatically find its transitive dependencies
     dependencies=["./lambda_example.py"],
     # only include the boto3 requirement, any sources are packaged in `:function`
@@ -198,8 +237,8 @@ python_sources()
 pex_binary(
     name="lambda",
     entry_point="lambda_example.py",
-    # specify an appropriate platform(s) for the targeted Lambda runtime (complete_platforms works too)
-    platforms=["linux_x86_64-cp39-cp39"],
+    # specify an appropriate platform for the targeted Lambda runtime:
+    complete_platforms=["path/to:platform-json-target"],
 )
 ```
 
diff --git a/docs/docs/python/integrations/google-cloud-functions.mdx b/docs/docs/python/integrations/google-cloud-functions.mdx
@@ -33,7 +33,7 @@ First, add your Cloud function in a Python file like you would [normally do with
 
 Then, in your BUILD file, make sure that you have a `python_source` or `python_sources` target with the handler file included in the `sources` field. You can use [`pants tailor ::`](../../getting-started/initial-configuration.mdx#5-generate-build-files) to automate this.
 
-Add a `python_google_cloud_function` target and define the `runtime`, `handler`, and `type` fields. The `type` should be either `"event"` or `"http"`. The `runtime` should be one of the values from [https://cloud.google.com/functions/docs/concepts/python-runtime](https://cloud.google.com/functions/docs/concepts/python-runtime). The `handler` has the form `handler_file.py:handler_func`, which Pants will convert into a well-formed entry point. Alternatively, you can set `handler` to the format `path.to.module:handler_func`.
+Add a `python_google_cloud_function` target and define `handler` and `type` fields. The `type` should be either `"event"` or `"http"`.  The `handler` has the form `handler_file.py:handler_func`, which Pants will convert into a well-formed entry point. Alternatively, you can set `handler` to the format `path.to.module:handler_func`.
 
 For example:
 
@@ -43,7 +43,6 @@ python_sources(name="lib")
 
 python_google_cloud_function(
     name="cloud_function",
-    runtime="python38",
     # Pants will convert this to `project.google_cloud_function_example:example_handler`.
     handler="google_cloud_function_example.py:example_handler",
     type="event",
@@ -67,6 +66,49 @@ Use [layout](../../../reference/targets/python_google_cloud_function.mdx#layout)
 `file` / `files` targets will not be included in the built Cloud Function because filesystem APIs like `open()` would not load them as expected. Instead, use the `resource` / `resources` target. See [Assets and archives](../../using-pants/assets-and-archives.mdx) for further explanation.
 :::
 
+### Specifying a runtime explicitly
+
+When building an Cloud function artifact, Pants and the underlying Pex tool need to know details about target runtime to be able to choose appropriate artifacts for third-party dependencies that have native code. These details can be inferred or provided in three ways, from highest precedence to lowest precedence:
+
+1. An explicit value for [the `complete_platforms` field](../../../reference/targets/python_google_cloud_function.mdx#complete_platforms). The "complete platforms" are the underlying source of truth.
+    ```python title="BUILD"
+    file(name="gcf-platform", source="gcf-platform.json")
+
+    python_google_cloud_function(
+        name="cloud_function",
+        handler="google_cloud_function_example.py:example_handler",
+        type="event",
+        # Explicit complete platforms:
+        complete_platforms=[":gcf-platform"],
+    )
+    ```
+2. An explicit value for [the `runtime` field](../../../reference/targets/python_google_cloud_function.mdx#runtime): Pants uses this to pick an appropriate "complete platforms" value, from options that Pants has pre-packaged. These are static exports from docker images provided by GCP, relying on the environment being relatively stable. (If Pants doesn't have an appropriate "complete platforms" default built-in, you will be prompted to use option 1 above.)
+    ```python title="BUILD"
+    python_google_cloud_function(
+        name="cloud_function",
+        handler="google_cloud_function_example.py:example_handler",
+        type="event",
+        # Explicit runtime, `complete_platforms` taken from Pants' built-in defaults:
+        runtime="python312",
+    )
+    ```
+3. Inferred from [the relevant interpreter constraints](../overview/interpreter-compatibility.mdx): the interpreter constraints may unambiguously imply a value for the `runtime` and thus `complete_platforms` fields. For example, `interpreter_constaints = ["==3.12.*"]` implies `runtime="python312"`. This only works with interpreter constraints that cover all patch versions of a given minor release series: `>=3.11,<3.13` is too wide (it covers both 3.11 and 3.12), while `==3.12.0` is too specific (GCF's `python312` runtime may not use that exact patch version).
+
+    ```toml tab={"label":"pants.toml"}
+    [python]
+    interpreter_constraints = ["==3.12.*"]
+    ```
+    ```python tab={"label":"project/BUILD"}
+    python_google_cloud_function(
+        name="cloud_function",
+        handler="google_cloud_function_example.py:example_handler",
+        type="event",
+        # `runtime` inferred and `complete_platforms` from built-in defaults
+    )
+    ```
+
+This guide is written using the last option, but you can add `runtime` or `complete_platforms` to any examples using the `python_google_cloud_function` target.
+
 ## Step 3: Run `package`
 
 Now run `pants package` on your `python_google_cloud_function` target to create a zipped file.
@@ -119,8 +161,8 @@ python_sources()
 pex_binary(
     name="gcf",
     entry_point="gcf_example.py",
-    # specify an appropriate platform(s) for the targeted GCF runtime (complete_platforms works too)
-    platforms=["linux_x86_64-cp39-cp39"],
+    # specify an appropriate platform for the targeted GCF runtime:
+    complete_platforms=["path/to:platform-json-target"],
 )
 ```
 
