-
Notifications
You must be signed in to change notification settings - Fork 251
Add smithy-cfn-json build plugin #3095
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| { | ||
| "type": "feature", | ||
| "description": "Added smithy-cfn-json plugin that serializes Smithy models to JSON AST with CloudFormation Fn Sub intrinsic functions", | ||
| "pull_requests": [ | ||
| "[#3095](https://github.com/smithy-lang/smithy/pull/3095)" | ||
| ] | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,147 @@ | ||
| .. _smithy-cfn-json: | ||
|
|
||
| ========================================================== | ||
| Converting Smithy to CloudFormation JSON | ||
| ========================================================== | ||
|
|
||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I like this!! The NodeVisitor approach and path-matching with wildcards are clean. One note, the core Fn::Sub injection logic is generic, it's not really API Gateway-specific. Any trait with ARN-valued fields could benefit. Would it make sense to put this in a more general module (e.g., smithy-aws-cloudformation) and make the paths configurable via the plugin config? The API Gateway paths could be defaults, but users could add their own for custom traits. This would also avoid needing to modify the class every time we add a new trait with an ARN field (like we just did with providerARNs on authorizers)! |
||
| This guide describes how Smithy models can be serialized to JSON AST with | ||
| CloudFormation ``Fn::Sub`` intrinsic function wrapping using the | ||
| ``smithy-cfn-json`` plugin. | ||
|
|
||
| ------------ | ||
| Introduction | ||
| ------------ | ||
|
|
||
| The ``smithy-cfn-json`` plugin serializes a Smithy model to its JSON AST | ||
| representation with automatic CloudFormation ``Fn::Sub`` substitution wrapping. | ||
| The output is intended for use as the ``Body`` property of an | ||
| ``AWS::ApiGateway::RestApi`` CloudFormation resource, enabling direct Smithy | ||
| model import without OpenAPI conversion. | ||
|
|
||
| String values containing ``${...}`` variable syntax at known trait paths are | ||
| automatically wrapped in ``{"Fn::Sub": "..."}`` objects so that CloudFormation | ||
| resolves the references at deploy time before passing the body to the API | ||
| Gateway SmithyImporter. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. what is SmithyImporter? (˵ ¬ᴗ¬˵) |
||
|
|
||
| .. _smithy-cfn-json-configuration: | ||
|
|
||
| ----------------------------------------------- | ||
| Converting to JSON AST with smithy-build | ||
| ----------------------------------------------- | ||
|
|
||
| The ``smithy-cfn-json`` plugin contained in the | ||
| ``software.amazon.smithy:smithy-aws-apigateway-cfn`` package can be used with | ||
| smithy-build to produce CloudFormation-ready JSON from Smithy models. | ||
|
|
||
| The following example shows how to configure the plugin in | ||
| ``smithy-build.json``: | ||
|
|
||
| .. code-block:: json | ||
| :caption: smithy-build.json | ||
|
|
||
| { | ||
| "version": "1.0", | ||
| "plugins": { | ||
| "smithy-cfn-json": { | ||
| "service": "com.example#MyService" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| The plugin writes ``{ServiceName}.smithy.json`` to the build output directory. | ||
|
|
||
| .. _smithy-cfn-json-settings: | ||
|
|
||
| ---------------------- | ||
| Configuration settings | ||
| ---------------------- | ||
|
|
||
| .. _smithy-cfn-json-setting-service: | ||
|
|
||
| service (``string``) | ||
| ==================== | ||
|
|
||
| **Required**. The Smithy service :ref:`shape ID <shape-id>` to export. | ||
|
|
||
| .. code-block:: json | ||
| :caption: smithy-build.json | ||
|
|
||
| { | ||
| "version": "1.0", | ||
| "plugins": { | ||
| "smithy-cfn-json": { | ||
| "service": "com.example#MyService" | ||
| } | ||
| } | ||
| } | ||
|
|
||
| .. _smithy-cfn-json-setting-disableCloudFormationSubstitution: | ||
|
|
||
| disableCloudFormationSubstitution (``boolean``) | ||
| =============================================== | ||
|
|
||
| Set to ``true`` to disable automatic ``Fn::Sub`` wrapping of string values | ||
| that contain ``${...}`` variable references. Defaults to ``false``. | ||
|
|
||
| .. code-block:: json | ||
| :caption: smithy-build.json | ||
|
|
||
| { | ||
| "version": "1.0", | ||
| "plugins": { | ||
| "smithy-cfn-json": { | ||
| "service": "com.example#MyService", | ||
| "disableCloudFormationSubstitution": true | ||
| } | ||
| } | ||
| } | ||
|
|
||
| .. _smithy-cfn-json-substitution: | ||
|
|
||
| ------------------------------ | ||
| CloudFormation substitution | ||
| ------------------------------ | ||
|
|
||
| When ``disableCloudFormationSubstitution`` is ``false`` (the default), string | ||
| values containing ``${...}`` variable syntax at the following trait paths are | ||
| automatically wrapped in ``{"Fn::Sub": "..."}`` objects: | ||
|
|
||
| * ``aws.apigateway#integration`` — ``uri``, ``credentials``, ``connectionId``, | ||
| ``integrationTarget`` | ||
| * ``aws.apigateway#authorizers`` — ``*/uri``, ``*/credentials`` | ||
| * ``aws.auth#cognitoUserPools`` — ``providerArns/*`` | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this should be apigateway cognito authorizer providerarns: #3085 I am yet to discuss with the smithy team re: aws auth's cognito user pools trait: https://smithy.io/2.0/aws/aws-auth.html#aws-auth-cognitouserpools-trait There's a lot of confusion between these two authorizing traits. |
||
|
|
||
| CloudFormation resolves ``Fn::Sub`` at deploy time before passing the body to | ||
| the API Gateway SmithyImporter. | ||
|
|
||
| .. _smithy-cfn-json-example: | ||
|
|
||
| ------- | ||
| Example | ||
| ------- | ||
|
|
||
| Given the following Smithy model input: | ||
|
|
||
| .. code-block:: smithy | ||
|
|
||
| @integration( | ||
| type: "aws_proxy" | ||
| uri: "${MyLambdaFunction.Arn}" | ||
| httpMethod: "POST" | ||
| credentials: "${ApiGatewayRole.Arn}" | ||
| ) | ||
|
|
||
| The plugin produces the following in the generated JSON AST: | ||
|
|
||
| .. code-block:: json | ||
|
|
||
| { | ||
| "aws.apigateway#integration": { | ||
| "type": "aws_proxy", | ||
| "uri": {"Fn::Sub": "${MyLambdaFunction.Arn}"}, | ||
| "httpMethod": "POST", | ||
| "credentials": {"Fn::Sub": "${ApiGatewayRole.Arn}"} | ||
| } | ||
| } | ||
|
|
||
| Values that do not contain ``${...}`` syntax are left as plain strings. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,67 @@ | ||
| # Smithy AWS API Gateway CloudFormation JSON Plugin | ||
|
|
||
| This module provides a `smithy-build` plugin that serializes a Smithy model to | ||
| JSON AST with CloudFormation `Fn::Sub` intrinsic function wrapping. The output | ||
| is intended for use as the `Body` property of an `AWS::ApiGateway::RestApi` | ||
| CloudFormation resource, enabling direct Smithy model import without OpenAPI | ||
| conversion. | ||
|
|
||
| ## Usage | ||
|
|
||
| Add the following to your `smithy-build.json`: | ||
|
|
||
| ```json | ||
| { | ||
| "version": "1.0", | ||
| "plugins": { | ||
| "smithy-cfn-json": { | ||
| "service": "com.example#MyService" | ||
| } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Configuration | ||
|
|
||
| | Property | Type | Required | Default | Description | | ||
| |----------|------|----------|---------|-------------| | ||
| | `service` | ShapeId | Yes | — | The service shape to export | | ||
| | `disableCloudFormationSubstitution` | boolean | No | `false` | Disable `Fn::Sub` wrapping | | ||
|
|
||
| ### Output | ||
|
|
||
| The plugin writes `{ServiceName}.smithy.json` to the build output directory. | ||
|
|
||
| ## CloudFormation Substitution | ||
|
|
||
| String values containing `${...}` variable syntax at the following trait paths | ||
| are automatically wrapped in `{"Fn::Sub": "..."}` objects: | ||
|
|
||
| - `aws.apigateway#integration` → `uri`, `credentials`, `connectionId`, `integrationTarget` | ||
| - `aws.apigateway#authorizers` → `*/uri`, `*/credentials` | ||
| - `aws.auth#cognitoUserPools` → `providerArns/*` | ||
|
|
||
| ### Example | ||
|
|
||
| Input (Smithy IDL): | ||
| ```smithy | ||
| @integration( | ||
| type: "aws_proxy" | ||
| uri: "${MyLambdaFunction.Arn}" | ||
| httpMethod: "POST" | ||
| credentials: "${ApiGatewayRole.Arn}" | ||
| ) | ||
| ``` | ||
|
|
||
| Output (in the generated JSON AST): | ||
| ```json | ||
| "aws.apigateway#integration": { | ||
| "type": "aws_proxy", | ||
| "uri": {"Fn::Sub": "${MyLambdaFunction.Arn}"}, | ||
| "httpMethod": "POST", | ||
| "credentials": {"Fn::Sub": "${ApiGatewayRole.Arn}"} | ||
| } | ||
| ``` | ||
|
|
||
| CloudFormation resolves `Fn::Sub` at deploy time before passing the body to | ||
| the API Gateway SmithyImporter. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| /* | ||
| * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
| plugins { | ||
| id("smithy.module-conventions") | ||
| } | ||
|
|
||
| description = "This module provides a smithy-build plugin that serializes a Smithy model " + | ||
| "to JSON AST with CloudFormation Fn::Sub substitution for use as a CFN RestApi Body." | ||
|
|
||
| extra["displayName"] = "Smithy :: Amazon API Gateway CloudFormation JSON" | ||
| extra["moduleName"] = "software.amazon.smithy.aws.apigateway.cfn" | ||
|
|
||
| dependencies { | ||
| api(project(":smithy-build")) | ||
| api(project(":smithy-model")) | ||
| testImplementation(project(":smithy-aws-apigateway-traits")) | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is used for testing, but i guess we can use another trait for testing and remove this dependency
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If we want this plugin to work beyond API Gateway, we could test with a simple inline trait instead of depending on the apigateway-traits module. That also proves it works for any trait with ARN fields. |
||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| /* | ||
| * Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. | ||
| * SPDX-License-Identifier: Apache-2.0 | ||
| */ | ||
| package software.amazon.smithy.aws.apigateway.cfn; | ||
|
|
||
| import java.util.ArrayDeque; | ||
| import java.util.ArrayList; | ||
| import java.util.Arrays; | ||
| import java.util.Deque; | ||
| import java.util.Iterator; | ||
| import java.util.LinkedHashMap; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
| import java.util.logging.Logger; | ||
| import java.util.regex.Pattern; | ||
| import java.util.stream.Collectors; | ||
| import software.amazon.smithy.model.SourceLocation; | ||
| import software.amazon.smithy.model.node.ArrayNode; | ||
| import software.amazon.smithy.model.node.Node; | ||
| import software.amazon.smithy.model.node.NodeVisitor; | ||
| import software.amazon.smithy.model.node.ObjectNode; | ||
| import software.amazon.smithy.model.node.StringNode; | ||
|
|
||
| /** | ||
| * Walks a Smithy JSON AST node tree and wraps string values containing | ||
| * CloudFormation variable syntax in Fn::Sub intrinsic function objects. | ||
| */ | ||
| final class CloudFormationFnSubInjector extends NodeVisitor.Default<Node> { | ||
|
|
||
| private static final Logger LOGGER = Logger.getLogger(CloudFormationFnSubInjector.class.getName()); | ||
| private static final String SUBSTITUTION_KEY = "Fn::Sub"; | ||
| private static final Pattern SUBSTITUTION_PATTERN = Pattern.compile("\\$\\{.+}"); | ||
|
|
||
| static final List<String> PATHS = Arrays.asList( | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These paths are hardcoded to API Gateway traits. If someone adds a new trait with an ARN field, they will have to modify this class. can we make these configurable via |
||
| "shapes/*/traits/aws.apigateway#integration/uri", | ||
| "shapes/*/traits/aws.apigateway#integration/credentials", | ||
| "shapes/*/traits/aws.apigateway#integration/connectionId", | ||
| "shapes/*/traits/aws.apigateway#integration/integrationTarget", | ||
| "shapes/*/traits/aws.apigateway#authorizers/*/uri", | ||
| "shapes/*/traits/aws.apigateway#authorizers/*/credentials", | ||
| "shapes/*/traits/aws.auth#cognitoUserPools/providerArns/*"); | ||
|
|
||
| private final Deque<String> stack = new ArrayDeque<>(); | ||
| private final List<String[]> paths; | ||
|
|
||
| CloudFormationFnSubInjector() { | ||
| this(PATHS); | ||
| } | ||
|
|
||
| CloudFormationFnSubInjector(List<String> paths) { | ||
| this.paths = paths.stream() | ||
| .map(path -> path.split(Pattern.quote("/"))) | ||
| .collect(Collectors.toList()); | ||
| } | ||
|
|
||
| @Override | ||
| protected Node getDefault(Node node) { | ||
| return node; | ||
| } | ||
|
|
||
| @Override | ||
| public Node arrayNode(ArrayNode node) { | ||
| List<Node> result = new ArrayList<>(); | ||
| for (int i = 0; i < node.size(); i++) { | ||
| Node member = node.get(i).get(); | ||
| stack.addLast(String.valueOf(i)); | ||
| result.add(member.accept(this)); | ||
| stack.removeLast(); | ||
| } | ||
| return new ArrayNode(result, SourceLocation.NONE); | ||
| } | ||
|
|
||
| @Override | ||
| public Node objectNode(ObjectNode node) { | ||
| Map<StringNode, Node> result = new LinkedHashMap<>(); | ||
| for (Map.Entry<StringNode, Node> entry : node.getMembers().entrySet()) { | ||
| stack.addLast(entry.getKey().getValue()); | ||
| result.put(entry.getKey(), entry.getValue().accept(this)); | ||
| stack.removeLast(); | ||
| } | ||
| return new ObjectNode(result, SourceLocation.NONE); | ||
| } | ||
|
|
||
| @Override | ||
| public Node stringNode(StringNode node) { | ||
| if (SUBSTITUTION_PATTERN.matcher(node.getValue()).find() && isInPath()) { | ||
| LOGGER.fine(() -> String.format( | ||
| "Wrapping CloudFormation variable in Fn::Sub at path %s: %s", | ||
| String.join("/", stack), | ||
| node.getValue())); | ||
| return Node.objectNode().withMember(SUBSTITUTION_KEY, node); | ||
| } | ||
| return node; | ||
| } | ||
|
|
||
| private boolean isInPath() { | ||
| return paths.stream().anyMatch(this::matchesPath); | ||
| } | ||
|
|
||
| private boolean matchesPath(String[] path) { | ||
| Iterator<String> iterator = stack.iterator(); | ||
| for (String segment : path) { | ||
| if (!iterator.hasNext()) { | ||
| return false; | ||
| } | ||
| String current = iterator.next(); | ||
| if (!segment.equals(current) && !segment.equals("*")) { | ||
| return false; | ||
| } | ||
| } | ||
| return true; | ||
| } | ||
| } | ||
Uh oh!
There was an error while loading. Please reload this page.