Skip to content

StepFunctions: tasks with apigateway:invoke fail without issuing an HTTP request #10163

Description

@Smelton01

Summary

With the parser backend enabled stepfunctions.execute_state_machine=True, any state machine that contains arn:aws:states:::apigateway:invoke fails, and no request is ever sent:

States.Runtime | TypeError(FailureEvent.__init__() missing 1 required positional argument: 'env')

There are actually three separate bugs in moto/stepfunctions/parser/asl/component/state/exec/state_task/service/state_task_service_api_gateway.py, and each one seems to hide the next:

  1. _from_error() doesn't pass FailureEvent's required env argument (L246), so the error path itself blows up with the States.Runtime above.
  2. _eval_service_task() still declares its last parameter as task_credentials (L264), but the base class calls it with state_credentials= (state_task_service.py L354). This would be a TypeError on every invocation, but surfaces only once bug 1 is fixed.
  3. _SUPPORTED_API_PARAM_BINDINGS only lists {"ApiEndpoint", "Method"} (L102-L104), and _eval_parameters() silently drops every parameter not in that set. This means Stage, Path, QueryParameters, RequestBody and Headers are all thrown away before the task reads them.

Bugs 1 and 2 look like a missed migration. #9142 added FailureEvent.env and renamed task_credentials to state_credentials, and updated every sibling integration (state_task_service_sqs.py, state_task_service_sns.py, ...) except this file.

Reproduction

Starts a throwaway HTTP server as the API Gateway target so you can see what request (if any) actually arrives.

import json
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer

import boto3
from moto import mock_aws

RECEIVED = []


class Handler(BaseHTTPRequestHandler):
    protocol_version = "HTTP/1.1"

    def do_POST(self):
        length = int(self.headers.get("Content-Length") or 0)
        body = self.rfile.read(length) if length else b""
        RECEIVED.append({"path": self.path, "body": body.decode()})
        payload = b'{"ok": true}'
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.send_header("Content-Length", str(len(payload)))
        self.end_headers()
        self.wfile.write(payload)

    def log_message(self, *args):
        pass


server = ThreadingHTTPServer(("127.0.0.1", 0), Handler)
PORT = server.server_address[1]
threading.Thread(target=server.serve_forever, daemon=True).start()

DEFN = {
    "StartAt": "Call",
    "States": {
        "Call": {
            "Type": "Task",
            "Resource": "arn:aws:states:::apigateway:invoke",
            "Parameters": {
                "ApiEndpoint": f"http://127.0.0.1:{PORT}",
                "Method": "POST",
                "Stage": "prod",
                "Path": "/things",
                "QueryParameters": {"mode": ["fast"]},
                "RequestBody": {"hello": "world"},
                "Headers": {"Content-Type": ["application/json"]},
            },
            "End": True,
        }
    },
}


@mock_aws(config={"stepfunctions": {"execute_state_machine": True}})
def run():
    sfn = boto3.client("stepfunctions", region_name="us-east-1")
    arn = sfn.create_state_machine(
        name="apigw-repro",
        definition=json.dumps(DEFN),
        roleArn="arn:aws:iam::123456789012:role/sf",
    )["stateMachineArn"]

    ex = sfn.start_execution(stateMachineArn=arn, input="{}")["executionArn"]
    for _ in range(50):
        d = sfn.describe_execution(executionArn=ex)
        if d["status"] != "RUNNING":
            break
        time.sleep(0.2)

    print("status :", d["status"])
    print("error  :", d.get("error"))
    print("cause  :", d.get("cause"))
    print("requests received by the API:", RECEIVED)

    assert d["status"] == "SUCCEEDED", d.get("cause")
    assert RECEIVED, "the task never issued an HTTP request"
    assert RECEIVED[0]["path"] == "/prod/_user_request_/things/?mode=fast", RECEIVED[0]["path"]
    assert json.loads(RECEIVED[0]["body"]) == {"hello": "world"}, RECEIVED[0]["body"]
    print("\nOK")


run()

Expected: the execution reaches SUCCEEDED and the API receives one POST with Stage/Path/QueryParameters in the URL and RequestBody as the JSON body.

Actual:

status : FAILED
error  : States.Runtime
cause  : TypeError(FailureEvent.__init__() missing 1 required positional argument: 'env')
requests received by the API: []

After patching bug 1, the failure becomes ApiGateway.TypeError | _eval_service_task() got an unexpected keyword argument 'state_credentials'.
After fixing 1 and 2, the request finally goes out, but as POST /_user_request_// with no query string and no body, while the execution reports SUCCEEDED.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions