Description
Describe the bug
I want to have Integration tests run both as standalone tests using integ-runner and as part of the regular deployment. Right now, I'm able to do this, but can't get the deployments to fail when tests fail.
Expected Behavior
Failed integration tests that are part of a regular deployment should make the deployment fail.
Current Behavior
Deployment shows the failed test output, but succeeds anyways.
Reproduction Steps
Given the following stack:
#import * as cdk from "aws-cdk-lib";
import { Construct } from "constructs";
import * as lambda from "aws-cdk-lib/aws-lambda";
import createAssertions from "./assertions";
export interface StackProps extends cdk.StackProps {
runAssertions?: boolean
envName?: string
}
export class IdemoStack extends cdk.Stack {
readonly sampleFunction: lambda.Function;
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const {runAssertions = true, envName=""} = props??{}
console.log("props: ", props)
this.sampleFunction = new lambda.Function(this, `sampleFunction${envName? "-" + envName:""}`, {
runtime: lambda.Runtime.NODEJS_LATEST,
handler: "index.handler",
code: lambda.Code.fromInline(`
exports.handler = async (event) => {
return true;
};
`),
});
this.sampleFunction.currentVersion
if (runAssertions) {
createAssertions(this)
}
}
}the following integration test file
import { App } from "aws-cdk-lib";
import { IdemoStack } from "../lib/idemo-stack";
import { IntegTest } from "@aws-cdk/integ-tests-alpha";
import createAssertions from "../lib/assertions";
import { DeployAssert } from '@aws-cdk/integ-tests-alpha/lib/assertions/private/deploy-assert';
const app = new App();
const stack = new IdemoStack(app, "StackUnderTest", {runAssertions:false, envName:"integration"});
const integTest = new IntegTest(app, "IntegrationTests", {
testCases: [stack],
diffAssets: true,
stackUpdateWorkflow: false,
cdkCommandOptions: {
destroy: {
args: {
force: true,
},
},
},
});
const deployAssert = integTest.assertions;
if (!DeployAssert.isDeployAssert(deployAssert)) {
throw new Error('Expected DeployAssert');
}
const assertionScope = deployAssert.scope;
createAssertions(stack, assertionScope);
app.synth()and the assertions defined like this:
import * as integ_tests from "@aws-cdk/integ-tests-alpha";
import { IdemoStack } from "./idemo-stack";
import { CustomResource, Stack } from "aws-cdk-lib";
export default function createAssertions(
stack: IdemoStack,
testStack?: Stack,
) {
const testStack_ = testStack ?? stack;
const val = new integ_tests.LambdaInvokeFunction(
testStack_,
"testSampleFunction",
{
functionName: stack.sampleFunction.functionName,
},
);
val.expect(integ_tests.ExpectedResult.objectLike({ Payload: 'false' }));
}If I run the tests with integ-runner, they fail as expected. But if I do a cdk deploy, the test fails but the deployment succeeds
Possible Solution
I find a workaround to make the deployment fail as I expect it to. if I add the following to the end of the createAssertions function
new integ_tests.EqualsAssertion(testStack_, "assertAssertionPassed", {
failDeployment: true,
expected: integ_tests.ExpectedResult.objectLike({status: "success"}),
actual: integ_tests.ActualResult.fromCustomResource(val.node.findChild("Default") as CustomResource, "assertion")
}).node.tryRemoveChild("AssertionResults")Then the deployment fails when the tests fail
Additional Information/Context
Ideally, I think there should be a way to add the failDeployment: true to the regular expect instead of having to check for the assertion value in a separate assertion
CDK CLI Version
2.143.1
Framework Version
No response
Node.js Version
20.12.2
OS
MacOs
Language
TypeScript
Language Version
5.4.5
Other information
No response