Skip to content
This repository was archived by the owner on Jan 8, 2024. It is now read-only.
This repository was archived by the owner on Jan 8, 2024. It is now read-only.

AWS ECS releaser reports empty target group as healthy #4900

Description

@pcjun97

Describe the bug
The releaser for AWS ECS plugin checks for unhealthy targets in a target group, but reports healthy if there is no target in the target group at all. The deploy step does not wait for the tasks to be in "running" state before moving to the release step, which at that point in time the tasks (which are in "pending" state) has yet to be registered as targets in the new target group.

Steps to Reproduce
Run waypoint deploy with aws-ecs apps.

project = "example"

app "httpbin" {
  build {
    use "docker-pull" {
      image = "kennethreitz/httpbin"
    }
  }

  deploy {
    use "aws-ecs" {
      ...
    }
  }
}

Despite the ECS task is still starting up and the target group being empty, the release step will reports all targets are healthy:

» Releasing...

» Performing operation locally
-> Running release v383
-> Initializing release...
-> Release initialized
-> Checking that all targets are healthy...
-> All targets are healthy!
-> Modifying load balancer to introduce new target group <REDACTED>
-> Finished ECS release

Expected behavior
The releaser should treat a target group with no targets as "unhealthy", and report that accordingly.

Waypoint Platform Versions
Additional version and platform information to help triage the issue if
applicable:

  • Waypoint CLI Version: 0.11.3
  • Waypoint Server Platform and Version: 0.11.3
  • Waypoint Plugin: aws/ecs

Additional context
Relevant code snippet:

// Check each target to see if any one of them isn't healthy, before we
// route 100% of traffic to it!
for _, targetHealthDescription := range targetHealth.TargetHealthDescriptions {
log.Debug("checking target health", "target", targetHealthDescription.Target.Id)
// Possible states are: initial, healthy, unhealthy, unused, draining,
// and unavailable.
if *targetHealthDescription.TargetHealth.State != "healthy" {
return nil, errors.Errorf("target (id: %s) is not healthy - will "+
"only release when all targets in group (ARN: %q) are healthy",
*targetHealthDescription.Target.Id, target.TargetGroupArn)
}
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions