(Nucleus): Shutdown script is not called before run script of new components

[megakoresh]

Describe the bug
Since this behavior is not documented anywhere, I am reporting it as a bug.

When you deploy new components to a device, their startup and run scripts will be executed before old no-longer-deployed component's shutdown scripts are called.

To Reproduce
Create a database component, e.g.

RecipeFormatVersion: "2020-01-25"
ComponentName: dev.development.QuestDB
ComponentVersion: 8.3.3
ComponentType: "aws.greengrass.generic"
ComponentDescription: QuestDB database
ComponentPublisher: SwagCorp
ComponentConfiguration:
ComponentDependencies:
  aws.greengrass.DockerApplicationManager:
    VersionRequirement: ">=2.0.0 <2.1.0"
    DependencyType: "HARD"
Manifests:
- Platform:
    os: "linux"
    architecture: "amd64"
  Lifecycle:
    Setenv:
      WORKDIR: "{work:path}"
    run: "docker run --rm --network host --name questdb -v questdb-data:/var/lib/questdb questdb/questdb:8.3.3"
    shutdown: "docker stop questdb || true"

Then, say you want to modify the image and you decide to make your own version of this database image with its own component:

RecipeFormatVersion: "2020-01-25"
ComponentName: dev.development.MyQuestDB
ComponentVersion: 1.0.0
ComponentType: "aws.greengrass.generic"
ComponentDescription: QuestDB database
ComponentPublisher: SwagCorp
ComponentConfiguration:
ComponentDependencies:
  aws.greengrass.DockerApplicationManager:
    VersionRequirement: ">=2.0.0 <2.1.0"
    DependencyType: "HARD"
Manifests:
- Platform:
    os: "linux"
    architecture: "amd64"
  Lifecycle:
    Setenv:
      WORKDIR: "{work:path}"
    run: "docker run --rm --network host --name questdb -v questdb-data:/var/lib/questdb -e  QDB_LOG_W_STDOUT_LEVEL=ERROR 133769420.dkr.ecr.eu-west-1.amazonaws.com/questdb:8.3.3"
    shutdown: "docker stop questdb || true"
Artifacts:
- Uri: "docker:133769420.dkr.ecr.eu-west-1.amazonaws.com/questdb:8.3.3"
  Unarchive: "NONE"
  Permission:
  Read: "OWNER"
  Execute: "NONE"

Now when you deploy this second component to your target, it will fail to run because they use the same container name. And if you give them different container names, the new component will still fail because it uses the same volume so new container won't be able to lock the table registry file.

Expected behavior
The old component's shutdown script completes before new component is started.

Actual behavior
The new component is started before old component's shutdown script runs.

Environment

• OS: Ubuntu 22.04
• JDK version: OpenJDK 11.0.27
• Nucleus version: 2.14.3

Additional context

Why is the lifecycle behavior so badly documented in general? There is zero information on when and how different lifecycle scripts are called, relative to other components and deployments. The only thing documentation says is what lifecycle script runs when a component enters a certain lifecycle stage. Anyone who wants to find out when the lifecycle stages actually change and under which conditions is left scouring forums and running experiments.

[saranyailla]

Hi,
What you are seeing is the expected behavior where, during a deployment, new/updated services are started before the removable services are stopped and removed. Deployment service waits for all the startable services to reach their desired state (Running/Finished) before it is safe to remove the removable ones.

Also, the ordering for starting/stopping the services is guaranteed only if the components have some dependency relation between them, which is not applicable in your case.

Can you check if your usecase can accommodate any of these workarounds?

1. Create and deploy a new version of the same component instead of a different component. That way, the older version of the component is fully stopped before starting the new version.
2. If you want to enforce your new component to have higher precedence over any of the the previously deployed components, you can modify your new component such that is stops the old component or all the resources it needs before it attempts to use them.
3. Create two deployments - one deployment to remove the old component and follow that up with another deployment to add the new component.

Thanks for your feedback on the documentation. We will look into it and improve the documentation as applicable.