Feature Request: Automatic Job Dependencies for Houdini Archive Export and Redshift Rendering

[skarpree]

Describe the problem

Summary

Add automatic job dependency setup in the Deadline Cloud Houdini submitter to create a two-step workflow where Houdini exports a Redshift archive, followed by a dependent redshiftCmdLine rendering task, similar to the behavior in Deadline 10.

Background

In Deadline 10, the Deadline ROP intelligently sets up job dependencies between:

1. A Houdini task that exports a Redshift archive
2. A redshiftCmdLine job that renders the exported archive file

This automatic dependency setup is standard practice because it avoids using a Houdini license as a wrapper to shell out to a terminal command. The rendering phase only requires a Redshift license, not a Houdini license.

Proposed Solution

Current Behavior in Deadline Cloud

When submitting the same Houdini/Redshift scene through the deadline-cloud ROP, the submitter creates a single task for Houdini to export the archive file but does not automatically set up the dependent redshiftCmdLine task for rendering.

Requested Feature

The Deadline Cloud Houdini submitter should automatically create a job with two steps when detecting a Houdini/Redshift archive export workflow:

Step 1: Archive Export

• Runs Houdini to export the Redshift archive
• Uses Houdini license only

Step 2: Redshift Rendering (dependent on Step 1)

• Runs redshiftCmdLine to render the archived scene
• Uses Redshift license only
• Automatically receives the archive file output from Step 1

Business Value

1. License Optimization: Avoids unnecessary Houdini license consumption during the rendering phase
2. Cost Efficiency: Reduces overall rendering costs by using appropriate licenses for each phase
3. Workflow Parity: Maintains feature parity with Deadline 10, reducing friction for customers migrating to Deadline Cloud
4. Pipeline Efficiency: Eliminates manual job template modification for a common rendering workflow

Technical Notes

• Deadline Cloud supports step dependencies through the Open Job Description specification
• The underlying capability exists (see: https://docs.aws.amazon.com/deadline-cloud/latest/developerguide/using-files-output-from-a-step-in-a-dependent-step.html)
• The feature gap is in the automatic setup by the Houdini submitter

Additional Context

This workflow is commonly used in production environments where Houdini is used for scene setup and export, but the actual rendering is handled by Redshift's command-line renderer for efficiency. The automatic dependency setup in Deadline 10 made this workflow seamless, and customers expect similar functionality in Deadline Cloud.

Example Use Cases

Use Case 1: VFX Studio Rendering Pipeline

A VFX studio has 100 shots that need to be rendered. Each shot requires:

• 5 minutes of Houdini time to export the Redshift archive
• 2 hours of Redshift rendering time per shot

Without automatic dependencies:

• Artists must manually create two separate jobs or modify job templates
• Risk of starting render jobs before archives are complete
• Potential for using Houdini licenses during the 2-hour render phase

With automatic dependencies:

• Single submission from Houdini creates both steps automatically
• Archive export completes first, then rendering begins
• 200 hours of rendering uses only Redshift licenses, not Houdini licenses
• Significant cost savings on license usage

[edwards-aws]

Thanks for the feature request @skarpree! Export workflows are definitely useful and we'll add it to our list of priorities, however we cannot provide an estimate for when this work will be worked on.