Adding support for building engine for buildable profiles

[visheshtanksale]

Added support for NIMBuild CRD to start pod that can build a engine

To Do

• Add Unit test Coverage
• Add support for adding details of the new local build engine on NIMCache Status

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[varunrsekar]

Some more minor comments...

[varunrsekar]

Follow-up: How do we account for a NIMService referencing a NIMCache for a buildable profile that is currently being built via a NIMBuild CR? In NIMService, we only check if the NIMCache is Ready before starting, but in this case we'd likely need more fine-grained checks on the profile status. And thinking crudely, we'd probably need the nimcache status to reflect the state of new profiles being built.

[visheshtanksale]

Follow-up: How do we account for a NIMService referencing a NIMCache for a buildable profile that is currently being built via a NIMBuild CR? In NIMService, we only check if the NIMCache is Ready before starting, but in this case we'd likely need more fine-grained checks on the profile status. And thinking crudely, we'd probably need the nimcache status to reflect the state of new profiles being built.

We cannot block NIMService from using NIMCache if there is a NIMBuild action pending. NIMCache is ready to use when its downloads the profile. There can be profile in the cache which user wants to run without building them. NIMBuild is an optional action being performed on the cache. Adding details of currently running NIMBuilds is mostly redundant because the details about relation between NIMCache and NIMBuild can also be obtained by querying NIMBuild. We add the details of successfully builds because those profile are part of the cache.

[varunrsekar]

Follow-up: How do we account for a NIMService referencing a NIMCache for a buildable profile that is currently being built via a NIMBuild CR? In NIMService, we only check if the NIMCache is Ready before starting, but in this case we'd likely need more fine-grained checks on the profile status. And thinking crudely, we'd probably need the nimcache status to reflect the state of new profiles being built.

We cannot block NIMService from using NIMCache if there is a NIMBuild action pending. NIMCache is ready to use when its downloads the profile. There can be profile in the cache which user wants to run without building them. NIMBuild is an optional action being performed on the cache. Adding details of currently running NIMBuilds is mostly redundant because the details about relation between NIMCache and NIMBuild can also be obtained by querying NIMBuild. We add the details of successfully builds because those profile are part of the cache.

So I was thinking of this scenario:

• a NIMService is created without specifying any model profiles on a node that has no matching optimized profiles
• The NIM would automatically choose the buildable profile for the node's GPU Type and attempt to build it - with insufficient resources, the NIMService would go to Failed state.
• In parallel, there is a NIMBuild running to build the profile for the node's GPU Type.
• Once the NIMBuild completes, if the exact same NIMService spec is attempted, this time it will go to Ready state.

I was thinking that this might cause confusion. But I agree that we shouldn't complicate the design and keep your current expectation. Thanks for the clarification.