Enable rolling inference and cutoff_frame functionalities #156
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Jibin Varghese <[email protected]>
begining -> beginning
chore: update inference_utils.py
Minor fixes on typos and grammars
Co-authored-by: Jinwei Gu <[email protected]>
Signed-off-by: Jibin Varghese <[email protected]>
Signed-off-by: Jibin Varghese <[email protected]>
* feat: add post-training and custom-training support * feat: add separate model definitions supporting tp/sp for training; update configs * feat: add example Dataset class, add data augmentors, update config * feat: add example data class, add misc improvements to data loading and config, add script to convert ckpt to tp * fix: fix conflict in DiTEncoder * cleanup * feat: compelete README in examples/ for post/pre-training; update the main README * fix: multiple minor fixes on example dataset * fix: multiple minor fixes + improve example dataset performance * feat+fix: multiple fixes + refinements to README
…oid ignoring the usage of the prompt upsampler (nvidia-cosmos#43)
…mos#55) * [fead-cicd]: add-linting, formatting and check video tags * fix comments
* Adding a News section * Small fix
* replacing aegis with llamaGuard3 * updating moderation output parsing * adding llama3 licensing * linter fixes --------- Co-authored-by: Andrew Wang <[email protected]>
* minor update on how to deal with ema field in the ckpt loading * minor fix
…vidia-cosmos#60) * Add lidar & Hdmap feature to example dataloader Signed-off-by: yu chen <[email protected]> * hdmap/lidar ctrl posttrain config * ckpt loading fixes * fix input channel config to new convention * add short-video ctrlnet model, fix config for sample av post train * add 57 frame configs * adapt to new av-specific dataloader * add readme for postraining sample-av * fix typo * add back script * adapt to latest toolkit data format * resolve review items --------- Signed-off-by: yu chen <[email protected]> Co-authored-by: yu chen <[email protected]> Co-authored-by: Tianshi Cao <[email protected]>
…smos#73) * Add lidar & Hdmap feature to example dataloader Signed-off-by: yu chen <[email protected]> * hdmap/lidar ctrl posttrain config * ckpt loading fixes * fix input channel config to new convention * add short-video ctrlnet model, fix config for sample av post train * add 57 frame configs * adapt to new av-specific dataloader * add readme for postraining sample-av * fix typo * add back script * adapt to latest toolkit data format * resolve review items * update readme link for sample AV pyt example --------- Signed-off-by: yu chen <[email protected]> Co-authored-by: yu chen <[email protected]> Co-authored-by: Tianshi Cao <[email protected]>
* fix issue 82 * fix file header * fix lint --------- Co-authored-by: Jinwei Gu <[email protected]> Co-authored-by: Jinwei Gu <[email protected]>
* minor fix for the keypoint control inference * lint
* add spatial-temporal weight adding code * add robot augmentation examples * update spatial temporal processing code and example * add readme * recover * recover * add prompts and update readme * fix comments * visualization of the mp4 * update readme * linting
Signed-off-by: Jibin Varghese <[email protected]>
Signed-off-by: Jibin Varghese <[email protected]>
This MR adds two new features that improve cosmos-transfer1's support for AV use cases. 1. Cutoff frame : This allows the user to specify a cutoff frame to the input video, to demarcate the point from which cosmos-transfer generates subsequent frames. 2. Save Intermediaries : This allows the user to save the output of the intermediate diffusion steps as a video, and a combined (tiled) GIF. This helps understand diffusion models better. Signed-off-by: Jibin Varghese <[email protected]>
This MR adds two new features that improve cosmos-transfer1's support for AV use cases. 1. Cutoff frame : This allows the user to specify a cutoff frame to the input video, to demarcate the point from which cosmos-transfer generates subsequent frames. 2. Save Intermediaries : This allows the user to save the output of the intermediate diffusion steps as a video, and a combined (tiled) GIF. This helps understand diffusion models better. Signed-off-by: Jibin Varghese <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Enable rolling inference for closed-loop based simulation. To support this functionality, cutoff_frame is also implemented, where frames before the cutoff_frame are kept constant during the denoising process