-
Notifications
You must be signed in to change notification settings - Fork 210
Add Code Examples to Video Tutorial #1348
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Ao Tang <[email protected]>
Greptile SummaryThis PR significantly expands the video tutorial documentation by adding comprehensive programmatic API usage examples. The changes transform a minimal CLI-focused tutorial into a complete guide that demonstrates how to build video curation pipelines using NeMo Curator's Python API directly. Major additions:
Issue found:
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant Pipeline
participant VideoReader
participant Extractor
participant Transcoder
participant FrameExtractor
participant Filter
participant Embedder
participant Captioner
participant Writer
User->>Pipeline: Initialize pipeline
User->>Pipeline: Add VideoReader stage
Pipeline->>VideoReader: Read videos from directory
VideoReader-->>Pipeline: Return video metadata
User->>Pipeline: Add FixedStrideExtractor/TransNetV2
Pipeline->>Extractor: Split videos into clips
Extractor-->>Pipeline: Return clip segments
User->>Pipeline: Add ClipTranscodingStage
Pipeline->>Transcoder: Transcode clips
Transcoder-->>Pipeline: Return encoded clips
opt Frame Extraction for Embeddings/Filtering
User->>Pipeline: Add ClipFrameExtractionStage
Pipeline->>FrameExtractor: Extract frames from clips
FrameExtractor-->>Pipeline: Return frame data
end
opt Quality Filtering
User->>Pipeline: Add AestheticFilter/MotionFilter
Pipeline->>Filter: Score and filter clips
Filter-->>Pipeline: Return filtered clips
end
opt Embedding Generation
User->>Pipeline: Add CosmosEmbed1/InternVideo2 stages
Pipeline->>Embedder: Generate embeddings from frames
Embedder-->>Pipeline: Return embeddings (768-dim or 512-dim)
end
opt Caption Generation
User->>Pipeline: Add CaptionPreparation/Generation
Pipeline->>Captioner: Generate captions for clips
Captioner-->>Pipeline: Return captions
end
User->>Pipeline: Add ClipWriterStage
Pipeline->>Writer: Write clips, metadata, embeddings
Writer-->>Pipeline: Save to output directory
User->>Pipeline: Execute with XennaExecutor
Pipeline-->>User: Complete (results in OUTPUT_DIR)
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1 file reviewed, 1 comment
| ### Cosmos-Embed1 (Recommended) | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Section header for Cosmos-Embed1 is empty/incomplete.
| ### Cosmos-Embed1 (Recommended) | |
| ### Cosmos-Embed1 (Recommended) | |
| Cosmos-Embed1 provides high-quality video embeddings with 768 dimensions. It offers better performance than InternVideo2 for most use cases. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1 file reviewed, 1 comment
| ### Cosmos-Embed1 (Recommended) | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Empty section for Cosmos-Embed1. Add description of features, output dimensions (768-dim embeddings), or usage details.
| ### Cosmos-Embed1 (Recommended) | |
| ### Cosmos-Embed1 (Recommended) | |
| Cosmos-Embed1 provides high-quality video embeddings with support for multiple resolution variants. It's the recommended choice for most video embedding tasks. | |
| - Output: 768-dimensional embeddings | |
| - Variants: `224p`, `336p`, `448p` (higher resolution = better quality, more memory) | |
| - Best for: similarity search, clustering, and general video understanding tasks | |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Signed-off-by: Lawrence Lane <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1 file reviewed, 1 comment
| ### Cosmos-Embed1 (Recommended) | ||
|
|
||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
syntax: Empty Cosmos-Embed1 section. Add content describing the model's features, capabilities, and embedding dimensions (768-dim).
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Description
Usage
# Add snippet demonstrating usageChecklist