Conversation
Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com>
| "img_url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/cat_style_layout.png", | ||
| "text_prompt": "Can you describe the image in detail.", | ||
| "num_layers": 6, | ||
| "num_layers": 2, |
There was a problem hiding this comment.
This model has to run for minimum 6 layers
There was a problem hiding this comment.
Layer 0 to 5 : full attention and layer 6: sliding window
There was a problem hiding this comment.
No need of 6 layers, I have changed the layer types. In 2 layers, we can check the model with full_attention and sliding_attention.
| "img_url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/datasets/cat_style_layout.png", | ||
| "text_prompt": "Can you describe the image in detail.", | ||
| "num_layers": 6, | ||
| "num_layers": 2, |
There was a problem hiding this comment.
same here 6 layers minimum
There was a problem hiding this comment.
No need of 6 layers, I have changed the layer types. In 2 layers, we can check the model with full_attention and sliding_attention.
| "additional_params": { | ||
| "text_config": { | ||
| "_sliding_window_pattern": 2, | ||
| "sliding_window_pattern": 2, |
There was a problem hiding this comment.
Won't this differ from original config file? And since you are updating this in code itself below, do we need this param here?
There was a problem hiding this comment.
No, it will not differ the original config. I have checked the config when I tested it.
| "text_config": { | ||
| "_sliding_window_pattern": 2, | ||
| "sliding_window_pattern": 2, | ||
| "head_dim": 256, |
There was a problem hiding this comment.
How head_dim :256 is computed? Also why sliding_window_pattern is 2?
There was a problem hiding this comment.
sliding window pattern, we can change according to the layer types.
| ) | ||
| config = set_num_layers_vlm(config, n_layer=n_layer) | ||
| if hasattr(config, "model_type") and config.model_type in ["gemma3"]: | ||
| config.text_config._sliding_window_pattern = 2 |
There was a problem hiding this comment.
Original model has slidding window length as 6
There was a problem hiding this comment.
Yes, that's true. But we can modify, right?
3 participants
No description provided.