Klarity v0.2 - Visual Language Models Support 🖼️ (LLaVA)

Klarity: Now with Visual Language Model Support!

We're excited to introduce VLM support to Klarity, our toolkit now support visual attention analysis capabilities. This update enables deep insights into how your models process and reason about images.

New Features:

• Track visual attention patterns with detailed heatmap analysis
• Measure token-level uncertainty in relation to visual features
• Monitor visual grounding quality and alignment scores
• Identify potential visual reasoning failures and hallucinations

Structured JSON output:

• visual_analysis: {attention_quality, key_regions[], missed_regions[]}
• token_attention: array of {word, focused_spot, relevance, uncertainty}
• uncertainty_analysis: {problem_spots[], improvement_tips[]}
• scores: {visual_grounding, overall_uncertainty, confidence}

Integration Support:

Full support for LLaVA models (tested with llava-onevision-qwen2-0.5b-ov-hf)
Together AI integration for enhanced analysis using Llama-3.2-90B-Vision
Compatible with Hugging Face Transformers pipeline

Join our Discord community for discussions and support!
https://discord.gg/7WrMqv7T

Klarity v0.1 - Reasoning Models Support 🎉 (DeepSeek)

Klarity: Now with Reasoning Model Support!
We're excited to introduce reasoning model support to Klarity, our toolkit for LLM behavior analysis. This update brings Chain-of-Thought entropy analysis to gain insights to improve RL performances.

New Features:

• Identify where your model's reasoning goes off track with step-by-step entropy analysis
• Get actionable scores for coherence and confidence at each reasoning step
• Training data insights: Identify which reasoning data lead to high-quality outputs

Structured JSON output:

• steps: array of {step_number, content, entropy_score, semantic_score, top_tokens[]}
• quality_metrics: array of {step, coherence, relevance, confidence}
• reasoning_insights: array of {step, type, pattern, suggestions[]}
• training_targets: array of {aspect, current_issue, improvement}

Join our Discord community for discussions and support!