Update sitemap and enhance usage documentation with depth pruning details

- Updated last modified dates in sitemap.xml for all URLs to 2025-09-24.
- Modified sitemap.xml.gz to reflect changes in sitemap.xml.
- Expanded usage/index.html to include comprehensive sections on depth pruning, covering:
  - Overview of depth pruning and its benefits.
  - Detailed Python API examples for basic depth pruning, pruning by percentage, and pruning with specific layer indices.
  - Command-line interface examples for depth pruning.
  - Comparative analysis of MLP GLU vs Depth Pruning, including when to use each method.
  - Added evaluation methods for assessing pruned models.

Author: peremartra

docs/index.md

@@ -23,11 +23,12 @@ Understanding bias in language models is crucial for:
 
 - **GLU Architecture-Aware Pruning**: Maintains the paired nature of gate_proj and up_proj layers
 - **Depth Pruning**: Remove entire transformer layers for aggressive efficiency gains
+- **Layer Importance Analysis**: Analyze transformer layers using cosine similarity to inform pruning decisions
+- **Bias Visualization Tools**: Comprehensive analysis of how models process demographic attributes
+- **Quantitative Bias Metrics**: Numeric measurements for consistent evaluation
 - **Multiple Neuron Selection Methods**: MAW, VOW, and PON for different pruning strategies
 - **Flexible Pruning Targets**: Support for both pruning percentage and target expansion rate
 - **Layer Selection Methods**: Choose specific layers or use automatic selection strategies
-- **Bias Visualization Tools**: Comprehensive analysis of how models process demographic attributes
-- **Quantitative Bias Metrics**: Numeric measurements for consistent evaluation
 - **Simple API and CLI**: Easy to use interfaces for Python and command line
 - **Progress Tracking**: Visual progress bars and detailed statistics
 

docs/usage.md

@@ -1,4 +1,5 @@
 # Usage Guide
+This guide provides detailed instructions on how to use the core functionalities of OptiPFair, from pruning models to analyzing bias.
 
 ## Python API
 
@@ -267,4 +268,102 @@ comparison = compare_models_inference(
 
 print(f"Speedup: {comparison['speedup']:.2f}x")
 print(f"Tokens per second improvement: {comparison['tps_improvement_percent']:.2f}%")
+```
+
+## Layer Importance Analysis
+
+OptiPFair includes functionality to analyze the importance of transformer layers using cosine similarity. This helps identify which layers contribute most to the model's transformations, informing depth pruning decisions.
+
+### Basic Usage
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+from torch.utils.data import DataLoader
+from optipfair import analyze_layer_importance
+
+# Load model and tokenizer
+model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B")
+tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-0.6B")
+
+# Prepare your dataset (user responsibility)
+# Example with a simple dataset
+from datasets import load_dataset
+dataset = load_dataset('wikitext', 'wikitext-2-raw-v1', split='train[:100]')
+
+def tokenize_function(examples):
+    return tokenizer(
+        examples['text'],
+        truncation=True,
+        padding='max_length',
+        max_length=512,
+        return_tensors='pt'
+    )
+
+tokenized_dataset = dataset.map(tokenize_function, batched=True)
+dataloader = DataLoader(tokenized_dataset, batch_size=8)
+
+# Analyze layer importance
+importance_scores = analyze_layer_importance(model, dataloader)
+
+# Results: {0: 0.890395, 1: 0.307580, 2: 0.771541, ...}
+print(importance_scores)
+```
+
+### Advanced Usage
+
+```python
+# With manual architecture specification
+importance_scores = analyze_layer_importance(
+    model=model,
+    dataloader=dataloader,
+    layers_path='transformer.h',  # For GPT-2 style models
+    show_progress=True
+)
+
+# Analyze specific layers for depth pruning
+# Higher scores indicate layers that transform data more significantly
+# Lower scores indicate "passive" layers that could be candidates for removal
+sorted_layers = sorted(importance_scores.items(), key=lambda x: x[1], reverse=True)
+print("Most important layers:", sorted_layers[:3])
+print("Least important layers:", sorted_layers[-3:])
+```
+
+### Multi-Architecture Support
+
+The function automatically detects transformer layers for different architectures:
+
+- **LLaMA/Qwen/Mistral**: `model.layers`
+- **GPT-2/DistilGPT2**: `transformer.h`  
+- **T5**: `encoder.block` or `decoder.block`
+- **BERT**: `encoder.layer`
+
+If automatic detection fails, specify the path manually:
+
+```python
+# Manual specification for custom architectures
+importance_scores = analyze_layer_importance(
+    model=model,
+    dataloader=dataloader,
+    layers_path='model.custom_transformer_layers'
+)
+```
+
+### Integration with Depth Pruning
+
+Use importance scores to inform depth pruning decisions:
+
+```python
+# Analyze layer importance
+importance_scores = analyze_layer_importance(model, dataloader)
+
+# Identify least important layers
+sorted_layers = sorted(importance_scores.items(), key=lambda x: x[1])
+layers_to_remove = [layer_idx for layer_idx, score in sorted_layers[:4]]
+
+# Apply depth pruning to remove least important layers
+pruned_model = prune_model(
+    model=model,
+    pruning_type="DEPTH",
+    layer_indices=layers_to_remove
+)
 ```