A complete implementation and comparison of three state-of-the-art Graph Neural Network architectures: GCN, GraphSAGE, and GAT on the Cora citation network dataset.
This project demonstrates the implementation and comparative analysis of Graph Neural Networks for node classification tasks. Using the Cora citation network dataset, we train and evaluate three different GNN architectures to understand their strengths and performance characteristics.
- π₯ GAT (Graph Attention Networks): 81.9% test accuracy
- π₯ GCN (Graph Convolutional Networks): 79.3% test accuracy
- π₯ GraphSAGE: 76.8% test accuracy
- 2,708 nodes (machine learning papers)
- 10,556 edges (citation relationships)
- 1,433 features per node (bag-of-words from abstracts)
- 7 classes (research areas: Neural Networks, Rule Learning, etc.)
- Semi-supervised setup: 140 training, 500 validation, 1000 test nodes
Cora citation network structure with nodes colored by research area
| Model | Parameters | Key Innovation | Convergence | Test Accuracy |
|---|---|---|---|---|
| GCN | 46,119 | Spectral graph convolution | 90 epochs | 79.3% |
| GraphSAGE | 92,199 | Sampling and aggregation | 187 epochs | 76.8% |
| GAT | 369,429 | Multi-head attention | 46 epochs | 81.9% |
Loss and accuracy curves showing training progression for all three models
- GAT: Fastest convergence (46 epochs) with highest final accuracy
- GCN: Steady, reliable convergence with good performance
- GraphSAGE: Slower start but strong final performance, took longest to converge
t-SNE visualization of learned node embeddings showing class separation quality
The embeddings visualization reveals:
- GAT: Best class separation with clear clustering
- GCN: Good separation with some overlap
- GraphSAGE: Decent clustering with more mixed regions
# Clone the repository
git clone https://github.com/GruheshKurra/GraphNeuralNetworks-GNN-.git
cd GraphNeuralNetworks-GNN-
# Install dependencies
pip install torch torchvision torchaudio
pip install torch-geometric torch-scatter torch-sparse torch-cluster torch-spline-conv
pip install matplotlib seaborn pandas numpy scikit-learn networkx# The code automatically detects and uses MPS acceleration
pip install torch torchvision torchaudio
pip install torch-geometric torch-scatter torch-sparse torch-cluster torch-spline-conv
pip install matplotlib seaborn pandas numpy scikit-learn networkx!pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
!pip install torch-geometric torch-scatter torch-sparse torch-cluster torch-spline-conv -f https://data.pyg.org/whl/torch-2.0.0+cpu.html
!pip install matplotlib seaborn pandas numpy scikit-learn networkxpython gnn_comparison.py- π― Three GNN Architectures: Complete implementations of GCN, GraphSAGE, and GAT
- π Comprehensive Evaluation: Accuracy, precision, recall, F1-score, confusion matrices
- π Visualization: Training curves, t-SNE embeddings, network structure
- π‘οΈ Robust Training: Early stopping, model checkpointing, cross-platform compatibility
- π Detailed Logging: Complete training logs instead of code comments
- πΎ Artifact Saving: Models, results, and visualizations saved automatically
βββ gnn_comparison.py # Main training script
βββ best_*_model.pth # Best model checkpoints
βββ *_full_model.pkl # Complete model objects
βββ training_curves.png # Loss and accuracy visualizations
βββ embeddings_tsne.png # t-SNE embedding visualizations
βββ graph_visualization.png # Network structure visualization
βββ results_summary.json # Comprehensive metrics
βββ gnn_training.log # Complete training logs
βββ README.md # This file
-
Graph Convolutional Networks (GCN)
- Spectral approach to graph convolutions
- Simple and effective baseline
- Fast convergence with good performance
-
GraphSAGE (Sample and Aggregate)
- Sampling-based approach for scalability
- Inductive learning capability
- Handles large graphs efficiently
-
Graph Attention Networks (GAT)
- Multi-head attention mechanism
- Dynamic neighbor weighting
- Best performance but highest complexity
config = {
'hidden_dim': 32, # Compact representation
'num_layers': 2, # Avoids over-smoothing
'dropout': 0.5, # Strong regularization
'learning_rate': 0.001, # Conservative learning
'weight_decay': 5e-4, # L2 regularization
'epochs': 200, # Maximum training
'patience': 20, # Early stopping
'attention_heads': 8 # Multi-head attention (GAT)
}| Model | Test Acc | Precision | Recall | F1-Score | Parameters |
|---|---|---|---|---|---|
| GCN | 79.3% | 0.791 | 0.793 | 0.792 | 46K |
| GraphSAGE | 76.8% | 0.765 | 0.768 | 0.766 | 92K |
| GAT | 81.9% | 0.819 | 0.819 | 0.819 | 369K |
- GAT's Superior Performance: Attention mechanism provides significant advantage
- Efficiency vs Performance: GCN offers good performance with fewer parameters
- Convergence Speed: GAT converges fastest despite higher complexity
- Regularization Impact: Strong dropout (0.5) crucial for small training set
The project automatically generates comprehensive visualizations:
Shows the Cora citation network with:
- Nodes colored by research area (7 classes)
- Spring layout for optimal visualization
- Clear community structure visible
Displays for each model:
- Loss curves: Training and validation loss progression
- Accuracy curves: Training and validation accuracy trends
- Overfitting analysis: Gap between train/validation performance
t-SNE visualization showing:
- Class separation: How well models distinguish between research areas
- Embedding quality: Clustering strength in learned representations
- Model comparison: Visual comparison of representation learning
- Apple Silicon MPS: Automatic detection and acceleration
- NVIDIA CUDA: GPU acceleration support
- CPU Fallback: Universal compatibility
- Early stopping to prevent overfitting
- Model checkpointing for reproducibility
- Comprehensive logging for debugging
- Cross-platform compatibility
- Memory-efficient implementations
This implementation demonstrates:
-
Graph Neural Network Fundamentals
- Message passing framework
- Neighborhood aggregation
- Semi-supervised node classification
-
Architecture Comparison
- Spectral vs spatial approaches
- Attention mechanisms in graphs
- Scalability considerations
-
Best Practices
- Hyperparameter selection for graphs
- Regularization techniques
- Evaluation methodologies
All experiments are fully reproducible:
- Fixed random seeds for consistent results
- Complete configuration saved in
results_summary.json - Model checkpoints saved at best validation performance
- Comprehensive logging of all training steps
Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.
git clone https://github.com/GruheshKurra/GraphNeuralNetworks-GNN-.git
cd GraphNeuralNetworks-GNN-
pip install -r requirements.txtThis project is licensed under the MIT License - see the LICENSE file for details.
- PyTorch Geometric team for excellent graph learning library
- Cora Dataset creators for benchmark citation network
- Graph Neural Network researchers for foundational work
For questions or collaborations, please open an issue or reach out through GitHub.
β If you find this project helpful, please consider giving it a star! β