Merge pull request #3 from Klus3kk/main

pulling changes from main

Author: Klus3kk

__init__.py .github/workflows/benchmark.yml__init__.py renamed to .github/workflows/benchmark.yml

@@ -2,46 +2,43 @@ name: CI
 
 on:
   push:
-    branches: [ main, master, dev ]
+    branches: [main, develop]
   pull_request:
-    branches: [ main, master ]
+    branches: [main]
 
 jobs:
-  test:
+  lint:
     runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.10'
+      - name: Install dependencies
+        run: |
+          pip install pre-commit
+          pre-commit install
+      - name: Run pre-commit
+        run: pre-commit run --all-files
+
+  test:
+    needs: lint
+    runs-on: ${{ matrix.os }}
     strategy:
       matrix:
-        python-version: [3.8, 3.9, '3.10']
-
+        os: [ubuntu-latest, macos-latest, windows-latest]
+        python-version: ['3.8', '3.9', '3.10', '3.11']
+    
     steps:
-    - uses: actions/checkout@v3
-    - name: Set up Python ${{ matrix.python-version }}
-      uses: actions/setup-python@v4
-      with:
-        python-version: ${{ matrix.python-version }}
-    - name: Install dependencies
-      run: |
-        python -m pip install --upgrade pip
-        if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
-        pip install pytest pytest-cov flake8 black
-        pip install -e .
-    - name: Format with black
-      run: |
-        # Format the code with black before checking
-        black .
-        # Check if any changes were made
-        git diff --exit-code || { echo "Code was not properly formatted before commit. Black has fixed the formatting issues."; exit 0; }
-    - name: Lint with flake8
-      run: |
-        # stop the build if there are Python syntax errors or undefined names
-        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
-        # exit-zero treats all errors as warnings
-        flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
-    - name: Test with pytest
-      run: |
-        pytest --cov=./ --cov-report=xml
-    - name: Upload coverage to Codecov
-      uses: codecov/codecov-action@v3
-      with:
-        file: ./coverage.xml
-        fail_ci_if_error: false
+      - uses: actions/checkout@v3
+      - uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+      - name: Install dependencies
+        run: |
+          pip install -e ".[dev]"
+      - name: Run tests
+        run: |
+          pytest tests/ --cov=fit --cov-report=xml
+      - name: Upload coverage
+        uses: codecov/codecov-action@v3

.github/workflows/ci.yml

@@ -1,6 +1,9 @@
 test.py
-*/__pycache__
+__pycache__/
 logs
-fit.egg-info
 .venv
 .idea
+venv
+mnist_model.pkl
+*.png
+fit.egg-info

monitor/logger.py .github/workflows/docs.ymlmonitor/logger.py renamed to .github/workflows/docs.yml

@@ -9,21 +9,21 @@ repos:
     -   id: debug-statements
     -   id: check-merge-conflict
 
-#-   repo: https://github.com/psf/black
-#    rev: 23.3.0
-#    hooks:
-#    -   id: black
-#        args: [--line-length=100]
+-   repo: https://github.com/psf/black
+    rev: 23.3.0
+    hooks:
+    -   id: black
+        args: [--line-length=80]
 
-#-   repo: https://github.com/pycqa/flake8
-#    rev: 6.0.0
-#    hooks:
-#    -   id: flake8
-#        args: [--max-line-length=100]
-#        additional_dependencies: [flake8-docstrings]
+-   repo: https://github.com/pycqa/flake8
+    rev: 6.0.0
+    hooks:
+    -   id: flake8
+        args: [--max-line-length=80]
+        additional_dependencies: [flake8-docstrings]
 
-#-   repo: https://github.com/pycqa/isort
-#    rev: 5.12.0
-#    hooks:
-#    -   id: isort
-#        args: [--profile=black]
+-   repo: https://github.com/pycqa/isort
+    rev: 5.12.0
+    hooks:
+    -   id: isort
+        args: [--profile=black]

monitor/profiler.py .github/workflows/release.ymlmonitor/profiler.py renamed to .github/workflows/release.yml

@@ -0,0 +1,107 @@
+# Changelog
+
+## Why This Major Restructuring?
+
+When I started developing FIT framework I wanted it to be as an educational project to understand deep learning from first principles. But after some initial development, I was coming up with more and more ideas, so tgat the project would be useful for the public usage:
+
+1. **Scale Beyond Educational Use**: Transform from a learning project into a production-ready framework
+2. **Meet Professional Standards**: Align with industry best practices for ML frameworks
+3. **Ensure Long-term Viability**: Build a sustainable project architecture
+
+### Complete Project Restructuring
+
+#### **Previous Issues**
+- **Flat Structure**: Mixed organization with some modules in root, others scattered
+- **Unclear Imports**: Inconsistent import paths (`from train.optim import Adam` vs `from nn.linear import Linear`)
+- **Limited Testing**: Basic test coverage without proper organization
+- **No CI/CD**: Manual testing and deployment
+- **Minimal Documentation**: Lack of comprehensive docs and examples
+- **Code Quality**: Inconsistent style, missing type hints, print-based debugging
+
+### Added
+
+#### **Core Improvements**
+- **Exception Hierarchy**: Custom exceptions for better error handling
+  - `TensorError`, `ShapeError`, `AutogradError` for specific error types
+  - Detailed error messages with context
+- **Logging System**: Professional logging replacing print statements
+  - Colored console output
+  - File logging support
+  - Performance tracking
+- **Configuration Management**: Centralized configuration system
+  - YAML config support
+  - Environment variable integration
+  - Validation and type checking
+- **Type Annotations**: Full type hints throughout the codebase
+- **Memory Management**: Tensor caching and memory profiling
+
+#### **Development Infrastructure**
+- **CI/CD Pipeline**: GitHub Actions for automated testing
+  - Multi-OS testing (Ubuntu, macOS, Windows)
+  - Multiple Python versions (3.8-3.11)
+  - Automated benchmarking
+  - Coverage reporting
+- **Code Quality Tools**:
+  - Pre-commit hooks (black, isort, flake8, mypy)
+  - Automated code formatting
+  - Style consistency enforcement
+- **Documentation System**:
+  - Sphinx-based API documentation
+  - Tutorials and guides
+  - Example gallery
+
+#### **New Features**
+- **Enhanced Optimizers**:
+  - SAM (Sharpness-Aware Minimization) with adaptive mode
+  - Lion optimizer with memory efficiency
+  - Learning rate schedulers
+- **Advanced Layers**:
+  - Spectral Normalization
+  - Neural ODEs with multiple solvers
+  - Attention mechanisms (planned)
+- **Improved Data Pipeline**:
+  - Better DataLoader with sampling strategies
+  - Transform pipeline
+  - Memory-efficient data handling
+
+### Changed
+
+#### **Module Organization**
+- **Before**: `from train.optim import Adam`
+- **After**: `from fit.optim.adam import Adam`
+- Clear separation between stable and experimental features
+- Consistent import paths
+
+#### **API Improvements**
+- Tensor operations now support method chaining
+- Consistent parameter names across modules
+- Better default values based on best practices
+
+#### **Error Messages**
+- **Before**: `ValueError: shapes not aligned`
+- **After**: `ShapeError: Cannot multiply tensors with shapes (2, 3) and (4, 5). Expected shapes to be compatible for matrix multiplication.`
+
+### Fixed
+- Memory leaks in autograd graph
+- Numerical instability in softmax computation
+- Broadcasting issues in tensor operations
+- Gradient accumulation bugs
+- Thread safety issues in data loading
+
+### Testing
+- Unit tests for all core functionality
+- Integration tests for common workflows
+- Performance benchmarks
+- Gradient checking tests
+
+### Breaking Changes
+- Import paths have changed 
+- Some internal APIs renamed for consistency
+- Removed deprecated functions
+- Default dtype changed from float64 to float32
+
+### Future Plans
+- GPU support via CuPy 
+- Distributed training 
+- JIT compilation 
+- Model deployment utilities