Skip to content

CHANGELOG.md

Latest commit

 

History

History
134 lines (104 loc) · 6.49 KB

CHANGELOG.md

File metadata and controls

134 lines (104 loc) · 6.49 KB
Raw

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[1.0.3.3] - 2026-07-20

Added

  • FSDP (Fully Sharded Data Parallel) trainer for distributed training across multiple GPUs with model sharding
  • FSDP1vsSampleDataset for FSDP-compatible 1-vs-sample scoring technique
  • New embedding models in dicee/models/real.py:
    • TransH: translation-based embedding on relation-specific hyperplanes (addresses TransE's limitations on 1-N/N-1/N-N relations)
    • MuRE: multi-relational embedding with relation-specific diagonal scaling, translation, and per-entity biases
  • Claude Code integration: project memory, specialized subagents, and skills for model development and training workflows
  • Deterministic unit test for Parquet reading regression (test_unit_read_parquet.py)
  • Comprehensive unit test suite for KGE inference API validation (test_unit_kge_inference.py):
    • 32 unit tests covering input validation, device management, entity embedding extraction
    • Tests validate exact exception messages for predict_topk() and predict() methods
    • Validation tests verify conditional branching logic for missing head/relation/tail predictions
    • Addresses IMPROVEMENTS.md #3 ("Core modules lack dedicated unit tests")
  • Type hint improvements and gradual mypy enforcement roadmap (IMPROVEMENTS.md #5):
    • Fixed Optional type annotations in dicee/config.py (11 fields now properly annotated as Optional[T])
    • Created docs/TYPING_ROADMAP.md documenting 3-tier type hint enforcement strategy
    • CI matrix now tests Python 3.11, 3.12, and 3.13 (IMPROVEMENTS.md #6)
  • TODO/FIXME backlog organization and categorization (IMPROVEMENTS.md #7):
    • Created docs/TODO_BACKLOG.md cataloging the 42 TODO/FIXME markers in dicee/ by priority
    • Categorized by impact: 6 Medium-priority items (performance/design), remaining Low-priority (refactoring)

Changed

  • Breaking change: Converted assert statements guarding user input at public API boundaries to explicit exceptions (ValueError, TypeError):
    • dicee/knowledge_graph_embeddings.py (__init__, predict, predict_topk, to, answer_multi_hop_query, find_missing_triples, predict_literals)
    • dicee/read_preprocess_save_load_kg/preprocess.py (all preprocessing-pipeline validation)
    • User-facing validation now works in Python optimized mode (-O), where assert is stripped
    • Internal tensor-shape/invariant checks (not user input) intentionally remain as assert — ~246 remain across dicee/ by design
  • Breaking change: Migrated print() calls to Python's logging module across trainers, models, evaluation code, and core utilities:
    • Per-module logging.getLogger(__name__) lets consumers control verbosity, silence output, or redirect logs without monkeypatching stdout
  • Improved TP (Tensor Parallel) trainer error message when insufficient GPUs are available
  • Updated README with FSDP trainer documentation

Fixed

  • #410: --read_only_few now efficiently loads only the requested rows from Parquet files
    • Previously: full Parquet file was loaded into memory, then .head(n) was applied (~26.9 MB peak for 200k rows requesting 10 rows)
    • Now: uses pyarrow.parquet.ParquetFile.iter_batches() to stop loading row groups early (~0.2 MB peak, ~135x improvement)
    • CSV/text inputs already used nrows= and were unaffected
    • Added regression test: test_unit_read_parquet.py with 6 deterministic tests covering row count, content, and order validation
  • Fixed TexLive latest version compatibility in FSDP trainer documentation
  • Fixed trainer model type consistency and precision handling in FSDP trainer

Deprecated

  • Direct use of print() in dicee modules — use logging module instead

Removed

  • (none in this release)

Security

  • (none in this release)

Known Issues (from IMPROVEMENTS.md)

The following issues are being tracked for future resolution:

Priority: Medium

  • Core modules lack dedicated unit tests (#3) — Partially Resolved
    • ✅ dicee/knowledge_graph_embeddings.py now has comprehensive unit test suite (32 tests in test_unit_kge_inference.py)
    • ⏳ executer.py, static_funcs.py, query_generator.py, knowledge_graph.py still only exercised indirectly through integration tests
    • Scoring-logic bugs in predict() and predict_topk() now caught by unit tests

Priority: Low

  • mypy is non-blocking in CI (#5)

    • .github/workflows/github-actions-python-package.yml runs with continue-on-error: true
    • 1358 errors currently reported (non-fatal); no mechanism yet to enforce gradual tightening — see docs/TYPING_ROADMAP.md

  • 42 accumulated TODO/FIXME markers, cataloged but not resolved (#7)

    • Several flag uncertainty about existing logic rather than pending work
    • Examples: abstracts.py:721, knowledge_graph.py:132, query_generator.py:226
    • Cataloged by priority in docs/TODO_BACKLOG.md; still needs original authors' input to actually resolve

Migration Guide

For users updating to this release

1. Exception handling instead of asserts

If you were relying on try/except AssertionError to catch KGE validation errors, update to catch the specific exception type:

# Old (may fail in optimized Python)
try:
    model = KGE(path="...")
except AssertionError as e:
    handle_error(e)

# New (recommended)
try:
    model = KGE(path="...")
except (ValueError, TypeError) as e:
    handle_error(e)

2. Capturing logs from dicee

If you were suppressing dicee output via monkeypatching print(), use Python's logging instead:

import logging

# Suppress dicee logs
logging.getLogger("dicee").setLevel(logging.WARNING)

# Or silence all dicee modules
for module in ["dicee.trainer", "dicee.models", "dicee.evaluation"]:
    logging.getLogger(module).setLevel(logging.ERROR)

Changelog Status

✅ Items marked DONE in IMPROVEMENTS.md:

  • Item #1: --read_only_few Parquet loading
  • Item #2: Assert validation → exceptions
  • Item #4: print() → logging

📋 Items still TODO:

  • Item #3: Core user-facing modules unit test coverage
  • Item #5: mypy blocking CI with ratchet
  • Item #6: Multi-version Python CI matrix
  • Item #7: TODO/FIXME documentation pass