Skip to content

test: add unit tests for prepare_benchmark_for_insert field coverage #192

@anfredette

Description

@anfredette

Context

PR #190 fixed a bug where model_uri was missing from the _INSERT_QUERY in loader.py. This class of bug (missing column in the INSERT query) is easy to introduce because there are no unit tests that verify prepare_benchmark_for_insert produces a dict with all the keys referenced by _INSERT_QUERY.

Proposal

Add unit tests for prepare_benchmark_for_insert in tests/unit/ that:

  1. Assert that calling prepare_benchmark_for_insert with minimal input includes all expected keys (including model_uri)
  2. Assert that optional fields default to None when not provided
  3. Assert that provided values (like model_uri) are preserved
  4. Bonus: Add a structural test that parses _INSERT_QUERY for %(...)s parameter names and asserts they're all present in the output of prepare_benchmark_for_insert — this would catch any future column additions that forget to update the preparation function

Example

def test_prepare_benchmark_defaults_model_uri_to_none():
    bench = {"model_hf_repo": "org/model", "hardware": "A100",
             "hardware_count": 1, "prompt_tokens": 512, "output_tokens": 256}
    result = prepare_benchmark_for_insert(bench)
    assert result["model_uri"] is None

def test_prepare_benchmark_preserves_model_uri():
    bench = {"model_hf_repo": "org/model", "hardware": "A100",
             "hardware_count": 1, "prompt_tokens": 512, "output_tokens": 256,
             "model_uri": "oci://registry.example.com/model:latest"}
    result = prepare_benchmark_for_insert(bench)
    assert result["model_uri"] == "oci://registry.example.com/model:latest"

def test_all_insert_query_params_present_in_prepared_output():
    """Structural test: every %(key)s in _INSERT_QUERY must appear in prepared output."""
    import re
    from planner.knowledge_base.loader import _INSERT_QUERY, prepare_benchmark_for_insert
    
    params = set(re.findall(r'%\((\w+)\)s', _INSERT_QUERY))
    bench = {"model_hf_repo": "org/model", "hardware": "A100",
             "hardware_count": 1, "prompt_tokens": 512, "output_tokens": 256}
    result = prepare_benchmark_for_insert(bench)
    missing = params - set(result.keys())
    assert not missing, f"Keys in _INSERT_QUERY but missing from prepared output: {missing}"

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions