Context
PR #190 fixed a bug where model_uri was missing from the _INSERT_QUERY in loader.py. This class of bug (missing column in the INSERT query) is easy to introduce because there are no unit tests that verify prepare_benchmark_for_insert produces a dict with all the keys referenced by _INSERT_QUERY.
Proposal
Add unit tests for prepare_benchmark_for_insert in tests/unit/ that:
- Assert that calling
prepare_benchmark_for_insert with minimal input includes all expected keys (including model_uri)
- Assert that optional fields default to
None when not provided
- Assert that provided values (like
model_uri) are preserved
- Bonus: Add a structural test that parses
_INSERT_QUERY for %(...)s parameter names and asserts they're all present in the output of prepare_benchmark_for_insert — this would catch any future column additions that forget to update the preparation function
Example
def test_prepare_benchmark_defaults_model_uri_to_none():
bench = {"model_hf_repo": "org/model", "hardware": "A100",
"hardware_count": 1, "prompt_tokens": 512, "output_tokens": 256}
result = prepare_benchmark_for_insert(bench)
assert result["model_uri"] is None
def test_prepare_benchmark_preserves_model_uri():
bench = {"model_hf_repo": "org/model", "hardware": "A100",
"hardware_count": 1, "prompt_tokens": 512, "output_tokens": 256,
"model_uri": "oci://registry.example.com/model:latest"}
result = prepare_benchmark_for_insert(bench)
assert result["model_uri"] == "oci://registry.example.com/model:latest"
def test_all_insert_query_params_present_in_prepared_output():
"""Structural test: every %(key)s in _INSERT_QUERY must appear in prepared output."""
import re
from planner.knowledge_base.loader import _INSERT_QUERY, prepare_benchmark_for_insert
params = set(re.findall(r'%\((\w+)\)s', _INSERT_QUERY))
bench = {"model_hf_repo": "org/model", "hardware": "A100",
"hardware_count": 1, "prompt_tokens": 512, "output_tokens": 256}
result = prepare_benchmark_for_insert(bench)
missing = params - set(result.keys())
assert not missing, f"Keys in _INSERT_QUERY but missing from prepared output: {missing}"
Related
Context
PR #190 fixed a bug where
model_uriwas missing from the_INSERT_QUERYinloader.py. This class of bug (missing column in the INSERT query) is easy to introduce because there are no unit tests that verifyprepare_benchmark_for_insertproduces a dict with all the keys referenced by_INSERT_QUERY.Proposal
Add unit tests for
prepare_benchmark_for_insertintests/unit/that:prepare_benchmark_for_insertwith minimal input includes all expected keys (includingmodel_uri)Nonewhen not providedmodel_uri) are preserved_INSERT_QUERYfor%(...)sparameter names and asserts they're all present in the output ofprepare_benchmark_for_insert— this would catch any future column additions that forget to update the preparation functionExample
Related