Gate ZAYA1-VL tools on the actual template contract - #2540
Conversation
|
Current live gate result — keep the fail-closed policy ( I tested the tempting relaxation against the exact published bundle and rejected it. Paired integration under test:
The lower engine boundary passes a strict one-tool request: valid ZAYA XML, one structured
Therefore a valid stamped runtime fallback is not sufficient proof of harness tool capability. I reverted the relaxation in |
Summary
supports_tools=falsebefore treating a parser name as tool capabilityzaya1_vlbundles whose JANG metadata accidentally inheritedsupports_tools=true/zaya_xmlBefore / causal evidence
The installed
JANGQ-AI/ZAYA1-VL-8B-JANGTQ4bundle currently has:config.json:model_type=zaya1_vljang_config.json:supports_tools=true,tool_parser=zaya_xmlchat_template.jsonandtokenizer_config.json: notoolsinput and notool_callemission contractThe matched 3-trial AgentLoop run produced 0/3 valid
spawn_batchcalls. A bounded raw trace on the native upstream-shaped schema selectedspawn_batchbut emitted schema declarations in place of values and omitted requiredtarget/input; the parser correctly rejected it. This is not safely repairable with an argument-parser heuristic.Artifacts from that reproduction:
/private/tmp/zaya-pr336-spawn-batch-repeat3.json/private/tmp/zaya-pr336-spawn-batch-repeat3.transcripts//private/tmp/zaya-pr336-native-template-trace.json/private/tmp/zaya-pr336-native-template-trace.transcripts/Future bundle prevention is separately scoped in JANG PR #22. vMLX template truthfulness remains separately scoped in vMLX PR #336.
Fix contract
supports_tools=falseis authoritative even when a parser string is present.zaya1_vl/zaya_vl, a legacy true stamp is insufficient: the actual template must consumetoolsand emittool_callsyntax.Current-head verification
Head:
cf1622679b5b8336cebde328669464d583f41d76MLXServiceRuntimePolicyTests: 18/18 PASS.OSAURUS_ZAYA_VL_TEST_MODEL=/Users/eric/models/JANGQ-AI/ZAYA1-VL-8B-JANGTQ4; the real installed bad-stamp bundle resolves tool support to false.Honest status
PARTIAL / draft. Source and installed-bundle capability resolution are proven. Remaining live gates:
This PR prevents broken tool exposure for current users; it does not claim that the present ZAYA1-VL checkpoint has learned valid tool calling.