Skip to content

Commit 94efb12

Browse files
committed
lightmetal.model optional override; lightmetal owns model config
1 parent 70425da commit 94efb12

3 files changed

Lines changed: 14 additions & 6 deletions

File tree

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,10 +145,12 @@ LM Studio (default port 1234), llama.cpp `--api`, and vLLM expose the same Chat
145145
[LightMetal](https://github.com/AdamBien/lightmetal) is a Java 25 GGUF runner that talks to Apple Silicon's Metal via the Foreign Function & Memory API. zsmith reaches it via the `UnaryOperator<String>` SPI (`lm.generation.boundary.LightMetalChat`), so the only compile-time dependency is `java.base` — drop `lightmetal.jar` on the classpath at runtime and the provider is **auto-selected**, overruling `llm.provider` whatever it is set to. The classpath is the explicit signal; no extra config is needed. The GGUF is loaded once on the first call and reused for every subsequent turn.
146146

147147
```properties
148-
lightmetal.model=/abs/path/to/model.gguf # requiredGGUF file
148+
lightmetal.model=/abs/path/to/model.gguf # optionaloverrides lightmetal's own config
149149
lightmetal.max.tokens=4096 # optional — default 4096
150150
```
151151

152+
`lightmetal.model` is **optional** in zsmith. When unset, zsmith omits `model` from the request payload entirely — lightmetal then sources it from its own eager-loaded `~/.lightmetal/app.properties` (or `-Dmodel=...`). So a user who already runs `lmprompt`/`lmserve` against a configured `~/.lightmetal/app.properties` needs zero zsmith-side model config. Set `lightmetal.model` in zsmith only when you want one agent to override the lightmetal-wide default.
153+
152154
LightMetal natively understands Anthropic-shaped `tools` and emits `tool_use` content blocks, so the Agent loop works the same as with Claude. Run agent scripts with `--enable-native-access=ALL-UNNAMED` so the FFM call into `libllama.dylib` is allowed:
153155

154156
```java

zsmith/src/main/java/airhacks/zsmith/lightmetal/control/LightMetal.java

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
package airhacks.zsmith.lightmetal.control;
22

3+
import java.util.Optional;
34
import java.util.ServiceLoader;
45
import java.util.function.UnaryOperator;
56

@@ -16,8 +17,9 @@ public interface LightMetal {
1617
String MAX_TOKENS_PROPERTY = "lightmetal.max.tokens";
1718
int DEFAULT_MAX_TOKENS = 4096;
1819

19-
static String modelPath() {
20-
return ZCfg.requiredString(MODEL_PROPERTY);
20+
static Optional<String> modelOverride() {
21+
var value = ZCfg.string(MODEL_PROPERTY);
22+
return (value == null || value.isBlank()) ? Optional.empty() : Optional.of(value);
2123
}
2224

2325
static int maxTokens() {
@@ -36,7 +38,7 @@ static JSONObject invoke(String system, JSONArray messages, JSONArray tools, flo
3638

3739
var event = new LightMetalAPICallEvent();
3840
event.begin();
39-
event.model = payload.getString("model");
41+
event.model = payload.optString("model", "(from lightmetal config)");
4042
Log.agent("invoking lightmetal model: " + event.model);
4143

4244
var chat = ChatHolder.lookup();
@@ -47,6 +49,7 @@ static JSONObject invoke(String system, JSONArray messages, JSONArray tools, flo
4749
}
4850
var answer = chat.apply(payloadString);
4951
var response = new JSONObject(answer);
52+
event.model = response.optString("model", event.model);
5053
populateUsage(event, response);
5154
logTokens(event);
5255
if (event.shouldCommit()) {
@@ -65,11 +68,14 @@ static JSONObject invoke(String system, String user, float temperature) {
6568

6669
static JSONObject anthropicPayload(String system, JSONArray messages, JSONArray tools, float temperature) {
6770
var payload = new JSONObject()
68-
.put("model", modelPath())
6971
.put("system", system == null ? "" : system)
7072
.put("messages", messages)
7173
.put("max_tokens", maxTokens())
7274
.put("temperature", temperature);
75+
// Omit `model` in the default case so lightmetal owns it via its own ZCfg.
76+
// Only include it when zsmith is explicitly overriding (e.g. running a
77+
// different GGUF for one agent without touching ~/.lightmetal/app.properties).
78+
modelOverride().ifPresent(override -> payload.put("model", override));
7379
if (tools != null && !tools.isEmpty()) {
7480
payload.put("tools", tools);
7581
}

zsmith/version.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
2026.06.12.02
1+
2026.06.12.04

0 commit comments

Comments
 (0)