1.7.3 Working on OSX Mac Metal?

[rpattcorner]

Using an install that followed the form:

cargo install shimmy --features apple --version 1.7.3

I see issues from the sample query:

• The published form has issues with the jq parsing

% curl -s http://127.0.0.1:11435/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
        "model":"llama-3.2-3b-instruct-q8-0",
        "messages":[{"role":"user","content":"Say hi in 5 words."}],
        "max_tokens":32
      }' | jq -r '.choices[0].message.content'
jq: parse error: Invalid numeric literal at line 1, column 5

• Stripping out the jq magic yields what appear to be debug messages, but no response:

% curl -s http://127.0.0.1:11435/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
        "model":"llama-3.2-3b-instruct-q8-0",
        "messages":[{"role":"user","content":"Say hi in 5 words."}],
        "max_tokens":32
      }' 
                                    
data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":null,"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"MLX ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"generated ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"response ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"for ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"prompt: ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"'<|start_header_id|>user<|end_header_id|> ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"Say ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"hi ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"in...' ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"(max_tokens: ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":"32) ","role":null},"finish_reason":null}]}

data: {"id":"chatcmpl-7c9ee5ca8d1b45359b2d2b506e71346e","object":"chat.completion.chunk","created":1760365737,"model":"llama-3.2-3b-instruct-q8-0","choices":[{"index":0,"delta":{"content":null,"role":null},"finish_reason":"stop"}]}

data: [DONE]