Skip to content

evals(community): OsaurusAI/ZAYA1-8B-MXFP4 on Apple M5 Max (partial, aborted) - #2558

Open
jax-0n-git wants to merge 1 commit into
osaurus-ai:mainfrom
jax-0n-git:evals/compat-apple-m5-max-OsaurusAI-ZAYA1-8B-MXFP4-20260830
Open

evals(community): OsaurusAI/ZAYA1-8B-MXFP4 on Apple M5 Max (partial, aborted)#2558
jax-0n-git wants to merge 1 commit into
osaurus-ai:mainfrom
jax-0n-git:evals/compat-apple-m5-max-OsaurusAI-ZAYA1-8B-MXFP4-20260830

Conversation

@jax-0n-git

Copy link
Copy Markdown
Contributor

Crowdsourced model-compatibility run (see COMMUNITY_EVALS.md).

Partial / aborted — not a full matrix. Do not treat 36/95 as a comparable leaderboard cell.

Why it was aborted

  • Wall clock ~10.7 h on Apple M5 Max / 128GB before abort (contributor jax-0n-git).
  • Initial 14-suite batch wrote almost nothing; loop fell back to serial per-suite retries.
  • agentdb.queryable-saved-view hit the 1800s watchdog after a single step dumped ~47k reasoning chars and no tool call (stop=unknown).
  • Model kept unbounded reasoning at ~14 tok/s; DefaultAgent was still on 24/49 after ~1.4 h on that suite alone with CPU ~120% (not a 0% hang).
  • Remaining suites not run: DefaultAgent (incomplete), HTTPAPI, Memory, MicroPerf, PromptInjection, ReasoningChannel, Subagent, SandboxFrontier.

What is in this file

Completed suite JSONs only: AgentDB (watchdog-terminated), AgentLoop, AgentLoopFrontier, AppleScript, CacheProof, CapabilityClaims, ComputerUseLoop. Aggregated via osaurus-evals matrix + compat --validate. Self-judge.

Environment: Apple M5 Max · RAM 128GB · see contribution file.

One contribution file; no other changes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant