Skip to content

evals(community): OsaurusAI/gemma-4-E2B-it-8bit on Apple M5 Max - #2511

Open
jax-0n-git wants to merge 1 commit into
osaurus-ai:mainfrom
jax-0n-git:evals/compat-apple-m5-max-OsaurusAI-gemma-4-E2B-it-8bit-20260827
Open

evals(community): OsaurusAI/gemma-4-E2B-it-8bit on Apple M5 Max#2511
jax-0n-git wants to merge 1 commit into
osaurus-ai:mainfrom
jax-0n-git:evals/compat-apple-m5-max-OsaurusAI-gemma-4-E2B-it-8bit-20260827

Conversation

@jax-0n-git

Copy link
Copy Markdown
Contributor

Crowdsourced model-compatibility run (see COMMUNITY_EVALS.md).

One contribution file; no other changes.

Environment: chip=Apple M5 Max · RAM 128GB · osVersion=26.6.2 · commit=f4b6ffa · judge=self-judge · catalogHash=f2f4dab21df21cfe · contributor=jax-0n-git

@jjang-ai

Copy link
Copy Markdown
Contributor

Cross-family audit: this E2B contribution ran only 17 AgentLoop rows and scored 0/17, with action coverage 0 and one model step; it is not comparable to the full-catalog Laguna/Qwen/DSV4 runs and cannot establish a Gemma model-quality verdict. It predates #2505/#2528, and the public aggregate omits exits, parser evidence, and cache counters. PR #2546 adds privacy-safe exit/recovery/cache taxonomy. Required follow-up: same current catalog and pin, explicit load proof, bundle generation config, tool schema received, tool call → result → continuation, 10-turn/Stop/restart, tok/s/TTFT/footprint, and visible Release-app confirmation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants