Skip to content

Add cherry blossoms experiment report (CC vs Codex)#1039

Open
sonichi wants to merge 1 commit intomassgen:mainfrom
sonichi:add-cherry-blossoms-experiment
Open

Add cherry blossoms experiment report (CC vs Codex)#1039
sonichi wants to merge 1 commit intomassgen:mainfrom
sonichi:add-cherry-blossoms-experiment

Conversation

@sonichi
Copy link
Copy Markdown
Collaborator

@sonichi sonichi commented Apr 1, 2026

Summary

  • Side-by-side experiment report: Claude Code (Sonnet 4.6) vs Codex (GPT-5.4) generating a 1200x600 cherry blossom SVG
  • CC won 2-0 — both agents voted for CC after finding Codex's SVG had unresolved template placeholders
  • Interactive HTML report with SVG comparison, coordination timeline, vote reasoning, and stats
  • Placed in docs/presentation/experiments/cherry-blossoms/

Files

  • index.html — experiment report page (dark theme, inline CSS)
  • cc-initial.svg — Claude Code's initial output
  • codex-initial.svg — Codex's initial output
  • final.svg — winning output (CC)

Stats

  • Cost: $4.95
  • Tokens: 3.8M input, 102K output
  • Rounds: 6 total (CC: 3, Codex: 2, + final)
  • Config: separate cwd per agent, no --cwd-context rw

Live preview

Also hosted at: https://sutando.ai/massgen-demo/

🤖 Generated with Claude Code

Summary by CodeRabbit

Release Notes

  • Documentation
    • Added comprehensive experiment documentation for the Cherry Blossoms SVG project, featuring detailed reports with side-by-side SVG comparisons, agent performance metrics, vote results analysis, generation timeline, statistical data, and experiment findings.

Side-by-side comparison of Claude Code (Sonnet 4.6) vs Codex (GPT-5.4)
generating a 1200x600 cherry blossom SVG illustration.

CC won 2-0 — both agents voted for CC after Codex's SVG had unresolved
template placeholders in blossom elements.

Includes: coordination timeline, vote reasoning, stats ($4.95 cost,
3.8M input tokens, 6 rounds).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Apr 1, 2026

📝 Walkthrough

Walkthrough

A new static HTML documentation page was added to report on the "MassGen Experiment: Cherry Blossoms SVG," featuring experiment details, agent comparison cards, SVG visualizations, coordination timeline, vote results, and statistics with inline CSS styling.

Changes

Cohort / File(s) Summary
Documentation Report
docs/presentation/experiments/cherry-blossoms/index.html
New static HTML page documenting an SVG generation experiment with styled sections for prompt details, agent comparisons, embedded SVG outputs, event timeline, vote results, final winner display, and metrics grid.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Description check ❓ Inconclusive The PR description provides a clear summary and context but does not follow the required template structure with type prefix, checklist items, or pre-commit status. Add the required docs: type prefix to the title and include the template sections: Type of change checkbox, Checklist, Pre-commit status, and How to Test.
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: adding an experiment report document comparing Claude Code vs Codex for cherry blossom SVG generation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Documentation Updated ✅ Passed PR adds a static HTML experiment documentation page without introducing code changes, new features, breaking changes, or undocumented functionality that would require additional documentation updates.
Capabilities Registry Check ✅ Passed Custom check conditionally triggered only when backend or model changes detected. PR contains only documentation files (HTML/SVG), no Python backend modifications or model changes.
Config Parameter Sync ✅ Passed This custom check is not applicable to the pull request. The PR exclusively adds static documentation files (HTML and SVG assets) to docs/presentation/experiments/cherry-blossoms/ without modifying any YAML configuration files or Python code. The custom check specifically states "If new YAML parameters are added," which is a conditional requirement that is not triggered by this PR.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@docs/presentation/experiments/cherry-blossoms/index.html`:
- Around line 107-112: Replace the unsafe <object> embeds for generated SVG
previews with inert <img> tags: locate the <object data="cc-initial.svg"
type="image/svg+xml"> and <object data="codex-initial.svg" type="image/svg+xml">
(and the other occurrence at the later SVG embed) and change them to <img
src="..."> elements preserving the surrounding container classes/attributes and
adding meaningful alt text (e.g., "CC SVG preview", "Codex SVG preview") so the
SVG is rendered as a static image with no active script surface.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 367ca19a-af62-4fc8-a7e7-00299e784481

📥 Commits

Reviewing files that changed from the base of the PR and between 3353fb7 and fecf654.

⛔ Files ignored due to path filters (3)
  • docs/presentation/experiments/cherry-blossoms/cc-initial.svg is excluded by !**/*.svg, !**/*.svg
  • docs/presentation/experiments/cherry-blossoms/codex-initial.svg is excluded by !**/*.svg, !**/*.svg
  • docs/presentation/experiments/cherry-blossoms/final.svg is excluded by !**/*.svg, !**/*.svg
📒 Files selected for processing (1)
  • docs/presentation/experiments/cherry-blossoms/index.html

Comment on lines +107 to +112
<object data="cc-initial.svg" type="image/svg+xml">CC SVG</object>
</div>
<div class="svg-panel">
<h3 class="agent-name codex" style="padding-left:1rem">Codex</h3>
<object data="codex-initial.svg" type="image/svg+xml">Codex SVG</object>
</div>
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Use <img> instead of <object> for generated SVG previews.

Line 107, Line 111, and Line 184 embed model-generated SVG using <object>, which can expose active SVG content risks. For static report rendering, use <img> to reduce scriptable surface.

Suggested hardening diff
-    <object data="cc-initial.svg" type="image/svg+xml">CC SVG</object>
+    <img src="cc-initial.svg" alt="Claude Code initial cherry blossom SVG output" loading="lazy">

-    <object data="codex-initial.svg" type="image/svg+xml">Codex SVG</object>
+    <img src="codex-initial.svg" alt="Codex initial cherry blossom SVG output" loading="lazy">

-  <object data="final.svg" type="image/svg+xml" style="width:100%; height:400px; background:white;">Final SVG</object>
+  <img src="final.svg" alt="Final winning cherry blossom SVG from Claude Code" style="width:100%; height:400px; background:white;" loading="lazy">

Also applies to: 184-184

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/presentation/experiments/cherry-blossoms/index.html` around lines 107 -
112, Replace the unsafe <object> embeds for generated SVG previews with inert
<img> tags: locate the <object data="cc-initial.svg" type="image/svg+xml"> and
<object data="codex-initial.svg" type="image/svg+xml"> (and the other occurrence
at the later SVG embed) and change them to <img src="..."> elements preserving
the surrounding container classes/attributes and adding meaningful alt text
(e.g., "CC SVG preview", "Codex SVG preview") so the SVG is rendered as a static
image with no active script surface.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant