Skip to content

Refresh competitor comparisons and docs from current sources - #9

Merged
BeLazy167 merged 1 commit into
mainfrom
docs/compare-refresh-pub
Sep 21, 2026
Merged

BeLazy167 merged 1 commit into
mainfrom
docs/compare-refresh-pub

Conversation

@BeLazy167

@BeLazy167 BeLazy167 commented Sep 21, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Re-verified all 10 comparison entries against official pricing/docs. Plan renames (CodeRabbit Essentials/Team/Advanced $24/$48/$72), credit pricing (Qodo, Greptile, Macroscope), new capabilities flipped: Cubic diagrams/ticketing/static analysis, SonarQube/Gitar memory+ticketing, Copilot ensemble+memory, Semgrep Multimodal BYOK, Codacy Jira tickets. Stale claims removed; all entries dated 2026-09-20.
  • Docs rewritten against codebase truth: real rules implementation, correct rate limits, hosted auto-review default off / self-hosted on, memory threshold defaults, settings paths, diagram gates, comment formats.
  • Bug fixes surfaced by verification: MEMORY_DEFAULTS mirrored stale backend thresholds; ratelimit.go comments listed wrong limits; auto_run toggle claimed "on by default" while hosted treats unset as off.
  • MCP docs intentionally excluded — MCP server is a private-repo feature.

Test plan

  • pnpm lint — no new warnings in touched files
  • pnpm typecheck — clean
  • pnpm build — all 69 pages, all 10 /compare/* routes (verified on private-base tree)
  • Spot-verified CodeRabbit + Qodo pricing against live pages

Summary by cubic

Refreshes competitor comparisons and docs to match current pricing, features, and actual codebase behavior.

Docs

  • Re-verified all 10 /compare entries against official pricing pages; updates plan renames, credit pricing, and new capabilities.
  • Rewrote docs to reflect real implementation: rate limits, auto-review defaults, memory thresholds, settings paths, and diagram gates.

Bug fixes

  • MEMORY_DEFAULTS now matches backend thresholds.
  • ratelimit.go comments now list correct limits.
  • Auto-review toggle now shows the correct default for hosted vs self-hosted.

Written for commit dd7d36f. Summary will update on new commits.

Review in cubic

Re-verified all 10 comparison entries against official pricing/docs:
plan renames (CodeRabbit Essentials/Team/Advanced), credit pricing
(Qodo, Greptile, Macroscope), new capabilities (Cubic diagrams +
ticketing, SonarQube/Gitar memory, Copilot ensemble + memory, Semgrep
Multimodal BYOK, Codacy Jira tickets). Stale claims removed; every
entry re-dated.

Docs rewritten against codebase truth: real rules implementation,
correct rate limits (30/hr repo, 200/day org, 10/hr force), hosted
auto-review default off / self-hosted on, memory threshold defaults,
settings paths, diagram gates, comment formats.

Fixes three verified bugs: MEMORY_DEFAULTS mirrored stale backend
thresholds, ratelimit.go comments listed wrong limits, auto_run
toggle showed "on by default" while hosted treats unset as off.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7 issues found across 11 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="web/src/components/marketing/docs-content.tsx">

<violation number="1" location="web/src/components/marketing/docs-content.tsx:2449">
P2: Auto-resolve invokes the LLM-backed `AddressedJudge` before resolving each proximity candidate, so this description understates both behavior and token spend. Document the judge confirmation and its cost instead of promising a pure-diff, free operation.</violation>
</file>

<file name="web/src/app/docs/layout.tsx">

<violation number="1" location="web/src/app/docs/layout.tsx:20">
P2: The new `Overview` link points to `/docs`, but the new docs tree (`web/src/app/docs/`) has no `page.tsx` at that path — only `layout.tsx`. The `/docs` URL is still owned by the legacy marketing page `web/src/app/(marketing)/docs/page.tsx`, so clicking Overview leaves the new docs shell and lands on the marketing full-reference page rendered without this sidebar (and without `DocsPageViewedProbe`). Either add `web/src/app/docs/page.tsx` so Overview renders inside this layout, or the link is pointing users at an inconsistent standalone page that contradicts the new nav structure.</violation>
</file>

<file name="web/src/app/(dashboard)/settings/page.tsx">

<violation number="1" location="web/src/app/(dashboard)/settings/page.tsx:922">
P3: On self-hosted installs the Auto-review toggle now renders OFF by default even though reviews still run unconditionally. The new description says "self-hosted defaults to on", but the toggle's displayed value comes only from the stored setting or AUTO_RUN_TOGGLE.defaultValue (line 1465-1467 / 1915-1919), and the single defaultValue=false cannot reflect deployment type. Backend decideAutoRun/AutoRun short-circuits self-hosted to ALWAYS review regardless of stored settings (backend/internal/pipeline/persona.go), so a fresh self-hosted org shows "Auto-review every PR" off in the settings page while the backend keeps auto-running reviews — contradicting the new description. Previously defaultValue=true at least displayed self-hosted's actual behavior. Consider deriving the shown default from a self-hosted flag when the backend exposes one, or narrow the description wording.</violation>
</file>

<file name="web/src/app/docs/features/memory-tuning/page.tsx">

<violation number="1" location="web/src/app/docs/features/memory-tuning/page.tsx:50">
P2: The new Thresholds paragraph claims "An explicit 0 disables that gate entirely," but that is false for scenario_dedupe. The backend clamps a non-positive dedupe threshold to the 0.95 default (backend/internal/pipeline/scenarios.go scenarioDedupeThreshold), because similarity >= 0 is always true and a 0 would otherwise suppress every new scenario — the opposite of disabling the gate. Qualify the statement so a user setting scenario_dedupe to 0 is not misled into expecting dedup off.</violation>
</file>

<file name="web/public/docs.md">

<violation number="1" location="web/public/docs.md:245">
P3: The claim that auto-resolve involves "no LLM call" contradicts the codebase: the auto-resolve path runs an addressed-judge LLM pass that consumes model tokens (`backend/internal/pipeline/auto_resolve_tokens_test.go` records judge prompt/model/cost tokens), and README/CONTEXT both state "stale threads auto-resolve only after LLM judge confirms the change addressed the finding; line proximity alone never resolves" and "Proximity is prefilter only; addressed-judge must confirm fix." So a push that merely changes the anchored lines does not itself close the thread — a judge call is involved and the thread stays open on judge failure. Please reconcile the wording (drop "no LLM call" and the implication that the push alone resolves) with the actual `autoResolveOnSynchronize` behavior.</violation>
</file>

<file name="web/src/app/docs/faq/page.tsx">

<violation number="1" location="web/src/app/docs/faq/page.tsx:68">
P2: The FAQ now states checkbox-triggered reviews run under a 10/hour per-repo force bucket, but backend/internal/api/handlers_webhook.go:424-428 still documents the same dispatch flow as "the tighter `force=true` rate-limit path (3/hr/repo)". Verify the actual admission bucket size before rollout: if the limit is still 3/hr, these docs (and the matching claims in docs-content.tsx) repeat the misstatement this PR set out to fix; if it is 10/hr, the backend comment is stale and should be corrected alongside.</violation>
</file>

<file name="web/src/lib/pseo/competitors.ts">

<violation number="1" location="web/src/lib/pseo/competitors.ts:174">
P3: Greptile's `ticketing` flag was flipped to true while the same entry's strengths and summary say ticket handling is context-only: "Reviews code against linked Jira tickets, Linear issues, and Confluence pages (read-only — no ticket creation)" and "doesn't ... create tickets." Every other entry sets `ticketing: true` only where ticket creation exists (Cubic "creates Jira/Linear tickets", Codacy "creates tickets from findings", SonarQube/Gitar "creates follow-up issues"), while Greptile explicitly does not create tickets. If the flag means ticket creation/integration uniformly, this contradicts the copy; if it now means context-only, its semantics differ from the other entries. Align the flag with the written behavior or the other entries.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

},
{
label: "Auto-resolve",
desc: "On every push, checks whether the new commits changed lines Argus previously flagged (within \u00B13 lines) and resolves those threads. Pure-diff — no LLM call, no token cost — and runs even when auto-review is off.",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Auto-resolve invokes the LLM-backed AddressedJudge before resolving each proximity candidate, so this description understates both behavior and token spend. Document the judge confirmation and its cost instead of promising a pure-diff, free operation.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/src/components/marketing/docs-content.tsx, line 2449:

<comment>Auto-resolve invokes the LLM-backed `AddressedJudge` before resolving each proximity candidate, so this description understates both behavior and token spend. Document the judge confirmation and its cost instead of promising a pure-diff, free operation.</comment>

<file context>
@@ -2412,6 +2442,11 @@ export function DocsContent() {
+                },
+                {
+                  label: "Auto-resolve",
+                  desc: "On every push, checks whether the new commits changed lines Argus previously flagged (within \u00B13 lines) and resolves those threads. Pure-diff — no LLM call, no token cost — and runs even when auto-review is off.",
                   status: "on by default",
                 },
</file context>
Suggested change
desc: "On every push, checks whether the new commits changed lines Argus previously flagged (within \u00B13 lines) and resolves those threads. Pure-diff — no LLM call, no token cost — and runs even when auto-review is off.",
desc: "On every push, checks whether the new commits changed lines Argus previously flagged (within ±3 lines), then asks an LLM judge to confirm the finding was actually fixed before resolving the thread. Runs even when auto-review is off.",

</Link>
<h2 className="text-xs font-mono text-slate-500 uppercase mb-3 tracking-wider">Docs</h2>
<nav className="space-y-1">
<DocLink href="/docs">Overview</DocLink>

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The new Overview link points to /docs, but the new docs tree (web/src/app/docs/) has no page.tsx at that path — only layout.tsx. The /docs URL is still owned by the legacy marketing page web/src/app/(marketing)/docs/page.tsx, so clicking Overview leaves the new docs shell and lands on the marketing full-reference page rendered without this sidebar (and without DocsPageViewedProbe). Either add web/src/app/docs/page.tsx so Overview renders inside this layout, or the link is pointing users at an inconsistent standalone page that contradicts the new nav structure.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/src/app/docs/layout.tsx, line 20:

<comment>The new `Overview` link points to `/docs`, but the new docs tree (`web/src/app/docs/`) has no `page.tsx` at that path — only `layout.tsx`. The `/docs` URL is still owned by the legacy marketing page `web/src/app/(marketing)/docs/page.tsx`, so clicking Overview leaves the new docs shell and lands on the marketing full-reference page rendered without this sidebar (and without `DocsPageViewedProbe`). Either add `web/src/app/docs/page.tsx` so Overview renders inside this layout, or the link is pointing users at an inconsistent standalone page that contradicts the new nav structure.</comment>

<file context>
@@ -17,9 +17,18 @@ export default function DocsLayout({ children }: { children: ReactNode }) {
           </Link>
           <h2 className="text-xs font-mono text-slate-500 uppercase mb-3 tracking-wider">Docs</h2>
           <nav className="space-y-1">
+            <DocLink href="/docs">Overview</DocLink>
+
+            <p className="pt-4 pb-1 text-[10px] font-mono uppercase tracking-wider text-slate-600">
</file context>

more relevant matches). Lower = more permissive (more context, more noise). Scores are raw
cosine similarity over the configured embedding space — they sit high, so the defaults
are higher than intuition suggests. An explicit{" "}
<code className="bg-slate-900 px-1 text-amber">0</code> disables that gate entirely.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The new Thresholds paragraph claims "An explicit 0 disables that gate entirely," but that is false for scenario_dedupe. The backend clamps a non-positive dedupe threshold to the 0.95 default (backend/internal/pipeline/scenarios.go scenarioDedupeThreshold), because similarity >= 0 is always true and a 0 would otherwise suppress every new scenario — the opposite of disabling the gate. Qualify the statement so a user setting scenario_dedupe to 0 is not misled into expecting dedup off.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/src/app/docs/features/memory-tuning/page.tsx, line 50:

<comment>The new Thresholds paragraph claims "An explicit 0 disables that gate entirely," but that is false for scenario_dedupe. The backend clamps a non-positive dedupe threshold to the 0.95 default (backend/internal/pipeline/scenarios.go scenarioDedupeThreshold), because similarity >= 0 is always true and a 0 would otherwise suppress every new scenario — the opposite of disabling the gate. Qualify the statement so a user setting scenario_dedupe to 0 is not misled into expecting dedup off.</comment>

<file context>
@@ -52,92 +44,86 @@ export default function MemoryTuningPage() {
+				more relevant matches). Lower = more permissive (more context, more noise). Scores are raw
+				cosine similarity over the configured embedding space — they sit high, so the defaults
+				are higher than intuition suggests. An explicit{" "}
+				<code className="bg-slate-900 px-1 text-amber">0</code> disables that gate entirely.
 			</p>
 
</file context>
Suggested change
<code className="bg-slate-900 px-1 text-amber">0</code> disables that gate entirely.
<code className="bg-slate-900 px-1 text-amber">0</code> disables the similarity gates — except <code className="bg-slate-900 px-1 text-amber">scenario_dedupe</code>, where 0 is clamped back to the default 0.95 because a zero floor would otherwise treat every candidate as a duplicate.

Ticking the box requires <strong className="text-slate-200">repository
write access</strong> — Argus checks the editor&apos;s permission, not
just the comment body. The review runs under a tighter{" "}
<strong className="text-slate-200">10/hour per-repo</strong> cap for

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: The FAQ now states checkbox-triggered reviews run under a 10/hour per-repo force bucket, but backend/internal/api/handlers_webhook.go:424-428 still documents the same dispatch flow as "the tighter force=true rate-limit path (3/hr/repo)". Verify the actual admission bucket size before rollout: if the limit is still 3/hr, these docs (and the matching claims in docs-content.tsx) repeat the misstatement this PR set out to fix; if it is 10/hr, the backend comment is stale and should be corrected alongside.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/src/app/docs/faq/page.tsx, line 68:

<comment>The FAQ now states checkbox-triggered reviews run under a 10/hour per-repo force bucket, but backend/internal/api/handlers_webhook.go:424-428 still documents the same dispatch flow as "the tighter `force=true` rate-limit path (3/hr/repo)". Verify the actual admission bucket size before rollout: if the limit is still 3/hr, these docs (and the matching claims in docs-content.tsx) repeat the misstatement this PR set out to fix; if it is 10/hr, the backend comment is stale and should be corrected alongside.</comment>

<file context>
@@ -62,10 +62,11 @@ export default function FAQPage() {
+          Ticking the box requires <strong className="text-slate-200">repository
+          write access</strong> — Argus checks the editor&apos;s permission, not
+          just the comment body. The review runs under a tighter{" "}
+          <strong className="text-slate-200">10/hour per-repo</strong> cap for
+          checkbox-triggered reviews (same bucket as the{" "}
           <code className="bg-slate-900 px-1 text-amber">--force</code> command flag).
</file context>

"On by default. When off, Argus posts a task-list checkbox on opened PRs (and on new pushes) with an estimated token / cost preview. Ticking the box runs the review on demand.",
defaultValue: true,
"Off by default on hosted Argus (self-hosted defaults to on). When off, Argus posts a task-list checkbox on opened PRs (and on new pushes) with an estimated token / cost preview. Ticking the box runs the review on demand.",
defaultValue: false,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: On self-hosted installs the Auto-review toggle now renders OFF by default even though reviews still run unconditionally. The new description says "self-hosted defaults to on", but the toggle's displayed value comes only from the stored setting or AUTO_RUN_TOGGLE.defaultValue (line 1465-1467 / 1915-1919), and the single defaultValue=false cannot reflect deployment type. Backend decideAutoRun/AutoRun short-circuits self-hosted to ALWAYS review regardless of stored settings (backend/internal/pipeline/persona.go), so a fresh self-hosted org shows "Auto-review every PR" off in the settings page while the backend keeps auto-running reviews — contradicting the new description. Previously defaultValue=true at least displayed self-hosted's actual behavior. Consider deriving the shown default from a self-hosted flag when the backend exposes one, or narrow the description wording.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/src/app/(dashboard)/settings/page.tsx, line 922:

<comment>On self-hosted installs the Auto-review toggle now renders OFF by default even though reviews still run unconditionally. The new description says "self-hosted defaults to on", but the toggle's displayed value comes only from the stored setting or AUTO_RUN_TOGGLE.defaultValue (line 1465-1467 / 1915-1919), and the single defaultValue=false cannot reflect deployment type. Backend decideAutoRun/AutoRun short-circuits self-hosted to ALWAYS review regardless of stored settings (backend/internal/pipeline/persona.go), so a fresh self-hosted org shows "Auto-review every PR" off in the settings page while the backend keeps auto-running reviews — contradicting the new description. Previously defaultValue=true at least displayed self-hosted's actual behavior. Consider deriving the shown default from a self-hosted flag when the backend exposes one, or narrow the description wording.</comment>

<file context>
@@ -918,8 +918,8 @@ const AUTO_RUN_TOGGLE = {
-		"On by default. When off, Argus posts a task-list checkbox on opened PRs (and on new pushes) with an estimated token / cost preview. Ticking the box runs the review on demand.",
-	defaultValue: true,
+		"Off by default on hosted Argus (self-hosted defaults to on). When off, Argus posts a task-list checkbox on opened PRs (and on new pushes) with an estimated token / cost preview. Ticking the box runs the review on demand.",
+	defaultValue: false,
 } as const;
 
</file context>

Comment thread web/public/docs.md
| Scenario memory | ON | Surfaces past bugs when you touch the same files |
| Code simulation | OFF | Tests known failure scenarios against your diff |
| Auto-review every PR | Off (hosted) / on (self-hosted unset) | Reviews run automatically vs. on-demand via checkbox |
| Auto-resolve stale threads | On | Pushes that fix flagged lines resolve the thread — no LLM call |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The claim that auto-resolve involves "no LLM call" contradicts the codebase: the auto-resolve path runs an addressed-judge LLM pass that consumes model tokens (backend/internal/pipeline/auto_resolve_tokens_test.go records judge prompt/model/cost tokens), and README/CONTEXT both state "stale threads auto-resolve only after LLM judge confirms the change addressed the finding; line proximity alone never resolves" and "Proximity is prefilter only; addressed-judge must confirm fix." So a push that merely changes the anchored lines does not itself close the thread — a judge call is involved and the thread stays open on judge failure. Please reconcile the wording (drop "no LLM call" and the implication that the push alone resolves) with the actual autoResolveOnSynchronize behavior.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/public/docs.md, line 245:

<comment>The claim that auto-resolve involves "no LLM call" contradicts the codebase: the auto-resolve path runs an addressed-judge LLM pass that consumes model tokens (`backend/internal/pipeline/auto_resolve_tokens_test.go` records judge prompt/model/cost tokens), and README/CONTEXT both state "stale threads auto-resolve only after LLM judge confirms the change addressed the finding; line proximity alone never resolves" and "Proximity is prefilter only; addressed-judge must confirm fix." So a push that merely changes the anchored lines does not itself close the thread — a judge call is involved and the thread stays open on judge failure. Please reconcile the wording (drop "no LLM call" and the implication that the push alone resolves) with the actual `autoResolveOnSynchronize` behavior.</comment>

<file context>
@@ -178,55 +204,74 @@ Example: *"All API endpoints must validate authentication tokens before processi
-| Scenario memory | ON | Surfaces past bugs when you touch the same files |
-| Code simulation | OFF | Tests known failure scenarios against your diff |
+| Auto-review every PR | Off (hosted) / on (self-hosted unset) | Reviews run automatically vs. on-demand via checkbox |
+| Auto-resolve stale threads | On | Pushes that fix flagged lines resolve the thread — no LLM call |
+| Deep Review | Off | 4 specialist reviewers per file; required for simulation |
+| Simulation & Scenarios | Off | Tests stored failure scenarios against the diff |
</file context>

testGeneration: true,
ideIntegration: false,
ticketing: false,
ticketing: true,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Greptile's ticketing flag was flipped to true while the same entry's strengths and summary say ticket handling is context-only: "Reviews code against linked Jira tickets, Linear issues, and Confluence pages (read-only — no ticket creation)" and "doesn't ... create tickets." Every other entry sets ticketing: true only where ticket creation exists (Cubic "creates Jira/Linear tickets", Codacy "creates tickets from findings", SonarQube/Gitar "creates follow-up issues"), while Greptile explicitly does not create tickets. If the flag means ticket creation/integration uniformly, this contradicts the copy; if it now means context-only, its semantics differ from the other entries. Align the flag with the written behavior or the other entries.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At web/src/lib/pseo/competitors.ts, line 174:

<comment>Greptile's `ticketing` flag was flipped to true while the same entry's strengths and summary say ticket handling is context-only: "Reviews code against linked Jira tickets, Linear issues, and Confluence pages (read-only — no ticket creation)" and "doesn't ... create tickets." Every other entry sets `ticketing: true` only where ticket creation exists (Cubic "creates Jira/Linear tickets", Codacy "creates tickets from findings", SonarQube/Gitar "creates follow-up issues"), while Greptile explicitly does not create tickets. If the flag means ticket creation/integration uniformly, this contradicts the copy; if it now means context-only, its semantics differ from the other entries. Align the flag with the written behavior or the other entries.</comment>

<file context>
@@ -168,87 +171,89 @@ export const competitors: Competitor[] = [
       testGeneration: true,
       ideIntegration: false,
-      ticketing: false,
+      ticketing: true,
       complianceCert: true,
     },
</file context>
Suggested change
ticketing: true,
ticketing: false,

@BeLazy167
BeLazy167 merged commit d169f80 into main Sep 21, 2026
5 checks passed
@BeLazy167
BeLazy167 deleted the docs/compare-refresh-pub branch September 21, 2026 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant