Skip to content

Enrich truncated OpenRouter descriptions#10

Merged
dzarlax merged 1 commit into
mainfrom
codex/openrouter-full-description-enrichment
Jun 16, 2026
Merged

Enrich truncated OpenRouter descriptions#10
dzarlax merged 1 commit into
mainfrom
codex/openrouter-full-description-enrichment

Conversation

@dzarlax

@dzarlax dzarlax commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • detect OpenRouter descriptions that already arrive truncated with ellipses
  • fetch full model-page SSR descriptions for visible catalog rows
  • persist enriched descriptions back into capability cache

Tests

  • go test ./...

Summary by CodeRabbit

Release Notes

  • New Features
    • Enhanced OpenRouter catalog with full model descriptions—previously truncated descriptions are now automatically fetched and displayed in their complete form.

@coderabbitai

coderabbitai Bot commented Jun 15, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

Adds bounded OpenRouter model description enrichment: a new maxVisibleDescriptionEnrich constant caps how many visible models are enriched per render. Two new exported functions in capabilities.go detect truncated descriptions and fetch full text by scraping the OpenRouter model page via SSR payload extraction. Both buildIndexData paths in handlers.go call the new enrichVisibleDescriptions helper before UI decoration.

Changes

OpenRouter description enrichment

Layer / File(s) Summary
Truncation detection, page fetch, and SSR extraction
internal/llm/capabilities.go, internal/llm/capabilities_test.go
Adds a shared HTTP client, base URL, and compiled regexp in a var block. Implements OpenRouterDescriptionLooksTruncated (detects .../ suffixes) and FetchOpenRouterModelDescription (fetches model page, extracts candidates from SSR payload, decodes HTML entities, normalizes whitespace, returns the longest match). Two unit tests cover extraction and truncation detection.
Bounded enrichment wired into buildIndexData
internal/adminapi/handlers.go
Adds maxVisibleDescriptionEnrich constant. Inserts enrichVisibleDescriptions calls in both the preset and manual-filter index data paths before decorateModelDisplay. Implements Server.enrichVisibleDescriptions: iterates visible models up to the limit, skips non-truncated descriptions, fetches full text, updates models[i].Description and persists updated capabilities via capStore.PutCapabilities; logs errors without aborting.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 Hop hop, the descriptions were clipped,
Just dots where the full text had slipped.
I fetch each model's true name,
Unescape, normalize, no shame—
Now the catalog stands complete,
Every description: replete!
🌿✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 27.27% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main change: enriching truncated OpenRouter descriptions with full text.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/openrouter-full-description-enrichment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/adminapi/handlers.go`:
- Around line 672-677: The lookupCapsFor function can return a zero-value struct
when a lookup misses or encounters an error, and the current code proceeds to
modify and persist this empty struct via PutCapabilities, which overwrites
existing capability data (such as pricing, flags, and context length) with
zeros. After the lookupCapsFor call on line 672, add a check to validate that
the returned capabilities struct is valid and not a zero-value before proceeding
with the modification and PutCapabilities call on line 677. If the lookup fails
or misses, skip the persistence operation entirely to prevent data loss.

In `@internal/llm/capabilities.go`:
- Around line 191-193: The prefix variable is being sliced at the byte level
with prefix[:120], which can split multi-byte UTF-8 characters and cause
subsequent strings.HasPrefix comparisons to fail. Convert the prefix string to
runes using []rune(), slice to 120 runes instead of 120 bytes, then convert back
to a string using string() to ensure proper UTF-8 character boundaries are
respected.
- Around line 160-168: Replace the simple defer statement for resp.Body.Close()
with explicit error handling that captures and checks for close errors, rather
than silently ignoring them. Additionally, ensure the error variable used in the
fmt.Errorf call for the io.ReadAll operation matches the actual variable name
being assigned (if you're using readErr for the read operation error, update the
error message to reference readErr instead of err). This applies to the HTTP
request handling block where openRouterHTTPClient.Do(req) is called and
resp.Body is read with io.LimitReader.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 63200344-4167-4d6e-8400-bd07da78a8c2

📥 Commits

Reviewing files that changed from the base of the PR and between cbbcc32 and 16fd067.

📒 Files selected for processing (3)
  • internal/adminapi/handlers.go
  • internal/llm/capabilities.go
  • internal/llm/capabilities_test.go

Comment on lines +672 to +677
caps := s.lookupCapsFor(ctx, provider, models[i].ID)
caps.Description = full
if caps.Name == "" {
caps.Name = models[i].Name
}
if err := s.capStore.PutCapabilities(ctx, provider, models[i].ID, caps); err != nil {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Prevent cache row clobbering when capability lookup misses.

At Line 672, lookupCapsFor can return a zero-value struct on miss/error; then Line 677 upserts that struct, which can overwrite persisted capability fields (pricing, flags, context length) with zeros.

Proposed fix
-		caps := s.lookupCapsFor(ctx, provider, models[i].ID)
+		caps, found, err := s.capStore.GetCapabilities(ctx, provider, models[i].ID)
+		if err != nil {
+			s.logger.Debug("openrouter description cache lookup failed", "model", models[i].ID, "err", err)
+			continue
+		}
+		if !found {
+			continue
+		}
 		caps.Description = full
 		if caps.Name == "" {
 			caps.Name = models[i].Name
 		}
 		if err := s.capStore.PutCapabilities(ctx, provider, models[i].ID, caps); err != nil {
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/adminapi/handlers.go` around lines 672 - 677, The lookupCapsFor
function can return a zero-value struct when a lookup misses or encounters an
error, and the current code proceeds to modify and persist this empty struct via
PutCapabilities, which overwrites existing capability data (such as pricing,
flags, and context length) with zeros. After the lookupCapsFor call on line 672,
add a check to validate that the returned capabilities struct is valid and not a
zero-value before proceeding with the modification and PutCapabilities call on
line 677. If the lookup fails or misses, skip the persistence operation entirely
to prevent data loss.

Comment on lines +160 to +168
resp, err := openRouterHTTPClient.Do(req)
if err != nil {
return "", fmt.Errorf("openrouter model page: %w", err)
}
defer resp.Body.Close()

body, err := io.ReadAll(io.LimitReader(resp.Body, 4*1024*1024))
if err != nil {
return "", fmt.Errorf("openrouter model page: read: %w", err)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

# Read the file around the specified lines
cat -n internal/llm/capabilities.go | sed -n '155,175p'

Repository: Dzarlax-AI/personal_assistant

Length of output: 1061


🏁 Script executed:

# Check for other similar patterns in the file to understand error handling patterns
rg -n "defer.*Close\(\)" internal/llm/capabilities.go -A 5 | head -40

Repository: Dzarlax-AI/personal_assistant

Length of output: 467


🏁 Script executed:

# Search for proper error handling of resp.Body.Close() in the codebase
rg -n "closeErr\s*:=\s*resp\.Body\.Close\(\)" --type go -A 2

Repository: Dzarlax-AI/personal_assistant

Length of output: 55


🏁 Script executed:

# Search for patterns where Close() errors are explicitly handled
rg "\.Close\(\)" --type go -A 2 | grep -E "(closeErr|errClose|err.*Close)" | head -20

Repository: Dzarlax-AI/personal_assistant

Length of output: 55


🏁 Script executed:

# Look for other HTTP response body reading patterns in the codebase
rg "io\.ReadAll.*resp\.Body" --type go -B 2 -A 3 | head -60

Repository: Dzarlax-AI/personal_assistant

Length of output: 3010


🏁 Script executed:

# Check if there's any linter configuration for errcheck in the repo
find . -name ".golangci.yml" -o -name ".golangci.yaml" -o -name "golangci.yml" | head -5

Repository: Dzarlax-AI/personal_assistant

Length of output: 55


🏁 Script executed:

# Look for linting configuration
fd -e "golangci*" -o -e "*.lint*"

Repository: Dzarlax-AI/personal_assistant

Length of output: 190


🏁 Script executed:

# Look for linting configuration files
find . -maxdepth 2 -type f \( -name "*.golangci*" -o -name ".lint*" \) 2>/dev/null

Repository: Dzarlax-AI/personal_assistant

Length of output: 55


🏁 Script executed:

# Check the exact proposed fix in the review - verify the error message issue
# Let me see if there are any other close error patterns in the repo
rg "resp\.Body" --type go -A 5 | grep -E "(Close|err)" | head -30

Repository: Dzarlax-AI/personal_assistant

Length of output: 2158


🏁 Script executed:

# Check if there's any pattern of handling close errors elsewhere in Go files
rg "defer.*Close" --type go -B 2 -A 8 | grep -E "(defer|closeErr|Close|\.Close)" | head -40

Repository: Dzarlax-AI/personal_assistant

Length of output: 2000


Handle resp.Body.Close() errors in the new fetch path.

Line 164 currently ignores close errors via defer, which can mask transport/resource issues. The proposed fix has a variable name mismatch—the error message still references err instead of readErr.

Proposed fix
-	defer resp.Body.Close()
-
-	body, err := io.ReadAll(io.LimitReader(resp.Body, 4*1024*1024))
-	if err != nil {
-		return "", fmt.Errorf("openrouter model page: read: %w", err)
+	body, readErr := io.ReadAll(io.LimitReader(resp.Body, 4*1024*1024))
+	closeErr := resp.Body.Close()
+	if readErr != nil {
+		return "", fmt.Errorf("openrouter model page: read: %w", readErr)
+	}
+	if closeErr != nil {
+		return "", fmt.Errorf("openrouter model page: close: %w", closeErr)
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
resp, err := openRouterHTTPClient.Do(req)
if err != nil {
return "", fmt.Errorf("openrouter model page: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 4*1024*1024))
if err != nil {
return "", fmt.Errorf("openrouter model page: read: %w", err)
resp, err := openRouterHTTPClient.Do(req)
if err != nil {
return "", fmt.Errorf("openrouter model page: %w", err)
}
body, readErr := io.ReadAll(io.LimitReader(resp.Body, 4*1024*1024))
closeErr := resp.Body.Close()
if readErr != nil {
return "", fmt.Errorf("openrouter model page: read: %w", readErr)
}
if closeErr != nil {
return "", fmt.Errorf("openrouter model page: close: %w", closeErr)
}
🧰 Tools
🪛 golangci-lint (2.12.2)

[error] 164-164: Error return value of resp.Body.Close is not checked

(errcheck)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/llm/capabilities.go` around lines 160 - 168, Replace the simple
defer statement for resp.Body.Close() with explicit error handling that captures
and checks for close errors, rather than silently ignoring them. Additionally,
ensure the error variable used in the fmt.Errorf call for the io.ReadAll
operation matches the actual variable name being assigned (if you're using
readErr for the read operation error, update the error message to reference
readErr instead of err). This applies to the HTTP request handling block where
openRouterHTTPClient.Do(req) is called and resp.Body is read with
io.LimitReader.

Source: Linters/SAST tools

Comment on lines +191 to +193
if len(prefix) > 120 {
prefix = prefix[:120]
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

cd /dev/null || true
# Search for the file and examine the context
fd -t f capabilities.go internal/llm

Repository: Dzarlax-AI/personal_assistant

Length of output: 170


🏁 Script executed:

# Read the relevant section around lines 191-193
cat -n internal/llm/capabilities.go | sed -n '170,210p'

Repository: Dzarlax-AI/personal_assistant

Length of output: 1706


🏁 Script executed:

# Get more context to understand what prefix is and how it's used
cat -n internal/llm/capabilities.go | sed -n '145,220p'

Repository: Dzarlax-AI/personal_assistant

Length of output: 3206


Avoid byte-slicing UTF-8 when capping the prefix.

At line 192, prefix = prefix[:120] slices at the byte level, not the rune level. For any multi-byte UTF-8 characters in the description, this can split a rune and cause the subsequent strings.HasPrefix comparison at line 208 to fail, preventing valid description enrichment.

Proposed fix
	if len(prefix) > 120 {
-		prefix = prefix[:120]
+		r := []rune(prefix)
+		prefix = string(r[:120])
	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if len(prefix) > 120 {
prefix = prefix[:120]
}
if len(prefix) > 120 {
r := []rune(prefix)
prefix = string(r[:120])
}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/llm/capabilities.go` around lines 191 - 193, The prefix variable is
being sliced at the byte level with prefix[:120], which can split multi-byte
UTF-8 characters and cause subsequent strings.HasPrefix comparisons to fail.
Convert the prefix string to runes using []rune(), slice to 120 runes instead of
120 bytes, then convert back to a string using string() to ensure proper UTF-8
character boundaries are respected.

@dzarlax dzarlax merged commit ded31ff into main Jun 16, 2026
6 checks passed
@dzarlax dzarlax deleted the codex/openrouter-full-description-enrichment branch June 16, 2026 08:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant