Skip to content

Track image generation method and display in UI#347

Merged
mucsi96 merged 8 commits into
mainfrom
claude/charming-lovelace-1r89qt
Jun 17, 2026
Merged

Track image generation method and display in UI#347
mucsi96 merged 8 commits into
mainfrom
claude/charming-lovelace-1r89qt

Conversation

@mucsi96

@mucsi96 mucsi96 commented Jun 16, 2026

Copy link
Copy Markdown
Owner

Summary

This change adds tracking of how images were generated (directly or via description) and displays this information in the UI with tooltips showing the description when applicable.

Key Changes

  • Backend: Modified ImageService.generateImage() to return a new GeneratedImage object containing both the image data and an optional description, instead of just the byte array
  • Backend: Created new GeneratedImage model class to encapsulate image data and description
  • Backend: Updated ImageController to extract and store the description in ExampleImageData
  • Backend: Updated ExampleImageData model to include optional description field
  • Frontend: Enhanced image carousel and grid components to display generation method ("direct" or "described") next to the model name
  • Frontend: Added tooltips to show the full description text when hovering over "described" label
  • Frontend: Updated type definitions across CarouselImage, GridImageValue, ExampleImage, and ImageResponse to include optional description field
  • Frontend: Added MatTooltipModule imports to carousel and grid components
  • Tests: Added assertions to verify "direct" and "described" labels appear correctly and that descriptions are properly stored in the database

Implementation Details

  • The description is generated by describeScene() when the describe flag is true, otherwise it remains null
  • Descriptions are only stored when images are generated via the description method
  • The UI displays "described" with italic styling and reduced opacity, with a help cursor to indicate tooltip availability
  • Tooltips are only enabled when a description exists

https://claude.ai/code/session_01PYCevjj6DcZL7kAUS5xtRG

Persist the AI-generated scene description on each example image and surface
it in the image grid: a 'direct'/'described' label is shown next to the model
name, with the description revealed on hover via tooltip.
@claude

claude Bot commented Jun 16, 2026

Copy link
Copy Markdown

Code Review

Overview

This PR tracks how example images were generated (directly from input or via an intermediate description/scene) and surfaces that information in the UI with "direct"/"described" labels and tooltips. The approach is clean and the feature is well-scoped.


Issues

1. Template and CSS duplication between carousel and grid components

The model-name block with the generation-method span is identical in both image-carousel.component.html and image-grid.component.html, and .generation-method CSS is duplicated across both stylesheets. Per project conventions (DRY + functional patterns), this should live in a shared component — e.g. <app-image-model-badge> — so there is a single source of truth.

2. description variable doubles as prompt in ImageService

The variable named description is silently reused as the prompt. This is a subtle semantic mismatch — when describe is true the description IS the prompt, which is not immediately obvious. Consider renaming for clarity:

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = sceneDescription != null ? sceneDescription : input;

Then return GeneratedImage.builder().data(data).description(sceneDescription).build().

3. Misplaced assertion in test

In generates described image with Gemini model, this line was inserted in the middle of the test flow:

await expect(page.getByText('Card updated successfully')).toBeVisible();

There is no preceding user action in that section of the test that would trigger a "Card updated successfully" toast. It appears to be in the wrong position and may cause flakiness.

4. Missing tooltip and DB verification in OpenAI described-image test

The Gemini test verifies tooltip content and DB description storage, but generates described image with OpenAI models only asserts the "described" label is present. For parity and confidence, the same tooltip hover and DB assertions should be added there too.

5. byte[] in Lombok @Value class

GeneratedImage uses @Value which generates equals()/hashCode(). Lombok handles byte[] correctly via Arrays.equals/Arrays.hashCode, so this is safe — just worth noting if the class ever appears in collections or test assertions.


What is Good

  • GeneratedImage wrapper model is a clean, idiomatic approach — Lombok @Value + @Builder is exactly right here.
  • @JsonInclude(Include.NON_NULL) on description is consistent with the existing model field pattern.
  • [matTooltipDisabled]="!imageResource.value()?.description" correctly gates the tooltip so no tooltip appears for direct images.
  • Test coverage is solid: label visibility, tooltip content, and DB persistence are all exercised.
  • The description field flows correctly end-to-end from describeScene() through GeneratedImage to ExampleImageData to JSON to the frontend.

claude added 3 commits June 16, 2026 18:56
- Extract the model-name/method label into a reusable ImageModelBadgeComponent
  shared by the image grid and carousel, removing duplicated template and CSS.
- Rename the description variable in ImageService for clarity.
- Add tooltip and persisted-description assertions to the OpenAI described test
  for parity with the Gemini test.
Assert the auto-dismissing 'Card updated successfully' toast immediately
after generation instead of after the slow image download/decode, where it
may have already vanished. Use exact text matching for the direct/described
method labels.
…ace-1r89qt

# Conflicts:
#	client/src/app/shared/types/image-generation.types.ts
#	server/src/main/java/io/github/mucsi96/learnlanguage/controller/ImageController.java
@claude

claude Bot commented Jun 16, 2026

Copy link
Copy Markdown

Code Review

Overview

This PR tracks how images were generated (directly vs. via scene description) by adding a description field through the full stack: DB → entity → job status response → frontend types → UI badge component. The ImageModelBadgeComponent is a clean extraction of inline model display logic into a reusable component.

Positive Aspects

  • Clean separation: GeneratedImage value object properly encapsulates data + description, keeping the return type honest
  • The ImageModelBadgeComponent extraction reduces duplication between carousel and grid components
  • DB migration is correct and well-formed
  • Good test coverage: both Gemini and OpenAI paths tested, with DB assertions and tooltip interaction testing

Issues / Suggestions

Backend

ImageService.generateImage() — logic clarity

The prompt selection now indirectly uses sceneDescription != null as a proxy for describe, which obscures intent:

// Current
final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = sceneDescription != null ? sceneDescription : input;

// Clearer — condition matches the `describe` flag directly
final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = describe ? sceneDescription : input;

More importantly: if describeScene() ever returns null, the current code silently falls back to input (producing a "direct" generation when "described" was requested), while the description stored would be null. The describe flag should be the authoritative condition.


Frontend

image-model-badge.component.html — redundant null coalescing

[matTooltip]="description() ?? ''"
[matTooltipDisabled]="!description()"

The ?? '' is unnecessary when the tooltip is disabled. Angular Material handles null/undefined gracefully. Simplify to:

[matTooltip]="description()"
[matTooltipDisabled]="!description()"

Tests

Missing negative tooltip assertion for "direct" images

The test verifies hovering on "described" shows a tooltip, but doesn't verify that hovering on "direct" does not show a tooltip (or shows nothing). Given that matTooltipDisabled controls this, a quick assertion would round out the coverage:

await page.getByText('direct', { exact: true }).first().hover();
await expect(page.getByRole('tooltip')).not.toBeVisible();

Summary

Overall this is a clean, well-structured change. The data flow is correct end-to-end and the component extraction is a net improvement. The two main suggestions are: prefer describe over sceneDescription != null as the branching condition in ImageService, and remove the ?? '' from the tooltip binding. Neither is a blocker.

…ltip test

Branch the image prompt on the describe flag directly rather than on whether
a scene description was produced, avoiding a silent fallback to a direct
prompt. Add an assertion that direct images show no description tooltip.
@claude

claude Bot commented Jun 16, 2026

Copy link
Copy Markdown

Code Review

Overview

This PR adds tracking of how images were generated ("direct" vs "described" via AI scene description) and surfaces this in the UI with tooltips. The implementation is clean and well-scoped.


Positives

  • Good component extraction: ImageModelBadgeComponent correctly eliminates duplicated .model-name CSS and template logic between image-carousel and image-grid.
  • GeneratedImage value object: A solid abstraction — wrapping byte[] + description in a typed object is better than returning a tuple or using an output parameter.
  • Correct use of Lombok: @Value @Builder on GeneratedImage follows project conventions.
  • Test coverage: E2E tests verify both the UI (label, tooltip presence/absence) and the DB storage of description — good layered assertion approach.

Issues / Suggestions

Minor: Redundant ternary in ImageService.java

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = describe ? sceneDescription : input;

The second line re-checks describe only to resolve sceneDescription (which is already null when describe == false). A cleaner equivalent that avoids the double branch:

final String prompt = describe ? describeScene(input, context) : input;
final String sceneDescription = describe ? prompt : null;

This calls describeScene exactly once and is easier to follow.

Minor: Unnecessary ?? '' in tooltip binding

In image-model-badge.component.html:

[matTooltip]="description() ?? ''"
[matTooltipDisabled]="!description()"

Since matTooltipDisabled already suppresses the tooltip when description() is falsy, the ?? '' fallback is redundant. [matTooltip]="description()" is sufficient.

Observation: byte[] mutability in GeneratedImage

@Value (Lombok) makes the wrapper immutable, but byte[] data remains a mutable reference. In this context it is fine (the array is immediately passed to ffmpegService.resizeImage and not shared further), but worth being aware of if GeneratedImage is ever retained longer or passed across threads.


No Issues Found With

  • Migration: The Liquibase changeset correctly adds a nullable text column to image_generation_jobs. No migration needed for the JSONB card data.
  • markFailed passing null description: Correct — there is no description to store on failure.
  • Data flow: Description correctly flows from describeScene() -> GeneratedImage -> job completion -> status API -> frontend -> card JSONB save.
  • Security: Description originates from controlled AI output, not raw user input.
  • Test selectors: Uses getByText and getByRole('tooltip') — consistent with project accessibility-based selector conventions.

Summary

Solid, well-structured PR. The two minor template/logic nits above are low-priority. The core change is correct and the test coverage is thorough.

Material renders tooltips with aria-hidden, so getByRole('tooltip') never
matches them. Assert the tooltip's visible text via getByText on the
description's distinctive phrase instead, and make the direct-image negative
check assert no description text appears.
@claude

claude Bot commented Jun 16, 2026

Copy link
Copy Markdown

Code Review

Overview

This PR cleanly tracks how images were generated (direct prompt vs. AI-described scene) and surfaces that in the UI via a new ImageModelBadgeComponent. The refactoring is well-scoped and test coverage is solid.


Issues

Bug: cursor: help applied unconditionally

In image-model-badge.component.css, cursor: help is set on .generation-method for all badges, but the tooltip only fires on "described". Hovering over "direct" shows a question-mark cursor with no tooltip — misleading to users.

Fix: conditionally apply the cursor via a CSS class:

<span
  class="generation-method"
  [class.has-tooltip]="description()"
  [matTooltip]="description() ?? ''"
  [matTooltipDisabled]="!description()">
  {{ description() ? "described" : "direct" }}
</span>
.has-tooltip { cursor: help; }

Minor Suggestions

Redundant conditional in ImageService.java

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = describe ? sceneDescription : input; // same condition checked twice

Since sceneDescription is already null when describe is false, the second check can use the value directly:

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = sceneDescription != null ? sceneDescription : input;

Redundant null-fallback in tooltip binding

[matTooltip]="description() ?? ''" combined with [matTooltipDisabled]="!description()" — when disabled, the tooltip value is irrelevant. The fallback is harmless but unnecessary.


What is Well Done

  • Component extraction: pulling ImageModelBadgeComponent out of both carousel and grid is a clean DRY refactoring.
  • GeneratedImage value object: returning a typed object from generateImage() instead of a bare byte[] makes the method signature self-documenting.
  • DB migration: Liquibase changeset is correct — nullable text column with no default is the right choice for a backfill-free migration.
  • Test coverage: E2E tests verify both the UI labels and DB state for direct vs. described flows. Checking the negative case (hover on "direct" shows no tooltip) is a nice touch.
  • Data flow: description travels through the polling mechanism (waitForImageReady return type change) cleanly and call sites destructure it naturally.

Summary

One real UX bug (help cursor without tooltip on "direct" badges) to fix; everything else is optional cleanup. The overall design is sound.

The help cursor was applied to all method badges, but direct badges have no
tooltip; gate it on the presence of a description.
@claude

claude Bot commented Jun 16, 2026

Copy link
Copy Markdown

Code Review

Overview

This PR adds tracking of image generation method ("direct" vs "described") and surfaces it in the UI via a new reusable ImageModelBadgeComponent with tooltip support. The implementation is clean and well-structured across all three layers.


Positives

  • Good extraction: Moving the model badge into its own ImageModelBadgeComponent is a proper DRY refactor — carousel and grid were duplicating identical inline HTML + CSS.
  • Clean return type: Wrapping byte[] + description in a GeneratedImage value object is the right Java pattern for returning multiple values.
  • Correct use of @JsonInclude(Include.NON_NULL) on description in both ExampleImageData and ImageJobStatusResponse avoids polluting stored JSONB with null fields.
  • Angular signals: The new component correctly uses input() signals, consistent with the project conventions.
  • Migration: The Liquibase changeSet is well-formed and correctly adds an optional text column.

Issues

Minor: Redundant variable in ImageService.java

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = describe ? sceneDescription : input;  // re-checks describe flag

The second line re-checks describe when it could use sceneDescription directly:

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = sceneDescription != null ? sceneDescription : input;

This makes the intent clearer — use the description as the prompt when it exists.


Test: Checking for tooltip absence is too narrow

await page.getByText('direct', { exact: true }).first().hover();
await expect(page.getByText('Detailed scene:')).toHaveCount(0);

This only asserts the absence of the string 'Detailed scene:', not the absence of any tooltip. If the description format ever changes, this test won't catch a regression. Consider using getByRole('tooltip') to assert no tooltip appears:

await expect(page.getByRole('tooltip')).toHaveCount(0);

Test: AI-generated content assertion is fragile

await expect(page.getByText('A train platform with a large clock')).toBeVisible();

This asserts specific wording from an AI-generated description, which could be non-deterministic. If the model returns semantically equivalent but differently-worded output, the test will flake. Consider asserting that some description tooltip is visible (e.g., via getByRole('tooltip')) or checking a stable prefix pattern rather than a specific substring.


Missing standalone: true declaration in new component

@Component({
  selector: 'app-image-model-badge',
  imports: [MatTooltipModule],
  ...
})
export class ImageModelBadgeComponent {

Angular 19+ defaults to standalone, so this works, but all other components in the project that are visible in the diff explicitly declare standalone: true. For consistency, add it here too.


Summary

The implementation is solid — clean abstraction, correct data flow end-to-end, and good test coverage for the happy path. The two test concerns (fragile AI content assertion and narrow tooltip absence check) are the main things worth addressing before merge.

@claude

claude Bot commented Jun 17, 2026

Copy link
Copy Markdown

Code Review

Overview

This PR tracks image generation method (direct vs. described) through the full stack — from backend job entity to a new frontend ImageModelBadgeComponent — and displays the method as a badge with a tooltip. The approach is clean and well-scoped.


Positives

  • Good abstraction: GeneratedImage wrapper cleanly separates image data from metadata, keeping ImageService return type self-contained.
  • Proper component extraction: ImageModelBadgeComponent removes duplication between image-carousel and image-grid and is a correct application of DRY.
  • waitForImageReady return type change: Returning { description? } instead of void is minimal and correct — the recursive polling propagates the value naturally.
  • DB migration included: The Liquibase changeset is present and the column type matches the entity.
  • Test coverage: Both Gemini and OpenAI paths are tested, including DB assertions for stored descriptions.

Issues & Suggestions

ImageService.java — minor redundancy

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = describe ? sceneDescription : input;

The describe flag is checked twice. Prefer:

final String sceneDescription = describe ? describeScene(input, context) : null;
final String prompt = sceneDescription != null ? sceneDescription : input;

This eliminates the implicit coupling between describe and sceneDescription.

Fragile tooltip assertions in tests

await expect(
  page.getByText('Detailed scene: Wann fährt der Zug ab? A train platform with a large clock, no visible text.').first()
).toBeVisible();

This asserts on AI-generated content verbatim. If this is mocked in the test environment, it's fine — but if it hits a real model it will be non-deterministic. Verify the test mock returns this exact string; if not, assert on a stable prefix or a partial match with { exact: false }.

ExampleImageData.java — missing @JsonInclude consistency
id and isFavorite fields (if present) might not follow the same @JsonInclude(NON_NULL) pattern. The new description field correctly includes it, but worth checking the full class for consistency.

matTooltip empty string fallback

[matTooltip]="description() ?? ''"
[matTooltipDisabled]="!description()"

This is correct and safe, but matTooltip with an empty string and matTooltipDisabled is slightly redundant. An empty string won't render a visible tooltip anyway, so the ?? '' fallback is just defensive. Not a bug, just noise.


No Issues Found In

  • DB migration and entity alignment
  • ImageGenerationJobRepository.updateStatus signature extension
  • markCompleted / markFailed call sites in AsyncImageGenerationService
  • Angular component imports and module declarations
  • Type propagation from ImageJobStatusResponsewaitForImageReadyExampleImage

Summary

Solid PR. The only actionable items are the minor redundancy in ImageService.java and verifying the tooltip assertion relies on a deterministic mock. Everything else is clean.

@mucsi96 mucsi96 merged commit da39926 into main Jun 17, 2026
9 checks passed
@mucsi96 mucsi96 deleted the claude/charming-lovelace-1r89qt branch June 17, 2026 06:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants