Skip to content

bug: log_file_info() filename parsing produces garbled task names for deduplication #4

Description

@Troubladore

Summary

Even after fixing the header fallback (#1), the Inspect View UI may still hide files due to deduplication by task_id values parsed from filenames. The log_file_info() function populates EvalLogInfo.task and EvalLogInfo.task_id from the filename, and the client-side logs IndexedDB store uses these garbled values for grouping/deduplication — separate from the correct values in log_previews.

Context

This was observed during the investigation for #1. After renaming eval files to g2a_simlex999_{model}.eval, 13 of 17 files appeared (up from 1), but 4 remained hidden because:

  • voyage_3_largetask_id="3"
  • openai_3_smalltask_id="3"
  • openai_3_largetask_id="3"

All three collide on task_id="3", so the UI deduplicates and shows only one.

Root Cause

The logs store in IndexedDB is populated from the /api/log-files response, which uses log_file_info() filename parsing. The log_previews store is populated from actual header reads. The UI uses the logs store for list rendering and applies deduplication/filtering on task_id — which contains the garbled filename-parsed values, not the correct header values.

Relationship to #1

Issue #1 fixes the log_file_info() return values by falling back to header reads. Once #1 is implemented, log_file_info() will return correct task/task_id from the header, which means the logs store will also have correct values. This issue may be fully resolved by #1.

However, if the client-side code separately caches or derives task_id from the filename (outside of what log_file_info returns), this issue would persist.

Verification Plan

After #1 is merged:

  1. Confirm that /api/log-files returns correct task/task_id for all files
  2. Confirm that the logs IndexedDB store has correct values
  3. Confirm no deduplication hides files
  4. If all pass → close this issue as resolved by fix: Inspect View filename parsing breaks for custom .eval files #1
  5. If deduplication still occurs → investigate client-side grouping logic

Acceptance Criteria

  • All eval files with unique content appear as separate entries in the viewer
  • No files hidden due to task_id collisions from filename parsing

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    design-neededRequires design work before implementation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions