Skip to content

Conversation

@jaegeral
Copy link
Collaborator

@jaegeral jaegeral commented Jan 6, 2026

This pull request resolves an issue where archived sketches containing failed timelines could not be unarchived, thereby preventing their proper management and deletion. The changes enable administrators to unarchive these sketches, allowing them to address the underlying timeline issues and ultimately facilitate the removal of the sketches. The update involves a modification to the unarchive API logic and the addition of new tests to ensure robustness across different timeline states.

It changes how to tackle unarchiving indexes that have a OpenSearchIndex that is not found. I tiwll continue to unarchive but set the corresponding Timeline etc to fail so the Sketch is not in a blocked mode.

@jaegeral
Copy link
Collaborator Author

jaegeral commented Jan 6, 2026

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly implements the ability to force-delete archived sketches. The changes are applied consistently across the API client, the server-side resource, and the end-to-end tests. The logic is sound, preventing soft-deletion of archived sketches while allowing force-deletion, and the error messages have been updated to guide the user. The accompanying test modifications properly validate the new functionality. I've added a couple of comments regarding outdated docstrings that should be updated to reflect these changes.

@jaegeral jaegeral marked this pull request as draft January 6, 2026 11:11
       * Server-side (`timesketch/api/v1/resources/archive.py`): Modified the unarchive logic to allow
         sketches with failed or processing timelines to be unarchived. Instead of aborting with a 500
         error, the system now logs a warning and proceeds to open all healthy indices. This enables
         users to recover the sketch and then fix or delete the problematic timelines.
   * `test_unarchive_sketch_with_failed_timeline`: Updated the docstring to clarify that the test
     verifies unarchiving is allowed for sketches with failed timelines to enable recovery and subsequent
     fixing.
   * `test_unarchive_sketch_with_mixed_states`: Enhanced the docstring to explain that the test confirms
     robustness against multiple problematic timeline states (fail and processing) during unarchive.
@jaegeral
Copy link
Collaborator Author

jaegeral commented Jan 6, 2026

/gemini review

@jaegeral jaegeral changed the title fix: Allow force_delete to also delete archived sketches fix: Allow archived sketches with failed timelines to be unarchived Jan 6, 2026
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request addresses an issue preventing sketches with timelines in 'fail' or 'processing' states from being unarchived, which is a great improvement. However, the pull request title and description are misleading. They claim that force_delete is now allowed on archived sketches, but the updated tests confirm the opposite: an archived sketch cannot be force-deleted and must be unarchived first. Please update the title and description to accurately reflect the changes, which are mainly about improving the unarchiving logic.

@jaegeral
Copy link
Collaborator Author

jaegeral commented Jan 6, 2026

/gemini summary

@gemini-code-assist
Copy link
Contributor

Summary of Changes

This pull request resolves an issue where archived sketches containing failed timelines could not be unarchived, thereby preventing their proper management and deletion. The changes enable administrators to unarchive these sketches, allowing them to address the underlying timeline issues and ultimately facilitate the removal of the sketches. The update involves a modification to the unarchive API logic and the addition of new tests to ensure robustness across different timeline states.

Highlights

  • Unarchiving Sketches with Failed Timelines: The primary change allows archived sketches to be unarchived even if they contain timelines in a 'failed' status. This is crucial for administrators to regain access to such sketches, remove problematic timelines, and then proceed with force-deleting the sketch if necessary.
  • API Logic Adjustment: The _unarchive_sketch method in timesketch/api/v1/resources/archive.py has been modified. Instead of blocking the unarchive operation due to failed timelines, it now logs a warning and proceeds, allowing the sketch to become 'ready' while the problematic timelines retain their 'fail' status.
  • New Test Cases: Comprehensive test cases have been added and updated to validate the new behavior. This includes test_unarchive_sketch_with_failed_timeline and test_unarchive_sketch_with_mixed_states in timesketch/api/v1/resources_test.py, which specifically test unarchiving sketches with various non-ready timeline states. Additionally, end-to-end tests were updated to reflect the new unarchive-then-force-delete workflow.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog
  • end_to_end_tests/client_test.py
    • Updated test_delete_archived_sketch to include steps for unarchiving a sketch and then force-deleting it, reflecting the new intended workflow.
  • timesketch/api/v1/resources/archive.py
    • Modified _unarchive_sketch to change error handling for failed timelines during unarchiving from blocking to logging a warning, allowing the unarchive operation to complete.
  • timesketch/api/v1/resources_test.py
    • Added necessary imports for Timeline, SearchIndex, Sketch, User, and db_session to support new test cases.
    • Introduced test_unarchive_sketch_with_failed_timeline to verify that sketches with failed timelines can be successfully unarchived.
    • Added test_unarchive_sketch_with_mixed_states to confirm that sketches with a mix of failed and processing timelines can also be unarchived.
    • Updated test_attempt_to_delete_archived_sketch to demonstrate the new process of unarchiving an archived sketch before a successful force-delete operation.
Activity
  • jaegeral initiated two /gemini review commands to request code review.
  • gemini-code-assist[bot] provided feedback on docstring inaccuracies in timesketch/api/v1/resources/archive.py and timesketch/api/v1/resources_test.py related to the updated deletion logic.
  • jaegeral clarified that a timeline remains in a 'fail' state after unarchiving, with the unarchive process skipping its index.
  • gemini-code-assist[bot] suggested refactoring duplicated test setup code in timesketch/api/v1/resources_test.py into helper methods for better maintainability.
  • jaegeral dismissed the refactoring suggestion, stating 'na that is fine.'
  • jaegeral initiated a /gemini summary command to request a pull request summary.

@jaegeral jaegeral self-assigned this Jan 6, 2026
@jaegeral jaegeral marked this pull request as ready for review January 6, 2026 12:21
@jaegeral jaegeral requested a review from jkppr January 6, 2026 12:21
@jkppr jkppr added the Backend label Jan 6, 2026
Copy link
Collaborator

@jkppr jkppr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First round. Please take a look below.

"Timeline '%s' status set to 'ready'.",
timeline.id,
)
# Only set timeline to ready if its index was successfully opened
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ran into the following issue:

  • A searchindex can be shared between a "ready" and a "fail" timeline. E.g. when you have 1 Plaso file that works and 1 that does not.
  • Somehow this situation leads to a "fail" timeline after unarchive where it was "ready" before.

Steps to reproduce:

  1. Create new Sketch
  2. Upload a working timeline
  3. Archive & unarchive work without issue.
  4. Now add a failing timeline.
  5. Try to archive the sketch will be prevented (as expected)
  6. Delete the broken timeline
  7. Archive sketch (now works)
  8. Unarchive sketch. But now the timeline that was working before is in a fail state! This should not happen.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok I tried to reproduce it and it should be fixed now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still able to reproduce this issue. Again here are the exact steps I did:

  1. Create new Sketch
  2. Upload a working Plaso timeline
  3. Upload a broken Plaso timeline (I have shared the one I use via DM)
  4. Delete the broken timeline
  5. Archive the sketch
  6. Unarchive sketch. But now the timeline that was working before is in a fail state! This should not happen.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok will try again, sorry for wasting your time :-/

@jaegeral jaegeral requested a review from jkppr January 7, 2026 09:10
Copy link
Collaborator

@jkppr jkppr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still having the same issue.

"Timeline '%s' status set to 'ready'.",
timeline.id,
)
# Only set timeline to ready if its index was successfully opened
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still able to reproduce this issue. Again here are the exact steps I did:

  1. Create new Sketch
  2. Upload a working Plaso timeline
  3. Upload a broken Plaso timeline (I have shared the one I use via DM)
  4. Delete the broken timeline
  5. Archive the sketch
  6. Unarchive sketch. But now the timeline that was working before is in a fail state! This should not happen.

@jaegeral
Copy link
Collaborator Author

jaegeral commented Jan 8, 2026

[2026-01-08T15:51:36.204709] [INSERT] <Sketch id=4 name='new test'>
[2026-01-08T15:51:36.204709] [INSERT] <SketchStatus id=16 status='new'>
[2026-01-08T15:51:36.204709] [UPDATE] <User id=1 name='dev'>
    CHANGED: sketches: [] -> [<Sketch id=4 name='new test'>]
[2026-01-08T15:51:36.222337] [INSERT] <SketchAccessControlEntry id=10>
[2026-01-08T15:51:36.222337] [UPDATE] <Sketch id=4 name='new test'>
    CHANGED: acl: [] -> [<SketchAccessControlEntry id=10>]
[2026-01-08T15:51:36.233965] [INSERT] <SketchAccessControlEntry id=11>
[2026-01-08T15:51:36.233965] [UPDATE] <Sketch id=4 name='new test'>
    CHANGED: acl: [] -> [<SketchAccessControlEntry id=11>]
[2026-01-08T15:51:36.244273] [INSERT] <SketchAccessControlEntry id=12>
[2026-01-08T15:51:36.244273] [UPDATE] <Sketch id=4 name='new test'>
    CHANGED: acl: [] -> [<SketchAccessControlEntry id=12>]
[2026-01-08T15:51:36.713639] [INSERT] <View id=4>
[2026-01-08T15:51:36.713639] [UPDATE] <Sketch id=4 name='new test'>
[2026-01-08T15:51:36.713639] [UPDATE] <User id=1 name='dev'>
    CHANGED: views: [] -> [<View id=4>]
[2026-01-08T15:51:36.719614] [UPDATE] <View id=4>
[2026-01-08T15:51:49.511401] [UPDATE] <View id=4>
[2026-01-08T15:51:49.528364] [INSERT] <SearchIndex id=8 name='pinfo_test'>
[2026-01-08T15:51:49.528364] [UPDATE] <User id=1 name='dev'>
    CHANGED: searchindices: [] -> [<SearchIndex id=8 name='pinfo_test'>]
[2026-01-08T15:51:49.542496] [INSERT] <SearchIndexAccessControlEntry id=22>
[2026-01-08T15:51:49.542496] [UPDATE] <SearchIndex id=8 name='pinfo_test'>
    CHANGED: acl: [] -> [<SearchIndexAccessControlEntry id=22>]
[2026-01-08T15:51:49.550691] [INSERT] <SearchIndexAccessControlEntry id=23>
[2026-01-08T15:51:49.550691] [UPDATE] <SearchIndex id=8 name='pinfo_test'>
    CHANGED: acl: [] -> [<SearchIndexAccessControlEntry id=23>]
[2026-01-08T15:51:49.562366] [INSERT] <SearchIndexAccessControlEntry id=24>
[2026-01-08T15:51:49.562366] [UPDATE] <SearchIndex id=8 name='pinfo_test'>
    CHANGED: acl: [] -> [<SearchIndexAccessControlEntry id=24>]
[2026-01-08T15:51:49.572661] [INSERT] <SearchIndexLabel id=7 label='plaso'>
[2026-01-08T15:51:49.572661] [UPDATE] <SearchIndex id=8 name='pinfo_test'>
    CHANGED: labels: [] -> [<SearchIndexLabel id=7 label='plaso'>]
[2026-01-08T15:51:49.592793] [INSERT] <Timeline id=10 name='pinfo_test'>
[2026-01-08T15:51:49.592793] [UPDATE] <SearchIndex id=8 name='pinfo_test'>
    CHANGED: timelines: [] -> [<Timeline id=10 name='pinfo_test'>]
[2026-01-08T15:51:49.592793] [UPDATE] <Sketch id=4 name='new test'>
[2026-01-08T15:51:49.592793] [UPDATE] <User id=1 name='dev'>
    CHANGED: timelines: [] -> [<Timeline id=10 name='pinfo_test'>]
[2026-01-08T15:51:49.602454] [INSERT] <TimelineStatus id=49 status='processing'>
[2026-01-08T15:51:49.602454] [UPDATE] <Timeline id=10 name='pinfo_test'>
    CHANGED: status: [] -> [<TimelineStatus id=49 status='processing'>]
[2026-01-08T15:51:49.621920] [INSERT] <DataSource id=11>
[2026-01-08T15:51:49.621920] [INSERT] <DataSourceStatus id=28 status='queueing'>
[2026-01-08T15:51:49.621920] [UPDATE] <Timeline id=10 name='pinfo_test'>
[2026-01-08T15:51:49.621920] [UPDATE] <User id=1 name='dev'>
    CHANGED: datasources: [] -> [<DataSource id=11>]
[2026-01-08T15:51:49.621920] [UPDATE] <Sketch id=4 name='new test'>
[2026-01-08T15:51:49.629908] [UPDATE] <Timeline id=10 name='pinfo_test'>
[2026-01-08T15:51:49.629908] [UPDATE] <DataSource id=11>
[2026-01-08T15:51:54.362679] [UPDATE] <View id=4>
[2026-01-08T15:51:54.367925] [UPDATE] <View id=4>
[2026-01-08T15:51:54.549837] [UPDATE] <View id=4>
[2026-01-08T15:52:13.548800] [UPDATE] <View id=4>
[2026-01-08T15:52:13.577932] [INSERT] <Timeline id=11 name='plaso_will_fail_to_import'>
[2026-01-08T15:52:13.577932] [UPDATE] <User id=1 name='dev'>
    CHANGED: timelines: [] -> [<Timeline id=11 name='plaso_will_fail_to_import'>]
[2026-01-08T15:52:13.577932] [UPDATE] <Sketch id=4 name='new test'>
    CHANGED: timelines: [] -> [<Timeline id=11 name='plaso_will_fail_to_import'>]
[2026-01-08T15:52:13.577932] [UPDATE] <SearchIndex id=8 name='pinfo_test'>
    CHANGED: timelines: [] -> [<Timeline id=11 name='plaso_will_fail_to_import'>]
[2026-01-08T15:52:13.585710] [INSERT] <TimelineStatus id=52 status='processing'>
[2026-01-08T15:52:13.585710] [UPDATE] <Timeline id=11 name='plaso_will_fail_to_import'>
    CHANGED: status: [] -> [<TimelineStatus id=52 status='processing'>]
[2026-01-08T15:52:13.601942] [INSERT] <DataSource id=12>
[2026-01-08T15:52:13.601942] [INSERT] <DataSourceStatus id=31 status='queueing'>
[2026-01-08T15:52:13.601942] [UPDATE] <Sketch id=4 name='new test'>
[2026-01-08T15:52:13.601942] [UPDATE] <User id=1 name='dev'>
    CHANGED: datasources: [] -> [<DataSource id=12>]
[2026-01-08T15:52:13.601942] [UPDATE] <Timeline id=11 name='plaso_will_fail_to_import'>
[2026-01-08T15:52:13.609922] [UPDATE] <Timeline id=11 name='plaso_will_fail_to_import'>
[2026-01-08T15:52:13.609922] [UPDATE] <DataSource id=12>
[2026-01-08T15:52:14.394644] [UPDATE] <View id=4>
[2026-01-08T15:52:14.418564] [UPDATE] <View id=4>
[2026-01-08T15:52:40.105725] [UPDATE] <View id=4>
[2026-01-08T15:52:40.109751] [UPDATE] <View id=4>
[2026-01-08T15:52:40.302867] [UPDATE] <View id=4>
[2026-01-08T15:52:46.383702] [UPDATE] <Sketch id=4 name='new test'>
    CHANGED: timelines: [<Timeline id=11 name='plaso_will_fail_to_import'>] -> []
[2026-01-08T15:52:46.383702] [UPDATE] <Timeline id=11 name='plaso_will_fail_to_import'>
    CHANGED: sketch_id: [<int id=N/A>] -> [<NoneType id=N/A>]
    CHANGED: sketch: [<Sketch id=4 name='new test'>] -> [<NoneType id=N/A>]
[2026-01-08T15:52:46.393097] [UPDATE] <View id=4>
[2026-01-08T15:52:46.663352] [UPDATE] <View id=4>
[2026-01-08T15:52:46.666357] [UPDATE] <View id=4>
[2026-01-08T15:52:55.555839] [INSERT] <SketchStatus id=17 status='archived'>
[2026-01-08T15:52:55.555839] [UPDATE] <Sketch id=4 name='new test'>
    CHANGED: status: [<SketchStatus id=16 status='new'>] -> [<SketchStatus id=17 status='archived'>]
[2026-01-08T15:52:55.569420] [INSERT] <TimelineStatus id=54 status='archived'>
[2026-01-08T15:52:55.569420] [UPDATE] <Timeline id=10 name='pinfo_test'>
    CHANGED: status: [<TimelineStatus id=51 status='ready'>] -> [<TimelineStatus id=54 status='archived'>]
[2026-01-08T15:52:55.824624] [UPDATE] <View id=4>
**[2026-01-08T15:53:03.523814] [INSERT] <TimelineStatus id=55 status='fail'>**
[2026-01-08T15:53:03.523814] [UPDATE] <Timeline id=10 name='pinfo_test'>
    CHANGED: status: [<TimelineStatus id=54 status='archived'>] -> [<TimelineStatus id=55 status='fail'>]
[2026-01-08T15:53:03.534317] [INSERT] <SketchStatus id=18 status='ready'>
[2026-01-08T15:53:03.534317] [UPDATE] <Sketch id=4 name='new test'>
    CHANGED: status: [<SketchStatus id=17 status='archived'>] -> [<SketchStatus id=18 status='ready'>]
[2026-01-08T15:53:03.873326] [UPDATE] <View id=4>

@jkppr jkppr merged commit c14bc88 into google:master Jan 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants