feat: implement visual pipeline debugger#269
Open
Grayking1905 wants to merge 1 commit intoc2siorg:mainfrom
Open
feat: implement visual pipeline debugger#269Grayking1905 wants to merge 1 commit intoc2siorg:mainfrom
Grayking1905 wants to merge 1 commit intoc2siorg:mainfrom
Conversation
Author
|
@ivantha |
Member
|
Thanks for the PR! It currently has merge conflicts with main (or is behind). Could you rebase onto the latest main and push the updated branch? Happy to merge once it's auto-rebasable. |
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Feature Description
This PR introduces the Visual Pipeline Debugger, allowing users to visually inspect their image processing pipelines step-by-step. By capturing intermediate state images after each operator executes, users can scrub forward and backward through their pipeline history. Corresponding Blockly blocks actively glow on the canvas to help trace execution logic visually.
Fix - #268
Full Approach in Detail
The approach relies on modifying the backend to pass back intermediate
DebugStepStateoutputs when adebug=Trueflag is appended to the Pipeline Request. Instead of returning raw high-definition PNGs for every intermediate step (which would bloat the response size), the backend compresses these snapshots aggressively. Each is downscaled to a max dimension of 512px and encoded to JPEG with a quality limit of 70. This ensures snappy UI feedback.On the frontend, the feature pivots around the React
usePipelineStore(via Zustand). A newDebugScrubberUI component intercepts the standard output preview and enables navigation using thedebugCursoranddebugStatesarray. We rely on theblock.idnatively provided by Blockly to match each state with its authoring node, dispatching CSS animations onto the active SVG path inLayout.tsxusing a ReactuseEffectcallback without modifying the core blockly library.State Diagram
stateDiagram-v2 [*] --> Idle Idle --> Executing : Click "Run" (Debug On) state Executing { DecodingImage --> ProcessingStep1 ProcessingStep1 --> CompressThumb1 : Extract State CompressThumb1 --> ProcessingStep2 ProcessingStep2 --> CompressThumb2 : Extract State CompressThumb2 --> EncodingFinalImage } Executing --> DebugModeActive : Return PipelineResponse Executing --> ErrorState : Operator Error (Return Partial States) state DebugModeActive { ScrubForward ScrubBackward HighlightActiveBlock } DebugModeActive --> Idle : Exit Debug Mode (Esc) ErrorState --> DebugModeActive : Error Intercepted (Debug Rendered)Task List Implemented
PipelineStepto accept optionalidmapping.PipelineRequestto handledebugflag.PipelineResponseto includedebug_states._thumbnail_for_debug), and JPEG encoding.image.pyto support variablequalityfor JPEG export encoding.DebugStepState.usePipelineStoreto manage debug modes, cursor states, and iteration controls.usePipelinehook to exportblock.idfrom Blockly inputs.DebugScrubber.tsxcomponent to parse intermediate results visually.PreviewPane.tsxto conditionally render debug UI or standard UI.Layout.tsxfor.debug-highlighted-block.Command/Ctrl + D,Arrow Left,Arrow Right,Esc).Manual Testing
pipelinebranch.Gaussian Blur,Convert to Grayscale,Sobel Edge Detection).DebugScrubberappears on the right pane instead of a single output.Right ArrowandLeft Arrowto scrub through the outputs. Check that the highlighted block on the canvas moves in parallel.Escapeor the exit button to successfully drop back to standard execution view.Video Result
Screen.Recording.2026-03-22.164728.mp4