Refractor Probes

[disconcision]

Unrelated changes:

• Fix scroll bug?
• Scroll to caret now keeps a buffer around the caret
• Hyperlink underline hover on references when command is held to indicate jump to def
• Helper to generate a string of all builtins for documentation and prompting purposes
• Better emoji support in strings (view only)

Instructions:

• You can edit the term even with the probe on.
• You can have probes inside other probes now.
• Right-click or use Cmd/Ctrl-E to toggle a manual refractor probe on the indicated term.
• Right-click or use Cmd/Ctrl-Shift-E to put an autoprober on the indicated term.
• You can mass remove probes in a selection with Cmd/Ctrl-E
• Trying to apply either of these to the other will remove the current probe
• To pin a call, click and hover over sample and use Pin icon, or Command-click the sample
• To step into a call, right-click on the function application or function name, and use the 'step into' context menu item.
• See the Probe (magnifying glass) sidebar for a summary of existing probes. Click on them to jump to them.

Features:

• Smart probe placement. A general principle: If you'd be probing a term of arrow type, try to probe subterms containing or contained by that term instead. We currently consider 4 cases: (a) If probing a function literal, probe parameters and body instead. (b) If probing a variable in function position of an application, probe the application instead. (c) If probing a partially applied function, in function position wrt an application (typically but not limited to a reverse application), probe the application instead. (d) (combining (b) and (c)) Variable in fun pos of partial ap in fun pos of ap (typically reverse). In a separate case, probing a let definition just probes the definition term. The following expression cases do not support probing: type alias definition (no point), deferrals in partial aps (no point + breaks parse), and labels (same).

Workarounds:

• Some aps in the dynamic cursor occur inside builtins. We filter these out of the cursor display.

DONE:

• Trigger/deserialize probe projector invocation into refractor probes
• Only cap closure length if there is /actually/ more than one closure on that line
• Make autoprobe line change color on hover/indication to clarify which expression correspond to which value
• Don't use same red underline for both indication and focus
• Indet styling (in Many mode, cursor version isn't distinguished)
• Autoprobe: Disable redundant variable ref suppression for now... was annoying when e.g. the ref is a branch of an a if expression

DONE Dyncursor:

• Dyncursor: Don't show frames from inside builtins for now
• Bug: DynCursor isn't retaining partial stem when moving up (or the sample selector isn't using it)
• Dyncursor: Make unrelated samples invisible in Single mode...
• Dyncursor: ...but clicking on ellipses exposes the 'best' sample from the probe, i.e. if there's none directly related to the cusor, pick the one with the maximal suffix match(so to maximizes amount of other samples kept stationary)

DONE Probe add/remove:

• Make sure refractors on projectors work
• Remove double click to toggle probe Many/Single (too easy to accidentally trigger)
• Replace Probe projector in Panel with refractor
• Command to remove all probes within a selection (reuse Cmd-E)
• Make manual and auto probes mutually exclusive when toggling

DONE Step-into:

• Make step into not toggle if already probed
• Make step into auto instead of manual
• Make step-into work on ap not just fn name

DONE Cleanup:

• Split up Dynamics into Sample and DynCursor
• Rename 'closures' to 'samples'
• Consolidate cursor naming as 'dynamic cursor'

DONE Context menu:

• Context menu: Add context menu
• Context menu: Add Goto definition
• Context menu: Indicate current probe state of term
• Context menu: Only show probe options for expressions/patterns
• Context menu: Restrict autoprobe to expressions
• Context menu: Restrict step-into to aps with var in function pos, and function pos vars

DONE Pinning:

• Reinstate pinning affordance
• Move pin icon to left instead of right so visible for large values
• Pinning: borked in Single mode; try pinning an ADT test and clicking on a param closure
• Pinning: If remove manual probe, remove pin
• Pinning: Context menu: add pin action
• Pinning: Disallow pinning constructor aps
• Pinning: Pins inside pins probably shouldn't hide outer pinned closures
• Pinning: If remove auto probe, remove pin
• Pinning: When bulk-removing probes via selections, remove pin if within selection
• Don't allow pinning builtins

DONE Sidebar:

• Add Sample legend in sidebar
• Make call cursor use legend style
• Show indicated call if any as indication in call cursor (red outline)
• Show index in call cursor (brown outline), dim entries below index
• Show pinned call in call cursor
• Make sure indicated call is always in call cursor? i would think it would be but maybe not
• Combine manual and auto probe lists in sidebar
• Nest auto probes together in sidebar
• Partition probes in sidebar by top level definitions
• Sidebar: Toggle: Single/Many
• Sidebar: Toggle: Step/Call
• Sidebar: Toggles for below/above colors
• Sidebar: Toggles for caller/callee colors
• Sidebar: 1 versus Many toggles for below/above/caller/callee

DONE Bugs:

• Bug: Indet cases/ifs shouldn't be multi-line
• Bug: Fix closure cursor (logic)
• Bug: probes on scrutinee of an inexhaustive case are doubled #1885
• Bug: Similar to 1885 but for deferrals
• Bug: Nested thread ops have weird closure counts
• Bug: Updating is one action behind
• Bug: Probe dynamics init has broken again, will need to bisect...
• Bug: when i probe setCell function in PaintCell case, parameter is not indicated as being in same closure as body...
• Bug: only one repl showing up at a time
• Bug: moving dyncursor by arrow keys loses indicated_ap
• Bug: Editor caret should be hidden when sample is focussed
• Bug: pattern probe collection is bugged; the indicated sample red outline is duplicated, movement is borked
• Bug: left/right arrow sample movement doesn't work when there's a pin
• Bug: samples reversed relative to dev
• Bug: leaving hole in place of j in setcell inner comparison crashes chrome (memory issue?) (fixed by sharing PR)

REPL PLACEMENT LOGIC (DONE):

• Want to implement more sophisticated logic for deciding which term on a line to put a probe on in repl mode (automatic per-line probe insertion). Instead of just putting it on the 'last largest term' on a line as we do now, I want to take (up to and including) the syntax, statics, and dynamics of terms to decide whether to probe them or not. only we'll skip the dynamics for now to avoid complex re-calculation. but we can look at the statics if necessary (though i don't think any of the below need it?), and definitely look at the term for a given id; can get this via the Terms.t map in the syntax cache.
• So we're going to want to find/write/factor out a fn that gets the term ids on a line, ordered by priority (ie first is largest ending on that line, following ones are others ending at the same place if any, followed by others ending earlier on the line. we are going to go through these accumulating a current best candidate, possibly None. as we go through term ids, we have a predicate to decide whether to replace the current best with the candidate under analysis.
• First special case is motivated by typing lets and function literals left to right. when we start this form, the whole form is on the line, and gets a probe. this feels okay except that after we start the pattern of the let/fun, it feels like that should get the probe. we could special case these, but i'm wondering if there's a more general, possibly more local pattern.
• Like maybe really the issue above is the probed term finishing in a hole (something else useless) when it could be something else. So a predicate could just be don't probe things that evaluate to holes, but without dynamics we could approximate this as don't probe holes (UNLESS there are only holes on the line).
• Or alternatively maybe there is a 'useless' predicate, which includes but is not limited to holes. this might get some of the other not particularly categories:
• Other not particularly interesting categories (besides empty holes): function literals (we don't show values for these atm), references of variables already seen through the patterns (either singly as a probe on a pattern variable, or in composite as in a tuple or list pattern, e.g. if we have a probe on the pattern (a, b) then we've seen and can tell the bindings a and b)
• Another special case is for tuples and list literals where the literal and the last element but not the second last element end on the same line to put the probe on the last element. This is perhaps an awkward description. the idea is that when we want this behavior: (a, // show value of a here [linebreak] b) // show value of b here, NOT the value of (a,b)
• Another special case. Technically in hazel tuples don't require parens around them, ie the parens counts as a separate term, the same as disambiguation parens. but if we have closing parens on a line following the last element's last line, we don't want to bother putting a probe on the parens (we are assuming that given that the last element will be probed, other elements of the tuple that the user wants to probe can or are being probed as well; not 100% sure about this logic, tbh)
• another special case: new bit of logic; if we have a let definition: let <pat> = <definition_body> in it's always going to be redundant to put probes on both the and the <definition_body>, as these will always be the same.

HOPEFULLY LOW HANGING:

• Pinning: When a probe would otherwise have samples shown, but doesn't because of pinning, put an un/pin icon in the place of the ellipses, and interacting with it can unpin the pin
• Pinning: Maybe step-into should pin?
• Step-into causes issues if no closure is selected for that ap; put step on sample for now
• Context menu: Maybe step-into should be in sample menu, with only jump to def in context menu
• Bug: arrow movement in single mode in recursive functions sometimes doesn't have the window align to the dyn cursor sample... first index of interest issue probably.
• Dyncursor: Adding a probe should set dyncursor to that probe's initial sample
• Autoprobe: Try making this a separate mode
• Bug: Adding a probe from any tuple comma other than the first results in no samples being shown
• Pinning: Show modifier keyboard shortcut somewhere?
• Cleanup: Re-style Many-mode closures to approximate how they were before (hover effect is lost)

Sidebar:

• Sidebar: Info popups for various features
• Sidebar: Legend: Add indet
• Sidebar: Toggle: Show env menu on Hover or in Sidebar
• Sidebar: Call cursor: make values shown contextual to level of call cursor
• Sidebar: Allow unpin via dynamic cursor pin icon
• Sidebar: Refine toggles (0/1/inf?)

Challenging:

• Dev scroll jumping bug makes debugging hard....
• State: Serialize refractor probes into invocation
• State: Rebuild doc slides (depends on persistence)
• Figure out how to get live feedback when inserting new defs in fn literal; Concave grout in fun body (for example when starting to write let) breaks semantics as precedence breaks the fun
• Step into: Focus closure on jump
• Should dyn cursor incoporate current ap in some way? example... when in single mode, when on one of the calls to cases in the probes slide, would be nice if the samples shown in the cases function body would align to the ap being indicated....

Nice to haves:

• Perf: why is clicking between samples kinda slow?
• Maybe require click (after selection click) to show env/menu?
• Context menu: Keyboard shortcuts
• Consider only showing Many on current top-level definition?
• Show function argument value somehow somewhere... maybe in env dropdown?
• Add remove affordance for probes in sidebar, maybe also a remove-all
• Sidebar: Make autoprobe map collapsable (collapsed shows first and last maybe so shows params and body for fn lit)
• Autoprobe: For multiholes, the first (or all) elements, not the multihole itself
• Autoprobe: maybe don't put probes on values (is there a predicate for syntactic values?)
• Autoprobe: given [1] probe should go on whole list not 1
• Make adding a probe on same line as another remove the first
• Prevent adding manual probe inside autoprobe, or un-autoprobe on manual add?
• Sidebar: Probe displays in sidebar should always be Single not Many
• Context menu: Further restrict what we can apply an auto/probe to
• Autoprobe: Maybe only on currently indicated top level definition
• Naming: Time Cursor?

SPECULATIVE / FOR FUTURE PRs:

• Grey out lines of code not associated with cursor-unrelated probes
• Samples: Option to mark a sample as red, perhaps autopopulated if put on a test (matching a certain form)?
• Cmd-D should expand to parent term if term already selected
• Samples: Explore time-based coloring
• Pinning: Subtractive pins
• Samples: Consider icon instead of color for up/down callstack
• Multi-line probe views
• State: garbage collect refractors map when persisting to localstore or exporting
• Decide if probe should still exist in projector panel

[codecov]

Codecov Report

❌ Patch coverage is 26.51279% with 1178 lines in your changes missing coverage. Please review.
✅ Project coverage is 48.92%. Comparing base (cf38c33) to head (74dd13b).

Files with missing lines	Patch %	Lines
.../haz3lcore/projectors/implementations/ProbeProj.re	0.38%	260 Missing ⚠️
src/language/dynamics/Sample.re	11.80%	142 Missing ⚠️
src/haz3lcore/Refractors.re	0.00%	117 Missing ⚠️
src/language/term/Abbreviate.re	32.35%	69 Missing ⚠️
src/haz3lcore/projectors/ProbeText.re	0.00%	57 Missing ⚠️
src/web/app/editors/code/ContextMenu.re	0.00%	55 Missing ⚠️
src/web/app/editors/decoration/Arms.re	0.00%	55 Missing ⚠️
src/haz3lcore/derived/AutoProbe.re	76.54%	38 Missing ⚠️
src/web/app/editors/code/CodeEditable.re	0.00%	38 Missing ⚠️
src/language/statics/StaticsBase.re	0.00%	35 Missing ⚠️
... and 39 more

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #1879      +/-   ##
==========================================
- Coverage   49.87%   48.92%   -0.95%     
==========================================
  Files         218      227       +9     
  Lines       23103    24245    +1142     
==========================================
+ Hits        11522    11862     +340     
- Misses      11581    12383     +802     

Files with missing lines	Coverage Δ
src/b2t2/Datasheet.re	100.00% <100.00%> (ø)
src/haz3lcore/CachedSyntax.re	47.05% <100.00%> (ø)
src/haz3lcore/MkRefractor.re	100.00% <100.00%> (ø)
src/haz3lcore/derived/Indentation.re	93.97% <ø> (+1.11%)	⬆️
src/haz3lcore/derived/TermData.re	51.06% <100.00%> (+33.99%)	⬆️
src/haz3lcore/lang/Form.re	92.59% <100.00%> (+0.03%)	⬆️
src/haz3lcore/lang/Precedence.re	94.23% <ø> (ø)
src/haz3lcore/projectors/ProjectorCore.re	76.74% <100.00%> (-2.81%)	⬇️
src/haz3lcore/tiles/Segment.re	66.17% <100.00%> (-0.91%)	⬇️
src/haz3lcore/tiles/Skel.re	84.88% <100.00%> (+6.18%)	⬆️
... and 68 more

... and 14 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[disconcision]

@7h3kk1d do you know any more about what the deal is with those regexp errors? i don't get them locally...

[7h3kk1d]

@7h3kk1d do you know any more about what the deal is with those regexp errors? i don't get them locally...

It's an issue with the regexp code plus the bisect_ppx code. I haven't minimized it to create an issue yet.

[dm0n3y]

lgtm except you should probably extend the grout precedence changes to Grout.shapes and then just defer to Piece.shapes on line 83 of Skel.re as it was originally