Use CodeMirror for rendering code/diffs

[rohitpaulk]

Currently, our code rendering system is built in-house using Shiki for code highlighting. Some features we've added on top of this are:

• Support for rendering diffs (green/red background for lines, +/- icons etc.)
• Support for expanding/collapsing unchanged diff sections
• Support for rendering comments (comment icons, expand/collapse comments etc.)

This is getting hard for us to extend, mostly because there isn't a solid foundation that is oriented around code editing. Let's transition away from this and use CodeMirror instead.

Rationale

We're already running into some limitations with the current system.

Current work

One example of a current limitation is the way we handle comments.

Our comment buttons are currently anchored to the right of a line, so in most cases they're hidden by default. Even if you scroll to them, the disappear when you scroll back to read the comment:

Screen.Recording.2024-01-29.at.17.44.47.mov

We want to them to be anchored to the right-most end of the editor container (not the line), like with Repl.it:

Screen.Recording.2024-01-29.at.17.43.05.mov

This is non-trivial to implement from scratch, since they need to be absolutely positioned wrt. the editor container in such a way that they're aware of a line's current Y-index. With CodeMirror, this'll be easier to do by using a system similar to their "gutter" feature (that renders icons on the left side, not right). You'll see that repl-it is using some kind of gutters plugin too.

Future work

A lot of the features we want to build in the future will be easier to build if we're using CodeMirror.

• Select text and click on "explain this"
• CodeIntelligence like GitHub, where you hover over a method and it shows you references, go-to defn. etc.
• Displaying line numbers along with diffs

We might also add an in-browser editor at some point (thought it's unlikely that'll happen soon).

Scope

For starters, let's not add any new features but transition our existing features to use CodeMirror. I'm ordering these based on least used -> most used (we should ship PRs for the least used first, so we can iron out any issues early on before shipping)

1. Rendering buildpack code: URL

This only renders code, no diffs. No expand/collapse functionality, no comments functionality.

2. Rendering Submission Diffs for admins: URL

This renders diffs, but without comments etc. Needs to support expanding/collapsing unchanged lines, but doesn't need to support comments.

3. Rendering first stage solution diffs: URL

Similar to the above.

4. Community Solution Diffs: URL

This is the most complicated one: needs diffs + comments + collapse/expand unchanged sections.

### MVP Implementation
- [x] Barebones component to show read-only syntax-highlighted code
- [x] Handling changes to all "options" arguments passed to CodeMirror
- [x] Handling changes to the document inside CodeMirror
- [x] Handling changes to document argument passed to CodeMirror
- [x] Language auto-detection
- [x] Basic Tests
- [x] Add a Github Light-looking theme
- [x] Styling adjustments
- [x] Add capability to show diffs
- [x] Enable editing mode
- [ ] Support option presets

### Line Comments
- [ ] Add a Right-Side Gutter extension for CodeMirror
- [ ] Show comments count in the comments gutter
- [ ] Show Add comment button in the comments gutter
- [ ] Show comments themselves below the lines
- [ ] Allow adding new comments

### Diff Viewer
- [x] Collapse unmodified lines in diffs
- [ ] Toggle between inline / side-to-side modes

[rohitpaulk]

Feel free to break this up however you see fit, and let's try to deploy incrementally for these 4 use-cases

[VasylMarchuk]

@rohitpaulk Broke up all the steps into several simple task-lists above. I have a lot of unfinished/experimental changes for many of those steps, already splitting them into half a dozen little branches, and preparing my barebones branch to be made into the first PR.