Gui

[Kethku]

I noticed that your website references creating a gui with skia/skulpin. That sounds very similar to my architecture for neovide which is a gui for neovim.

A not broadly stated goal for neovide is to eventually expose the graphical parts as a swappable front end for other text editors such as what helix looks like. I was curious if you are interested in such a collaboration. Getting a gui right is hard work (as I've found out), I think collaborating would be great for both use cases!

Are you planning on exposing a gui protocol ala neovim's? If so what would that look like/what features do you think would be useful for helix that are unique to helix? The editor looks very cool btw!

[archseer]

Hey! The skulpin mention was specifically after I looked at neovide, it seemed like a good fit. @norcalli wanted to experiment on a new frontend as well.

I'm a bit apprehensive on exposing a GUI protocol, they're either too vague and the client needs to re-implement a lot of the handling, or they're very specific and every client ends up the same. I don't have a solid opinion yet so we'll see. Right now the idea is that helix-view would provide most of the logic, and a frontend would just be another crate wrapping everything with a UI -- that needs some work right now though because commands are currently sitting directly in helix-term.

[Kethku]

That makes good sense. One thing thats been on my list for a while is to split neovide into the neovim gui, and a front end renderer so that other apps could build their own rendering. Then you could integrate it as a crate without having to do all the painful font rendering and such. Let me know if something like that would be interesting for your use cases or not. In any case, if you end up going the skia route, I'm happy to help out if you run into problems. Awesome project

[Kethku]

The more I think about it, the more I am interested in trying out something like this. You mentioned that commands are currently located in the helix-term. Are you suggesting that they would need to be moved into helix-view in order for a gui crate to consume and produce them? Are you interested in a PR which attempts that kind of refactoring?

[archseer]

So there's a few sets of changes necessary:

• helix-view needs to be refactored to have it's own generic key/style/color structs. Fix previous broken refactor key into helix-view #345 is a start but it depends on crossterm. Using generic structs would let us drop the crossterm dependency from helix-view.

• helix-term commands.rs need to be moved into helix-view. The problem there is that the context depends on the compositor written for the TUI, and some commands directly manipulate it by pushing in new components.

  The solution here is to make the compositor a trait so that each frontend can implement it's own. The new trait would have methods like cx.compositor.popup() so that we can manipulate UI without directly referencing the types. But I'm not sure how to handle cases like these

  helix/helix-term/src/commands.rs

  Lines 3084 to 3086 in fd1ae35

  	let contents = ui::Markdown::new(contents, editor.syn_loader.clone());
  	let mut popup = Popup::new(contents);
  	compositor.push(Box::new(popup));

• There's a bit of lsp code in term/src/application.rs that would be better off shared somewhere.

With these changes helix-view would end up holding most of the code, and -term would simply be a tiny frontend implementing the compositor & components.

[Kethku]

I'm looking into step 2 now that step 1 has merged. Let me first say back to you what I think I understand you to mean for how we might move the commands out of helix-term. Then I will propose a slightly different strategy with some benefits and drawbacks.

Gui as a component drawer

The solution here is to make the compositor a trait so that each frontend can implement it's own. The new trait would have methods like cx.compositor.popup() so that we can manipulate UI without directly referencing the types. But I'm not sure how to handle cases like these

So what I think you are suggesting is that each frontend would create a custom compositor type which implements the Compositor trait. This trait would have functions on it for each of the custom components that are implemented in the helix-term crate today. Your example is popup, but I'm guessing this would also require picker, prompt, spinner, menu etc.

Some pros for this approach:

• Custom more native UI can be implemented for each of these effectively giving more control to the gui
• The gui can implement surfaces however they like. Doesn't have to have everything on a strict grid
• This is a relatively small change assuming all of the components can be implemented as functions

Some negatives that come to mind:

• Adding new components requires implementing them in every front end (probably just term and gui, but who knows maybe multiple front ends are created)
• Guis could be very different in feel from each other. Would configs work the same in all of them? Neovim has this problem because often a config that works great in one gui is a complete mess in another
• Theres a relatively large implementation burden for the guis. Even though popup and prompt are basically just rendering text in a window of some sort, they would both have to be implemented in the front end

Gui as a render surface

If we look at Neovim's approach instead, it works by defining a basic set of rendering capabilities that the gui must provide and then implements components and such on top of those basic systems. At the simplest, a neovim gui renders a single grid of characters and all of the components and such are drawn by neovim on top of that. A quick and dirty gui can be built in a couple days if you target that system.

If however more gui extensions are implemented, the rendering becomes slightly more complicated. Rather than drawing a single grid, multiple grids can be defined and drawn to. This gives the gui some more freedom to effect the size and positioning of the rendered windows, but components are still done just using that basic framework. Popups are just smaller windows which render text into them. Floating terminals are implemented on floating windows with text just the same as a dialog box.

This also separates innovation in the editor from implementation in the gui. Components can be introduced without requiring them to be implemented in every front end.

Some benefits:

• Guis are pretty simple. Basically they are just a rendering system which components can depend on to do their drawing
• Guis don't need to be updated to add new components
• Many guis can be created which benefit from upstream updates and changes. Can start to build an ecosystem if there is desire to

Some drawbacks:

• Guis can only innovate so far as the interface allows. It would require some creativity to let guis do special rendering for a given type of component
  -- maybe this can be fixed by giving each rendered window a "purpose" flag which lets the gui special case a certain component
• Pretty significantly different approach to the current architecture. Would maybe require some intermediate steps
• Likely locks guis into a grid based interface. I personally feel this is nice because it simplifies design and such, but I can understand why others would prefer more flexibility.

Thoughts

This has been a bit ramble-y. I hope my point was at least somewhat understandable. I'm interested in your thoughts.

[pickfire]

@Kethku If anyone were to take on GUI probably you are the one that is most familiar with it since you did neovide. My thought is that we may want it as a render surface but with custom components. It is useful in doom emacs where you can draw something not available in the terminal.

Like you can see on the left, green for new lines, yellow for modified line. And one more you can see is a single red dash for removed lines, I don't think we can easily render these in terminal.

Before getting into GUI, like neovide I see an issue is that users may not be able to access terminal. Which will break some users workflow. I usually do edit and then ctrl-z to run some shell commands. If we only do a GUI for helix, what if users want to run commands? Do we want to provide like sort of rendering for terminal?

There is also another possibility, getting neovide to have helix backend? Not sure how hard is that but from current viewpoint, helix does not have anything, but yeah neovide currently seemed to only be drawing the same thing as terminal, in some places it would be good to have certain way of drawing which is not possible in terminal.

[Kethku]

Before getting into GUI, like neovide I see an issue is that users may not be able to access terminal. Which will break some users workflow. I usually do edit and then ctrl-z to run some shell commands. If we only do a GUI for helix, what if users want to run commands? Do we want to provide like sort of rendering for terminal?

To be honest, a good terminal emulator inside of helix sounds wonderful. I personally rarely dip into a terminal emulator outside of the one contained within neovim. But I think thats a large can of worms that doesn't need to happen now. What I'm interested in helping with is just making sure the design of helix is conducive to gui front ends in the long run even if its not ready yet.

There is also another possibility, getting neovide to have helix backend? Not sure how hard is that but from current viewpoint, helix does not have anything, but yeah neovide currently seemed to only be drawing the same thing as terminal, in some places it would be good to have certain way of drawing which is not possible in terminal.

I'm definitely thinking about ways to make more full fledged guis using Neovide's rendering system. What I would LOVE is to split Neovide in half so that the rendering part is just a crate which provides channels for sending draw commands and receiving input/events from the window. Then the renderer would handle everything else. Integrating that crate with an editor backend would just require writing some glue.

Neovide isn't there yet because there are lots of neovim specific-isms that would need to be moved around, but I think it would be valuable to do because features built for one editor backend would likely be useful for others.

To be honest though its kinda a pipe dream at the moment. What I was thinking in the short run is to just recreate something like neovide's rendering system for helix specifically in order to get off the ground, and then think about unifying the two later if there is interest. That way I don't impose further constraints on this project that don't really benefit this project long term

[archseer]

Like you can see on the left, green for new lines, yellow for modified line. And one more you can see is a single red dash for removed lines, I don't think we can easily render these in terminal.

This is doable using box drawing characters (I imagine emacs is using the same thing). https://github.com/archseer/snowflake/blob/e616d7cb24d85cf6b17b77c3e0cb1ef6c4212414/profiles/develop/neovim/init.vim#L294-L299

[archseer]

If we look at Neovim's approach instead, it works by defining a basic set of rendering capabilities that the gui must provide and then implements components and such on top of those basic systems. At the simplest, a neovim gui renders a single grid of characters and all of the components and such are drawn by neovim on top of that. A quick and dirty gui can be built in a couple days if you target that system.

I was a bit undecided between the two approaches. While the neovim approach works, it leads to most frontends looking very similar. (For example, while I'm aware of Neovide and looked at it's source, I haven't actually used it yet because I didn't have enough reasons to switch.)

A character grid is a good start but is limiting if we want to experiment with inline decorations (see the CodeMirror 6 examples). These might be variable sized or won't fit perfectly in the character grid. Another example would be rust-analyzer's clickable code action links that are embedded into the view.

We're also limited by the terminal in that it's hard to render 0 width multiple selections (cursors) so selections are always 1-width or more (see #362 for more context).

I'm not even sure if the component drawer approach is flexible enough, since we might want to use completely different component types on different frontends. I guess we could allow for frontends to provide their own sets of commands that would augment the defaults with different UI. Maybe it's okay for the frontends to share the same core/view but end up being slightly different (similar to scintilla, or codemirror embedding)

I've been keeping the built-in component list short: https://github.com/helix-editor/helix/tree/master/helix-term/src/ui

Feel free to join us on Matrix to discuss further! (https://matrix.to/#/#helix-community:matrix.org, make sure to join #helix-editor:matrix.org if you can't see it on the list)

---

Before getting into GUI, like neovide I see an issue is that users may not be able to access terminal. Which will break some users workflow. I usually do edit and then ctrl-z to run some shell commands. If we only do a GUI for helix, what if users want to run commands? Do we want to provide like sort of rendering for terminal?

We won't only be doing a GUI, a helix-term is still going to be the primary implementation. (Well, one of two, I'd like to have the skulpin implementation as "officially supported" too). Down the road we'd want to embed a terminal, but that's quite a bit of work.

[archseer]

I'd also like to get @cessen's opinion here as well

[Kethku]

I was a bit undecided between the two approaches. While the neovim approach works, it leads to most frontends looking very similar. (For example, while I'm aware of Neovide and looked at it's source, I haven't actually used it yet because I didn't have enough reasons to switch.)

This is a very reasonable critique of Neovim's approach. To me the difference between the two depends almost entirely on if you want to maintain the canonical gui for helix or if you want there to be many different guis which use helix as the backend. If you imagine the main helix gui is basically the only one people will use, then I agree it makes a lot of sense to have component aware logic in the gui because it can tweak the experience to feel just right. There would still be reimplimentation of logic between the terminal and gui front ends, but hopefully that wouldn't be too bad.

However if you want to enable many different front ends, I think that strategy falls apart a bit because any time you want to tweak the behavior of a given component, you have to convince all the front ends to also update their implementations. Its harder to innovate like that.

I'm definitely biased given that neovim is what I've spent the most time with, but I think the vision of a gui api which can be implemented very simply as just a grid, but then augmented with optional extensions if the gui developer wants to customize a given type of behavior is beautiful. It means that if somebody wants to make a gui which is just simple tweaks off the terminal experience, they can pretty easily. And if they want to make something that completely reworks the user experience, thats possible too with a little more effort.

A middle ground

If you are interested in building an ecosystem where multiple groups can implement guis if they want to, then it seems to me that a great way to do this would be to implement the grid based approach. Basically move all the component logic into helix-view, and tern helix-term into a super thin rendering surface. Basically just cursor moves and text on a grid. Then each of the components could be implemented as extensions optionally. If the gui wants to customize the dialog box, theres a trait they can implement for their gui which takes over drawing of dialog boxes completely. And a different trait for taking over markdown renders etc etc etc

This solution would be very similar to neovim's extension model where guis can re implement the tabline or message rendering if they want to, but the difference is that helix's implementation would be implemented with components as the basis from the start. Rather than hacking in extensions for features editors would like, helix has the benefit of starting from scratch and thinking about components as first class extension points.

---

All that said, these are just some quick thoughts. I'm happy to help out whichever way you end up going.

[CBenoit]

I'll pin this issue because it seems to me it could use more visibility. We currently pin two issues out of the maximum of three, so I went ahead.