Ergot Vs 9p

[sminez]

I brought this up on your morning stream this morning: I'd like to compare how you have things set up in ergot with what I've ended up putting together for implementing 9p servers / clients. The crate I've got is ninep and there's an example of writing a simple server here. The crate supports both sync and async clients/servers using an underlying sansIO layer based on coroutines. That works but involves some "interesting" tricks that abuse the Waker API for futures.

9p is specifically a filesystem protocol so there are a fixed set of message types rather than what you've got in ergot so that's already going to put a different spin on things but I'd be interested to learn more about how you've designed things.

[jamesmunns]

Let me know if you have any specific questions :) I'll paste some of the diagrams I showed this morning, and definitely check out the major concepts section I wrote yesterday.

So the main "trick" of how things work, is that all messages are sent to the netstack. Since things are immediate, the flow goes something like:

• Figure out if the destination is "here", or "not here", based on the address (net id + node id)
• If the destination is "here": then we go through our list of sockets, and attempt to find the right one(s) to give the message to.
  • If we find one, give it to the socket. This is done with tricky unsafe code and a vtable, but it's sort of like having a dyn Socket, which has various "receive" methods.
  • If this is a "raw" serialized message, the socket deserializes the message. If this is a "typed" message (e.g. from the local system), we just place the item in the socket
• If the destination is "not here", we pass the message to the "Profile", which has any external interfaces. The profile figures out if there's a matching interface, if it finds one:
  • If this is a "raw" serialized message already, it just puts the message in the outgoing queue
  • If this is a "typed" message, we serialize the message into the outgoing queue

Part of the trick is that there are ways for sockets to declare what "type" they accept. Whereas 9p is (I think?) more of a "file system protocol that you can happen to do other things with", ergot is much more like a rust channel: there's a sender and a receiver, and it's always a given type. It just HAPPENS to sometimes go over some kind of network link.

[sminez]

Sorry @jamesmunns, I've managed to raise this right as I've ended up without any free time to properly engage with it(!)

I'm reading through the ergot docs still for the time being but I suppose the core of what I'd like to talk to you about is using ergot less for communication between devices specifically, and more about communication between processes that may (or may not) be running on different devices. I suspect that this amounts to basically the same thing at the end of the day but it's worthwhile making the distinction 🙂

To be more concrete about my use case: in my text editor ad I'm reproducing a version of the standard plan9 API model where programs present themselves to clients as a 9p filesystem. So things like editor plugins, and querying the editor state can all be done with standard Unix tools that understand how to interact with files. As it stands today this all works fine but I've been wondering about breaking the current editor apart into a core file server that then accepts RPC requests from arbitrary clients, with the UI just being "another client".

I think something along the lines of ergot might be a good fit for this as a way of allowing multiple clients to interact with both the editor and other clients attached to files/buffers that they are interested in.

(If this sounds a little half baked, it probably is!)

As an end goal, I really like the idea from plan9 that you can build up an aggregate system out of multiple machines and merging different physical resources together. 9p is great for a bunch of things but it does have the issue that it assumes that everything is a filesystem even when you'd like to be able to expose more structured RPC semantics. So you typically end up with interfaces that amount to "write some bytes to this file path and cross your fingers".

I'm still unsure as to whether I want to:

• have an ergot based API that I can still mount as a file system
• have a 9p file system API that exposes files that are actually ergot endpoints
• ditch 9p and only use ergot (or an alternative)
• or something else entirely!

If this makes any sort of sense to you id be interested in your thoughts on how to tackle this sort of orchestration problem and whether or not ergot would be worthwhile exploring as part of the implementation.

[jamesmunns]

Hey @sminez, no worries, happy to have you interested :).

So, ergot is definitely usable for making a "network of entities, where each entity has one or more services". Each entity (could be a device, could be a process) is given an address, and each service on an entity is given a port.

Today, I really only give you tools for setting up a network in more or less a star topology: you have a "router" in the center, and each entity connects to the router. On the desktop, that probably means that the router sets up a TCP listening socket, where client entities can join the network by opening a TCP connection to that listening socket. In the demos/std folder, check out the ergot-std-router and ergot-std-client projects.

I think you could implement your approach using ergot, I have less clear of a picture of how, but i'm happy to keep chatting about it!

I think you are likely to run into limitations of what ergot can do today, vs what I want it to be able to do. It's very useful to me personally to have you run into those limitations, and for at least the next four weeks, I'm happy to help solve those limitations.

In particular, today, "entity/service discovery" isn't a solved problem, and you will probably want things like "exclusive" services for things like limiting concurrent access. You might also want these things for more efficient streaming of data, etc.

I can't really comment on "plan9 over ergot" or "ergot over plan9", ergot is exceedingly flexible however you want to integrate it, but I'm not sure if there is immediate value in trying to mesh the two models and their assumptions. If you have concrete questions, I'm happy to discuss!