Typed ports/channels

[bjmdev]

Hi there,

the current DAG and its channels are not typed, i.e. all nodes can be connected to each other regardless of what type their actions produce. This has the consequence, that actions must check on receiving an IP, that the content can be downcast to the expected type it needs for its logic. If that is not the case it needs to deal with it somehow (drop? report error? ...)

Maybe it makes sense to have typed channels. Nodes could be configured to have specific in-/out channels of specific type and one can only connect nodes with matching in-/out-type. Therefore, it is enforced by the dag, that nodes can receive only specific types via respective input channels and can send only specific types to respective output channels.

See https://github.com/teetime-framework/TeeTime as an example of what i mean.

Any thoughts on this?

Many thanks and best regards!

[191220029]

Hi @bjmdev,

@genedna and I discussed your idea and we think it would be great to introduce typed channels in dagrs. We plan to make channels as a trait so we can customize the input/output types of channels. The original untyped channels will be kept as special implementations of this new trait.

If you have more ideas on this, feel free to let me know.

[bjmdev]

Hi,

that sounds great. One more thought on this: one of the basic ideas of FBP is, that the nodes are completely independent black boxes. They receive an IP over one or more input channels (which are ideally typed) and send results to one more, ideally typed output channels.

Right now, IPs can either be broadcast to all output channels (which in future must be filtered to the matching type) or send to a specific channel which has to be identified by the nodeId of the connected node. In other words, the nodes (actions) logik must have knowledge of other (connected) nodes in the graph, which is not good.

The action logik should be completely independent of what nodes are connected to its channels. The logik should be:

1. receive IP(s) of input channel(s) of their respective types
2. do something with those
3. send IP(s) to output channel(s) of respective types according to its logik

That way, the node doesn't care what nodes are connected to its output channels.

Best regards!

[191220029]

Sorry for the late response, because genedna and I were at the 10 years of Rust Celebration yesterday.

So you want the nodes don't need to care where the data is from and where it is going to, so we don't need to know other nodes in the graph when implementing actions for a specific node? Maybe you can show us an example to provide more details about this idea.

Looking forward to your reply :)

Best

[bjmdev]

Hi,

thank you for your thoughts and in my opinion quick reply! :-)

maybe the problem is a limitation of how channels in tokio work. Unfortunately, I lack the detailed knowledge about that matter.

So let me explain in more detail, what I mean: again, I refer to TeeTime, a pipe&filter framework for Java, which embraces the idea of FBP.

There, each node is independently defined by its execution logic and typed input and output ports. In that way, the node is a complete blackbox and its input ports can be connected to any output port of another node as long as the type matches.

The connection works via a pipe which is basically a queue. So when the node is scheduled for exection, it can take an IP from the connected queue (if there is one), act on it and can then send IPs to its output port. The node doesn't have to know, which nodes are connected to its ports, because the IPs are pulled and sent from/to the connected queues.

The graph (network) is just for building the network, i.e. connecting ports and for the scheduler to know, which nodes to schedule.

That way, an implemented node can be reused in any network and is characterized by its execution logik and its typed in/out ports.

Example:

1. a source node which has no input ports but a single output port of type usize. Each time its execute method is scheduled, it sends the next number from e.g. a vector to its ouput port.
2. a multiplication node, which has one input and one output port, both of type usize. On execution, it receives the next number from its input port, multiplies by 2 and sends the result to its output port.
3. a sink with a single input port of type usize, no output ports. Receives a number from its input port and stores it in a vector.

Now, these can be connected and run. If the source vector contain 1,2,3, then the sink vector will contain 2,4,6.
Then, one can implement independently a different mapping node, which adds instead of multiplies, and the multiplier node can be replaced in the network.

Since the implementations are independent of the other nodes, each node is reusable in any network without having to change its implementation (e.g. changing node-ids to which IPs are sent).

I hope I could clarify my idea.

Best regards!

[191220029]

Thanks for your explanation, and I understand your idea now.

Currently, the dagrs team is planning to start a new phase of work, and the idea you put forward here will become part of our goal. We will write an issue in the next few days to introduce this matter in detail.

Best

[bjmdev]

Thank you, sounds good!