Do we still need the provider indirection at all in V5?

[stefanhaustein]

I think I can experiment with implementing a context without providers for the MCP2221 Usb bridge, and if that works out, it might be worth trying the same for mainline / FFM....

[stefanhaustein]

Still a bit crude but I think this should be workable:

https://github.com/stefanhaustein/pi4j-usb-bridge/blob/main/src/main/java/com/pi4j/usbbridge/DirectContextBase.java

[eitch]

That makes sense

[stefanhaustein]

@IAmNickNack would it be feasible to refactor your "remote Pi" work using a similar approach, dropping providers?

[IAmNickNack]

Maybe. What's your beef with the current implementation?

I'd agree that the current provider implantation can be problematic, with priority handling and service loader dependency. Maybe that's an unnecessary complexity. (I can expand on this point when I'm not on mobile).

From a broader design point of view, if we were to consider them factories they make a lot of sense and fit with the S in SOLID. Their single responsibility is to manage the lifecycle of the component at startup and shutdown and allow the device component to focus on the actual doing of the work at runtime.

It's possible to move the lifecycle stuff elsewhere such as, as you suggest, the registry. But that clouds the registry's responsibility.

I'd say a big yes to removing the priority stuff and maybe also the service loader, but I'd question removing the abstraction for device specific lifecycle management.

[stefanhaustein]

Maybe. What's your beef with the current implementation?

It just looks like the opposite of KISS... Overuse of generics in too many places, reinventing a Map and calling it "Metadata", fluff Interfaces without any clear purpose such as "Digital" all over the place, unclear responsibilities paired with overly-generic class names. What is a "Descriptor" and why does it have "parent" and "quantity"?

I am not complaining -- it just looks quit a bit grown. And complexity is really hard to refactor for new challenges. So if the loss of the provider requirement presents an opportunity to cut back radically, I think we should take it.

From a broader design point of view, if we were to consider them factories they make a lot of sense and fit with the S in SOLID. Their single responsibility is to manage the lifecycle of the component at startup and shutdown and allow the device component to focus on the actual doing of the work at runtime.

I was assuming the providers were the factories whereas the lifecycle was managed by some combination of the Context, (Runtime-?)Registry and Runtime?

It's possible to move the lifecycle stuff elsewhere such as, as you suggest, the registry. But that clouds the registry's responsibility.

What is the registry's responsibility apart from keeping track of IO instances for life cycle management?

I'd say a big yes to removing the priority stuff and maybe also the service loader, but I'd question removing the abstraction for device specific lifecycle management.

I have adjusted the create call structure to allow us to keep the life cycle management responsibilty for individual ports out of concrete implementations.

[IAmNickNack]

It just looks like the opposite of KISS... Overuse of generics in too many places, reinventing a Map and calling it "Metadata", fluff Interfaces without any clear purpose such as "Digital" all over the place, unclear responsibilities paired with overly-generic class names. What is a "Descriptor" and why does it have "parent" and "quantity"?

I 100% agree with this. The complexity makes the framework unnecessarily complicated to extend or add new device types. I would also call out the depth of inheritance as a contributor to complexity.

I don't think this is a problem with the logical model of registry -> factory -> device though. I think it's more a style issue.

Maybe I've misunderstood your original question 🤔

[FDelporte]

This is a returning question... I agree the plugin/provider approach makes the Pi4J library more difficult to understand. But it has proven over time that it allowed us to get new implementations that added support for RPi5 and later added FFM, without losing previous implementations. So we always had backward compatibility. And we don't know what is next (other board types, other gpio use,...)?

So my idea is:

• Yes, for simplification within the current architecture
• No, for a total removal of the plugin approach

[stefanhaustein]

This is a returning question... I agree the plugin/provider approach makes the Pi4J library more difficult to understand. But it has proven over time that it allowed us to get new implementations that added support for RPi5 and later added FFM, without losing previous implementations. So we always had backward compatibility. And we don't know what is next (other board types, other gpio use,...)?

I am assuming that our current working assumption is that FFM+linux will be able to cover new "pi-like" boards

Any new type of IO should be covered by this, too -- but would probably be more difficult to integrate if it has to be piped through the plugin mechanism

I have put up a concrete example for a simplification supporting a quite different use case that still fits into how I would expect users wanting to use the API. We can try this out in a subproject without touching the core for now. I have called out @IAmNickNack specifically because I know he has a similar case and I wanted to know if it could theoretically work there, too

Suggesting we improve without touching too much sounds like asking for improvements to a spaghetti ball by shortening individual sphaghetti. I mean surely it's possible and easy to suggest in the abstract -- but it's likely a LOT more work than a more radical cut. And we don't have anything concrete that would show that the extra work would be worthwhile.

A plugin mechanism might be useful in the future for something -- but I doubt pulling the current one through a time where we don't have a concrete need will improve how it fits for new challenges as they arise

[eitch]

If we go a bit down memory lane, then the priorities were implemented because we had multiple providers for multiple IOs, but one IO was better served by provider A, while another by Provider B. The providers were created because we had to replace old third library party libraries which became deprecated and didn't work on newer kernels/boards, etc.

If i understand Robert Savage correctly, the plugin/provider architecture was originally created so that multiple boards could one day be supported, even though this was never completed. It started with plugins for testing and mocking etc., but didn't progress beyond that.

FFM brings us everything we need if we want to support other boards which use the Linux kernel. I don't really think that we want to support something other than the Linux kernel, or are we thinking of BSD kernels here? Otherwise FFM will work for all use cases down the road.

What FFM doesn't solve is mocking for tests and of course the use-case where we want to communicate with devices from our development machines, i.e. standard x64 systems on Windows etc. Here we need something, and this i what @stefanhaustein has been working on with bridges for I2C over USB etc. or am i mistaken?
So perhaps the whole plugin system can be replaced with a bridge?

And up to today i don't understand the difference between the runtime and the registry. For me both do the same thing or at least can be folded into the context class. In the end we need a map for the IO instances and they need to have a life cycle management. Three classes for this is not KISS imho. =)

If i had the time, then i would consolidate those three classes, remove the plugin infrastructure but provide interfaces so that it is possible to bridge the IO where needed.

[IAmNickNack]

I have called out @IAmNickNack specifically because I know he has a similar case and I wanted to know if it could theoretically work there, too

I think anything could work so long as the underlying abstractions are generic enough.

From my point of view, this is the key point. If you lose the abstractions, you run the risk of losing the extensibility of the framework and possibly run the risk that implementing or integrating extensions becomes more intrusive in the core APIs and more busy-work than necessary.

Just my 2 cents.

[DigitalSmile]

I am assuming that our current working assumption is that FFM+linux will be able to cover new "pi-like" boards

I think it can be just FFM, since there is nothing left for linux plugin, all is covered

A plugin mechanism might be useful in the future for something -- but I doubt pulling the current one through a time where we don't have a concrete need will improve how it fits for new challenges as they arise

Let's try to think in specific examples. There is a device with hardware interfaces. We cover most of them (or even all?) with FFM implementation. But there are plenty of application level APIs, that relies on a hardware. The most specific example is long story with one-wire -> #435 May be we can left the plugins mechanics as a part of application level protocols? Still I am not aware how many of them exists, except of some popular, like CAN.

What FFM doesn't solve is mocking for tests and of course the use-case where we want to communicate with devices from our development machines, i.e. standard x64 systems on Windows etc.

I have been working on this some time ago:

1. Mocking on a testing level works great with mock linux kernel drivers. I implemented couple of them here -> https://github.com/Pi4J/pi4j/pull/539/changes#diff-3299467305952d05e58defbd4ff6bacc1f6f473a49a7d8c28154c735b819441e They are simple and provides "echo" functionality for testing with CI or in any linux machine (does not matter x86 or arm64). I need to recap the work here but it was done by 85%. So mocking will be covered.

2. As for local development, I was thinking about test-containers https://java.testcontainers.org/ - it's solid and reliable way to test your code in a local environment. It requires Docker though (oh shit, here we go again). But the good thing it can be easily configured with desktop app (https://testcontainers.com/desktop/) or from code. I can research this part and it will be cool to integrate it with @IAmNickNack solution for communication. So we can decide.

---

tldr; I totally agree we need to simplify the architecture and class hierarchy, it is quite redundant and feeling useless now.

[eitch]

I find dockers are totally fine for such test cases. But it must be totally optional for development and building.

[stefanhaustein]

Let's try to get clarification on 1-wire on this thread: #429

What FFM doesn't solve is mocking for tests and of course the use-case where we want to communicate with devices from our development machines, i.e. standard x64 systems on Windows etc. Here we need something, and this i what @stefanhaustein has been working on with bridges for I2C over USB etc. or am i mistaken?
So perhaps the whole plugin system can be replaced with a bridge?

I am not super concerned about mocking, I think a more direct architecture would still allow us to re-use the existing mocks for testing with a mock context (now wondering if they aren't really fakes just named mocks?). It would make it harder to "mix" ports, but I'd tend to argue that if people really want/need this for some obscure reason, they could implement a "dispatch" context that delegates.

What I have been working on is basically trying to demonstrate that bridges would work with a simpler architecture (illustrated in DirectContextBase.java) as an example -- to make the case that everything we are currently aware of should work with a simpler architecture.

[DigitalSmile]

But it must be totally optional for development and building.

I don't thing there is any way you can develop on Windows without docker. But if it's a linux machine you can go straightforward without docker.

I think a more direct architecture would still allow us to re-use the existing mocks for testing with a mock context (now wondering if they aren't really fakes just named mocks?

Mocks context could be just a wrapper around FFM which creates a container and run the code inside. Simple as that.