Feature Request: SNI/Archipelago support

[benserwa]

I play in multi-worlds using Archipelago, a service that allows ROM randomizers (such as Link to the Past and Super Metroid randomizers) to connect to a network and interface with each other. The way that SNES games do this is through Super Nintendo Interface, or SNI, as is shown here: SNI on Github

If the SNES core on Mister had SNI support, I could play my Archipelago games with accurate emulation on my CRT! I imagine SNI support would allow for other uses too, perhaps scripting smart devices to respond to in-game events.

The way I connect now is through a Lua bridge via Bizhawk. I'm not sure if that'd be the way to go here or not.

[littlegamer87]

There are already experiments happening on discord for SNI support, but it requires the ARM to run a SNI server application/binary. Not sure if that is admissible. Think they already reached out to sorelig for discussion.

If it is admissible, I would like to request a stable release before SNI is committed. Current unstable is an excellent candidate for a stable release with save states and rumble, so SNI can be tested separately.

[sorgelig]

I don't remember anyone reached me with SNI support.
Need to check how heavy is that binary. It may affect the core performance, especially input lagging if it requires to much processing power.

[NobodyNada]

@sorgelig Hi there! I'm currently working on SNI support. A few weeks ago I sent some test builds to people in speedrunning/randomizer communities for some initial debugging, and I posted about it in the MiSTer discord for feedback/design review, where I was advised to contact you and @paulb-nl. I have not done so yet because I was hoping to finish addresssing the first round of bugs/feedback and get things in a more presentable state first :) I'm a software guy, not an EE -- and I don't have any Verilog experience outside of one class in college -- so my progress is quite slow.

My implementation adds a UART mode in which the SNES core accepts memory read/write requests over UART. Then, a server daemon on the ARM side listens to requests over the network (via the standard SNI and QUsb2snes protocols used for this purpose) from applications running on a PC. The added binary is lightweight in terms of CPU power, using about 3% under heavy load. Also, it can run on the other core from the main MiSTer binary (which is usually almost totally idle on my system). It's written in Rust, so its dependencies are statically linked.

It would be great to eventually upstream the changes to the SNES core (and adding the UART mode to the main binary + uartmode script), once it's done and the quality is acceptable. I'm happy to provide the server daemon to upstream as well, but I don't mind maintaining it as a separate download (so project devs don't need to add a Rust toolchain to compile the world & so it doesn't add 5MB to download size for the majority of users who don't need SNI).

---

Here's my current work if you're interested in taking a look:

• SNES core changes (refactoring out UART module and adding the SNI interface) -- note that this is nowhere near complete & I'm in the middle of reworking it heavily: master...NobodyNada:SNES_MiSTer:sni
• Changes to main binary (adding the UART mode): MiSTer-devel/Main_MiSTer@master...NobodyNada:Main_MiSTer:sni
• SNI daemon (running on the ARM side): https://github.com/NobodyNada/snid

The core changes I linked are a proof-of-concept that implement memory access by "stealing" CPU access cycles where WRAM is idle. It seems to have some bugs (based on tester feedback) and does not support access to ROM. I'm currently in the middle of reworking this by moving the "cycle-stealing" logic into the SDRAM controller: adding a third access channel that can sneak in an access (on a best-effort basis) whenever one of the two primary channels is not accessing the target bank, without affecting access or refresh timings at all for the two primary channels.

---

I don't think any of this needs to affect your release process or schedule planning. It'll be a little while (at least a few weeks, likely longer) before my changes to the core are in what I'd consider an upstreamable state -- I have limited spare time to work on this, and I don't really know what I'm doing, so progress is slow.

[sorgelig]

According to description, it's quite a lot of change including additional service. So may be better to make it as a plug-in (using scripts) with separate development of service. It's quite a niche feature, so even SNES users won't need it in 99%.

I wonder one thing: since PC gets involved into the chain of communication, why don't connect PC to USER_IO and communicate it over UART without involving HPS part of MiSTer?

[NobodyNada]

According to description, it's quite a lot of change including additional service. So may be better to make it as a plug-in (using scripts) with separate development of service. It's quite a niche feature, so even SNES users won't need it in 99%.

Agreed -- that was my thinking exactly.

I wonder one thing: since PC gets involved into the chain of communication, why don't connect PC to USER_IO and communicate it over UART without involving HPS part of MiSTer?

Because then the user needs to buy/make additional hardware. Additionally, if I understand correctly this would prevent people from using the user I/O port for SNAC (the overlap between users who want SNI and users who want SNAC is likely to be quite high).

The standard SNI/QUsb2snes protocols already run over TCP, so hosting the server on the MiSTer itself is IMO the simplest solution from a software perspective and works with any hardware setup out of the box (the only requirement is a network connection to the MiSTer).

[sorgelig]

What is the role of PC in the chain?

[NobodyNada]

The PC is running the end user's applications. Common examples include:

• Autosplitters for a speedrun timer
• Item / map trackers for randomizers
• Race room integration
• Stream / chat integration features (guessing games, stats and room times, input display)
• Networked multiplayer (apps that synchronize elements of game state between multiple players)

The whole point of SNI is to allow applications running on a host PC to read/write SNES memory. And since users want to run multiple applications simultaneously and connect to a wide variety of hardware/emulators, these applications target standardized protocols that abstract away the underlying device behind a TCP socket.

[sorgelig]

Generally speaking this is beyond what MiSTer is targeted for. It has nothing to do with preservation. SNES core is already heavy core and i would rather to keep remaining room for main purpose - emulating original platform with high accuracy. So it's under a big question if this addon will be accepted.

[NobodyNada]

That's completely fair. I'll continue working on this and do my best to try to come to something that is acceptable. I'll send a set of PRs once I'm done, but if you don't want to merge them, that's fine. Hopefully the changes will end up being small enough to not be a problem. The UART interface is shared with the existing MIDI support (that's another advantage of using the HPS UART instead of user I/O), so the only additions are the state machine to handle read/write requests & the logic to arbitrate access to SDRAM.

IMO MiSTer is the best, lowest-cost, and most flexible platform for anyone looking for a high-quality SNES speedrunning setup -- but the lack of usb2snes/SNI is a deal-breaker for many people, which is why I've been working on this. I'd like to have the changes to the SNES core upstreamed so that I don't have to deal with the maintenance burden of a downstream fork, but I'll do that if I have to.

Happy to work with you however I can to support this use case in a way that is acceptable to the goals of the project.