WonderSwan support

[asiekierka]

Hello! As the WonderSwan has the interesting distinction of being a game console powered by an 8086-compatible CPU, I've sometimes been idly wondering if a port of ELKS to the platform would be possible, and would like to inquire with people who have more experience with ELKS about it.

The main catch is that the WonderSwan doesn't have a lot of RAM. The mono model comes with 16 KB, shared between the video hardware and the CPU; the later color model expands that to 64 KB. However, it's not uncommon for cartridge to provide additional RAM on-board; the largest of which having been 256 KB and 512 KB back in the console's heyday.

... But, it's banked. That is:

• the 16-64 KB of console RAM is hooked up to a 16-bit bus at 0x00000 - 0x0FFFF, but requires reserving some holes of RAM in the middle for video data. For C programs from the Swan's heyday, it is typically used to store the stack and other "fast" variables.
• the cartridge RAM is hooked up to an 8-bit bus and is available as a contiguous, manually banked 64 KB window at 0x10000 - 0x1FFFF.
• there is also cartridge ROM, available at 0x20000 - 0xFFFFF.

Of course, this also means that the WonderSwan is not IBM-compatible in the slightest. For a brief introduction to its video/sound/UART hardware, I encourage looking through my hardware overview, as well as the wiki I maintain.

In terms of CPU performance, the 3 MHz NEC V30MZ is a fairly accurate 80186 clone (unlike the V20/V30 family) with fast instruction timings, putting it in the ballpark of a 6-8 MHz, maybe? 286.

My question is, before I dive deep into it myself - does it appear feasible to adjust ELKS to work under these constraints?

[ghaerr]

Hello @asiekierka, nice to see around here! :)

I would say that the ELKS kernel with no applications might possibly be able to run in 20k+ RAM and 36k ROM, but then it really depends on what you want to do after that. Will you be running the kernel ELKS kernel CS segment and a filesystem from ROM, and then use some extra cartridge RAM to run some applications? I'll comment on how some of those scenarios might be made to work.

I haven't had time to look at your hardware overview yet, so we'll just stick to basic kernel, filesystem and RAM configurations for the moment.

The kernel itself is easily run from ROM (min ~36k?), and can be setup to run without any BIOS services. A HW timer helps a lot, but neither PIC nor PIT are necessarily mandatory, using special options. The kernel data segment is highly configurable based on options in elks/include/linuxmt/config.h, shown here for a small 19k data segment (normally 64k):

/* The following can be set for minimal systems or for QEMU emulation testing:
 * 10 buffers (@20 = 200), 2 ttyq (@80 = 160), 4k L1 cache, 512 heap free,
 * 10 tasks (@876 = 8760), 64 inodes (@80 = 5120), 64 files (@14 = 896) = ~19744.
 * Use buf=10 cache=4 task=10 inode=64 file=64  in /bootopts
 */
#if defined(CONFIG_HW_MK88)
#define SETUP_HEAPSIZE            19744         /* force kernel heap size */
...

An ELKS filesystem can be put in ROM and a ROMFS filesystem driver compiled into the kernel, using various .config options. The entire result can be emulated on the desktop using emu86 (see emu86.sh) and one of the emu-*.config prebuilt config options.

The kernel itself is going to be at least 36k, so running it from RAM will be more problematic with your limited memory, although I suppose possible out of the cartridge RAM. The kernel doesn't know anything about bank switching, although our XMS (extended memory) buffer system uses a similar mechanism for managing data buffers that aren't directly addressable.

The bigger problem comes after kernel startup, when /bin/init (unlikely) or /bin/sh or /bin/sash (standalone shell) are run. Our ASH (bash) shell is huge at 68k and sash at 50k:

# ps
  PID   GRP  TTY USER STAT CPU  HEAP  FREE   SIZE COMMAND
    1     0      root    S   0  1060  4020  12944 /bin/init 
    6     6    1 root    S   0  1184 13132  68640 -/bin/sh 
    9     9   S0 root    S   0  1184 13204  68640 -/bin/sh 
   11     9   S0 root    S   0  1330  6518  50144 sash 
   12     9   S0 root    R   0    50  2078   8848 ps 

Sash could possibly be made to run within a system composed of 32k of base RAM and 64K of cartridge RAM, but I'm wondering what else will?! Each application data or code segment has to be contigous within itself, so it can't be split across say, main memory and cartridge ROM, although the code or data segment(s) separately could.

This probably means you might just want to run a custom application instead of a shell. That would be straightforward as it'd be pretty much like writing a small Linux or POSIX program, as far as system calls are concerned.

Most all the kernel resources, # tasks, # buffers, # inodes, pipe, serial and keyboard input queue sizes, etc. can be sized way down to something like 15-20k kernel data. But even though our applications are seemingly tiny, some are still pretty big, when you're talking 64k RAM total.

Much time has been spent removing external dependencies from the ELKS system, especially BIOS and external hardware. More information about that is detailed in Documentation/text/porting-guide.txt. The WonderSwan's PIC and PIT should be ported, but that's fairly straightforward. A new console driver might have to be written, which is a bit of work, unless you just want serial I/O to start.

So yes, it is possibly feasible, but still could be a lot of work (and fun).

[asiekierka]

Thank you! That tells me it's very much within the realm of possibility.

As for available memory, here's a more helpful summary:

---

RAM provided by console

WonderSwan "mono"

The WonderSwan "mono" has 16K of fast RAM mapped at 0x00000 - 0x03FFF, but some of that has to be reserved for graphics if console I/O is to be shown on display. There are two routes:

• larger (8x8) font (for a 28x18 display) - 256 x 16 bytes for tiles, plus 1024 x 2 bytes for tilemap data. This means the area 0x02800 - 0x03FFF has to be reserved.
• smaller (4x8 or 4x6) font (for a 56x18 or 56x24 display) - 504 x 16 bytes for tiles, plus 1024 x 2 bytes for tilemap data. This means the area 0x01800 - 0x03F7F has to be reserved. This would also allow for a graphical "framebuffer"-esque mode to be implemented.

This leaves anywhere between 6 KiB and 10 KiB of contiguous "fast" RAM that can be allocated to various things.

WonderSwan Color

The color model has 64K of fast RAM mapped at 0x00000 - 0x0FFFF. The same tile restrictions apply as previously, though there's more flexibility in arranging where exactly the gap would be; in addition, there's potential for a 4bpp framebuffer, requiring (with tilemap) reserving an area from, say, 0x03800 - 0x07EFF. In addition, color palette data is stored at 0x0FE00 - 0x0FFFF fixed.

This gives two RAM holes: one somewhere between 6 KiB and 14 KiB in size, the other somewhere between 32.5 KiB and 48 KiB in size.

RAM provided by cartridge (and etc.)

Generally, WonderSwan cartridges came with RAM sizes up to 512 KiB and ROM sizes up to 16 MiB; however, the RAM and ROM are both banked. This means that up to 64 KiB of RAM and 896 KiB of ROM is accessible at any one time. This is a hardware limitation enforced by the console - addresses 0x20000 - 0xFFFFF cannot be written to.

WonderWitch

The WonderWitch is the official homebrew development kit for the WonderSwan; as such, for reasons of vaguely snobbish accuracy, I'd like to get ELKS running on it too.

This cartridge has 448 KiB of user-programmable NOR flash (and 64 KiB of fixed NOR flash containing the cartridge's own BIOS) which could house the romfs and kernel, as well as 256 KiB of cartridge RAM. As the BIOS reserves some console RAM for itself, that 64 KiB window would need to be sufficient for running ELKS on this platform (unless the display is left unused).

nileswan

nileswan is a flash cartridge I happen to be working on. It provides 16 MiB of ROM emulated using PSRAM, and 512 KiB of RAM in that 64 KiB window. In addition, it provides an SPI bus for communication with a removable storage card (which, thankfully, can be easily adapted to ELKS based on what I've seen of its code).

Interestingly, if code segments can be guaranteed to be read-only, they could be read into this emulated ROM area dynamically; that is, it can be read like standard 8086 memory, but it can only be written through the RAM window. I am, however, not sure if ELKS makes that guarantee at any level, given we're running in real mode :-)

WS Flash Masta

The WS Flash Masta is a popular flash cartridge for the console with 8 MiB of ROM (via NOR flash) and 512 KiB of RAM in that 64 KiB window. No special features are offered.

---

Overall, I feel the best thing to do to take full extent of the WonderSwan would be to figure out if it would be possible to adapt the kernel's process/thread subsystems to take advantage of SRAM banking (and nileswan ROM emulation), by allowing programs which don't share code/data segments to be bank-switched.

That is not necessary for an initial port, of course, but it would make ELKS on the platform substantially more usable.

Oh, and we also have a networking peripheral...

[ghaerr]

that 64 KiB window would need to be sufficient for running ELKS on this platform

Hopefully I'm not sticking my neck out in thinking a minimal ELKS could run in 64K RAM! Building the emu86-rom.config version yesterday, the kernel was ~36k, which if set up for a small kernel data segment +~20k = ~56k. The ROMFS would have to go somewhere else and the RAM for applications could be added to the main memory allocation list from other sources and managed by segment descriptor and length.

if code segments can be guaranteed to be read-only

not sure if ELKS makes that guarantee at any level, given we're running in real mode :-)

Well, certainly this depends on what "guarantee" means :) There are no hardware guarantees, 'cuz we're running real mode. But I can "guarantee" that all the previous code writing to code segments has been removed, and the system doesn't do so... until user applications start running when the guarantee is off... (BTW, application code segments are shared, for what it's worth, and there are no known cases of writing to a code segment).

to figure out if it would be possible to adapt the kernel's process/thread subsystems to take advantage of SRAM banking (and nileswan ROM emulation), by allowing programs which don't share code/data segments to be bank-switched.

Indeed, the kernel syscall/interrupt/process subsystem is the (elegant) core of ELKS (there are no threads). Understanding this is key to how the system operates, especially with nested interrupts and kernel re-entry. This has been previously contemplated during a discussion of adding process swapping, and it will be very tricky. The problem with general process swapping is that the swapin address has to be the same as swapout, since the code can't be relocated solely with a segment register switch. But in the bank-switch case, that's a non-issue since the address remains constant.

I wrote up a two-part article on exactly how the core task/interrupt processing works; though there's been a slight modification to the actual workings since.

[asiekierka]

Hopefully I'm not sticking my neck out in thinking a minimal ELKS could run in 64K RAM!

Ah, but on the Color machine, its total of ~128K of free RAM (64K game console + 64K cartridge) can be neatly subdivided between console emulation, kernel, and userland. And we can of course keep the ROMFS and kernel code in ROM; there's no problem with doing that.

[asiekierka]

SR
  Headless console
ttyS0 at b1, irq 11
syscaps 0, 128K base ram, 16 tasks, 64 files, 96 inodes
ELKS 0.9.0-dev (40592 text, 0 ftext, 4928 data, 1728 bss, 40960 heap)
Kernel text e062 data 3f8 end 1000 top 2000 64+0+0K free
VFS: Mounted root device 0600 (romfs filesystem).
/bin/init: Exec failed: /bin/sh (errno 2)


ELKS 0.9.0-dev

[0.13 secs] login:

Status update! Was easier than expected. I'll need to write real drivers for most things, though.

[ghaerr]

Status update! Was easier than expected.

Good to hear :) There's been a lot of work reducing the system's external exposure, and we've tried to keep all builds running, including the ROM build. I do see that /bin/sh exec failed, not sure if you're suing emu86-rom.config or something close. The kernel init by default runs /bin/init unless CONFIG_SYS_NO_BININIT is set; see elks/init/main.c.

You may also want/need to change the applications that are on ROMFS, that's done via script. Let me take a look to remember exactly how that is being done. Finally, you'll want to force reduce the kernel heap down from 40k using SETUP_HEAPSIZE in config.h and possibly other limits in limits.h, since you won't have a /bootopts to dynamically change them.

[asiekierka]

I've already done a bunch of tweaking to config.h; I've had to add a way to manually force membase to be a specific value, as the way I'm currently laying things out is that the kernel data + heap lives in 0x03F80 - 0x0FDFF, while the user heap lives in 0x10000 - 0x1FFFF; so, the user heap cannot immediately follow the kernel heap.

If there's one thing I'm wondering about, it's "why is it required to have both a console driver and a serial driver to build"? It feels like there should be an option to "Force serial console", at least for early development; in this platform's case, I want to use a serial console for initial bringup, but I hope to at least bring output to a direct console driver using the console's display.

EDIT: Oh, and a second thing - it seems all elkscmd applications try to compile always, even if a given target does not provide support for some of it (like basic or clock or nano-X).

[ghaerr]

why is it required to have both a console driver and a serial driver to build?

It isn't, there's two options that you'll need to set: CONFIG_CONSOLE_SERIAL which routes all kernel printk's to serial, and CONFIG_CONSOLE_HEADLESS, which it looks like you're already using. CONFIG_CONSOLE_HEADLESS is a much simplified console driver that uses a lower level set of functions that should be easily re-routable to serial I/O.

it seems all elkscmd applications try to compile always, even if a given target does not provide support for some of it (like basic or clock or nano-X).

Yes, it was deemed better to always keep all applications compiling to prevent bit rot. Then, in somewhat of a bad idea, it was allowed to have applications built differently based on platform. Bad idea, but had to be done; this could be improved. For now, the easiest solution is to edit the SUBDIR= line in elkscmd/Makefile and remove the directories for applications that won't currently build for you. For clock, you'll have to edit sys_utils/Makefile as well. We can fix this up after you've got a ways further into your port.

[rafael2k]

So nice to see ELKS running on another device! Yay! Very cool!


Thanks @asiekierka !

[asiekierka]

It has been a while! I've decided to come back to the project to implement some features to try and manuever past the 64 KiB process RAM limit:

• Directly using the in-ROM text segment of a program stored in romfs, avoiding the need to copy it to RAM;
• Bank switching; storing each process's data segment and heap in its own separate 64 KiB RAM bank.

With these two combined (and a few minor tweaks), I managed to make ELKS on the WonderSwan surprisingly usable:

test2.mp4

[rafael2k]

Wow!! This is nothing short than amazing @asiekierka !!!!
Thanks for such development!

[asiekierka]

My custom flashcart for the platform has a rather large amount of RAM onboard, as well as a SPI bus which I can use for external storage. I've used some of it to emulate 768 KiB of ROM (I call it ROM as it is writable, but only with some fiddling), which acts as an allocation space for code segments.

[rafael2k]

Lua on ELKS on the WonderSwan!
You use the big ROM space for the text and data segments of the executables/processes, and the ram for the heap and stack?

[asiekierka]

Text segments only. Data is read/write, so must be stored in RAM.

You can use the OpenWatcom -zc flag to put read-only data in the text segment, which helps.

[rafael2k]

Cool, thanks @asiekierka.

[rafael2k]

If I manage to put my hands on a WonderSwan, which kind of cartridge would I need to "burn" ELKS to it?