games/NXDoom: Add RGB565, module support, and crash fixes.

[aviralgarg05]

Note: Please adhere to Contributing Guidelines.

Summary

Update NXDoom for RGB565 framebuffer targets and optional loadable-module
builds, and fix crashes observed during ESP32-S3 bring-up.

The changes:

• make CONFIG_GAMES_NXDOOM tristate and derive MODULE from it;
• support FB_FMT_RGB16_565 while retaining FB_FMT_RGB32;
• validate framebuffer format and depth before drawing;
• keep renderer scratch buffers static by default, with heap allocation behind
  CONFIG_GAMES_NXDOOM_HEAP_BUFFERS;
• parse configuration one physical line at a time, retain defaults for
  malformed integer values, and clamp screenblocks;
• guard plane/span renderer indices and check visplane capacity before writing;
• handle supervisor SIGTERM at a safe point in the frame loop.

Impact

• New feature: YES. NXDoom can be built in with =y or as a loadable ELF
  with =m, and can render to RGB565.
• User adaptation: NO for existing RGB32/static-storage configurations.
• Build: YES. The Kconfig symbol becomes tristate and derives MODULE.
• Hardware: YES for framebuffer compatibility; RGB565 is added without
  changing a framebuffer driver.
• Documentation: NO user-documentation change is required for these
  compatibility and correctness fixes.
• Security: NO known security-boundary change; the renderer and parser bounds
  fixes prevent invalid memory accesses.
• Compatibility: Existing static renderer storage remains the default. Heap
  storage is opt-in. Unsupported framebuffer layouts now fail initialization.
• Other: A direct-framebuffer build can reserve nxstore's supervisor strip.

Testing

Build host:

• macOS 26.5, arm64
• xtensa-esp-elf-gcc 14.2.0
  (esp-14.2.0_20251107)

Target:

• Xtensa / ESP32-S3
• Waveshare ESP32-S3-Touch-LCD-7 custom configuration
• RGB565 framebuffer

Static verification:

• nxstyle and git diff --check
• all NXDoom sources cross-compiled for ESP32-S3
• independent review of the complete diff

The exact final source head was built and flashed. NXDoom reached gameplay on
RGB565, malformed configuration input retained safe defaults, and the process
returned cleanly to nxstore after SIGTERM.

Hardware bring-up photograph from July 11 (the later signed-tree checks are
recorded below):

Testing logs after change:

LD: nuttx
Generated: nuttx.bin
Hash of data verified.
NXDoom v0.0.0
W_Init: Init WADfiles.
 adding /mnt/sdcard/doom/doom1.wad
r_init: Init DOOM refresh daemon - ...................
p_init: Init Playloop state.
nxstore: close cb fired, active=1 pid=13
nxstore: close SIGTERM sent to 13
nxdoom: quit signal seen, calling i_quit
nxstore: close reaped=0 gone=1 after 32 tries

Malformed-configuration reproduction:

fixture: screenblocks has no value
fixture SHA-256: 8d4d67d909e257165e28478cd0ee42e18af7c0fa4c9ebb0f908fb68cb4bcf981
result: compiled default retained; gameplay started; SIGTERM close completed

The original divide-by-zero or renderer-exception log was not retained, so it
cannot be included as before-change output.

The final restored image remained stable while idle; the earlier glitch
coincided with a failed transfer whose Base64 payload was interpreted as NSH
commands, not ordinary nxpkg file writing.

PR verification Self-Check

• This PR introduces one focused NXDoom bring-up/correctness change set.
• I have updated all required description fields above.
• I have reviewed and signed every rewritten commit.
• I have attached the retained real-hardware evidence and imagery.
• This PR adheres to the current contribution and coding guidelines.
• My PR is still work in progress.
• My PR is ready for review and can be safely merged.

[linguini1]

The PR description needs to be revised and made more concise. It reads totally AI generated and is hard to follow, and also has some subtle mistakes (i.e. " happy to split further if any single commit above should be its own PR" when there is only one). Pretty much the entirety of this PR is just patches to NXDoom, we don't really need to know about the other module loading PRs here, just the build system change you made to load this as a module. You are expected to review the output of AI agents before submitting here, even as a draft :)
You should also use the Assisted-by: field in your commit messages, see the new updates to the contribution guide :)

Again, the testing section cannot just be a description of what you tested. Can you show the simulator playing DOOM after the changes, or your target device? Could you show some before/after output for the divide-by-zero and other issues reported here?

I don't see the need to malloc the arrays that were previously statically allocated. Using malloc on embedded devices is not a good practice and should be avoided where possible. I think those changes should be reverted.

[aviralgarg05]

Thanks for the thorough review @linguini1 going through each point:

PR description: You're right, that was sloppy, I squashed the commits down but didn't re-read the description afterward, so it still had a line referring to multiple commits that no longer existed. Rewriting it now

Assisted-by field: Didn't know this was now expected, will add it going forward, including updating this PR's commit message.

Testing section: Fair — I have this on real hardware (ESP32-S3 board, RGB565 framebuffer) but didn't capture it for the PR. I'll get a screen recording of it running plus the crash logs from before the fixes and attach both.

malloc vs static arrays: The reason was DRAM budget, these buffers (visplanes/openings/drawsegs/vissprites) are sized generously above vanilla DOOM's limits, and as static arrays they were eating into internal SRAM budget once this actually gets linked into a full firmware image alongside everything else. Heap allocation moves them onto this target's PSRAM-backed heap instead.

But you're right that malloc-by-default isn't great practice, I'll put it behind a Kconfig option instead, so boards that don't need it keep static allocation as the default, and boards that are DRAM-constrained can opt in.

Will push an update addressing all of this plus the inline comments one by one

[linguini1]

But you're right that malloc-by-default isn't great practice, I'll put it behind a Kconfig option instead, so boards that don't need it keep static allocation as the default, and boards that are DRAM-constrained can opt in.

Can we resolve this with FAR designators?

[aviralgarg05]

Can we resolve this with FAR designators?

FAR is defined as empty on this target (Xtensa/flat 32-bit memory model). It only expands to something real on segmented 8/16-bit architectures like 8051 (__xdata) or old x86 memory models (far/_Far), where it's about pointer segment qualification, not memory placement.

The actual problem here is static/BSS footprint at link time — a large static array is always-resident regardless of what pointer qualifier you put on it, FAR included. That's a different axis from what FAR addresses, so it won't help with the DRAM budget concern. The Kconfig-gated heap allocation is still the mechanism that actually moves the buffer's footprint onto the PSRAM-backed heap instead of internal SRAM.

[aviralgarg05]

Thanks for the detailed review. I went back through the PR and addressed the
main points:

• the description is now focused on the NXDoom changes;
• every rewritten commit has the required sign-off and Assisted-by trailer;
• renderer buffers remain static by default, with heap allocation behind the
  explicit Kconfig option;
• the Testing section includes a real RGB565 target photograph, startup and
  clean SIGTERM-close output, and the malformed-configuration result; and
• I removed the unsupported argv-sentinel change and corrected my earlier
  root-cause claim in that thread.

The original exception log was not retained, and the Testing section now says
that plainly. I have replied to each remaining inline point and resolved the
threads after checking the updated diff and evidence.