Upgrade makefiles to use MekkoGX and single top level Makefile

[FozzTexx]

Draft PR to review concept, do not merge!

Why I'm doing this now: I’m working on the MSX bring-up, and the old system means dropping yet another huge platform Makefile into the root. Mostly boilerplate, not fun to maintain, feels like clutter.

The biggest win here is having everything in a single, easy-to-read Makefile. Instead of a dozen (or more) platform-specific Makefiles scattered around, there’s just one main Makefile to look at. All per-platform tweaks (C64, CoCo, Apple II, Atari, and even the MSX stuff I’m working on) are visible right there, clear and explicit.

The makefiles/ directory handles all the messy boilerplate behind the scenes. Less-experienced developers don’t need to touch those files at all. Everything they need is explicit in the top-level Makefile. (One Makefile to rule them all!)

A concrete example: when Rich tried to get CoCo working with FUJINET_LIB, the old Makefile had no concept of it. He had to find the right spot in Makefile.coco, figure out what boilerplate to copy from which other Makefile, and troubleshoot download and linking issues. With this new structure, it’s trivial: if FUJINET_LIB points to a version built for the platform, the Makefile automatically sets up all the compiler and linker flags needed to use it.

Other nice touches: everything that is created/downloaded during build ends up in one of three directories: build, _cache, or r2r. Binaries and disk images end up in r2r (Ready 2 Run), you can swap FUJINET_LIB on the fly, and you can tweak a single build run by passing variables on the command line. This structure also includes the platform mapping feature in other FujiNet projects, exposed as a PLATFORM_COMBOS variable in the main Makefile, letting platforms share source without duplication (such as Dragon using CoCo). Overall it just makes contributing much easier and less error-prone, especially for people who haven’t memorized all the quirks of each platform.

Some highlights of what this change does:

• Single top-level Makefile for all platforms: all customization is done there: extra flags, library paths, special tweaks for Apple II, Atari, C64, CoCo, etc. You don’t need to touch anything inside makefiles/. That folder is just the engine running behind the scenes.
• Tiny platform-specific Makefiles: just a dozen lines to define what’s unique; the heavy lifting is handled automatically.
• FUJINET_LIB is easy to swap: point it to a directory, a version number, or a git URL. You can still override it on the make command line or with an environment variable, and now switching between local checkouts, releases, or remote repositories is trivial. This makes local development and testing even smoother.
• Standardized output: everything lands in r2r/ automatically.
• makefiles/ is the same for everyone: copied directly from MekkoGX and not customized for this project, so all projects share the same consistent build logic. When MekkoGX is updated, the makefiles/ folder can be refreshed in one step without touching the top-level Makefile.
• Less boilerplate, more consistency: common tasks are centralized in makefiles/, so you don’t have to look there unless you really want to tweak the underlying system.

This isn’t about reinventing the wheel. It’s about making the build process easier for everyone, whether you’ve been working on these platforms for years or are just getting started. You can run builds, swap libraries, or try out a new workflow without having to fight through a maze of copy-pasted Makefiles. And if you do need to tweak something, the structure is consistent, clear, and easy to reason about.

Do not merge!

[markjfisher]

https://github.com/FujiNetWIFI/fujinet-lib/tree/main/makefiles
https://github.com/FujiNetWIFI/fujinet-lib-examples/tree/main/makefiles
https://github.com/FujiNetWIFI/fujinet-build-tools/tree/main/makefiles
https://github.com/FujiNetWIFI/fujinet-iss-tracker/tree/main/atari/makefiles
https://github.com/FujiNetWIFI/fujinet-cater/tree/main/makefiles
https://github.com/FujiNetWIFI/fujinet-contiki-webbrowser/tree/main/makefiles
https://github.com/FujiNetWIFI/fujinet-tools/tree/master/post_migration/makefiles
https://github.com/FujiNetWIFI/fujinet-apps/tree/master/netcat/makefiles
https://github.com/FujiNetWIFI/fujinet-weather/tree/main/makefiles

it's like we've seen this all before.
is there any particular reason you chose to start again?

[FozzTexx]

is there any particular reason you chose to start again?

Motivation for this came from recurring pain points I’ve encountered (and seen others encounter), conversations across several Discord channels, and observing newer contributors struggle with some of the existing Makefiles. The goal is simply to make the build process easier to understand and smoother for everyone; experienced developers, newcomers, and anyone in between.

1. Using in-progress versions of FUJINET_LIB: your “make a zip and copy it” approach works fine, but this new setup lets FUJINET_LIB point directly to a directory, release, or git URL and builds against it automatically. That makes it easier to switch between multiple library versions without extra scripts.
2. Adding new toolchains: the existing makefiles are centered on cc65, which makes it harder to add or test non-cc65 targets. This setup isolates the toolchain logic so other toolchains (like CoCo, Adam, and soon MSX) can be added more easily without touching unrelated code.
3. All project-specific tweaks in one visible place: everything lives in the top-level Makefile, with documentation for what each setting does. Nothing in makefiles/ needs to be touched unless you’re bringing up a new platform.

I realize this isn’t an incremental change, which is why I opened it as a draft PR. My goal is to explore how we can adopt these improvements in a way that works for everyone. I’d really like to hear ideas about what an incremental rollout could look like that achieves these goals (per-platform tweaks, FUJINET_LIB swapping, and easier toolchain additions) without disrupting workflows people are already comfortable with.

[markjfisher]

Adding new toolchains: the existing makefiles are centered on cc65

Simply not true. https://github.com/FujiNetWIFI/fujinet-lib/blob/main/makefiles/compiler-cmoc.mk and others in that folder.

Not every project uses the compiler extensions, but that was the point, they were brought in incrementally and tested in fujinet-lib by many authors (Eric, Thom, Jan, Geoff, and myself)

everything lives in the top-level Makefile

again, simply not true, and if you are saying it is because everything is put into the makefiles dirs, then so is current build. In fact, it couldn't get much simpler, you only have to change the list of targets and just add your platforms specific code in platform specific folders under the src folder. No need to edit mk files, just as you are claiming.

everything in a single, easy-to-read Makefile. Instead of a dozen (or more) platform-specific Makefiles scattered around

really? Does this statement align with this:

Nothing in makefiles/ needs to be touched unless you’re bringing up a new platform

exactly the design goal I have too. New platforms go in their own make file. There is no need for the user to edit any of the files in the makefiles/ folder unless improving the overall build system, e.g. adding a new platform or compiler

but this new setup lets FUJINET_LIB point directly to a directory, release, or git URL and builds against it automatically

Did that warrant ignoring the other build and writing your own?
There was already a bash script for this, all you've done is replace it with a python script that handles additional fetching. That script is a glorfied copy. Every build needs the library, it's usually downloaded from a release into _cache (a concept I see you copied into your build), all you have here is extra ways of doing exactly the same thing.

At the moment, most users simply update the version string of the fujinet-lib they wish to use. Those developing against fujinet-lib already have a mechanism for copying a lib being created locally, you've added some bells and whistles to that same mechanism.

The goal is simply to make the build process easier to understand and smoother for everyone; experienced developers, newcomers, and anyone in between

The current build also has all these points in mind. The user simply sets their target list, the lib version they want, and puts their src in the folder structures to support multi-target, within target, or standalone systems.
I don't see a single thing you're claiming to be doing here that wasn't already supported, or could easily be improved on.

My issue here is you're introducing another fragmentation. as xkcd observed https://xkcd.com/927/
There is nothing in this that couldn't have been incrementally brought to the existing builds.

This is a continuation of you creating something new rather than actually extending or improving on things that already exist.

[dillera]

Waiting to hear about about the Makefiles- either it's one single makefile or it's a collection of them. If this is more than one, it's no better (just different) than the existing one- of which I personally enjoy using and have adpated at least 3 apps to use it (while pulling them out of the dreaded fujinet-apps repo/graveyard).

So unless this really does move the ball forward and can be shown to make life easier for more than just one person, it should be rejected, and positive aspects of this PR (like the more flexible fn-lib targeting) should just be moved into the existing system as new features we all can use.

[FozzTexx]

If this is more than one, it's no better.

So unless this really does move the ball forward and can be shown to make life easier for more than just one person

Until someone is actually going to look at it and stop dismissing it out of hand with "well, we had a makefiles dir before and this also has a makefiles dir so it's not any different" then I don't know what to tell you. I've provided detailed documentation on how all the internals works in the README.md in the makefiles dir. I've listed what the differences are and what the pain points are that motivated me. However, I can list them again:

• Switching out FujiNet lib easily for other versions by changing a single variable that accepts multiple formats
• platforms and toolchains are separated and use very small .mk files with boilerplate stuff in common.mk and tc-common.mk making it extremely easy to add new platforms and new toolchains
• Per project customization is done entirely in the top-level Makefile, no changes to anything in the makefiles dir is required

As another experiment I converted the news app to use this and all of the different platforms are using these makefiles, including Atari Pascal: https://github.com/FozzTexx/fujinet-news/tree/feature/MekkoGX/clients

Maybe someone would like to try converting weather?

and positive aspects of this PR (like the more flexible fn-lib targeting) should just be moved into the existing system as new features we all can use.

I would love to do that, but I’ll need some assistance on how to incrementally upgrade fujinet-build-tools. As has been noted before, sometimes starting on a “green field” is the only way to experiment and validate a new approach, which naturally makes integration more challenging.

Let's also keep in mind that this project doesn't use the fujinet-build-tools at all, which is why it seemed like a good place to make this change.

[markjfisher]

Ok, I think you set a fair challenge converting the news app for clients, which I spent the last couple of hours doing to challenge myself to see how much effort it would be.

I did all the same things as you, moving src into its own src/ folders in some projects.

Whereas you've gone for clients/src/, I defaulted to clients/common + clients/platform, as to me clients/ is the exception in projects I've been trying to build, where they mostly share common source, using fujinet-lib.
But news is a bit different.

I copied in the makefiles, and created the top level Makefile, and set the target list in it.

I was able to build apple2 pretty much immediately.
c16 and c64 were also fairly simple, the folders don't quite follow the convention I'd set out for where c16/c64 should live (I defaulted to commodore after talks with devs in discord at some point).
I would have also refactored the c16 and c64 into a single folder, and either ifdef'd the differences between them (mostly just colours and the includes), or moved the 3 files into sub-target folder for the differences.

Where the build-tools based project fell short was compiling atari.
You've built madpascal into the tooling.
You needed a supporting python script to do so due to the way that compiler works, but you did it.

So I thought, how would I integrate it into build-tools? Then I realised, why should I even try?
You've done the work, and it is clean, it's extensible, and in eating my own food, it does have a lot of warts, to bring it up to the level you have. And I wouldn't want to burden others with trying to get the features you've already got working into build-tools based projects.

I also realised how many differences there still are inherent in the different versions of the build-tools versions; one supports multiple compilers, some of them have slightly different folder structures. It was a lot messier than I remember.

So, I'm retracting my objection.

I didn't look into how you handle shared folders so that you can have something like a single commodore folder and only differences held separately into c16/c64 folders, but I doubt you haven't thought of it. mostly ifdefs are good enough anyway.

[FozzTexx]

I didn't look into how you handle shared folders so that you can have something like a single commodore folder and only differences held separately into c16/c64 folders, but I doubt you haven't thought of it. mostly ifdefs are good enough anyway.

That's the PLATFORM_COMBOS variable which allows adding in multiple additional directories to be included when building a platform. It also is used as a fallback for FUJINET_LIB when there's no lib specifically targeting a particular platform. (Needed for example by msx to get the lib when the platform is msxrom or msxdos.) I made that work the way you had described it to Rich on discord, but made it so the mapping is contained entirely in the top level Makefile which allows adjusting it per project. I used a += syntax to attempt to keep it intuitive.

(I defaulted to commodore after talks with devs in discord at some point).

Which I put as one of the examples in Makefile. It's possible to move all the C16/C64/VIC20 common things into a commodore folder and have them share it. Heck, you could move all 6502 assembly routines that are common among 6502 platforms into a 6502 folder and do c64+=commodore,6502 apple2+=6502 etc.

Whereas you've gone for clients/src/, I defaulted to clients/common + clients/platform, as to me clients/ is the exception in projects I've been trying to build, where they mostly share common source, using fujinet-lib.

That could be done too. In fact, I could have largely left the source where it was as <platform>/src and changed it to SRC_DIRS = %PLATFORM%/src (a couple of platforms didn't have their source in a src subdir though). I just did it as src/%PLATFORM% since that's the standard FujiNet convention.

You needed a supporting python script to do so due to the way that compiler works, but you did it.

I ... was not happy about this, but I could see no other way. I was shocked to discover Mad-Pascal doesn't work with .o files or a linker. The script is a fairly simple wrapper for Mad-Pascal. I figured it was best to get it working with a Python wrapper first before tackling making it a shell script, or possibly an inline shell script in the .mk file itself. The way I made it work was to treat Mad-Pascal as the linker rather than a compiler, and once I did that, it fit into the toolchain abstraction (other than needing a wrapper).

I also realised how many differences there still are inherent in the different versions of the build-tools versions

FYI, as I add extra platforms (and fix bugs), I simply copy the entire makefiles dir from a project back into MekkoGX. Or if I know I have fixes, I just copy the latest MekkoGX makefiles contents into a project. Because all the customization exists only in the top level Makefile there's no issue with updating.