originating from the idea of @aherbert in #8 :
PS: You're right, maybe we should aim at detecting broken plugins on the Mastodon side.. although, I'm not sure how that should be achieved. Do you happen to have an idea?
I don't think this can be done from Mastodon's side. It would have to know all possible downstream users of the API. From the plugin side you could use GitHub's dependabot to notify you of new releases for Mastodon. This will raise a PR with the code updated to the latest Mastodon version whenever a release happens. In this case the PR would break as the API is incompatible and you should be sent a notification.
I've experienced dependabot working on repositories where it watches all your dependencies and can get very noisy with constant PRs for releases of minor revisions. For the mastodon-ctc repo there are only 3 direct dependencies so the noise should be low. You can configure dependabot to ignore dependencies if you only wish to watch mastodon.
See Configuration options for the dependabot.yml file
originating from the idea of @aherbert in #8 :