Skip to content

[major] replace crayfish derivative microservices with scyllaridae#432

Merged
nigelgbanks merged 20 commits intomainfrom
scyllaridae
Oct 11, 2025
Merged

[major] replace crayfish derivative microservices with scyllaridae#432
nigelgbanks merged 20 commits intomainfrom
scyllaridae

Conversation

@joecorall
Copy link
Copy Markdown
Contributor

@joecorall joecorall commented May 16, 2025

Replace crayfish derivative microservices with scyllaridae.

As seen in this PR and Islandora/Crayfish#188, scyllaridae offers significantly less code complexity to add new microservices. Between this PR and Islandora/Crayfish#188 around 40K lines of code can be removed by replacing crayfish with scyllaridae. scyllaridae is basically an http service with a YAML based spec to define how derivatives are created when events are received by the service. This allows for new microserves to easily be created by only focusing on the input file source, what software needs to manipulate the input, and then printing the resulting file to stdout. Islandora's specifics around JWT auth, how the source file is streamed, and how the output is streamed, are all handled by the core scyllaridae service, and the spec is how individual microservices can implement their specific logic.

Some reasoning why this PR was created:

If there is interest in this, we can build a milliner alternative using scyllaridae and remove crayfish algother from buildkit.

Related to Islandora/documentation#2354 Islandora/Crayfish#188

@joecorall joecorall changed the title [major] replace derivative microservices with scyllaridae [major] replace crayfish derivative microservices with scyllaridae May 16, 2025
@joecorall joecorall marked this pull request as ready for review October 8, 2025 16:58
Copy link
Copy Markdown
Contributor

@nigelgbanks nigelgbanks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other than the one change, this looks good to go in, great work!!!

@joecorall joecorall marked this pull request as draft October 10, 2025 11:15
@joecorall
Copy link
Copy Markdown
Contributor Author

gonna add some integration tests

@joecorall joecorall marked this pull request as ready for review October 10, 2025 11:49
We aren't testing FITS functionality in this test per-say so using the latest version should be fine
the test runs fine locally. It's probably our simplest scyllaridae microservice so we probably don't need to test this
Copy link
Copy Markdown
Contributor

@nigelgbanks nigelgbanks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still just the issue with scyllaridae/rootfs/etc/s6-overlay/s6-rc.d/scyllaridae/dependencies.d/container-environment but otherwise great! Awesome to have the tests!

@nigelgbanks nigelgbanks merged commit 9841a49 into main Oct 11, 2025
130 checks passed
@nigelgbanks nigelgbanks deleted the scyllaridae branch October 11, 2025 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants