UG Migration to Brain Brew Rust - Federation support and new workflows - #743
UG Migration to Brain Brew Rust - Federation support and new workflows#743jeprecated wants to merge 30 commits into
Conversation
|
Copying my previous comment to here to have a better throughline for the conversation: Hello sirs! Sorry for the delay in replying, I've been swamped with work. Both in the professional sense and also doing more work on this PR!
Big thinking work, but it actually didn't take long to do. I've thought about this problem on and off since my email to you two about this rewrite idea 10 months ago, but the actual effort for my initial PR was < 6 hours of actual work. More on that below.
No worries at all, sir. I am not perturbed. Next time don't feel the need to delay, I was not expecting to simply merge this in. I truly meant it when I said:
Though perhaps I could have been clearer on my expectations, the amount of effort I've actually put in here, and my workflow. Though that gets into some of my personal situation, AI/Agentic Development, and other stuff like that - which I don't want to turn this PR into a discussion/debate on these topics. But just for the context of what is to come, and to set your expectations on the state of my work and how I get it done (and why you can trust it) then here's a collapsed box explaining just that: How I developed and validated this migrationI did all of the work for this Brain Brew rust migration using Agentic Development (with AI). In my spare time, while running 5 other projects.
No! 😁 Allow me to explain by first adding context. I created my own startup 4 months ago: Self-Deprecated https://selfdeprecated.ai/ an AI Solutions company, to help bring clients up to speed on Agentic Development. On how to use Agents, what their strengths are, when not to trust them, and how to make sure your "self-healing loop" is the best it can be in order to get the output you desired, while remaining in a "human-out-the-loop" for as much as possible. I use Hunk (https://github.com/modem-dev/hunk/) to get the Agent to make PR comments on it's changes, and I review the actual code. Do I read absolutely every line always? Absolutely not, and nor should I. It depends on the system being created/changes. Hunk helps one have the Agent walk them through a change/PR step by step, while looking at the code, explaining in as much detail what the changes were for and how they relate to the other code segments. For this work (migrating an existing tool from one language to another, though with some bit feature changes too) I had an amazingly easy job. The bulk of the work is in thinking and designing the system, while the core migration is unusually straightforward to test as an output-equivalence problem. Brain Brew’s job (before and after the rewrite) is to produce CrowdAnki decks. Given the same accepted source, it either produces the same parsed JSON and media or it produces a regression, apart from differences we have explicitly reviewed and classified. The migration evidence compares deck and model identity, fields, cards/templates/CSS, notes/GUIDs/tags, descriptions, configuration, and media bytes. Yet at the same time I've add a lot more features! Though almost none of these change the output in the end, they just give better composition and flexibility to the data as it's stored in the repo. Such as combining decks together for the federation. In short:
So all of the above said:
Work Done to Address the commentsRather than line by line reply to each comments (as we three are want to do) I think it'd be faster for everyone if I simply address the things that I have improved. Some I always intended to do, and some are direct responses to good feedback from yourselves Translation overlay format cleanupExample: old and new translation overlay formatsBefore: old alpha schema mixed several intents under translations. "changes", "additions", each of which could be general or field-specific translations:
changes:
Taiwan: "台湾(Taiwan)"
Taipei: "台北(Taipei)"
Georgia:
notes.note.georgia.fields.field.country: Georgien
notes.note.us-georgia.fields.field.region: Georgia
Partially recognised state claimed by China.:
notes.note.taiwan.fields.field.country-info: "中国宣称对台湾拥有主权，但仅被部分国家承认"
additions:
notes.note.yellow-sea.fields.field.country-info: "虽然国际上普遍认为黄海包含了渤海，但在中国，它们通常被视为两个独立的海域。"After: reusable translations, contextual translations, and typed target adaptations are separate translations:
direct:
Taiwan: "台湾(Taiwan)"
Taipei: "台北(Taipei)"
contextual:
notes.note:
andorra.fields.field.flag-similarity.message.variables.country_1:
Moldova: "摩尔多瓦"
target_adaptations:
notes.note.taiwan.fields.field.country-info:
intent: adapt
ownership: translation
expected_source: Partially recognised state claimed by China.
target: "中国宣称对台湾拥有主权，但仅被部分国家承认"
reason: target-language geopolitical wording
notes.note.yellow-sea.fields.field.country-info:
intent: adapt
ownership: translation
expected_source: ""
target: "虽然国际上普遍认为黄海(Yellow Sea)包含了渤海(Bohai Sea)，但在中国(China)，它们通常被视为两个独立的海域。"
reason: "migrated from legacy translations.target_additions; review and describe its target-language purpose"
notes.note.czech-republic.fields.field.country-info:
intent: delete
ownership: translation
expected_source: Also known as Czechia.
reason: "migrated legacy target adaptation; review and describe its target-language purpose"Reviewed source text that should intentionally remain unchanged can be listed under Better authoring formats
Example: external description, CSS, and template includesdeck:
description: !include descriptions/ultimate-geography/en.html
note_types:
note-type.ultimate-geography:
styling: !include styles/ultimate-geography/card.css
card_templates:
template.country-capital:
question_format: !include templates/ultimate-geography/country-capital/question.html
answer_format: !include templates/ultimate-geography/country-capital/answer.html
Examples: reusable message parts and translated formatsBefore: one long translation key# src/data/flag_similarity.csv, represented as YAML for comparison
field.flag-similarity: Iceland (blue background, red and white cross), Norway (red background, blue and white cross)# The Norwegian translation was also one indivisible string
translations:
direct:
"Iceland (blue background, red and white cross), Norway (red background, blue and white cross)": "Island (blå bakgrunn, rødt og hvitt kors), Norge (rød bakgrunn, blått og hvitt kors)"After: reusable translated parts# deck.yaml, notes.note.faroe-islands
field.flag-similarity:
format: "{country_1} ({description_1}), {country_2} ({description_2})"
variables:
country_1:
ref: notes.note.iceland.fields.field.country
country_2:
ref: notes.note.norway.fields.field.country
description_1:
text: blue background, red and white cross
description_2:
text: red background, blue and white cross# overlays/languages/nb.yaml
translations:
direct:
Iceland: Island
Norway: Norge
blue background, red and white cross: "blå bakgrunn, rødt og hvitt kors"
red background, blue and white cross: "rød bakgrunn, blått og hvitt kors"Translating the format itselfBase source: field.flag-similarity:
format: "{country_1} ({description_1}), {country_2} ({description_2})"
variables:
country_1:
ref: notes.note.iceland.fields.field.country
country_2:
ref: notes.note.norway.fields.field.country
description_1:
text: blue background, red and white cross
description_2:
text: red background, blue and white crossThe pieces can be translated normally: translations:
direct:
Iceland: Island
Norway: Norge
blue background, red and white cross: "blå bakgrunn, rødt og hvitt kors"
red background, blue and white cross: "rød bakgrunn, blått og hvitt kors"But the format string itself can also be translated when the target language needs different punctuation, spacing, order, or glue. Example: Simplified Chinese punctuationtranslations:
direct:
Iceland: "冰岛"
Norway: "挪威"
blue background, red and white cross: "蓝底，红白交叉"
red background, blue and white cross: "红底，蓝白交叉"
"{country_1} ({description_1}), {country_2} ({description_2})": "{country_1}({description_1})、{country_2}({description_2})"Output: Example: description before countrytranslations:
direct:
"{country_1} ({description_1}), {country_2} ({description_2})": "{description_1}: {country_1}; {description_2}: {country_2}"Output: Example: sentence-style wordingtranslations:
direct:
"{country_1} ({description_1}), {country_2} ({description_2})": "{country_1}: {description_1}; {country_2}: {description_2}."Output: Example: contextual format translationUse contextual if only one field should use a different format: translations:
contextual:
notes.note:
faroe-islands.fields.field.flag-similarity.message.format:
"{country_1} ({description_1}), {country_2} ({description_2})": "{country_1} resembles {country_2}: {description_1} vs {description_2}"Output for that field only: Structured media references and integrity
Example: single/multiple images, media declarations, and strict verificationA field can contain one stable image reference or an ordered image sequence, such as UG's blurred and normal Bolivia flags: # deck.yaml
notes:
note.bolivia:
fields:
field.flag:
- !image media.ug-flag-bolivia-blur-svg
- !image media.ug-flag-bolivia-svg
field.map: !image media.ug-map-bolivia-png
media: !include media.yamlThe stable IDs resolve through the separately maintained declaration map: # media.yaml
media.ug-flag-bolivia-blur-svg:
path: ug-flag-bolivia-blur.svg
sha256: f13669cab4afb991b9851a9c55bb94be5a2c91303a6f8bbb4407a9ffd67951c7
media.ug-flag-bolivia-svg:
path: ug-flag-bolivia.svg
sha256: 3010bf58668ac58ae5a1b614867cf94c53b229f2d26679c0ca04cae6d936ced1
media.ug-map-bolivia-png:
path: ug-map-bolivia.png
sha256: d461a4cb0d4b845fdc61de1123efca9cc766aaea10dabd95202a1beb980fca7dThis means a file can be renamed by updating its declaration without rewriting every note-field reference. Brain Brew renders the references as safe Anki-compatible brainbrew media hash --manifest brainbrew.yaml --all-targets --media-root media
brainbrew verify --manifest brainbrew.yaml --all-targets --media-root media
brainbrew export crowdanki \
--manifest brainbrew.yaml \
--target en-standard \
--media-root media \
--out build/crowdanki/en-standard
Rust crate releaseI had never intend to put Nix/NixOS as a dev dependency, that's just what I happen to use (because it is truly excellent). I simply left it in my test PR rather than put in the effort to actually release anything properly, before it got approved and the effort was worth it 😁
Example: installation and normal CLI usageBefore # Mostly contributor/developer style usage
nix run . -- --help
cargo run -- compose --manifest brainbrew.yaml --target de-standardAfter cargo install brainbrew --version 1.0.0-alpha.3 --locked
brainbrew compose --manifest brainbrew.yaml --target de-standardLanguage-first project metadata
Example: language-first manifest metadataBefore targets:
de-standard:
overlays:
- overlay.translation.de
de-extended:
overlays:
- overlay.variant.extended
- overlay.translation.deTools had to infer language/variant meaning from target names. After languages:
en:
display_name: English
source: true
primary_target: standard
targets:
experimental: en-experimental
extended: en-extended
hardcore-extended: en-hardcore-extended
hardcore-standard: en-hardcore-standard
standard: en-standard
de:
display_name: German
translation_overlays:
base: overlay.translation.de
hardcore: overlay.translation.hardcore.de
primary_target: standard
targets:
experimental: de-experimental
extended: de-extended
hardcore-extended: de-hardcore-extended
hardcore-standard: de-hardcore-standard
standard: de-standard
translation_profile:
structural_fields:
- field.flag
- field.map
metadata_categories:
- key: deck-metadata
label: Deck metadata
paths:
- deck.name
- deck.descriptionThis means that all deck extensions can have their own translations, yet still the whole can be understood to be under one language group. This will help tools show all the translations for one language, for all decks/extensions. Safer extension composition
Examples: field additions, blank-only fills, and expected-base checksExperimental adds one field definition and supplies values only where needed: id: overlay.variant.experimental
kind: extension
field_additions:
note-type.ultimate-geography:
fields:
field.region-code: Region code
values:
note.afghanistan:
field.region-code: AF
note.albania:
field.region-code: ALHardcore can fill fields that already exist in the shared note model: id: overlay.extension.hardcore.field-fills
kind: extension
field_fills:
note.hardcore-bali:
field.capital: Denpasar
field.flag:
- !image media.ug-flag-bali-blur-png
- !image media.ug-flag-bali-pngA value that intentionally replaces existing source records exactly what it expects to replace: note_types:
note-type.ultimate-geography:
intent: merge
variables:
variant.name-suffix:
intent: replace
value: " [Extended]"
expected_base:
value: ""Translator workflows
Example: translator CLI workflowBefore brainbrew verify --manifest brainbrew.yaml --target da-standardThen manually inspect YAML failures and decide what was missing/stale/unchanged. After brainbrew translations --manifest brainbrew.yaml --all-targets --summary
brainbrew translations --manifest brainbrew.yaml --target da-standard --context --status missing
brainbrew translations --manifest brainbrew.yaml --target da-standard --context --status missing --apply --interactiveReport, summary, and context modes are read-only. Summary mode provides compact per-language and per-overlay counts; context mode shows missing or stale text with its source, target, note, field, card, and duplicate-source context.
When English source text changes, an outdated translation can be retained as an explicit stale record rather than being treated as current. Under UG’s lenient policy, Brain Brew warns about that record and continues using its target text until a maintainer resolves it after review; strict translation coverage rejects unresolved stale records. Orphaned dictionary keys, invalid contextual paths or target adaptations, and broken references still fail normal verification. Deck WorkbenchAdded a local "Workbench", a webpage one can run which shows the deck contents and allows for editing inline while previewing a Note/Card/Field. 
Example: Workbench workflow brainbrew workbench serve \
--manifest brainbrew.yaml \
--enable-writeThen use the local Workbench to review content and edit its underlying source or translation fields: This was a stretch goal I had in mind for a while, but I decided to take a crack at it now. Brain Brew 1.0.0-alpha.3 includes Workbench write support in the normal release. It starts read-only; The Workbench shows deck content in context, including source and translated text, cards, metadata, and other languages for comparison. Edits remain drafts until someone confirms Apply, which then updates the canonical YAML and owned translation overlays in the local working tree. The write workflow is still being hardened, so it should be used on a version-controlled checkout. My longer-term goal is to make this a complete translator-facing GUI where people can update translations, create new ones, review the resulting changes, and compare their work with other languages without needing to edit YAML directly. ❗ This tool is very much a work in progress! It still has some sharp edges and strange display stuttering. But these will be fixed! This is just how I imagine a tool could support the workflow much better. Anyways all of this is up for future consideration/work, but I hope you at least like the direction. I'm open to any and all ideas, it of course matters what people would want to do / how they work want to work. But the more different options the better, in my books! Discussion PointsThings here are up for discussion. I have taken my own liberties in designing a system I think is good, but am open to being wrong about! Support for the below in Brain Brew does not dictate that UG need use it too. As I said just above more options are better! Yaml as a formatI hope that the above improvements will help with yaml being the main storage format. Especially when taken as it being the git repo source of truth, not that everyone needs to work via the yaml files (though I personally think it's much nicer to do so now, compared to the csvs before). I have kept this Brain Brew rewrite working in the same way the old one did: a hub and spoke type of format system. The old system translated everything into "Deck Parts" before then translating to/from CrowdAnki/CSV. This new system has the same general hub-and-spoke shape, except I have made the Canonical Deck YAML itself the stored intermediate representation. That makes it straightforward to add CSV export and import back in, including a workflow where someone exports to CSV, edits it, and imports it again. The CSV adapter is not implemented in alpha.3 yet; it is an option I can add if that is something people want. I still think the yaml is much better than that old system. English as the Translation fieldI truly believe that this is a necessary evil (hopefully less evil now that I've gotten rid of the big long horrible strings). I've also considered using stable note IDs or country codes as translation keys, with English treated as just another translation. Those identifiers solve identity, but by themselves give translators even less context: they still need the source text, note, field, card, and related translations to understand what they are editing. The new context views and Workbench are intended to make the current English-source model much easier to work with. The workbench editing workflow will help a lot with this problem too. Hardcore geography
All I present for it is that it is doable to have it here in this repo! It was simpler for my demo to have it here too.
Oh this was totally an oversight on my part. I have preserved the historical UG note GUIDs rather than allowing them to be regenerated, as well as the 45 meaningful historical Hardcore note GUIDs. The canonical source uses readable stable note IDs for composition, while explicit I also reorganised Hardcore so that standalone and companion exports reuse the same 45-note content overlay. The companion keeps its own deck identity, while its note GUIDs, fields, tags, and note model stay aligned with the corresponding standalone deck. The migration evidence collector checks this relationship for all 12 localised Hardcore language pairs. That gives us an explicit structural check for the identity/model behaviour behind this import-update concern, rather than relying only on generated GUID overrides. I would still want the final workflow exercised in Anki before treating that as a complete end-to-end import guarantee. |
a934c93 to
5e96018
Compare
|
In an attempt to demonstrate the new setup for ordering/formatting the translations I have done a translation audit (in two separate follow-up PRs, so it does not muddy the migration and output-equivalence review). See them below:
This leaves #743 focused on the Brain Brew migration itself, #744 on output-preserving translation structure, and #745 on optional learner-visible translation review. #744 should help show how we can write the translations but with extra deck specific information to categorise/clarify for the future what the changes are and why. #745 is just a nice to have, and some may be wrong, and it will surely not get merged in. But! The great thing about this new setup is that if we deem any of the translations to be wrong then we can move them into the stale category, with extra information. This will keep that output generating the same, but allow us to merge in changes that mark specific translations as in need of native attention to categorise/fix. Thus allowing the deck to keep moving ahead without dozens of hanging open PRs for specific translators. If one English string changes then all other translations fields can be moved into stale - which does not claim they are wrong just that they need re-reviewed by a native. Flexible! |
|
Github support for Stacked Diffs has arrived! 👀 ❗ that will make all of this easier. I have split up the changes into a stack of PRs on my own UG fork. See the first here: jeprecated#17 @aplaice @axelboc I believe this is much easier to view the changes in chunks there. Happy to discuss over there or wherever - the specifics of the migration, how UG can/should look and feel (the deck content, the commands, the workflow), and what BB supports 👍 |
|
I'm going to be blunt: no way I'm merging any of this. I'm not opposed to AI when used reasonably, like for generating new translation, finding mistranslations, making a new map, or whatever ... but this is wild. I definitely don't appreciate having to read an AI-generated reply that's hundreds of lines long to find out that it doesn't even answer my questions (at least I don't think it does, can't really tell since it's all gibberish or the agent realising that it did stuff it shouldn't have while still trying to argue that it was for a good reason and it's not going to change it anyway). I guess I should use an AI to summarise that comment for me and generate a response... What's even the point of having a discussion? Let's just start our agents and go on holidays! 🤣 The only way I see us move forward is with a slow step-by-step process where the current setup changes as little as possible each time and where we take the time to discuss each design choice carefully, human to human. The design choices can be suggested by AI, I don't care, but I want a human to be able to explain them to me in their own words. If that's not something you're willing to do, then I'd prefer to stick with what we have, which works perfectly fine, is still maintainable by humans, and still satisfies hundreds of users worldwide as shown by the overwhelming number of positive feedback we get from the shared deck page to this day https://ankiweb.net/shared/info/2109889812 All that being said, while I still care deeply about this project, I haven't been actively maintaining it for a very long time so if you guys think going full AI is the way forward, I'm not going to get in the way. |
Lmao. I actually agree with your sentiment, believe it or not 😁 That's why it's a draft. This is a very very large change, akin to switching a working tool to another language, and it's very hard to gain the trust that the old system has. I don't expect that'll take 30 mins of a casual review 😅 my best imagination of a path forward would be to run two simultaneous systems: the current/legacy here (in this repo) and the new Rust generated version as a POC (in my fork?). Side be side to show they result in the same outputs, and one can compare the commits as changes happen. There's also the idea of features that the new system has that the old does not (like Federation) but that's a separate thing to test.
Most importantly, put upfront: I did not generate my response here (or in the previous thread) with AI! That's why it took so long to do 😅 I wrote the first draft, ran it through an agent for corrections (like if I claimed something was X and it was Y) and made those corrections myself. I then decided some of the points could just be addressed rather than discussing, made more updates to Brain Brew (like adding in the requested !include ability so the yaml could be split up into components). More updates happened after that, some of which I got the agent to generate (like demo yaml examples for the features) but I was human in the loop throughout 😄 Not much I can do to prove that though. Just know that I too hate AI slop! Perhaps even more so, since I'm more exposed to it 😁 If you remove the examples I can wholeheartedly claim 90% of the writing as purely my own. As you should know, I can be rather verbose 😅 🙇♂️
The improvements I made should have addressed all your concerns. Along with my explanation on the Yaml format. If not then I'm happy to elaborate. All I can see here is perhaps the variant vs extension distinction you asked about at the end of your last command (answer: there is no structural difference it's just a naming convention that's 100% up for discussion). But as I said after my intro: "Rather than line by line reply to each comments (as we three are want to do) I think it'd be faster for everyone if I simply address the things that I have improved." 😅 I hope you'll agree these conversations have gone long in the past. Was just trying to head that off. If another medium (a call, an interactive chat, etc) would be better I'm up for it whenever. I was aware there could be this type of stigma / negative view since I used LLMs to do the update. That's why I tried to head it off with the big "How I developed and validated this migration" section 😅 My intention here is not to turn the repo into an AI generated slopfest, but to craft (with an Agent as my tool) a better Brain Brew that will make things easier and more featureful for maintainers and users. The point in Anki is to help users remember information! That's something we can't export to AI 😁
It's your deck, sir. I'm just a contributor that like a hard problem to solve 😁 I cannot and will not claim any ownership of UG. I'm here because I have some duty to the system (as a contributor and a user) and the issues of the old Brain Brew have nagged at me for years (because the code and architecture is shit, lol). The idea of getting Federation working has been a treat that I knew was possible but have never had the time to do. Well, thanks to Agentic Development, I found the time! 😁 Totally how I wrote my replyReply to Axelboc. Make no mistakes! |
I still disagree with the proposed translation storage format. One file per language with every English strings duplicated in every language file is not viable in my opinion. And that's probably just one of the many objections I would have if I could actually review something without browsing through hundreds of unrelated changes like hardcord geo deck extensions, media files being moved, and even changes to the real content of the deck like translations!
Seeing that the the output is the same does not give me any confidence in the system that generates that output. Two core design requirements for me:
If the goal of these giant PRs is to brainstorm and show proofs of concept, that's okay. But to actually move forward, I would expect first to focus on improving what we already have (i.e. mostly switching from CSV to YAML, one note per file, etc.) and to have deep discussions about repo structure, file format, translation process, contribution process, command line interface, etc. Once we're happy with that, and only then, we can move on to federation features. For this, I would expect:
In other words, an iterative development process that's been proven to lead to good quality software that is maintainable and usable by humans. |
|
@axelboc thanks very much for the comment and saying much of what I wanted to say! :D @jeprecated I appreciate your continued efforts to improve our tooling and workflows! I think there are three (partially linked) topics here —† the technical design, process and the question of AI/LLM use. † I've long been a fan of the em-dash and I won't allow LLMs to take it from me... AI policyI'd been hoping that we wouldn't need an AI-use policy/guidelines, but it's probably necessary (not even just for this, but in general — I trust you @jeprecated to not do anything crazy, but I don't trust all future contributors). Given that to a large extent it's a matter of preference/ethics I'll open a separate discussion for the AI question. ProcessI haven't given this much thought until now.. In terms of motivation, I understand that the goal of the migration is to rely on a BrainBrew that's easier to extend? @axelboc's template seems generally sensible, though I'd keep it as a suggestion rather than a prescription. (I also have some technical quibbles, but I'll leave this for the next section.) Two parallel instances
That's an interesting idea, especially since my perception of the new/suggested format is that it's better for reading/reviewing than editing. However, I'm not sure how it would work (mono- vs. bi-directional?) (as a GH workflow that would automatically update the tree on new commits? (works for the mono-case, but not really the bi-case (due to GH workflow permissions))). I'm also not sure what would be the end-game. We might end-up deciding that the new format is nice for viewing diffs and a nightmare for editing (or even worse have no feedback on the editing side since everyone will stick to the old format out of familiarity). What then? I'd also feel uncomfortable since you (@jeprecated) would be putting in effort/tokens into a system that we might end up discarding. Your changes to BrainBrew are independent of AUG (any project could end up using it) and AUG is at least a good test-case, but a parallel repos systems would be bespoke. Technical(Subset of potential topics.)
|
|
Thanks @aplaice, I agree that an AI policy would be beneficial — I would suggest Numpy's policy as a starting point but that can be discussed in a dedicated issue.
I was more thinking of per-note grouping as I initially described here #143 (comment) Should we reopen this issue and continue the discussion there? |
|
Alright I have worked some magic, in a way that will hopefully allow us to split up all of this into entirely separate issues. Specifically I have introduced CSV support into the new Brain Brew (in a way that doesn't make me sad) so that it functions merely as a data source for the note contents (including translations). This will allow us (when wanted) to do the move from the legacy/python Brain Brew over to the updated/rust Brain Brew entirely separately from the data model changes. In fact, I have worked magic to make the csv and yaml live together, so one could move over any specific content from the csv into the yaml, and have it all combine in the end. So the flag similarities csv/column could be deleted and the data placed into the yaml with the nice variables which make things easier to read and translate, resulting in the same output in the end. Or a specific country / note could be moved into yaml, and it would all just work. I think a note could even live in the csv but be translated via the yaml too! See the PR stack here: jeprecated#28 11 PRs in total, split up to be minimal and easy to review. The changes progressively add in the deck content into the new system, while comparing the output, and in the end removing the old system. Just to be super clear: the PRs are to my own fork, and can be pulled in iteratively over time. No rush, no big deal. I hope you'll both find this a more acceptable method to move forward 🙏 The rest
I disagree! As Aplaice said there are pros and cons of each system. But we can get into things like this individually in another change/discussion 👍 rather than bloating this one. Same with Aplaice's comments on English as keys and Yaml vs csv, lets take that elsewhere / have that conversation another time. With my current changes it's not needed just yet anyways 👍
I still feel like it is both these things! But I can see that we can take the process changes more iteratively. I hope my recent changes assuage your concerns here, sir.
Sounds sensible!
Pretty much. The old python Brain Brew is a mess that I do not wish to update or maintain. The design is bad, and I've wanted to do a rewrite for years. This is my rewrite, which is much more maintainable and really better in every way. Axelboc's comments on slowly and iteratively changing things are perfectly fair for the process, but not for the tool itself. There's no world in which a slow change of Brain Brew would serve anyone better, it just needed to be torn out and replaced.
Yea I meant that I'd 'manually' mirror any updates over to my fork. It would not take much effort at all. The goal would be to use the tool naturally and find the pitfalls. Having other maintainers also try and duplicate the changes they make there would be a nice experiment too, if they (or you) are willing. It would just be a way to try the new system without... committing to anything (not sorry for the pun).
Splitting? This was a manual process, one that Brain Brew does not do. Instead of the field being the manual string "Moldova (wider, coat of arms with eagle)" the data itself was changed by me/agents into: field.flag-similarity:
- country: note.moldova
description: wider, coat of arms with eaglewhich gets stitched together via the message pattern in the field definition in the note-types.yaml: field.flag-similarity:
name: Flag similarity
message_pattern:
kind: list
item_format: '{country} ({description})' <--- The stitching is just the variables below
separator: ', '
parameters:
country:
type: note_field_ref <--- 'country' above is a note reference. If the name changes the id will still work
field: field.country <--- Which then resolves to a specific field of that note
description:
type: textThe above example only has one entry, but if the list had multiple similarities then it would resolve each of them and join the results together with the separator. Simple but powerful stuff! This is all defined once in the note-types.yaml, and the maintainer/translator just sees the info that needs to be put in
Aplaice is AI, confirmed 😆 |
3 participants
Replacement PR for #736 which is on the right branch. See that PR for the full description.
Summary (as of the opening of this PR):
This is a Work in Progress. Things are subject to change.