Skip to content

[Draft, for discussion only] IDL gen improvements 3 - Fold all proc-macro code back into rs-matter-macros #239

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

ivmarkov
Copy link
Contributor

@ivmarkov ivmarkov commented May 22, 2025

(NOTE: #233 and #237 need to go in first or else this PR will show an unrealistically large change-set.)

This PR is addressing #236 in folding all of rs-matter-data-model, rs-matter-data-model-impl and rs-matter-macros into just rs-matter-macros. The original split was done because of (to my recollection) issues with tests in proc-macro crates.

What works in this "folded" setup:

  • Unit tests. I did not even had to move them into a separate tests folder

What does not work:

  • The "example" which is showing how a user can use the IDL parser - without the import! proc-macro - in their code. I don't think that would be a real use case any time soon (if ever) so I just removed this

  • The benchmark. We have a single simple benchmark (in the benches folder) which is benchmarking (a) the IDL parser parsing time, but is not benchmarking (b) the code-generation time, nor it is benchmarking (c) the time it takes rustc to actually compile the code generated by the import! proc-macro (which is relatively large).

I don't have a good answer (not yet at least) how to restore the existing "IDL parser timing" benchmark if we fold all code under the rs-matter-macros crate.

But in any case, I think we need at some point a more extensive benchmark anyway, which is also timing all three aspects (parsing; code generation; time it takes rustc to compile the generated code). As I have a suspicion that the proc-macro execution time might actually be dominated by (c).

@ivmarkov ivmarkov changed the title [Draft, for discussion only] IDL gen improvements 3 - Fold rs matter macros [Draft, for discussion only] IDL gen improvements 3 - Fold all proc-macro code back into rs-matter-macros May 22, 2025
@ivmarkov ivmarkov force-pushed the fold-rs-matter-macros branch from 614c75f to 191f046 Compare May 29, 2025 11:02
@ivmarkov ivmarkov force-pushed the fold-rs-matter-macros branch from 191f046 to 07000cd Compare June 3, 2025 12:29
@ivmarkov
Copy link
Contributor Author

ivmarkov commented Jun 3, 2025

@bjoernQ What do you think about this one? I'm unsure how to proceed.

  • On one hand, the current separation of the proc-macros implementation into non-proc-macro crate(s) do allow more flexibility. It is another topic that we don't need this flexibility for unit tests, as they just work, proc-macro crate or not. We might need this flexibility if we want to keep the current single benchmark that times the IDL parser
  • On the other, having a benchmark that benchmarks all three aspects of the import! macro as per the summary is what we might actually want so as to say that we have a real benchmark, but not sure how to even achieve it. Any ideas?

In any case, my feeling is three proc-macro-related crates is a bit too many, so either everything back into rs-matter-macros (as this PR does), or rs-matter-macros-impl + rs-matter-macros.

What is your feeling?

@bjoernQ
Copy link
Contributor

bjoernQ commented Jun 3, 2025

I probably need to think about this a bit more - so just my initial thoughts here.

My gut feeling is that having all the proc-macro code in just one crate is nicer for everyone. (Not sure how much users would really care but it's at least easier in terms of maintainence)

I recently learnt about https://crates.io/crates/trybuild - I haven't used it yet, but it looks quite interesting and might be something to consider here?

@ivmarkov
Copy link
Contributor Author

ivmarkov commented Jun 3, 2025

I probably need to think about this a bit more - so just my initial thoughts here.

My gut feeling is that having all the proc-macro code in just one crate is nicer for everyone. (Not sure how much users would really care but it's at least easier in terms of maintainence)

I recently learnt about https://crates.io/crates/trybuild - I haven't used it yet, but it looks quite interesting and might be something to consider here?

Let me spend some time with that crate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants