Proto generation for Vixen Render

[fernandodeluret]

Overview

The render now generates code needed to support proto feature of Vixen. Simplification of end-user API.

Changes

• Generate rust sdk within the vixen since always required and able to remove top level config. You can access the rust SDK from the Vixen crate.
• Support proto feature of Vixen.
• Generate open telemetry traces for parsing
• Handle more variants of IDL (primarily shank flavor).
• Regenerate e2e parsers and re-test.
• Expand unit test coverage based on changes to the render.

[changeset-bot]

🦋 Changeset detected

Latest commit: ec06b88

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@codama/renderers-vixen-parser	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[lorisleiva]

After a first pass on this PR, I can see this is a better approach than before. My main issue here is that this PR does way too much. It's tricky to review and make sure no bugs or misuse of the standard are being introduced.

Since this seems to add a new Codama to Protobuf generator, how would you feel about creating a @codama/renderers-protobuf package that only does this. Then you could use this package as a dependency for the vixen parser and only focus on the parts that are unique to vixen. Would that make sense or is the protobuf conversion too opinionated to be extracted as a generic tool?

[kespinola]

Since this seems to add a new Codama to Protobuf generator, how would you feel about creating a @codama/renderers-protobuf package that only does this. Then you could use this package as a dependency for the vixen parser and only focus on the parts that are unique to vixen. Would that make sense or is the protobuf conversion too opinionated to be extracted as a generic tool?

@lorisleiva for now the proto is an extension of Vixen for its stream feature. There are likely pieces that could be isolated but don't want to split out too early. It is cool that this generates a proto def for the IDL that may be useful in other situations like a grpc based RPC stack that returns parsed results.

Overall, @lorisleiva all these changes are isolated to the renderers-vixen-parser and we have 10+ new parsers based on these updates so we are happy with its capabilities right now and would like to get a release out so we can update community and our docs.

[lorisleiva]

Thanks for the update Kyle, I'm happy to get that merged if you guys are happy with the PR. Just need to get CI to green first and I'll approve. 🙏

[fernandodeluret]

Thanks for the update Kyle, I'm happy to get that merged if you guys are happy with the PR. Just need to get CI to green first and I'll approve. 🙏

Thanks @lorisleiva! 🙏 This is done, just updated the branch and the tests. Also now that we have the shank discriminators merged in, I updated the e2e script here to showcase that here eeb0198#diff-b9cdaef35149142caeaf31a6606a6891e1c88d37cb45e2c240549810487c0fa9

[sonicfromnewyoke]

overall looks great to me.

if you don't mind, implementing the next changes, which are described in this solana-sdk-crate-migration-guide

this migration should drastically reduce compilation time and resulting binary size

[kespinola]

overall looks great to me.

if you don't mind, implementing the next changes, which are described in this solana-sdk-crate-migration-guide

this migration should drastically reduce compilation time and resulting binary size

Great suggestions @sonicfromnewyoke and I agree with removing solana-program as a dependency. @fernandodeluret can you gather this info into a follow up issue for use to address. There is a mix requests for changes to the rust SDK and Vixen parser renderers.

[lorisleiva]

Thanks! I can see there are some ongoing comments but feel free to merge whenever you're happy with the state of the PR.