Skip to content

Comments

wip(syrup): syrup reader, writer, and codecs for efficient parsing#2751

Closed
kumavis wants to merge 31 commits intomasterfrom
kumavis-syrup-lazy-parse
Closed

wip(syrup): syrup reader, writer, and codecs for efficient parsing#2751
kumavis wants to merge 31 commits intomasterfrom
kumavis-syrup-lazy-parse

Conversation

@kumavis
Copy link
Member

@kumavis kumavis commented Mar 29, 2025

replaced by #2766

wip improvement to syrup -- working towards allowing a future consumer (eg OCapN) to lazily parse syrup messages without reifying all encountered values

see test file https://github.com/endojs/endo/blob/7632803791dc6160c39c1af33963f52e0f2730ec/packages/syrup/test/parse.test.js

@@ -0,0 +1,3 @@
// To be used as keys, syrup symbols must be javascript symbols.
// To avoid an otherwise meaningful symbol name, we prefix it with 'syrup:'.
export const SyrupSymbolFor = (name) => Symbol.for(`syrup:${name}`);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unfortunately registers every symbol encountered in a global table that isn’t necessarily garbage collected.

My intuition is that we’ll need to either wrap every value in a wrapper object like {type: 'symbol', symbol: 'hello'} or just wrap all non-primitive values, including list, struct, and record, like {type: 'record', record: [1, 2, 3]} and {type: 'list', list: [1, 2, 3]}.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also why we’re now calling Syrup symbols “Selector”. They do not necessarily correspond to implementation-language symbols.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i made them prefixed symbols so that they can be keys on dictionaries (represented as js objects) though I don't create them when they are Record labels, as used in ocapn messages.

could create a custom clearable symbol registry though Symbol('hi') !== Symbol.for('hi')

as well as other options

@kumavis kumavis changed the title wip(syrup): seeking and parser for lazy parsing wip(syrup): syrup reader, writer, and codecs for efficient parsing Apr 10, 2025
@kumavis
Copy link
Member Author

kumavis commented Apr 11, 2025

separate branch with symbol -> selector rename https://github.com/endojs/endo/compare/kumavis-syrup-lazy-parse...kumavis-syrup-lazy-parse-selector?expand=1

update: merged here

@kumavis
Copy link
Member Author

kumavis commented Apr 17, 2025

replaced by #2766

@kumavis kumavis closed this Apr 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants