Skip to content

vef: allow VDFs as type operations#59

Merged
malone-at-work merged 3 commits intomainfrom
maloney/vdf/1
Feb 26, 2026
Merged

vef: allow VDFs as type operations#59
malone-at-work merged 3 commits intomainfrom
maloney/vdf/1

Conversation

@malone-at-work
Copy link
Copy Markdown
Member

Extend vef_type_desc_t (protocol 2) with encode/decode/compare/hash_vdf_name fields so extensions can name a registered VDF as the implementation of each type operation instead of providing a raw function pointer.

Server-side: resolve VDF names at extension registration, validate signatures (including rejecting prerun/postrun hooks), and encapsulate fn-ptr vs VDF dispatch in new EncodeOp/DecodeOp/CompareOp/HashOp wrapper classes.

These wrappers are registered in the TypeDescriptor; all call sites use invoke() which hide the protocol differences

SDK: add overloaded .encode()/.decode()/.compare()/.hash() builder methods, that take the VDF. Convert vsql-complex and vsql-tvector examples to the new style.

@malone-at-work malone-at-work changed the title vef: allow VDFs as type functions vef: allow VDFs as type operations Feb 25, 2026
@malone-at-work malone-at-work force-pushed the maloney/vdf/1 branch 2 times, most recently from e8474ac to dc017d6 Compare February 25, 2026 15:56
Comment thread villagesql/types/type_op.cc Outdated
Comment thread villagesql/types/type_op.cc Outdated
Comment thread villagesql/types/type_op.cc Outdated
Comment thread villagesql/schema/descriptor/type_descriptor.h Outdated
Comment thread villagesql/sdk/include/villagesql/type_builder.h Outdated
Comment thread villagesql/veb/veb_file.cc Outdated
Comment thread villagesql/examples/vsql-tvector/src/tvector.cc Outdated
Comment thread villagesql/examples/vsql-complex/src/complex.cc
@malone-at-work malone-at-work force-pushed the maloney/vdf/1 branch 2 times, most recently from 5dfb1d6 to 36aa905 Compare February 25, 2026 20:18
Complex cx = load_complex(in->bin_value);
int written =
snprintf(out->str_buf, out->max_str_len, "(%g,%g)", cx.re, cx.im);
if (written < 0 || static_cast<size_t>(written) >= out->max_str_len) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does written < 0 mean the buffer is too small necessarily?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

according to man the only other reason is invalid characters in the format string, which would be a compile time error with most tool chains.

// Compare VDF for ORDER BY, indexes: (COMPLEX, COMPLEX) -> INT
void complex_compare(vef_context_t *ctx, vef_invalue_t *in_l,
vef_invalue_t *in_r, vef_vdf_result_t *out) {
if (in_l->bin_len < kComplexSize || in_r->bin_len < kComplexSize) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why we only checked this direction, instead of !=

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed to use the style I had previously changed the arithmetic functions below to.

// working correctly with hash joins and EXCEPT operations.
void complex2_hash(vef_context_t *ctx, vef_invalue_t *in,
vef_vdf_result_t *out) {
if (in->bin_len < kComplexSize) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we do this? Is it because we have no way to flag an error here? We could flag an error, but i don't think call sites would be able to handle it well.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'twas what the previous code did, I was trying not to change semantics.

This change does give us the opportunity to return an error, and fall back to the standard memory hash. So I will convert it, if you think this is best.


void ReturnError(const char *err_msg, vef_vdf_result_t *result) {
result->type = VEF_RESULT_ERROR;
snprintf(result->error_msg, VEF_MAX_ERROR_LEN, "%s", err_msg);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The implementations of ReturnError are so different from complex and this. Should we not be opinionated about which is better?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to replace the raw ABI types in all of these with more C++ equivalents.

*length = 0;
return true;
// STRING -> TVECTOR
// Dimension is inferred from out->max_bin_len / 4.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but this is going to change in my next change (well, once type params are plumbed). Just FYI.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack.

@malone-at-work
Copy link
Copy Markdown
Member Author

This PR is blocked on investigating empty key comparison in Temp Tables.

Extend vef_type_desc_t (protocol 2) with encode/decode/compare/hash_vdf_name
fields so extensions can name a registered VDF as the implementation of each
type operation instead of providing a raw function pointer.

Server-side: resolve VDF names at extension registration, validate signatures
(including rejecting prerun/postrun hooks), and encapsulate fn-ptr vs VDF
dispatch in new EncodeOp/DecodeOp/CompareOp/HashOp wrapper classes.

These wrappers are registered in the TypeDescriptor; all call sites use
invoke() which hide the protocol differences

SDK: add overloaded .encode()/.decode()/.compare()/.hash() builder
methods, that take the VDF.  Convert vsql-complex and vsql-tvector
examples to the new style.
@malone-at-work
Copy link
Copy Markdown
Member Author

Blocking issue was fixed in "fix custom type compare bug (#64)" I will merge when tests pass.

@malone-at-work malone-at-work merged commit a79bf9d into main Feb 26, 2026
3 checks passed
@malone-at-work malone-at-work deleted the maloney/vdf/1 branch February 26, 2026 18:37
@github-actions github-actions Bot locked and limited conversation to collaborators Feb 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants