Skip to content

Ephemeral relations, Refactor & Cleanup#34

Merged
MMagueta merged 26 commits into
masterfrom
ephemeral_relations
Dec 14, 2025
Merged

Ephemeral relations, Refactor & Cleanup#34
MMagueta merged 26 commits into
masterfrom
ephemeral_relations

Conversation

@MMagueta

@MMagueta MMagueta commented Dec 4, 2025

Copy link
Copy Markdown
Collaborator

Context

In this branch I did a major refactor to enforce a clear segregation between iterators and pure relational operations.

Implementations

Ephemeral Relations

Ephemeral relations are like views in the SQL standard, except that here they are evanescent objects generated out of running any query whatsoever. In an analogy, they are more closely related to the output of a query.

Provenance and lineage at the relation level

It didn't make sense to store the provenance and lineage at the tuple level. A lonely tuple is not bound by any relational scope and practically speaking, causes horrendous duplication of information on rendering for no reason. So I moved it all up to the relation record itself.

Segregation between iterators and operators

The distinction was not clear, so I made pure relational operations to spawn an iterator on demand as to generate its tuples. This is used by both ephemeral and persistent relations. Iterators just pull data, operations organize tuples.

Documentation

Purged old dummy AI-created docs

Example

operations:setup(), 
DB = operations:create_database(test),
{DB1, _} = operations:create_relation(DB, employees, #{id => integer, name => string, dept_id => integer}),
{DB2, _} = operations:create_relation(DB1, departments, #{id => integer, dept_name => string}), 
Employees = [#{id => I, name => "Emp" ++ integer_to_list(I), dept_id => (I rem 3) + 1} || I <- lists:seq(1, 10)],
Depts = [#{dept_id => 1, dept_name => "Eng"}, #{dept_id => 2, dept_name => "Sales"}, #{dept_id => 3, dept_name => "HR"}],
{DB3, E} = operations:create_all_tuples(DB2, employees, Employees),
{DB4, D} = operations:create_all_tuples(DB3, departments, Depts),
operations:collect_all(((relational_operators:join(E, D, dept_id))#relation.generator)(#{})).

Comment thread src/query_planner.erl Outdated
EduardoLR10
EduardoLR10 previously approved these changes Dec 8, 2025
@MMagueta MMagueta changed the title Ephemeral relations Ephemeral relations, Refactor & Cleanup Dec 14, 2025
@MMagueta MMagueta merged commit ff5691f into master Dec 14, 2025
4 of 5 checks passed
@MMagueta MMagueta deleted the ephemeral_relations branch December 14, 2025 03:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants