Skip to content

[TILES-V2-3]: implement postgres functions for row, column and table operations #974

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 5, 2025

Conversation

pregnantboy
Copy link
Contributor

@pregnantboy pregnantboy commented May 6, 2025

Changes

Added PostgreSQL support for Tiles with streaming capabilities and ULID for row IDs.

What changed?

  • Added new dependencies: pg-query-stream and ulid to support PostgreSQL streaming and monotonic ID generation
  • Created new PostgreSQL-specific implementation files for table operations:
    • table-functions.ts - For creating tables
    • table-column-functions.ts - For managing table columns
    • table-row-functions.ts - For CRUD operations on table rows
  • Implemented types for table operations in types.ts
  • Introduced ULID (Universally Unique Lexicographically Sortable Identifier) for row IDs

How to test?

  1. Install the new dependencies with npm install
  2. Test the PostgreSQL implementation by running the existing integration tests

@pregnantboy pregnantboy changed the title feat: implement postgres functions for row, column and table operations [TILES-V2-3]: implement postgres functions for row, column and table operations May 6, 2025
@datadog-opengovsg
Copy link

datadog-opengovsg bot commented May 6, 2025

Datadog Report

Branch report: tiles-v2/pg-functions
Commit report: 30bd0a2
Test service: plumber

✅ 0 Failed, 772 Passed, 0 Skipped, 2m 47.68s Total Time
🔻 Test Sessions change in coverage: 1 decreased (-0.29%)

🔻 Code Coverage Decreases vs Default Branch (1)

Copy link
Contributor

@kevinkim-ogp kevinkim-ogp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!
tested new dependencies and existing integration tests 👍
new tiles pg functions look good

Copy link
Contributor Author

pregnantboy commented Jun 5, 2025

Merge activity

  • Jun 5, 7:16 AM UTC: A user started a stack merge that includes this pull request via Graphite.
  • Jun 5, 7:20 AM UTC: Graphite rebased this pull request as part of a merge.
  • Jun 5, 7:22 AM UTC: @pregnantboy merged this pull request with Graphite.

@pregnantboy pregnantboy changed the base branch from tiles-v2/table-metadata-db to graphite-base/974 June 5, 2025 07:17
@pregnantboy pregnantboy changed the base branch from graphite-base/974 to tiles-v2/trunk June 5, 2025 07:19
@pregnantboy pregnantboy force-pushed the tiles-v2/pg-functions branch from 30bd0a2 to 6ae2644 Compare June 5, 2025 07:20
@pregnantboy pregnantboy merged commit 171f90d into tiles-v2/trunk Jun 5, 2025
2 checks passed
@pregnantboy pregnantboy deleted the tiles-v2/pg-functions branch June 5, 2025 07:22
Copy link

@cursor-com cursor-com bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: ULID Generation Inconsistency Breaks Monotonicity

Inconsistent ULID generation between createTableRow and createTableRows. createTableRow uses monotonicFactory() but creates it inside the function (defeating its purpose for calls across time), while createTableRows uses the global ulid() function. This inconsistency and incorrect usage breaks monotonic ordering guarantees for row IDs, particularly when creating multiple rows.

packages/backend/src/models/tiles/pg/table-row-functions.ts#L26-L59

}: CreateRowInput): Promise<TableRowItem> => {
const ulid = monotonicFactory()
try {
const res = await tilesClient(tableId)
.insert({
...data,
rowId: ulid(),
})
.returning('*')
return res[0]
} catch (e: unknown) {
logger.error(e)
throw e
}
}
export const createTableRows = async ({
tableId,
dataArray,
}: CreateRowsInput): Promise<string[]> => {
try {
const rows = dataArray.map((data, i) => ({
rowId: ulid(),
...data,
// manually bumping the createdAt timestamp to ensure that row order is preserved
createdAt: new Date(Date.now() + i),
}))
const res = await tilesClient(tableId).insert(rows).returning(['rowId'])
return res.map((row) => row.rowId)
} catch (e: unknown) {
logger.error(e)
throw e
}
}

Fix in Cursor


Bug: Column Type Mismatch Bug

Inconsistent column types: createTable uses string for columns, while createTableColumns uses text. This creates a type mismatch between initially created columns and later added columns.

packages/backend/src/models/tiles/pg/table-functions.ts#L5-L7

table.string('rowId').primary()
columnIds.forEach((columnId) => {
table.string(columnId)

packages/backend/src/models/tiles/pg/table-column-functions.ts#L5-L6

columnIds.forEach((columnId) => {
table.text(columnId)

Fix in Cursor


BugBot free trial expires on June 9, 2025
You have used $0.00 of your $1.00 spend limit so far. Manage your spend limit in the Cursor dashboard.

Was this report helpful? Give feedback by reacting with 👍 or 👎

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants