[TILES-V2-3]: implement postgres functions for row, column and table operations

[pregnantboy]

Changes

Added PostgreSQL support for Tiles with streaming capabilities and ULID for row IDs.

What changed?

• Added new dependencies: pg-query-stream and ulid to support PostgreSQL streaming and monotonic ID generation
• Created new PostgreSQL-specific implementation files for table operations:
  • table-functions.ts - For creating tables
  • table-column-functions.ts - For managing table columns
  • table-row-functions.ts - For CRUD operations on table rows
• Implemented types for table operations in types.ts
• Introduced ULID (Universally Unique Lexicographically Sortable Identifier) for row IDs

How to test?

1. Install the new dependencies with npm install
2. Test the PostgreSQL implementation by running the existing integration tests

[pregnantboy]

• [TILES-V2-11]: add unit and integration tests for tiles v2 #1033
• [TILES-V2-10]: add load testing scripts for tiles database #1011
• [TILES-V2-9]: store database type in TableContext on frontend #987
• [TILES-V2-8]: add env vars #986
• [TILES-V2-7]: update tiles actions to use factory class #985
• [TILES-V2-6]: use factory class for queries and mutations, minimise getAllRows payload #977
• [TILES-V2-5]: abstract functions into factory pattern #976
• [TILES-V2-4]: reorganise files and update import paths #975
• [TILES-V2-3]: implement postgres functions for row, column and table operations #974 👈 (View in Graphite)
• [TILES-V2-2]: add new column to table_metadata #973
• [TILES-V2-1]: add another postgres db to docker compose, new env vars #972
• tiles-v2/trunk

This stack of pull requests is managed by Graphite. Learn more about stacking.

[datadog-opengovsg]

Datadog Report

Branch report: tiles-v2/pg-functions
Commit report: 30bd0a2
Test service: plumber

✅ 0 Failed, 772 Passed, 0 Skipped, 2m 47.68s Total Time
🔻 Test Sessions change in coverage: 1 decreased (-0.29%)

🔻 Code Coverage Decreases vs Default Branch (1)

• vitest 32.36% (-0.29%) - Details

[kevinkim-ogp]

lgtm!
tested new dependencies and existing integration tests 👍
new tiles pg functions look good

[pregnantboy]

Merge activity

• Jun 5, 7:16 AM UTC: A user started a stack merge that includes this pull request via Graphite.
• Jun 5, 7:20 AM UTC: Graphite rebased this pull request as part of a merge.
• Jun 5, 7:22 AM UTC: @pregnantboy merged this pull request with Graphite.

[cursor-com]

Bug: ULID Generation Inconsistency Breaks Monotonicity

Inconsistent ULID generation between createTableRow and createTableRows. createTableRow uses monotonicFactory() but creates it inside the function (defeating its purpose for calls across time), while createTableRows uses the global ulid() function. This inconsistency and incorrect usage breaks monotonic ordering guarantees for row IDs, particularly when creating multiple rows.

packages/backend/src/models/tiles/pg/table-row-functions.ts#L26-L59

plumber/packages/backend/src/models/tiles/pg/table-row-functions.ts

Lines 26 to 59 in 6ae2644

	}: CreateRowInput): Promise<TableRowItem> => {
	const ulid = monotonicFactory()
	try {
	const res = await tilesClient(tableId)
	.insert({
	...data,
	rowId: ulid(),
	})
	.returning('*')
	return res[0]
	} catch (e: unknown) {
	logger.error(e)
	throw e
	}
	}
	export const createTableRows = async ({
	tableId,
	dataArray,
	}: CreateRowsInput): Promise<string[]> => {
	try {
	const rows = dataArray.map((data, i) => ({
	rowId: ulid(),
	...data,
	// manually bumping the createdAt timestamp to ensure that row order is preserved
	createdAt: new Date(Date.now() + i),
	}))
	const res = await tilesClient(tableId).insert(rows).returning(['rowId'])
	return res.map((row) => row.rowId)
	} catch (e: unknown) {
	logger.error(e)
	throw e
	}
	}

Fix in Cursor

---

Bug: Column Type Mismatch Bug

Inconsistent column types: createTable uses string for columns, while createTableColumns uses text. This creates a type mismatch between initially created columns and later added columns.

packages/backend/src/models/tiles/pg/table-functions.ts#L5-L7

plumber/packages/backend/src/models/tiles/pg/table-functions.ts

Lines 5 to 7 in 6ae2644

	table.string('rowId').primary()
	columnIds.forEach((columnId) => {
	table.string(columnId)

packages/backend/src/models/tiles/pg/table-column-functions.ts#L5-L6

plumber/packages/backend/src/models/tiles/pg/table-column-functions.ts

Lines 5 to 6 in 6ae2644

	columnIds.forEach((columnId) => {
	table.text(columnId)

Fix in Cursor

---

BugBot free trial expires on June 9, 2025
You have used $0.00 of your $1.00 spend limit so far. Manage your spend limit in the Cursor dashboard.

Was this report helpful? Give feedback by reacting with 👍 or 👎