Skip to content

sql: filter out undefined values in INSERT helper instead of treating as NULL #25830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cirospaciari merged 2 commits into from
Jan 5, 2026
+242 −83
Merged

sql: filter out undefined values in INSERT helper instead of treating as NULL #25830

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
36e5098
sql: filter out undefined values in INSERT helper instead of treating…
alii Jan 5, 2026
dc27bc1
refactor: consolidate INSERT undefined filtering into shared helper
alii Jan 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 26 additions & 25 deletions src/js/internal/sql/mysql.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import type { MySQLErrorOptions } from "internal/sql/errors";
import type { Query } from "./query";
import type { ArrayType, DatabaseAdapter, SQLArrayParameter, SQLHelper, SQLResultArray, SSLMode } from "./shared";
const { SQLHelper, SSLMode, SQLResultArray } = require("internal/sql/shared");
const { SQLHelper, SSLMode, SQLResultArray, getDefinedColumns } = require("internal/sql/shared");
const {
Query,
SQLQueryFlags,
Expand Down Expand Up @@ -455,11 +455,10 @@ class PooledMySQLConnection {
}
}

class MySQLAdapter implements DatabaseAdapter<
PooledMySQLConnection,
$ZigGeneratedClasses.MySQLConnection,
$ZigGeneratedClasses.MySQLQuery
> {
class MySQLAdapter
implements
DatabaseAdapter<PooledMySQLConnection, $ZigGeneratedClasses.MySQLConnection, $ZigGeneratedClasses.MySQLQuery>
{
public readonly connectionInfo: Bun.SQL.__internal.DefinedPostgresOrMySQLOptions;

public readonly connections: PooledMySQLConnection[];
Expand Down Expand Up @@ -1020,10 +1019,19 @@ class MySQLAdapter implements DatabaseAdapter<
// insert into users ${sql(users)} or insert into users ${sql(user)}
//

// Filter out columns with undefined values
// Check ALL items to build union of defined columns (any item having a value means include the column)
const definedColumns = getDefinedColumns(columns, items);
const definedColumnCount = definedColumns.length;
if (definedColumnCount === 0) {
throw new SyntaxError("Insert needs to have at least one column with a defined value");
}
const lastDefinedColumnIndex = definedColumnCount - 1;

query += "(";
for (let j = 0; j < columnCount; j++) {
query += this.escapeIdentifier(columns[j]);
if (j < lastColumnIndex) {
for (let j = 0; j < definedColumnCount; j++) {
query += this.escapeIdentifier(definedColumns[j]);
if (j < lastDefinedColumnIndex) {
query += ", ";
}
}
Expand All @@ -1034,15 +1042,12 @@ class MySQLAdapter implements DatabaseAdapter<
for (let j = 0; j < itemsCount; j++) {
query += "(";
const item = items[j];
for (let k = 0; k < columnCount; k++) {
const column = columns[k];
for (let k = 0; k < definedColumnCount; k++) {
const column = definedColumns[k];
const columnValue = item[column];
query += `?${k < lastColumnIndex ? ", " : ""}`;
if (typeof columnValue === "undefined") {
binding_values.push(null);
} else {
binding_values.push(columnValue);
}
query += `?${k < lastDefinedColumnIndex ? ", " : ""}`;
// If this item has undefined for a column that other items defined, use null
binding_values.push(typeof columnValue === "undefined" ? null : columnValue);
}
if (j < lastItemIndex) {
query += "),";
Expand All @@ -1053,15 +1058,11 @@ class MySQLAdapter implements DatabaseAdapter<
} else {
query += "(";
const item = items;
for (let j = 0; j < columnCount; j++) {
const column = columns[j];
for (let j = 0; j < definedColumnCount; j++) {
const column = definedColumns[j];
const columnValue = item[column];
query += `?${j < lastColumnIndex ? ", " : ""}`;
if (typeof columnValue === "undefined") {
binding_values.push(null);
} else {
binding_values.push(columnValue);
}
query += `?${j < lastDefinedColumnIndex ? ", " : ""}`;
binding_values.push(columnValue);
}
query += ") "; // the user can add RETURNING * or RETURNING id
}
Expand Down
55 changes: 30 additions & 25 deletions src/js/internal/sql/postgres.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import type { PostgresErrorOptions } from "internal/sql/errors";
import type { Query } from "./query";
import type { ArrayType, DatabaseAdapter, SQLArrayParameter, SQLHelper, SQLResultArray, SSLMode } from "./shared";
const { SQLHelper, SSLMode, SQLResultArray, SQLArrayParameter } = require("internal/sql/shared");
const { SQLHelper, SSLMode, SQLResultArray, SQLArrayParameter, getDefinedColumns } = require("internal/sql/shared");
const {
Query,
SQLQueryFlags,
Expand Down Expand Up @@ -672,11 +672,14 @@ class PooledPostgresConnection {
}
}

class PostgresAdapter implements DatabaseAdapter<
PooledPostgresConnection,
$ZigGeneratedClasses.PostgresSQLConnection,
$ZigGeneratedClasses.PostgresSQLQuery
> {
class PostgresAdapter
implements
DatabaseAdapter<
PooledPostgresConnection,
$ZigGeneratedClasses.PostgresSQLConnection,
$ZigGeneratedClasses.PostgresSQLQuery
>
{
public readonly connectionInfo: Bun.SQL.__internal.DefinedPostgresOrMySQLOptions;

public readonly connections: PooledPostgresConnection[];
Expand Down Expand Up @@ -1249,10 +1252,19 @@ class PostgresAdapter implements DatabaseAdapter<
// insert into users ${sql(users)} or insert into users ${sql(user)}
//

// Filter out columns with undefined values
// Check ALL items to build union of defined columns (any item having a value means include the column)
const definedColumns = getDefinedColumns(columns, items);
const definedColumnCount = definedColumns.length;
if (definedColumnCount === 0) {
throw new SyntaxError("Insert needs to have at least one column with a defined value");
}
const lastDefinedColumnIndex = definedColumnCount - 1;

query += "(";
for (let j = 0; j < columnCount; j++) {
query += this.escapeIdentifier(columns[j]);
if (j < lastColumnIndex) {
for (let j = 0; j < definedColumnCount; j++) {
query += this.escapeIdentifier(definedColumns[j]);
if (j < lastDefinedColumnIndex) {
query += ", ";
}
}
Expand All @@ -1263,15 +1275,12 @@ class PostgresAdapter implements DatabaseAdapter<
for (let j = 0; j < itemsCount; j++) {
query += "(";
const item = items[j];
for (let k = 0; k < columnCount; k++) {
const column = columns[k];
for (let k = 0; k < definedColumnCount; k++) {
const column = definedColumns[k];
const columnValue = item[column];
query += `$${binding_idx++}${k < lastColumnIndex ? ", " : ""}`;
if (typeof columnValue === "undefined") {
binding_values.push(null);
} else {
binding_values.push(columnValue);
}
query += `$${binding_idx++}${k < lastDefinedColumnIndex ? ", " : ""}`;
// If this item has undefined for a column that other items defined, use null
binding_values.push(typeof columnValue === "undefined" ? null : columnValue);
}
if (j < lastItemIndex) {
query += "),";
Expand All @@ -1282,15 +1291,11 @@ class PostgresAdapter implements DatabaseAdapter<
} else {
query += "(";
const item = items;
for (let j = 0; j < columnCount; j++) {
const column = columns[j];
for (let j = 0; j < definedColumnCount; j++) {
const column = definedColumns[j];
const columnValue = item[column];
query += `$${binding_idx++}${j < lastColumnIndex ? ", " : ""}`;
if (typeof columnValue === "undefined") {
binding_values.push(null);
} else {
binding_values.push(columnValue);
}
query += `$${binding_idx++}${j < lastDefinedColumnIndex ? ", " : ""}`;
binding_values.push(columnValue);
}
query += ") "; // the user can add RETURNING * or RETURNING id
}
Expand Down
33 changes: 33 additions & 0 deletions src/js/internal/sql/shared.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,38 @@ class SQLHelper<T> {
}
}

/**
* Get columns that have at least one defined value across all items.
* Used by INSERT to filter out columns where ALL items have undefined.
* If any item has a value for a column, that column is included.
*/
function getDefinedColumns<T>(columns: (keyof T)[], items: T | T[]): (keyof T)[] {
const definedColumns: (keyof T)[] = [];
const columnCount = columns.length;

for (let j = 0; j < columnCount; j++) {
const column = columns[j];
let hasDefinedValue = false;

if ($isArray(items)) {
for (let k = 0; k < items.length; k++) {
if (typeof items[k][column] !== "undefined") {
hasDefinedValue = true;
break;
}
}
} else {
hasDefinedValue = typeof items[column] !== "undefined";
}

if (hasDefinedValue) {
definedColumns.push(column);
}
}

return definedColumns;
}

const SQLITE_MEMORY = ":memory:";
const SQLITE_MEMORY_VARIANTS: string[] = [":memory:", "sqlite://:memory:", "sqlite:memory"];

Expand Down Expand Up @@ -911,6 +943,7 @@ export default {
assertIsOptionsOfAdapter,
parseOptions,
SQLHelper,
getDefinedColumns,
normalizeSSLMode,
SQLResultArray,
SQLArrayParameter,
Expand Down
42 changes: 22 additions & 20 deletions src/js/internal/sql/sqlite.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ import type * as BunSQLiteModule from "bun:sqlite";
import type { BaseQueryHandle, Query, SQLQueryResultMode } from "./query";
import type { ArrayType, DatabaseAdapter, OnConnected, SQLArrayParameter, SQLHelper, SQLResultArray } from "./shared";

const { SQLHelper, SQLResultArray } = require("internal/sql/shared");
const { SQLHelper, SQLResultArray, getDefinedColumns } = require("internal/sql/shared");
const {
Query,
SQLQueryResultMode,
Expand Down Expand Up @@ -433,10 +433,19 @@ class SQLiteAdapter implements DatabaseAdapter<BunSQLiteModule.Database, BunSQLi
// insert into users ${sql(users)} or insert into users ${sql(user)}
//

// Filter out columns with undefined values
// Check ALL items to build union of defined columns (any item having a value means include the column)
const definedColumns = getDefinedColumns(columns, items);
const definedColumnCount = definedColumns.length;
if (definedColumnCount === 0) {
throw new SyntaxError("Insert needs to have at least one column with a defined value");
}
const lastDefinedColumnIndex = definedColumnCount - 1;

query += "(";
for (let j = 0; j < columnCount; j++) {
query += this.escapeIdentifier(columns[j]);
if (j < lastColumnIndex) {
for (let j = 0; j < definedColumnCount; j++) {
query += this.escapeIdentifier(definedColumns[j]);
if (j < lastDefinedColumnIndex) {
query += ", ";
}
}
Expand All @@ -447,16 +456,13 @@ class SQLiteAdapter implements DatabaseAdapter<BunSQLiteModule.Database, BunSQLi
for (let j = 0; j < itemsCount; j++) {
query += "(";
const item = items[j];
for (let k = 0; k < columnCount; k++) {
const column = columns[k];
for (let k = 0; k < definedColumnCount; k++) {
const column = definedColumns[k];
const columnValue = item[column];
Copy link
Member

@cirospaciari cirospaciari Jan 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can just check the columnValue here instead of looping twice using getDefinedColumns + having the right index of the actually valid values instead of k

// SQLite uses ? for placeholders, not $1, $2, etc.
query += `?${k < lastColumnIndex ? ", " : ""}`;
if (typeof columnValue === "undefined") {
binding_values.push(null);
} else {
binding_values.push(columnValue);
}
query += `?${k < lastDefinedColumnIndex ? ", " : ""}`;
// If this item has undefined for a column that other items defined, use null
binding_values.push(typeof columnValue === "undefined" ? null : columnValue);
}
if (j < lastItemIndex) {
query += "),";
Expand All @@ -467,16 +473,12 @@ class SQLiteAdapter implements DatabaseAdapter<BunSQLiteModule.Database, BunSQLi
} else {
query += "(";
const item = items;
for (let j = 0; j < columnCount; j++) {
const column = columns[j];
for (let j = 0; j < definedColumnCount; j++) {
const column = definedColumns[j];
const columnValue = item[column];
// SQLite uses ? for placeholders
query += `?${j < lastColumnIndex ? ", " : ""}`;
if (typeof columnValue === "undefined") {
binding_values.push(null);
} else {
binding_values.push(columnValue);
}
query += `?${j < lastDefinedColumnIndex ? ", " : ""}`;
binding_values.push(columnValue);
}
query += ") "; // the user can add RETURNING * or RETURNING id
}
Expand Down
29 changes: 28 additions & 1 deletion test/CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ When callbacks must be used and it's just a single callback, use `Promise.withRe
```ts
const ws = new WebSocket("ws://localhost:8080");
const { promise, resolve, reject } = Promise.withResolvers();
const { promise, resolve, reject } = Promise.withResolvers<void>(); // Can specify any type here for resolution value
ws.onopen = resolve;
ws.onclose = reject;
await promise;
Expand Down Expand Up @@ -153,6 +153,33 @@ To create a repetitive string, use `Buffer.alloc(count, fill).toString()` instea
- Unit tests for specific features are organized by module (e.g., `/test/js/bun/`, `/test/js/node/`)
- Integration tests are in `/test/integration/`
### Nested/complex object equality
Prefer usage of `.toEqual` rather than many `.toBe` assertions for nested or complex objects.
<example>
BAD (try to avoid doing this):
```ts
expect(result).toHaveLength(3);
expect(result[0].optional).toBe(null);
expect(result[1].optional).toBe("middle-value"); // CRITICAL: middle item's value must be preserved
expect(result[2].optional).toBe(null);
```
**GOOD (always prefer this):**
```ts
expect(result).toEqual([
{ optional: null },
{ optional: "middle-value" }, // CRITICAL: middle item's value must be preserved
{ optional: null },
]);
```
</example>
### Common Imports from `harness`
```ts
Expand Down
Loading