Skip to content

Quack server crashes under parallel CREATE TABLE AS load and then refuses new inserts until restart #172

Description

@Xander089

What happens?

When several clients run a bulk CREATE OR REPLACE TABLE ... AS SELECT * FROM read_parquet(...) in parallel against a persistent (on-disk) database served via quack_serve, the server process dies mid-transfer. All clients fail at the same time with:

IO Error: Failed to send message: IO Error: Could not connect to server error for HTTP POST to 'http://localhost:9494/quack'

The server process is still listed as running in the OS task manager, but it no longer serves requests.

Restarting the same parallel insert does not recover: every client now fails immediately with a malformed/garbled error referencing the target table, e.g.:

Schema\0main\0main\0Table\0main\0BigTable1

(each client reports the same with its own target table name: BigTable2, BigTable3, …). A full server restart is required to make the endpoint usable again.

The failure is concurrency-dependent: with a lower number of parallel clients the same workload completes successfully. In my tests up to 7 clients in parallel finished without issues; the crash starts to appear at higher concurrency (reproduced here with 10 clients).

To Reproduce

Server (persistent, on-disk DuckDB database):

CALL quack_serve('quack:localhost:9494', token = 'super_secret');

Then start N clients in parallel (e.g. N = 10), each running:

CREATE SECRET (TYPE quack, TOKEN 'super_secret');
ATTACH 'quack:localhost:9494' AS remote;
CREATE OR REPLACE TABLE remote.BigTable<k> AS
  SELECT * FROM read_parquet('data.parquet') LIMIT 4000000;

where <k> is a per-client index (so each client writes to a distinct table) and data.parquet is a ~60 MB Parquet file (~3.7M rows). Any reasonably large Parquet file should reproduce it.

Observed:

  1. First run — all clients run for ~90–100 s, then fail together with Could not connect to server. The server process stays alive but unresponsive.
  2. Second run (server not restarted) — all clients fail almost immediately (~37–41 s) with the garbled Schema\0main\0main\0Table\0main\0BigTable<k> message.

A single client (N = 1) completes successfully, and so does a moderate level of concurrency (up to ~7 clients in my tests); the failure shows up only at higher concurrency.

Environment

OS Windows 11 (x64)
CPU Intel Core i5-1135G7 @ 2.40 GHz (4 cores / 8 threads)
RAM 16 GB
Setup DuckDB CLI + quack extension
DuckDB version v1.5.3 (Variegata)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions