ExecuteRawResultAsync / ClickHouseRawResult does not detect in-band mid-stream server exceptions

[claude]

Description

When a query fails after the ClickHouse HTTP interface has already committed 200 OK and started streaming rows (e.g. a runtime throwIf partway through a large result), the server appends an in-band exception block to the response body and closes the connection.

The native read path handles this: ClickHouseDataReader.FromHttpResponseAsync (ClickHouse.Driver/ADO/Readers/ClickHouseDataReader.cs:66-90) reads the X-ClickHouse-Exception-Tag response header and wraps the body in ExceptionTagAwareStream, so ExecuteReader raises a clean ClickHouseServerException (see MidStreamExceptionTests.ShouldDetectMidStreamException).

The raw / custom-FORMAT streaming path does not. ClickHouseClient.ExecuteRawResultAsync (ClickHouse.Driver/ClickHouseClient.cs:428-434) and ClickHouseCommand.ExecuteRawResultAsync (ClickHouse.Driver/ADO/ClickHouseCommand.cs:143-150) construct ClickHouseRawResult directly from the HttpResponseMessage, and ClickHouseRawResult (ClickHouse.Driver/ADO/Readers/ClickHouseRawResult.cs:54-72) hands response.Content straight to the caller via ReadAsStreamAsync / ReadAsByteArrayAsync / ReadAsStringAsync / CopyToAsync. The exception tag header is never consulted.

Consequences for a consumer streaming FORMAT CSV / JSONEachRow / Arrow / Parquet via ExecuteRawResultAsync:

• The consumer sees an System.Net.Http.HttpIOException: The response ended prematurely. (ResponseEnded) (or a truncated body for the buffered accessors) instead of the server's error.
• When the server-side setting http_write_exception_in_output_format=1 is enabled, the raw __exception__<token> ... <token>__exception__ block is injected verbatim into the caller's data stream — i.e. the caller's CSV/Arrow/Parquet parser is fed garbage bytes — and the stream still ends with HttpIOException.
• ReadAsStringAsync / ReadAsByteArrayAsync / CopyToAsync are affected the same way; the body is either truncated or contains the exception block, with no ClickHouseServerException raised.

This is the .NET analogue of clickhouse-connect#913 (Arrow streaming methods bypassing the in-band exception check).

Related but distinct: #333 covers the same ExceptionTagAwareStream-is-only-wired-into-ClickHouseDataReader root cause for ExecuteNonQueryAsync. This issue is about the raw/custom-FORMAT streaming surface.

ClickHouse server version

26.7.1.1315 (local server, HTTP interface). Verified against a running server, not code analysis only.

Reproduction

Scratch NUnit test added to ClickHouse.Driver.Tests/ADO/:

using System;
using System.Threading;
using System.Threading.Tasks;
using ClickHouse.Driver.ADO;
using NUnit.Framework;

namespace ClickHouse.Driver.Tests.ADO;

public class RawMidStreamTests : AbstractConnectionTestFixture
{
    private const string Query = @"
        SELECT toInt32(number) AS n, throwIf(number = 5000000, 'boom') AS e
        FROM system.numbers LIMIT 100000000 FORMAT CSV";

    [Test]
    public async Task RawResultStream_ShouldSurfaceMidStreamException()
    {
        using var command = connection.CreateCommand();
        command.CustomSettings["http_write_exception_in_output_format"] = 1;
        command.CommandText = Query;

        using var result = await command.ExecuteRawResultAsync(CancellationToken.None);
        using var stream = await result.ReadAsStreamAsync();

        var tail = string.Empty;
        long total = 0;
        Exception thrown = null;
        var buf = new byte[64 * 1024];
        try
        {
            int n;
            while ((n = await stream.ReadAsync(buf, 0, buf.Length)) > 0)
            {
                total += n;
                tail = System.Text.Encoding.UTF8.GetString(buf, 0, n);
            }
        }
        catch (Exception e)
        {
            thrown = e;
        }

        TestContext.WriteLine($"BYTES: {total}");
        TestContext.WriteLine($"THROWN: {thrown?.GetType().FullName ?? \"<none>\"}");
        TestContext.WriteLine($"MSG: {thrown?.Message}");
        TestContext.WriteLine($"BODY CONTAINS __exception__: {tail.Contains(\"__exception__\")}");

        Assert.That(thrown, Is.InstanceOf<ClickHouseServerException>(),
            "expected clean server exception carrying 'boom'");
    }
}

Expected: a ClickHouseServerException carrying Code: 395 ... boom, matching what command.ExecuteReader() raises for the same query.

Actual (dotnet test -f net10.0 --filter FullyQualifiedName~RawMidStream):

BYTES: 45983730
THROWN: System.Net.Http.HttpIOException
MSG: The response ended prematurely. (ResponseEnded)
BODY CONTAINS __exception__: True
TAIL:  boom: while executing 'FUNCTION throwIf(equals(__table1.number, 5000000_UInt32) :: 4, 'boom'_String :: 2)
 -> throwIf(equals(__table1.number, 5000000_UInt32), 'boom'_String) UInt8 : 0'.
 (FUNCTION_THROW_IF_VALUE_IS_NON_ZERO) (version 26.7.1.1315 (official build))|288 qurkqsppcevjzlmg

Failed RawResultStream_ShouldSurfaceMidStreamException
  Expected: instance of <ClickHouse.Driver.ClickHouseServerException>
  But was: <System.Net.Http.HttpIOException>

The same test without the http_write_exception_in_output_format custom setting also fails with HttpIOException: The response ended prematurely.

Note that if the error is raised before the server flushes any output (e.g. throwIf(number = 100000) with a small result), the response is a non-2xx and HandleError already produces a correct ClickHouseServerException — the bug only shows once streaming has begun.

Suggested fix

Plumb the same exception-tag handling used by ClickHouseDataReader into the raw path:

• Capture X-ClickHouse-Exception-Tag from the HttpResponseMessage in ClickHouseRawResult's constructor (ClickHouse.Driver/ADO/Readers/ClickHouseRawResult.cs:20-23).
• When the tag is present, wrap the content stream in ExceptionTagAwareStream in ReadAsStreamAsync and in the buffered accessors (ReadAsByteArrayAsync, ReadAsStringAsync, CopyToAsync), so the __exception__ block is stripped from the caller's data and re-thrown as ClickHouseServerException.
• Consider also surfacing a clean error when the tag is absent but the body ends prematurely, so callers get a ClickHouse-flavoured exception rather than a bare HttpIOException.

Link

Upstream report: ClickHouse/clickhouse-connect#913
Tracking: https://github.com/ClickHouse/integrations-ai-playground/issues/330