feat: add SearchAsync for vector, keyword, and hybrid search (#22)

* feat: add SearchAsync for vector, keyword, and hybrid search

Wraps POST /v1/search, which was previously unreachable from .NET without
hand-rolling the HTTP call. spice.js is the only other SDK that exposes it.

Only Text is required; Datasets, Limit, Where, AdditionalColumns and Keywords
are optional. Supplying Keywords pre-filters the embedding column with a lexical
search before the vector search, making the search hybrid.

SearchMatch maps the runtime's wire format, including the `_score` field name
and the objects the runtime omits when empty; those default to empty
dictionaries so callers can read them without a null check.

System.Text.Json was already resolved transitively via Apache.Arrow.Adbc; it is
now referenced explicitly at the version that dependency already selects, since
netstandard2.0 has no in-box System.Text.Json.

* fix: surface why authentication failed instead of a trailer error

AuthenticateAsync passed stream.GetTrailers() as an argument, so trailers were
read eagerly — before the handshake had completed. gRPC only makes trailers
available once a call finishes, so any handshake that fails throws

  System.InvalidOperationException: Can't get the call trailers because the
  call has not completed successfully.

which replaces the gRPC status describing the actual problem. Every integration
test in CI reports this, and it says nothing about what to fix.

Now the token is read from the response headers first, trailers are consulted
only if the headers carry none, and a failure to read them is caught rather
than propagated. The resulting SpiceException names the likely cause and
carries the originating RpcException as InnerException.

Reproduced against a TLS endpoint that is not a Flight service:
  before: InvalidOperationException: Can't get the call trailers ...
  after:  SpiceException: Failed to authenticate: the runtime returned no
          authorization token. Check that the API key is valid for this endpoint.

* fix: scope the System.Text.Json reference to netstandard2.0, name the failing member

Two issues from review:

- System.Text.Json was referenced unconditionally, pushing a dependency
  constraint onto net8.0+ consumers that already have it in-box. Now
  conditioned on netstandard2.0, which is the target that actually needs it.
  netstandard2.0 still resolves 9.0.9, so there is no NU1605 downgrade.
- Argument validation reported nameof(request) when it was request.Text that
  was empty, pointing callers at the wrong member.

Author: lukekim

README.md

@@ -215,6 +215,50 @@ using var client = new SpiceClientBuilder()
 await client.RefreshDatasetAsync("my_dataset");
 ```
 
+#### Search
+
+`SearchAsync` runs vector similarity, keyword, and hybrid search against datasets that have
+an embedding column and a loaded embedding model.
+
+```csharp
+using Spice;
+using Spice.Search;
+
+using var client = new SpiceClientBuilder().Build();
+
+var response = await client.SearchAsync(new SearchRequest("tickets to Tokyo")
+{
+    Datasets = new[] { "app_messages" },
+    Limit = 3,
+});
+
+Console.WriteLine($"{response.Results.Count} matches in {response.DurationMs}ms");
+foreach (var match in response.Results)
+{
+    Console.WriteLine($"{match.Dataset} {match.Score}");
+}
+```
+
+Only `Text` is required. `Datasets` restricts the search — leave it unset to search every
+dataset with an embedding column. `Limit` caps matches per dataset, `Where` applies an SQL
+predicate before the search, and `AdditionalColumns` names extra columns to return. Setting
+`Keywords` pre-filters the embedding column with a lexical search before the vector search
+runs, making the search hybrid:
+
+```csharp
+var response = await client.SearchAsync(new SearchRequest("tickets to Tokyo")
+{
+    Where = "city = 'Tokyo'",
+    AdditionalColumns = new[] { "timestamp" },
+    Keywords = new[] { "plane", "tickets" },
+});
+```
+
+Each `SearchMatch` carries the `Dataset` it was found in, its similarity `Score`, the
+matched column values in `Matches`, the row's `PrimaryKey`, the columns requested via
+`AdditionalColumns` in `Data`, and any `Metadata`. The runtime omits the last three when
+empty; they default to empty dictionaries, so they can be read without a null check.
+
 ### Memory Management
 
 The `SpiceClient` implements `IDisposable` and should be properly disposed to release network resources (gRPC channels, HTTP clients). Use the `using` statement or `using` declaration for automatic disposal:

Spice/Spice.csproj

@@ -55,4 +55,14 @@
         </AssemblyAttribute>
     </ItemGroup>
 
+    <!--
+        netstandard2.0 has no in-box System.Text.Json, so it needs the package. net8.0+
+        ship it in the framework, and referencing it there would push an unnecessary
+        dependency constraint onto consumers. The version matches what
+        Apache.Arrow.Adbc already resolves to, so netstandard2.0 sees no downgrade.
+    -->
+    <ItemGroup Condition="'$(TargetFramework)' == 'netstandard2.0'">
+        <PackageReference Include="System.Text.Json" Version="9.0.9" />
+    </ItemGroup>
+
 </Project>

Spice/src/Errors/SpiceException.cs

@@ -36,4 +36,10 @@ internal SpiceException(SpiceStatus status, string message) : base(message)
     {
         Status = status;
     }
+
+    internal SpiceException(SpiceStatus status, string message, Exception? innerException)
+        : base(message, innerException)
+    {
+        Status = status;
+    }
 }

Spice/src/Flight/SpiceFlightClient.cs

@@ -166,17 +166,83 @@ private async Task AuthenticateAsync()
     {
         var stream = _flightClient.Handshake();
 
-        var headers = await stream.ResponseHeadersAsync.ConfigureAwait(false);
-        var token = GetAuthToken(headers, stream.GetTrailers());
-        
+        Metadata headers;
+        try
+        {
+            headers = await stream.ResponseHeadersAsync.ConfigureAwait(false);
+        }
+        catch (RpcException ex)
+        {
+            throw new SpiceException(
+                SpiceStatus.FailedToAuthenticate,
+                $"Failed to authenticate: {DescribeRpcFailure(ex)}",
+                ex);
+        }
+
+        // Trailers are only readable once the call has completed. Reading them
+        // eagerly throws InvalidOperationException on a handshake that failed,
+        // which masks the gRPC status that says what actually went wrong.
+        RpcException? failure = null;
+        var token = headers.Get("authorization");
+        if (token == null)
+        {
+            token = TryGetTrailerToken(stream, out failure);
+        }
+
         if (token == null || _httpClient == null)
         {
-            throw new SpiceException(SpiceStatus.FailedToAuthenticate, "Failed to authenticate");
+            var reason = failure == null
+                ? "the runtime returned no authorization token."
+                : DescribeRpcFailure(failure);
+
+            throw new SpiceException(
+                SpiceStatus.FailedToAuthenticate,
+                $"Failed to authenticate: {reason} Check that the API key is valid for this endpoint.",
+                failure);
         }
 
         _httpClient.DefaultRequestHeaders.Authorization = AuthenticationHeaderValue.Parse(token.Value);
     }
 
+    /// <summary>
+    /// Reads the authorization token from the handshake trailers, if the call
+    /// completed far enough for them to be available.
+    /// </summary>
+    /// <param name="stream">The handshake call</param>
+    /// <param name="failure">The gRPC failure, when the trailers could not be read</param>
+    /// <returns>The token entry, or null</returns>
+    private static Metadata.Entry? TryGetTrailerToken(
+        AsyncDuplexStreamingCall<FlightHandshakeRequest, FlightHandshakeResponse> stream,
+        out RpcException? failure)
+    {
+        failure = null;
+        try
+        {
+            return stream.GetTrailers().Get("authorization");
+        }
+        catch (RpcException ex)
+        {
+            failure = ex;
+            return null;
+        }
+        catch (InvalidOperationException)
+        {
+            // The handshake never completed, so there are no trailers to read.
+            return null;
+        }
+    }
+
+    /// <summary>
+    /// Renders a gRPC failure as something a caller can act on.
+    /// </summary>
+    /// <param name="ex">The gRPC exception</param>
+    /// <returns>A short description of the failure</returns>
+    private static string DescribeRpcFailure(RpcException ex)
+    {
+        var detail = string.IsNullOrWhiteSpace(ex.Status.Detail) ? ex.Message : ex.Status.Detail;
+        return $"{ex.StatusCode} - {detail}.";
+    }
+
     internal async Task<FlightClientRecordBatchStreamReader> Query(string sql)
     {
         if (string.IsNullOrEmpty(sql))

Spice/src/Http/ISpiceHttpClient.cs

@@ -20,6 +20,8 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
 */
 
+using Spice.Search;
+
 namespace Spice.Http;
 
 /// <summary>
@@ -44,4 +46,15 @@ public interface ISpiceHttpClient : IDisposable
     /// <exception cref="System.ArgumentException">Thrown when datasetName is null or empty</exception>
     /// <exception cref="System.Net.Http.HttpRequestException">Thrown when the HTTP request fails</exception>
     Task RefreshDatasetAsync(string datasetName);
+
+    /// <summary>
+    /// Runs a vector, keyword, or hybrid search against datasets with an embedding column.
+    /// </summary>
+    /// <param name="request">The search to run</param>
+    /// <param name="cancellationToken">Token to cancel the request</param>
+    /// <returns>The matches, ordered by descending score</returns>
+    /// <exception cref="System.ArgumentNullException">Thrown when request is null</exception>
+    /// <exception cref="System.ArgumentException">Thrown when the search text is null or empty</exception>
+    /// <exception cref="System.Net.Http.HttpRequestException">Thrown when the HTTP request fails</exception>
+    Task<SearchResponse> SearchAsync(SearchRequest request, CancellationToken cancellationToken = default);
 }

Spice/src/Http/SpiceHttpClient.cs

@@ -22,7 +22,9 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 
 using System.Net.Http.Headers;
 using System.Text;
+using System.Text.Json;
 using Spice.Auth;
+using Spice.Search;
 
 namespace Spice.Http;
 
@@ -136,6 +138,89 @@ public async Task RefreshDatasetAsync(string datasetName)
         response.EnsureSuccessStatusCode();
     }
 
+    /// <summary>
+    /// Options used to serialize search requests and deserialize search responses.
+    /// </summary>
+    private static readonly JsonSerializerOptions SearchJsonOptions = new()
+    {
+        PropertyNamingPolicy = null,
+    };
+
+    /// <summary>
+    /// Runs a vector, keyword, or hybrid search against datasets with an embedding column.
+    /// </summary>
+    /// <param name="request">The search to run</param>
+    /// <param name="cancellationToken">Token to cancel the request</param>
+    /// <returns>The matches, ordered by descending score</returns>
+    /// <exception cref="System.ArgumentNullException">Thrown when request is null</exception>
+    /// <exception cref="System.ArgumentException">Thrown when the search text is null or empty</exception>
+    /// <exception cref="System.Net.Http.HttpRequestException">Thrown when the HTTP request fails</exception>
+    public async Task<SearchResponse> SearchAsync(SearchRequest request, CancellationToken cancellationToken = default)
+    {
+#if NET8_0_OR_GREATER
+        ObjectDisposedException.ThrowIf(_disposed, this);
+        ArgumentNullException.ThrowIfNull(request);
+        ArgumentException.ThrowIfNullOrWhiteSpace(request.Text, $"{nameof(request)}.{nameof(request.Text)}");
+#else
+        if (_disposed) throw new ObjectDisposedException(GetType().FullName);
+        ThrowHelper.ThrowIfNull(request, nameof(request));
+        ThrowHelper.ThrowIfNullOrWhiteSpace(request.Text, $"{nameof(request)}.{nameof(request.Text)}");
+#endif
+
+        var url = $"{_httpAddress}/v1/search";
+        var json = JsonSerializer.Serialize(request, SearchJsonOptions);
+
+        using var content = new StringContent(json, Encoding.UTF8, "application/json");
+        using var response = await _httpClient.PostAsync(url, content, cancellationToken).ConfigureAwait(false);
+
+#if NET8_0_OR_GREATER
+        var body = await response.Content.ReadAsStringAsync(cancellationToken).ConfigureAwait(false);
+#else
+        // netstandard2.0 has no CancellationToken overload for ReadAsStringAsync.
+        var body = await response.Content.ReadAsStringAsync().ConfigureAwait(false);
+#endif
+
+        if (!response.IsSuccessStatusCode)
+        {
+            // Surface the runtime's own message — it names what the caller needs to fix.
+            throw new HttpRequestException(
+                $"Search failed with status {(int)response.StatusCode}: {ExtractErrorMessage(body)}");
+        }
+
+        return JsonSerializer.Deserialize<SearchResponse>(body, SearchJsonOptions) ?? new SearchResponse();
+    }
+
+    /// <summary>
+    /// Pulls the runtime's error message out of a failed response body, falling back to
+    /// the raw body when it is not the expected shape.
+    /// </summary>
+    /// <param name="body">The response body</param>
+    /// <returns>The message to report</returns>
+    private static string ExtractErrorMessage(string body)
+    {
+        if (string.IsNullOrWhiteSpace(body))
+        {
+            return "(no response body)";
+        }
+
+        try
+        {
+            using var document = JsonDocument.Parse(body);
+            if (document.RootElement.ValueKind == JsonValueKind.Object
+                && document.RootElement.TryGetProperty("error", out var error)
+                && error.ValueKind == JsonValueKind.String)
+            {
+                return error.GetString() ?? body;
+            }
+        }
+        catch (JsonException)
+        {
+            // Not JSON — fall through and report the body verbatim.
+        }
+
+        return body;
+    }
+
     /// <summary>
     /// Releases all resources used by the <see cref="SpiceHttpClient"/>.
     /// </summary>