Skip to content

[Bug] results[0].partitions[0].relevance returns -Infinity when calling MemoryWebClient.SearchSummariesAsync with PipelineOnlySummary #1048

Open
@rainmakerho

Description

@rainmakerho

Context / Scenario

When calling MemoryWebClient.SearchSummariesAsync with PipelineOnlySummary, the value of results[0].partitions[0].relevance in the answer object becomes -Infinity.

The relevance value is: -1.7976931348623157E+308

Error Message

[15:45:16.251] trce: Microsoft.KernelMemory.Search.SearchClient[0] Fetching relevant memories by similarity, min relevance 0
[15:45:16.252] trce: Microsoft.KernelMemory.Search.SearchClient[0] 1 records processed
[15:45:16.253] fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1] An unhandled exception has occurred while executing the request. System.ArgumentException: .NET number values such as positive and negative infinity cannot be written as valid JSON. To make it work when using 'JsonSerializer', consider specifying 'JsonNumberHandling.AllowNamedFloatingPointLiterals' (see https://learn.microsoft.com/dotnet/api/system.text.json.serialization.jsonnumberhandling).    at System.Text.Json.ThrowHelper.ThrowArgumentException_ValueNotSupported()    at System.Text.Json.Utf8JsonWriter.WriteNumberValue(Single value)    at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, ...

What happened?

  1. start kernel-memory as service
  2. new MemoryWebClient
using Microsoft.KernelMemory;
using Microsoft.KernelMemory.Context;
using System.Net.Http;
var kmServiceEndpoint = "http://localhost:5000/";
var httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMinutes(100);
var kmClient = new MemoryWebClient(kmServiceEndpoint, client:httpClient);
  1. ImportTextAsync with Constants.PipelineOnlySummary
meetingTranscriptResult = """
WEBVTT

00:00:18.344 --> 00:00:22.544
Tony:好,那我們今天就針對大學面試要注意的事項來聊一下,看看大家有沒有什麼經驗或建議可以分享。

Cindy:好啊。我覺得第一個重點是要做功課,像是去查該系的課程、教授的研究方向、還有該科系重視的特質。

RM:嗯,沒錯。面試老師常常會問「你為什麼想讀我們這個科系?」這種問題,如果回答太空泛就不太加分。

Tony:對,我記得以前有人就說「因為分數剛好到這裡」,這種就...有點糟糕(笑)。

Cindy:哈哈,對。還有服裝儀容我覺得也不能忽略,乾淨整齊、得體就好,不用太正式,但一定要讓人覺得你是有準備、尊重這場面試的。

RM:那說到面試技巧,我會建議大家可以事先練習幾個常見的問題,比如「你的興趣是什麼?」、「你怎麼看待團隊合作?」之類的,甚至找朋友模擬一下會更有幫助。

Tony:對,還有自我介紹最好控制在一分鐘內,而且要有重點,不是從小講到大(笑)。

Cindy:哈哈沒錯。另外,如果有參加過比賽或專題,可以稍微提一下,重點是要跟申請的科系連結起來,才有說服力。

RM:最後我覺得也很重要的是要有禮貌,進出面試場都要記得打招呼、感謝,這些小細節其實老師都會注意到。

Tony:總結一下,大概就是:事前準備、儀容得體、自我介紹有重點、練習常見題、還有基本禮儀。這樣應該就能讓印象加分不少。

Cindy:嗯嗯,我覺得這樣一份小提醒給學生也很有幫助。

RM:沒錯,那我們就整理一下今天的重點,看看能不能做成一張 checklist 給學生參考。

Tony:好,今天的討論先到這邊,感謝大家!
""";
var index = "gss";
var text = meetingTranscriptResult;
var context = new RequestContext();
context.SetArg("custom_summary_prompt_str", """"
請根據以下逐字稿內容,撰寫一份結構清晰簡潔的會議記錄。會議記錄應包含:

1. 會議時間與地點(如有)
2. 參與人員
3. 會議主題
4. 討論重點(分條列)
5. 結論與決議事項
6. 待辦事項與負責人

請以正式、條理清晰的格式撰寫會議記錄。若逐字稿中未明確指出某些項目,請註記為「未提及」。
最後以英文方式輸出。
以下是逐字稿:
---
{{$input}}
"""");
context.SetArg("custom_summary_target_token_size_int", 1000); 
context.SetArg("custom_summary_overlapping_tokens_int", 0); 
var steps = Constants.PipelineOnlySummary;
TagCollection tags = new TagCollection();
tags.Add("test");

var documentId = await kmClient.ImportTextAsync(text, null, tags, index,steps,context);
  1. Wait for DocumentReady
bool isDocReady = false;
while (!isDocReady)
{
    isDocReady = await kmClient.IsDocumentReadyAsync(documentId, index);

    if (!isDocReady)
    {
        Console.WriteLine("Document not ready yet. Waiting 3 seconds...");
        await Task.Delay(3000); 
    }
}

Console.WriteLine("Document is ready!");
  1. call MemoryWebClient.SearchSummariesAsync
var result = await kmClient.SearchSummariesAsync(filter: MemoryFilters.ByDocument(documentId), index: index);

throw rror: System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error).
8. Kernel-Memory Service Show An unhandled exception has occurred while executing the request. System.ArgumentException: .NET number values such as positive and negative infinity cannot be written as valid JSON Error

  • Should we add a check for if (recordRelevance < float.MinValue) in the ProcessMemoryRecord method?
if (result.Mode == SearchMode.SearchMode)
        {
            // Relevance is `float.MinValue` when search uses only filters
            if (recordRelevance > float.MinValue) { this._log.LogTrace("Adding result with relevance {0}", recordRelevance); }
            if (recordRelevance < float.MinValue )
            {
                recordRelevance = float.MinValue;
            }
        }

Importance

I cannot use Kernel Memory

Platform, Language, Versions

C#, packages-0.97.250211.1, AOAI, Redis, .NET 8

Relevant log output

[15:45:16.251] trce: Microsoft.KernelMemory.Search.SearchClient[0] Fetching relevant memories by similarity, min relevance 0
[15:45:16.252] trce: Microsoft.KernelMemory.Search.SearchClient[0] 1 records processed
[15:45:16.253] fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1] An unhandled exception has occurred while executing the request. System.ArgumentException: .NET number values such as positive and negative infinity cannot be written as valid JSON. To make it work when using 'JsonSerializer', consider specifying 'JsonNumberHandling.AllowNamedFloatingPointLiterals' (see https://learn.microsoft.com/dotnet/api/system.text.json.serialization.jsonnumberhandling).    at System.Text.Json.ThrowHelper.ThrowArgumentException_ValueNotSupported()    at System.Text.Json.Utf8JsonWriter.WriteNumberValue(Single value)    at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)    at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.Converters.ListOfTConverter`2.OnWriteResume(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryWrite(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)    at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.Converters.ListOfTConverter`2.OnWriteResume(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonCollectionConverter`2.OnTryWrite(Utf8JsonWriter writer, TCollection value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.GetMemberAndWriteJson(Object obj, WriteStack& state, Utf8JsonWriter writer)    at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryWrite(Utf8JsonWriter writer, T value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)    at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.SerializeAsync(PipeWriter pipeWriter, T rootValue, Int32 flushThreshold, CancellationToken cancellationToken, Object rootValueBoxed)    at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.SerializeAsync(PipeWriter pipeWriter, T rootValue, Int32 flushThreshold, CancellationToken cancellationToken, Object rootValueBoxed)    at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.SerializeAsync(PipeWriter pipeWriter, T rootValue, Int32 flushThreshold, CancellationToken cancellationToken, Object rootValueBoxed)    at Microsoft.AspNetCore.Http.HttpResponseJsonExtensions.<WriteAsJsonAsync>g__WriteAsJsonAsyncSlow|5_0[TValue](HttpResponse response, TValue value, JsonTypeInfo`1 jsonTypeInfo, CancellationToken cancellationToken)    at Microsoft.AspNetCore.Http.RequestDelegateFactory.<ExecuteValueTaskOfObject>g__ExecuteAwaited|128_0(ValueTask`1 valueTask, HttpContext httpContext, JsonTypeInfo`1 jsonTypeInfo)    at Microsoft.AspNetCore.Http.RequestDelegateFactory.<>c__DisplayClass101_2.<<HandleRequestBodyAndCompileRequestDelegateForJson>b__2>d.MoveNext() --- End of stack trace from previous location ---    at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)    at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)    at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddlewareImpl.Invoke(HttpContext context)
Image

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingtriage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions