RequestPushMessageDeliveryAsync Fails with Unable to write content to request stream; content would exceed Content-Length.

[twurm]

We've been using the lib for a long time. In our .Net 8 upgrade testing we started seeing the following:

System.Net.Http.HttpRequestException: Unable to write content to request stream; content would exceed Content-Length.
   at Lib.Net.Http.EncryptedContentEncoding.Aes128GcmEncoding.EncryptContentAsync(Stream source, Stream destination, Int32 recordSize, Byte[] contentEncryptionKeyInfoParameterHash, Byte[] nonceInfoParameterHash)
   at Lib.Net.Http.EncryptedContentEncoding.Aes128GcmEncoding.EncodeAsync(Stream source, Stream destination, Byte[] salt, Byte[] key, Byte[] keyId, Int32 recordSize)
   at Lib.Net.Http.EncryptedContentEncoding.Aes128GcmEncodedContent.SerializeToStreamAsync(Stream stream, TransportContext context)
   at System.Net.Http.HttpContent.g__WaitAsync|56_0(ValueTask copyTask)
   at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
   at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingHttpMessageHandler.g__Core|5_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.g__Core|5_0(HttpRequestMessage request, Boolean useAsync, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
   at Lib.Net.Http.WebPush.PushServiceClient.RequestPushMessageDeliveryAsync(PushSubscription subscription, PushMessage message, VapidAuthentication authentication, VapidAuthenticationScheme authenticationScheme, CancellationToken cancellationToken) in /_/src/Lib.Net.Http.WebPush/PushServiceClient.cs:line 189

Any thoughts on what Content is blowing past the Content-Length. We see this issue with both Microsoft and Google's push notification services.

[tpeczek]

Hi @twurm,

Looks like it's coming from .NET HttpClient. Maybe some internal change/optimization to how content length is being handled which interferes with how the lib is doing things. I'll add .NET 8 to tests targets and see if I can catch it. If not, I'll probably ask for some minimal repro if possible.

[tpeczek]

Hi @twurm,

I've added .NET 8 to tests targets and they all passed. I've also migrated the demo project to .NET 8 and tested it - it worked as expected.

This suggests to me that the issue you are experiencing is context-specific. I can try to investigate it, but I'll need as many details as possible - ideally a minimal reproduction.

[twurm]

@tpeczek I've been walking back changes all day it looks at the moment like when we switched from NewtonSoft to System.Text.Json as part of the upgrade we introduced the error. I haven't nailed down the serialization issue just yet but going back to NewtonSoft for serialization has solved the issue. Sorry to have tied up your time into this. Thanks for the help and the quick response.

When we nail down the serialization issue I'll update the ticket for anyone else that stumbles across it.

[twurm]

@tpeczek I've been walking back changes all day it looks at the moment like when we switched from NewtonSoft to System.Text.Json we introduced the error. I haven't nailed down the serializationn issue just yet but going back to NewtonSoft for serialization has solved the issue. Sorry to

[tpeczek]

Sorry to have tied up your time into this. Thanks for the help and the quick response.

No problem, in fact I should have add .NET 8 to tests targets anyway so it's good that you've motivated me.

When we nail down the serialization issue I'll update the ticket for anyone else that stumbles across it.

Please do. I'm honestly intrigued myself. Also it might be something that the library should sanitize or validate, so there might be some work to do here on my end.

[tbdrz]

@tpeczek I got the same error, after a lot of debugging, testing, the exception happens when providing an empty string in the PushMessage constructor:

await _pushServiceClient.RequestPushMessageDeliveryAsync(item.Subscription, new PushMessage(""), cancellationToken);

This works fine:

await _pushServiceClient.RequestPushMessageDeliveryAsync(item.Subscription, new PushMessage("test"), cancellationToken);

[tpeczek]

Thank you @tbdrz - I'll take a look at this over the weekend.

[tpeczek]

Once again, thank you @tbdrz, your observation was absolutely correct.

It looks that the length computation for the encoded content has a bug and it incorrectly calculates the length when to content to be encrypted is empty (tpeczek/Lib.Net.Http.EncryptedContentEncoding#15). It was like this always, but it become issue only since .NET 8 (I'm guessing something in HttpClient has changed internally).

In @twurm case I guess that change from Newtonsoft.Json to System.Text.Json has caused something to be serialized into an empty string instead of empty object or array.

[tbdrz]

@tpeczek No problem! Thank you for the package and maintaining it!