[NETMF4.3.1] Memory leak while using “POST” method by HTTPS.

[Yuri-N]

I am facing the memory leak while using “POST” method by HTTPS.

I put below code to every after throwing “POST” in order to make sure the heap memory condition.
Debug.GC(true).ToString();

Here is the result.

• After throwing 1 of “POST”
  :
  Type 15 (FREEBLOCK ): 6102624 bytes
  :
  Type 1E (BINARY_BLOB_HEAD ): 144252 bytes
  :

• After throwing 20 of “POST”
  :
  Type 15 (FREEBLOCK ): 5395848 bytes
  :
  Type 1E (BINARY_BLOB_HEAD ): 850596 bytes
  :

  I got warning. It seems there is a lack of memory.
  memp_malloc: out of memory in pool TCP_PCB LWIP Assertion "tcp_pcb_purge: pcb->state == SYN_RCVD but tcp_listen_pcbs is NULL" failed at line 337 in C:\MicroFrameworkPK_v4_3\DeviceCode\PAL\lwip\LWIP\src\core\tcp.c

• After throwing 164 of “POST”
  :
  Type 15 (FREEBLOCK ): 69588 bytes
  :
  Type 1E (BINARY_BLOB_HEAD ): 6170388 bytes
  :

I got exception. It seems not to have enough memory for BINARY_BLOB_HEAD.

#### Exception System.Net.Sockets.SocketException - CLR_E_FAIL (1) ####
#### Message:
#### Microsoft.SPOT.Net.Security.SslNative::SecureConnect [IP: 0000] ####
#### Microsoft.SPOT.Net.Security.SslStream::Authenticate [IP: 0060] ####
#### Microsoft.SPOT.Net.Security.SslStream::AuthenticateAsClient [IP: 000c] ####
#### System.Net.HttpWebRequest::EstablishConnection [IP: 0247] ####
#### System.Net.HttpWebRequest::SubmitRequest [IP: 0019] ####
#### System.Net.HttpWebRequest::GetRequestStream [IP: 0008] ####

In this time, program does not run properly because of memory shortage.

 
[Temporary solution]
It seems that EstablishConnection() in System.Net.HttpWebRequest.cs causes memory leak.
Actually, EstablishConnection() is called by every time when “POST” is proceeded.
This is why memory leaks with every “POST” throwing.
I temporary changed code as below.
File: System.Net.HttpWebResponse.cs
Function name: protected override void Dispose(bool disposing)

--- original code
// If server had not send this header or value is not "close", then we keep connection. closeConnection = connValue == null || connValue.ToLower() == HttpKnownHeaderValues.close;

--- Modified code

// If server had not send this header or value is not "close", then we keep connection.
//closeConnection = connValue == null || connValue.ToLower() == HttpKnownHeaderValues.close;

if (connValue == null) closeConnection = false;                                                     
else if(connValue.ToLower() == HttpKnownHeaderValues.close) closeConnection = true;              
else closeConnection = false;                                                                    

This modification is related with what to dispose after receiving reply from server.

• In the case of original code
  Even if server had not sent "Connection:Close", the original code close socket.
• In the case of modified code
  If server had not sent "Connection:Close", never close socket.

I am not sure if this modification is acceptable, or right way to fix it.
At least, it is working very well in my end.

[smaillet-ms]

Unless my pre-caffeinated brain is missing something... that can be reduced to a simpler expression:

closeConnection = connValue.ToLower() == HttpKnownHeaderValues.close;

Since only one condition leads to setting closeConnection to true that makes for a simpler form

[Yuri-N]

Dear smaillet-ms,

Thanks for advice! I will try it.

Unfortunately, this solution works only during Keep-alive state. Once "Keep-alive" expires, need to connect to server by EstablisheConnection(). It makes memory leak again.
For example,, To connect to the server which has Keep-alive max is 100, the memory leak occurs every 100 POST messages.

In my feeling, HttpWebRequest.AllowWriteStreamBuffering should be "false" but NETMF4.3.1 does not support it.

So... need to find out the root cause of memory leak. maybe during buffering BLOB.
But it is too difficult for me...
If you help me to solve it, I would appreciate very very much.

Best Regards,
Yuri

[smaillet-ms]

The actual leak is likely the same one identified in #160. At present we don't have any idea where the actual leak is (It seems to be deep in the OpenSSL stack or the NETMF wrappers around it but the leak has proven to be very illusive and difficult o identify)