Having difficulty using WiFi to download a file

[jdeltoft]

I am wondering if anyone has tried using WifiClient with this repo? I'm trying a simple http download of a text file but hit the issue described by this:

espressif/arduino-esp32#6129

And there is a patch mentioned here:

https://github.com/espressif/arduino-esp32/pull/8541/files

But this is somewhat old so I was wondering if anyone else needed to do this or maybe something else to use WifiClient?

The error I get is [WiFiClient.cpp:517] flush(): fail on fd 48, errno: 11, "No more processes" and I have confirmed the url works from another computer on the same LAN with just Chrome

I see another wifi discussion here but it looks like its asking about noise/interference.

[jdeltoft]

Ok, applied that patch from https://github.com/espressif/arduino-esp32/pull/8541/files and it's working now.

Found the file to manually patch at ~/.packages/[email protected]/libraries/WiFi/src/WiFiClient.cpp

which was found when looking at a pio build as

PACKAGES: 
 - framework-arduinoespressif32 @ 3.20009.0 (2.0.9) 

Then I realized my HTTP GET was not going to work due to the file size. :-) The getString() body was empty. All working now, sorry to bug here.

[mrcodetastic]

Is there basically a bug in the esp-Arduino library?

[jdeltoft]

Yup, that patch mentioned in comment#1 was merged to main in Oct but I didn't see any release with it yet. So I just manually added it and it worked perfectly.

Now I'm playing with memory constraints of the esp32 for RAM, and also having a small issue I think with getting off by one nibble or byte in my rgb for setup of 4 of the 64x64 panels. I'm using this with Squeezebox and it's a lot of fun to setup a server to pull the art from the LMS (logitech media server), run through PIL with python to make 128x128 and then save to RGB and serve to the esp32 over wifi.

There is also one weird issue I'm not sure about yet with this function. I use 0,0 for x and y, but it seems that maybe this function drawPixelRGB888() is expecting 1,1 to be the top left pixel?

void vShowImage(VirtualMatrixPanel * vDisp, uint16_t x, uint16_t y, uint8_t* img, uint16_t w, uint16_t h) {
  uint16_t rgbCnt=0;
  for (uint16_t j=0; j<h; j++ ) {
    for (uint16_t i = 0; i<w; i++){
      vDisp->drawPixelRGB888( x+i, y+j, img[rgbCnt], img[rgbCnt+1], img[rgbCnt+2]);
      rgbCnt+=3;
    }
  }
} 

[jdeltoft]

I need to re-open this. In general the discussion might be just a pure wifi related discussion, but figured I would start here since I started this discussion.

So I'm having some further wifi issues with downloading album art, but only after some number of attempts. I added some interesting prints to the loop that does the stream->readBytes(). Here is the code snippet:

bool getAlbumArt() {
  HTTPClient http;
  int rxCount = -1;
  bool retVal = false;

  // first check if there is a new album art playing
  http.begin("http://192.168.1.123:1234/albumcount.txt");
  int httpCode = http.GET();
  if(httpCode > 0) {
    if(httpCode == HTTP_CODE_OK) {
      rxCount = (http.getString()).toInt();
      if (albumArtCount == -1 || rxCount != albumArtCount) {
        Serial.print("album count:");
        Serial.println(rxCount);
      }
    }
  } else {
    Serial.print("Error code: ");
    Serial.println(httpCode);
  }
  http.end();

  delay(100);

  for (int retry=0; retry < 1 && retVal == false; retry++) {
    if (albumArtCount == -1 || rxCount != albumArtCount) {
      http.begin("http://192.168.1.123:1234/albumart.bin");
      int httpCode = http.GET();
      if(httpCode > 0) {
        if(httpCode == HTTP_CODE_OK) {
          int len = http.getSize(); 
          int lenLeft = len;
          uint8_t buff[256] = { 0 };
          uint16_t offset = 0;
          WiFiClient * stream = http.getStreamPtr();

          stream->setTimeout(1900);

          printf("Len of image is %d\n", len);
          while(http.connected() && (lenLeft > 0 || lenLeft == -1)) {
            size_t size = stream->available();
            if(size) {
              int byteCount = ((size > sizeof(buff)) ? sizeof(buff) : size);

              if (byteCount == sizeof(buff)) {
                printf("o");
              } else {
                printf(".");
              }

              int c = stream->readBytes(buff, byteCount);

              if (c != byteCount) {
                printf("read less bytes than requested: attempt:%d actual:%d\n", byteCount, c);
              }

              for (int i=0; i<c; i++) {
                albumArtData[offset + i] = buff[i];
              }
              if(lenLeft > 0) {
                lenLeft -= c;
                offset += c;
              }
            }
            delay(1);
          }
          printf("\ntotal downloaded %d\n", offset);
          if (offset != len) {
            printf("failed download of art!!!!!\n");
            // TODO: put up some err artwork?
          } else {
            albumArtCount = rxCount;
            retVal = true;
          }
        }
      } else {
        Serial.print("Error code: ");
        Serial.println(httpCode);
      }
      http.end();
    }
  }

  return retVal;
}

This gives me this output, where the first several are fast , and then you and see it starts failing. The album count tells where the retries are happening.

lbum count:78
Len of image is 49152
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo
total downloaded 49152
album count:79
Len of image is 49152
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.ooooooooooooooooooo.
total downloaded 49152
album count:80
Len of image is 49152
ooooooooooooooo.ooooo.oooooooooooooooooooooooooooooooooooooooooooo.
total downloaded 16932
failed download of art!!!!!
album count:80
Len of image is 49152
ooooooooooooooooooooo.
total downloaded 5444
failed download of art!!!!!
album count:80
Len of image is 49152
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.oooo.ooooooooooooooooo.oooooooooooooo.oooooooooooooooooo.ooooooooo.ooooooooo.oooooooooooooooooooooooooooooooo.ooooooooooooooooooo.
total downloaded 49152
album count:81
Len of image is 49152
oooooooooooooooooooooooooooooooo.oooooooooooooooooooooooooooo.oooooooooooooooooooooooooooo.
total downloaded 22676
failed download of art!!!!!
album count:81
Len of image is 49152
ooooooooooooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.
total downloaded 21240
failed download of art!!!!!
album count:81
Len of image is 49152
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.ooooooooooooooooooooooooooooooooo.ooooo.oooooooooooooooooooooo.oooooooooooooooooooooooooooooooooooooooooooo.
total downloaded 42780
failed download of art!!!!!
album count:81
Len of image is 49152
oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.
total downloaded 24112
failed download of art!!!!!
album count:81
Len of image is 49152
oooo.ooooo.ooooo.ooooo.ooo.oooo.oooo.oooo.oooo.oooo.oooo.oooo.oooo.oooo.
total downloaded 17208
failed download of art!!!!!
album count:81
Len of image is 49152
ooooooooooooooo.ooooooooooooooooooooooooooooooooo.
total downloaded 12624
failed download of art!!!!!
album count:81
Len of image is 49152
ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.oooooooooooooooooooooooooooooooooooooooooooo.oooooooooooooooooooooooooooo.oooo.ooooooooooooooooo.ooooooooo.ooooooooooooooo.
total downloaded 49152
album count:82
Len of image is 49152
ooooooooooooooo.oooooooooooooooooooooo.ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.oooooooooooooooooooooooooooooooooooo.
total downloaded 49152
album count:83
Len of image is 49152
ooooooooooooooo.oooooooooooooooooooooooooooooooooooooooooooooooooo.ooooo.ooooo.ooooo.oooooooooooooooooooooooooooooooooooooooooooo.ooooo.ooooo.oooooooooooooooooooooooooooo.oooooooooooooooooooooo.oo.
total downloaded 49152
album count:84
Len of image is 49152
oooooooooooooooooooooooooooooooo.ooooooooooooooooooooooooooooooooo.
total downloaded 16930
failed download of art!!!!!
album count:84
Len of image is 49152
oooo.ooooooooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.
total downloaded 12624
failed download of art!!!!!
album count:84
Len of image is 49152
ooooooooooooooo.oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo.oooooooooooooooooooooo.oooooooooooooooo.ooooo.ooooooooooo.ooooooooooooooooooooooooooooooooooooooooo.
total downloaded 49152

[jdeltoft]

So I can't tell if it's a server issue or client issue, but if I restart the esp32 it works again for the first several just fine. I think that proves there is a client issue, but not sure where to look next.

[jdeltoft]

Well, there ya go. Its counterintuitive (to me anyway), but if I lower the timeout from 1900 to 500 then it works every time so far. I guess it's keeping the connection alive.

total downloaded 49152
album count:91
Len of image is 49152
oooo.ooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.oooooooooooooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.oo.
total downloaded 49152
album count:92
Len of image is 49152
oooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.oooooooo.
total downloaded 49152
album count:93
Len of image is 49152
oooooooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.oo.oooo.oooo.oooo.oooo.oo.oooo.oooo.o.oooo.o.
total downloaded 49152
album count:94
Len of image is 49152
oooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooo.ooooo.ooooooooooo.ooooo.ooooo.ooooo.ooooo.ooooo.ooooo.oo.
total downloaded 49152

[mrcodetastic]

Probably best to make sure you close and clean up after every connection as well to ensure no memoey leaks.

[jdeltoft]

I do call http.end() each time. Is there any part of that that needs more async time to complete?

[mrcodetastic]

That should be enough. So long as there's some form of cleanup.

[jdeltoft]

Just to update this, I did move over to the espessif wifi library after I realized the arduino one was 12 years old.

It works better, but I think two other main issues still are wifi antenna placement, and free heap. I have ordered S3 ESP32s to help free heap.