Remove workaround for 500 errors for empty response bodies

[mraible]

No description provided.

[mraible]

If I remove this workaround and deploy the app, the output from the first call to the Anomali API from the workflow seems the same as before:

{
  "files_created": 5,
  "job_id": "7f9b5ffc",
  "message": "Processed 1000 IOCs into 5 lookup files",
  "meta": {
    "limit": 1000,
    "next": "/api/v2/intelligence/?limit=1000&order_by=update_id&status=active&update_id__gt=0&search_after=4868898",
    "offset": 0,
    "took": 3182,
    "total_count": 583629510
  },
  "next": "4868898",
  "total_iocs": 1000,
  "upload_results": [
    {
      "file": "anomali_threatstream_hash_md5.csv",
      "message": "Upload failed: [{'message': 'Expecting value: line 1 column 1 (char 0)', 'code': 500}]",
      "status": "error"
    },
    {
      "file": "anomali_threatstream_domain.csv",
      "message": "Upload failed: [{'message': 'Expecting value: line 1 column 1 (char 0)', 'code': 500}]",
      "status": "error"
    },
    {
      "file": "anomali_threatstream_email.csv",
      "message": "Upload failed: [{'message': 'Expecting value: line 1 column 1 (char 0)', 'code': 500}]",
      "status": "error"
    },
    {
      "file": "anomali_threatstream_url.csv",
      "message": "Upload failed: [{'message': 'Extra data: line 1 column 2 (char 1)', 'code': 500}]",
      "status": "error"
    },
    {
      "file": "anomali_threatstream_ip.csv",
      "message": "Upload failed: [{'message': 'Expecting value: line 1 column 1 (char 0)', 'code': 500}]",
      "status": "error"
    }
  ]
}

I thought this was fixed by @subbu-cs in CrowdStrike/falconpy#1393? Any thoughts, @prvn or @jshcodes?

FWIW, I did whip up a function to test uploading 150MB, 200MB, and 250MB lookup files. When I run this locally, I can see the first response is a 200.

=== RAW RESPONSE ===
Type: <class 'dict'>
Status code: 200
Errors: [{'message': 'No content was received for this request.'}]

I suspect if I wired this same function up in Foundry, the message would be a 500.

[mraible]

Below are the responses from the 200MB and 250MB file upload requests. They're expected to fail since the max upload limit is 200MB.

============================================================
Testing 200MB file upload with FalconPy...
============================================================
FalconPy version: 1.6.0
Generating 200MB test file (2,097,152 rows)...
Test file size: 204.82 MB
Created test file: /var/folders/bf/g69vtl0x2d12fgm17358l9xc0000gn/T/test_upload_200mb_y1tgxa4t.csv
Uploading to repository: search-all

=== RAW RESPONSE ===
Type: <class 'dict'>
Status code: 413
Errors: [{'code': 413, 'message': 'request body size over limit of 209715200 bytes'}]

*** 200MB: UNEXPECTED STATUS: 413 ***
Cleaned up test file

============================================================
Testing 250MB file upload with FalconPy...
============================================================
FalconPy version: 1.6.0
Generating 250MB test file (2,621,440 rows)...
Test file size: 256.82 MB
Created test file: /var/folders/bf/g69vtl0x2d12fgm17358l9xc0000gn/T/test_upload_250mb_7jk8znbu.csv
Uploading to repository: search-all

=== RAW RESPONSE ===
Type: <class 'dict'>
Status code: 413
Errors: [{'code': 413, 'message': 'request body size over limit of 209715200 bytes'}]

*** 250MB: UNEXPECTED STATUS: 413 ***
Cleaned up test file

============================================================
SUMMARY
============================================================
  150MB: ✓ SUCCESS
  200MB: ✗ FAILED
  250MB: ✗ FAILED

[prvn]

Maybe the limitation is on Logscale itself? Helping out Zoom, I realized that not all customers and clusters there are provisioned are capable of 200MB+ lookup files.

[mraible]

@prvn The results are expected as 200MB is the max for uploaded lookup files. The unexpected part is the 500 parsing error that's still happening when running in Foundry:

Upload failed: [{'message': 'Expecting value: line 1 column 1 (char 0)', 'code': 500}]

I thought @subbu-cs fixed this in CrowdStrike/falconpy#1393.

[mraible]

I talked to @jshcodes yesterday and he said the fix for this did not make it into the latest release of FalconPy. I have a meeting set up with him and @subbu-cs to discuss how to fix it.