Skip to content

[BUG] Field with null value is omitted when write back to result index #139

Open
@seankao-az

Description

@seankao-az

What is the bug?
When writing back query result to query_execution_result_{datasource}, fields with null value is dropped. I encountered this with cloudwatch datasource.

Sample query:

SELECT srcPort FROM mycw.default.FLE_GLE_VPC_LoadGen LIMIT 10

Raw result:

+-------+
|srcPort|
+-------+
|  20641|
|  20641|
|   null|
|   null|
|  20641|
|  20641|
|   null|
|   null|
|  20641|
|  20641|
+-------+

Corresponding doc that is written back to query_execution_result_{datasource}

{
    "_index": "query_execution_result_mycw",
    "_id": "-BgYq4sBhRhnt4SbZiGq",
    "_score": 1,
    "_source": {
      "result": [
        "{'srcPort':20641}",
        "{'srcPort':20641}",
        "{}",
        "{}",
        "{'srcPort':20641}",
        "{'srcPort':20641}",
        "{}",
        "{}",
        "{'srcPort':20641}",
        "{'srcPort':20641}"
      ],
      "schema": [
        "{'column_name':'srcPort','data_type':'long'}"
      ],
      "jobRunId": "00feineujeco7g0a",
      "applicationId": "00feggo87noppi09",
      "dataSourceName": "mycw",
      "status": "SUCCESS",
      "error": "",
      "queryId": "cEZnRDVhVlVjWG15Y3c=",
      "queryText": "select srcPort from mycw.default.FLE_GLE_VPC_LoadGen limit 10",
      "sessionId": "dThKam9xTU5sVW15Y3c=",
      "updateTime": 1699382583256,
      "queryRunTime": 11805
    }
}

Expected result:

"result": [
        "{'srcPort':20641}",
        "{'srcPort':20641}",
        "{'srcPort':null}",
        "{'srcPort':null}",
        "{'srcPort':20641}",
        "{'srcPort':20641}",
        "{'srcPort':null}",
        "{'srcPort':null}",
        "{'srcPort':20641}",
        "{'srcPort':20641}"
      ]

Metadata

Metadata

Assignees

No one assigned

    Labels

    0.2bugSomething isn't working

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions