Skip to content

Set Processor will change field type in _source field when use Mustache #123234

Open
@kin122

Description

@kin122

Elasticsearch Version

all

Installed Plugins

No response

Java Version

bundled

OS Version

linux 4.18.0-348.7.1.el8_5.x86_64

Problem Description

we found this problem, when Mustache is used to refer to a field content, the Set Processor changes the field type in the _source field

Steps to Reproduce

# test index
PUT /test
{
  "mappings": {
    "properties": {
      "status": {
        "type": "integer"
      }
    }
  }
}
# test data
POST /test/_bulk
{"index":{}}
{"status":404}
{"index":{}}
{"status":500}

GET test/_search
# status's type is integer in _source
    "hits": [
      {
        "_index": "test",
        "_type": "_doc",
        "_id": "2ry5NpUBLvnTvXTp1F5z",
        "_score": 1,
        "_source": {
          "status": 404 
        }
      },
      {
        "_index": "test",
        "_type": "_doc",
        "_id": "27y5NpUBLvnTvXTp1F5z",
        "_score": 1,
        "_source": {
          "status": 500
        }
      }
    ]

# add subfield
PUT test/_mapping
{
  "properties": {
    "status": {
      "type": "integer",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    }
  }
}

# create set processor, use Mustache
PUT _ingest/pipeline/copy_status_to_keyword
{
  "description": "resets the value of status and subfields",
  "processors": [
    {
      "set": {
        "field": "status",
        "value": "{{{status}}}"
      }
    }
  ]
}

# update data
POST test/_update_by_query?pipeline=copy_status_to_keyword
{
  "query": {
    "bool": {
      "must_not": {
        "exists": {
          "field": "status.keyword"
        }
      },
      "must": {
        "exists": {
          "field": "status"
        }
      }
    }
  }
}


GET test/_search
{
  "query": {
    "exists": {
      "field": "status.keyword"
    }
  }
}

# results
   "hits": [
      {
        "_index": "test_set",
        "_type": "_doc",
        "_id": "G7zHNpUBLvnTvXTpVIC4",
        "_score": 1,
        "_source": {
          "status": "404" # status's type is changed ,being string
        }
      },
      {
        "_index": "test_set",
        "_type": "_doc",
        "_id": "HLzHNpUBLvnTvXTpVIC4",
        "_score": 1,
        "_source": {
          "status": "500"
        }
      }
    ]

Logs (if relevant)

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions