Description

The recently introduced _ignored meta field (https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-ignored-field.html) is helpful to detect issues with data ingestion.

However, it requires read permissions and potentially expensive search queries to get basic statistics.

Adding information about _ignored field usage to the index stats would allow to monitor it in a cheap way without requiring read access to the indices. Kibana is running a regular background job to collect basic telemetry about the data stored in the cluster. However, for security reasons the Kibana system user does not have read access to the indices and can only retrieve metadata. This is sufficient to collect metrics like the number of documents, but to monitor data quality it's important to be able to collect the number of "degraded documents" (documents that have _ignored set) as well.

The API could look like this:

GET my-index/_stats

{
  "_all": {
    "total": {
      "ignored_fields": {   
        "degraded_docs": 123
     }
  }
}

Implementation notes: #108092 (comment)