Data written by logbook-logstash is not in sync with ECS model

[kshepitko]

Hi!

I'm using Zalando Logbook v3.11.0, and its addon org.zalando:logbook-logstash.

Although logbook-logstash is supposed to provide compatibility with Elastic stack, it is not quite doing so.
When LogstashLogbackSink works, the serialized HTTP request/response details from "message" field are gone, and are placed into a separate "http" field.

Example:

  "http": {
    "origin": "local",
    "type": "response",
    "correlation": "e259dcb96eb6d47e",
    "duration": 379,
    "protocol": "HTTP/1.1",
    "status": 200,
    "headers": {
      "Cache-Control": [
        "no-cache, no-store, max-age=0, must-revalidate"
      ],
     ......
  }

Much better than serialized JSON, indeed.
However, this is quite different from HTTP fields specification in ECS: https://www.elastic.co/guide/en/ecs/8.17/ecs-http.html
For instance, the fields lack response suffix, and some names are different (e.g., http.status instead of http.response.status_code).
The same stands for request fields.

Is this a known issue?
It doesn't seem to be relevant to ECS version, as the same is actual for 1.x schema version (https://www.elastic.co/guide/en/ecs/1.12/ecs-http.html).