Uploaded image for project: 'ZABBIX FEATURE REQUESTS'
  1. ZABBIX FEATURE REQUESTS
  2. ZBXNEXT-8704

Ability to stream metrics and events to Elasticsearch

XMLWordPrintable

    • Icon: New Feature Request New Feature Request
    • Resolution: Unresolved
    • Icon: Trivial Trivial
    • None
    • None
    • Server (S)
    • None

      The new feature to create a connector and be able to stream events and systems is great however missing functionality. 

      Elasticsearch is able to receive data streams, however it expects always an @timestamp field (https://www.elastic.co/guide/en/elasticsearch/reference/current/set-up-a-data-stream.html)

      Documents must include a @timestamp field.

      Can you include a @timestamp field in the connector?

      Also the streaming of multiple events at once to Elasticsearch seems not possible in the current configuration, elasticsearch expects: (according to the same documentation)
      PUT my-data-stream/_bulk{ "create":{ } }

      { "@timestamp": "2099-05-06T16:21:15.000Z", "message": "192.0.2.42 - - [06/May/2099:16:21:15 +0000] \"GET /images/bg.jpg HTTP/1.0\" 200 24736" }

      { "create":{ } }

      { "@timestamp": "2099-05-06T16:25:42.000Z", "message": "192.0.2.255 - - [06/May/2099:16:25:42 +0000] \"GET /favicon.ico HTTP/1.0\" 200 3638" }

      However, zabbix is sending a POST request in stead of a PUT and is not sending a { "create":{ } } before each item. 
      POST /zabbix-data-stream/_doc HTTP/1.1
      Host: elasticsearch:9200
      Accept: /
      Accept-Encoding: deflate, gzip, br
      Content-Type: application/x-ndjson
      Content-Length: 595

      {"host":\{"host":"Zabbix server","name":"Zabbix server"}

      ,"groups":["Zabbix servers"],"item_tags":[\{"tag":"component","value":"internal-process"}],"itemid":28536,"name":"Zabbix server: Utilization of LLD manager internal processes, in %","clock":1694543316,"ns":686250492,"value":0,"type":0}

      {"host":\{"host":"Zabbix server","name":"Zabbix server"}

      ,"groups":["Zabbix servers"],"item_tags":[\{"tag":"component","value":"internal-process"}],"itemid":23256,"name":"Zabbix server: Utilization of escalator internal processes, in %","clock":1694543316,"ns":686310705,"value":0.0846740050804403,"type":0}

      It would be great to have a Elasticsearch connector or to customize the connector settings so this settings can be customized to also be compatible with other streaming receivers.

            wiper Andris Zeila
            Albert-Jan A.J. Goedhart
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: