[ZBXNEXT-3863] Extend item preprocessing with jsonpath function Created: 2017 May 11  Updated: 2024 Apr 10  Resolved: 2017 Aug 15

Status: Closed
Project: ZABBIX FEATURE REQUESTS
Component/s: Server (S)
Affects Version/s: 3.4.0alpha1
Fix Version/s: 3.4.0alpha1, 3.4 (plan)

Type: New Feature Request Priority: Minor
Reporter: Andris Zeila Assignee: Unassigned
Resolution: Fixed Votes: 3
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Causes
causes ZBX-13556 Not possible to use '-' character in ... Closed
causes ZBX-13778 Logically dead code in JSON Path parser Closed
Sub-task
depends on ZBXNEXT-3006 A cache to provide multiple metrics o... Closed
part of ZBXNEXT-3190 Add new REST API simple check with JS... Closed
Team: Team A
Sprint: Sprint 8, Sprint 9, Sprint 10, Sprint 11, Sprint 12
Story Points: 3

 Description   

Add item preprocessing jsonpath function to extract fragment from json data with a lightweight version of jsonpath (simple subset of functionality).



 Comments   
Comment by Andris Zeila [ 2017 May 15 ]

Server is ready for testing in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-3863

Comment by Rostislav Palivoda [ 2017 May 17 ]

Todo spec for frontend changes and implement frontend for this feature.

Comment by Rostislav Palivoda [ 2017 Jun 08 ]

Please include in test case - parsing of Ceph Rest API response - http://ceph.com/geen-categorie/experimenting-with-the-ceph-rest-api/
SoapUI could be used to emulate service - https://www.soapui.org/rest-testing-mocking/rest-mock-service-creation/rest-mock-from-service.html

Discussed with akucenko and vjaceslavs. Depents on items multipe items and json preprocessing. Then its possible to get JSON via page.get agent item. Still it's nice to have direct JOSN parsing item on server. We are going to monitor applications deployed across multiple servers. Proposed to raise priority of ZBXNEXT-3190.

Comment by Andris Zeila [ 2017 Jun 21 ]

Merged xpath support from ZBXNEXT-3864

Comment by Ivo Kurzemnieks [ 2017 Jun 21 ]

(1) Translation strings added:

  • JSON Path
  • Structured data
  • XML XPath
  • path

oleg.egorov CLOSED

Comment by Ivo Kurzemnieks [ 2017 Jun 21 ]

To avoid unnecessary conflicts in code and extra work, frontend development for ZBXNEXT-3864 has been implemented here.

Comment by Ivo Kurzemnieks [ 2017 Jun 21 ]

Frontend side together with ZBXNEXT-3864 is ready for testing svn://svn.zabbix.com/branches/dev/ZBXNEXT-3863

Comment by Dimitri Bellini [ 2017 Jun 22 ]

Hi Andris,
I have tested this new feature in the latest SVN but i could not understand how it work (I know is working in progress)
I have configure an item as an Extern Check that execute a cat over a json file:
File: JSON

{
  "firstName": "John",
  "lastName" : "doe",
  "age"      : 26,
  "address"  : {
    "streetAddress": "naist street",
    "city"         : "Nara",
    "postalCode"   : "630-0192"
  },
  "phoneNumbers": [
    {
      "type"  : "iPhone",
      "number": "0123-4567-8888"
    },
    {
      "type"  : "home",
      "number": "0123-4567-8910"
    }
  ]
}

I have configure the Item Preprocessing steps as follow:

JSON Path: $.phoneNumbers[*].type

But i receive this error:

25964:20170622:104027.632 error reason for "Zabbix server:test.sh" changed: Item preprocessing step #1 failed: cannot extract value from json by path "$.phoneNumbers[*].type": unsupported character in json path starting with: "[*].type"

Some help?
Thanks very much

Comment by Andris Zeila [ 2017 Jun 22 ]

Yes, currently only small part of json path features are supported. Basically you can spefify path using dot/bracket notation and access array elements by index. So you should be access first phone number by $.phoneNumbers[0], but you enumerate them.

Comment by Dimitri Bellini [ 2017 Jun 22 ]

Hi Andris and thank you for the reply but i'm confused
Follow you suggestion i have used the simple JSON Path "$.phoneNumbers[0]" on my JSON file, below the output received from Zabbix Server:

{
      "type"  : "iPhone",
      "number": "0123-4567-8888"
    }

But how i can deal with this common scenario? Are you still working to implement the Full version of JSON parsing?
Your work is proactive for the multiple value on a single check (ZBXNEXT-3006)?
Thanks very much

Comment by Dimitri Bellini [ 2017 Jun 22 ]

Ops, i solved to my first question....
Using a JSON Path like that "$.phoneNumbers[0].type", i was able to retrieve in Zabbix Server the below value:

iPhone

But what about to my second question (multiple values)?

Comment by richlv [ 2017 Jun 22 ]

dimitri.bellini, looks like lack of a specification is causing quite a lot of confusion. maybe specs will be written in the future

Comment by Andris Zeila [ 2017 Jun 22 ]

For 3.4 there are no plans to support full jsonpath functionality. Making one from scratch is quite an effort and using existing libraries would suggest replacing our current json parsing functionality.

We might even have embedded scripts before full jsonpath support.

Comment by Dimitri Bellini [ 2017 Jun 22 ]

Richlv: Yes, a specification is needed.. maybe non so far
Andris: Have made other test using JSON Path feature and everything seems point to solve the old problem "catch a single value", but ZBXNEXT-3006 is not related to have a single check for multiple values (maybe items)?

I hope to see "embedded scripts" as soon as possible, this feature give us a very helpful improvement in our work!

Comment by Andris Zeila [ 2017 Jun 22 ]

In the context of ZBXNEXT-3006 jsonpath is meant to extract values from some status report in json format. So basically when data has static format. For dynamical format we would require some kind of lld or something. In your example to discover the list of phone numbers and then retrieve the corresponding values.

Comment by Dimitri Bellini [ 2017 Jun 22 ]

Ok, so is not related to read multiple values from a single check, like in my example "type" and "number".

{
      "type"  : "iPhone",
      "number": "0123-4567-8888"
}
Comment by Oleg Egorov (Inactive) [ 2017 Jun 27 ]

Frontend successfully tested

Comment by Andris Zeila [ 2017 Jul 05 ]

Released in:

  • pre-3.4.0alpha1 r69806
Comment by Julien Leboeuf [ 2017 Sep 29 ]

It would be nice to allowed traversal of json when we know that there are no multiple values.

For exemple, with ElacticSearch, if in a template, we want to recover statistics for one node, we can use :

CURL /_nodes/_local/stats

which gives :

{
  "_nodes" : {
    "total" : 1,
    "successful" : 1,
    "failed" : 0
  },
  "cluster_name" : "es_cluster",
  "nodes" : {
    "iLlDeJ5MRyuBBGPBq4BAfQ" : {
      "timestamp" : 1506691036638,
      "name" : "server",
      "transport_address" : "172.31.227.1:9300",
      "host" : "172.31.227.1",
      "ip" : "172.31.227.1:9300",
      "roles" : [
        "master",
        "data",
        "ingest"
      ],
      "indices" : {
        "docs" : {
          "count" : 3226435589,
          "deleted" : 25
        },
		...

Here "iLlDeJ5MRyuBBGPBq4BAfQ" is the ID of the only node retrieved,it is not known in advance and in fact we don't care about this value.

We could use syntax like :

$.nodes.[*].indices.docs.count or $.nodes..indices.docs.count or $.nodes.*.indices.docs.count

to get unique values for this node, but it is currently impossible.

Generated at Fri Apr 26 19:49:06 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.