[ZBX-13782] Unicode in JSON From Dependent Item Broken Created: 2018 Apr 21  Updated: 2024 Apr 10  Resolved: 2018 May 13

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 3.4.8
Fix Version/s: 3.4.10rc1, 4.0.0alpha7, 4.0 (plan)

Type: Problem report Priority: Trivial
Reporter: Max Ried Assignee: Andris Mednis
Resolution: Fixed Votes: 0
Labels: json, preprocessing
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

zabbix_server (Zabbix) 3.4.8
Revision 79252 3 April 2018, compilation time: Apr 3 2018 10:48:29

Debian 9.4 x64

mysql Ver 15.1 Distrib 10.1.26-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2


Team: Team A
Sprint: Sprint 32, Sprint 33
Story Points: 6

 Description   

I use an external script to retrieve a JSON that contains a String, that has Unicode characters in it, e.g. ü, which is U+00FC, or ä, which is U+00E4. It is escaped as '\u00e4' inside the value returned by the external script. When I look into the history of the "External Check" item, I see the raw json string with the escaped unicode:

{"DeviceLog": "IPv6-Pr\u00e4fix"}

When this gets filtered by a "Dependent item" item with the "JSON Path" "$.DeviceLog", the item's value becomes "IPv6-Pr?fix". If it encounters a "\u00E4", the dependent item stops there.

 



 Comments   
Comment by Andris Mednis [ 2018 May 02 ]

Fixed in development branches:

  • svn://svn.zabbix.com/branches/dev/ZBX-13782-34 (3.4)
  • svn://svn.zabbix.com/branches/dev/ZBX-13782 (trunk).
Comment by Andris Mednis [ 2018 May 09 ]

Available in versions:

  • pre-3.4.10rc1 r80749
  • pre-4.0.0alpha7 r80751
Comment by Andris Mednis [ 2018 May 09 ]

No documentation changes required.

Comment by Glebs Ivanovskis [ 2019 Feb 21 ]

A comment regarding r80630 is hidden or missing.

Comment by Andris Mednis [ 2019 Feb 21 ]

I could not find reference to r80630 on this page.

Comment by Glebs Ivanovskis [ 2019 Feb 21 ]

And the question is why it is the only one referenced in "fixed in" comment. Compare

$ svn log -r80630 svn://svn.zabbix.com/trunk --diff

and

$ svn log -r80751 svn://svn.zabbix.com/trunk --diff

Perhaps, 3.4 is missing revision too.

Comment by Andris Mednis [ 2019 Feb 21 ]

Hi, Gleb!

Initially it was "fixed in" r80629 and r80630. Then you pointed to some issues with coding style and a typo. After fixing these issues I did another merge and updated numbers in "fixed in" comment to r80749 and r80751.

Comment by Glebs Ivanovskis [ 2019 Feb 21 ]

I must say that regular users don't see any of that. They don't have Subversion tab at all. In History tab they see that issue was reopened, but there is no information about comment editing in Activity.

My memory isn't perfect and it is sometimes challenging to find what you want by browsing svn log without a lead. As far as I remember we used to list all revisions when something was merged to trunk or stable branch, not just the revision when the issue was finally fixed. Knowing all merge revisions makes it easy to isolate changes made to fight the issue.

Comment by Andris Mednis [ 2019 Feb 21 ]

Knowing all merge revisions makes it easy to isolate changes made to fight the issue.

It could be discussed. Currently we list only one - the last revision number in "fixed in" comment.

One can find r80630, for example, as

$ svn log | grep -B2 ZBX-13782
r80751 | andris | 2018-05-11 17:00:39 +0300 (Pk, 11 mai 2018) | 1 line

...G...PS. [ZBX-13782] improved coding style of decoding of Unicode characters in JSON, fixed comment typo
--
r80630 | andris | 2018-05-09 19:19:01 +0300 (T , 09 mai 2018) | 1 line

...G...PS. [ZBX-13782] fixed decoding of Unicode characters in JSON
Comment by Glebs Ivanovskis [ 2019 Feb 21 ]

This is not described in development guidelines, so I must have learned this directly from more knowledgeable colleagues (I suspect asaveljevs ), that in such cases "fixed in" comments should look like this.

Comment by Andris Mednis [ 2019 Feb 22 ]

Ok, I see. Current recommendation is one, the latest rev. number.

Comment by Glebs Ivanovskis [ 2019 Apr 11 ]

There is a test case:

test case: ZBX-13782, escaped Unicode character \u00e4 which translates into 1 byte UTF-8
in:
  json: '{"DeviceLog": "AB-\u00e4-C"}'
  path: $.DeviceLog
out:
  result: succeed
  value: "AB-ä-C"

Character "ä" is a 2-byte UTF-8 sequence, not 1-byte.

Comment by Andris Mednis [ 2019 Apr 11 ]

Only test case description is incorrect ?

Comment by Glebs Ivanovskis [ 2019 Apr 11 ]

Yes. Or a character represented as 1-byte UTF-8 sequence should be chosen. There is another test case for a 2-byte sequence. (I would fix the description of this test case and would add 1-byte UTF-8 test as a new one.)

Comment by Andris Mednis [ 2019 Apr 11 ]

Thanks, Gleb!

Generated at Fri Apr 26 05:52:57 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.