[ZBX-10540] Japanese characters are changed to ?????? in "Trigger Description" have been created by the discovery Created: 2016 Mar 16  Updated: 2017 May 30  Resolved: 2016 May 07

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 3.0.1
Fix Version/s: 3.0.3rc1, 3.2.0alpha1

Type: Incident report Priority: Minor
Reporter: takahisa Assignee: Unassigned
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS Linux release 7.2.1511 (Core)
MySQL 5.6.27


Attachments: File ZBX-10540.patch     PNG File fig1.png     PNG File fig2.png     Text File trigger_table_mysql.txt    
Issue Links:
Duplicate
is duplicated by ZBX-10665 Cyrillic characters replaced by quest... Closed
is duplicated by ZBX-10857 when I use vsphere:https://ip/sdk tem... Closed
is duplicated by ZBX-11089 zabbix v 3.0.1 email bad utf-8 Closed
is duplicated by ZBX-10707 Double UTF-8 coding for Discovered Se... Closed
is duplicated by ZBX-10929 Can't use unicode in LLD JSON string Closed

 Description   

Japanese characters are changed to ?????? in "Trigger Description" that have been created by the discovery. (fig1.png)

In MySQL, Japanese characters are changed to ??????. ("trigger_table_mysql.txt" and "fig2.png RED BOX")

If edit template item , The "??????" are changed to normal (linked change. and fig1.png BLUE BOX)

If delete the item and re-discovery, Reproduce any number of times in all of the items.

If restart the zabbix-server (systemctl restart zabbix-server) , then operating normally.

But It relapse after a few weeks.

It looks like ZBX-9307



 Comments   
Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 11 ]

Did you use the correct command to create MySQL database as documented here?

mysql> create database zabbix character set utf8 collate utf8_bin;

Or maybe you have correct character set for database, but the default character set for connections is not utf8. Frontend does SET NAMES utf8; explicitly, but sever does not. So, if MySQL assumes Zabbix server connects in latin1 it will send back '?' instead of non-ASCII characters in results. Server simply takes these '???' from template and copies to discovered hosts.

I failed to reproduce your issue by messing around with /etc/mysql/my.cnf. Could you provide more details?

Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 11 ]

Managed to reproduce. I believe the decisive moment was when I added this line to /etc/mysql/my.cnf:

[mysqld]
...
init-connect='SET character_set_results = latin1'
Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 13 ]

Hm, server does set names utf8 too...

Comment by richlv [ 2016 Apr 14 ]

ZBX-10665 could be the same

Comment by takahisa [ 2016 Apr 14 ]

ADD INFO.

After restart the zabbix-server (systemctl restart zabbix-server) , LLD created normal entrys.
About 1 days later(maybe) from restart, LLD created normal entrys.
About 2 days later from restart, LLD created illegal("?????") entrys.

I think MySQL setting is normal.
But I tryed.

1st settings
character-set-server = utf8
skip-character-set-client-handshake

2nd settings
character-set-server = utf8
skip-character-set-client-handshake
collation-server=utf8_bin

Still relapsing....

Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 14 ]

Now it's clear to me. ZBX-6163 enabled auto-reconnection to MySQL, but result of set names utf8; is lost on reconnection. That's why this bug is so random. Whenever Zabbix process loses connection to database MySQL automatically reconnects and falls back to default encoding.

Simple test with mysql client:

mysql> select * from test_utf8;    <--- default settings
+------+------+
| id   | data |
+------+------+
|    1 | ???? |
|    2 | ???? |
+------+------+
2 rows in set (0.00 sec)

mysql> set names utf8;    <--- correct settings
Query OK, 0 rows affected (0.00 sec)

mysql> select * from test_utf8;
+------+----------+
| id   | data     |
+------+----------+
|    1 | фыва |
|    2 | олдж |
+------+----------+
2 rows in set (0.00 sec)

mysql> select * from test_utf8;
ERROR 2006 (HY000): MySQL server has gone away    <--- restarted MySQL server in the meantime
No connection. Trying to reconnect...
Connection id:    37
Current database: test_utf8

+------+------+
| id   | data |    <--- default settings again
+------+------+
|    1 | ???? |
|    2 | ???? |
+------+------+
2 rows in set (0.00 sec)
Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 14 ]

As a temporary workaround try to set character_set_connection, character_set_result and character_set_client to utf8 in MySQL server settings ("[client]" section).

Comment by takahisa [ 2016 Apr 15 ]

When the workaround has not applied,

1.
Connect mysql

2.
mysql> select * from triggers;
<<<RETURN IS OK>>>

3.
<<<reboot mysql server>>>

4.
mysql> set names latin1;
mysql> select * from triggers;
<<<RETURN IS NG "??????">>>

5.
mysql> set names utf8;
mysql> select * from triggers;
<<<RETURN IS OK>>>

I think "skip-character-set-client-handshake" is working.

But now the workaround has applied.
And Let's wait and see.

Comment by takahisa [ 2016 Apr 15 ]

I think the workaround does not working

1. Set the workaround ( set the my.cnf on the Zabbix Server = mysql client )
2. Exec "systemctl restart zabbix-server"
3. Reboot mysql server <----- maybe disconnect mysql connection?
4. LLD created illegal("?????") entrys.

Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 15 ]

Fix for version 3.0 is available in development branch svn://svn.zabbix.com/branches/dev/ZBX-10540 revision 59441.

Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 15 ]

Patch attached. Fix is really simple. To test I had to restart MySQL server in order to cause automatic reconnects. takahisa, BcTpe4HbIu, your help in testing on more or less loaded system where reconnects are normal will be highly appreciated.

Comment by Igor Golubkov [ 2016 Apr 18 ]

Gleb, with this patch everything seems to be normal. No more question marks spotted.

Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 18 ]

Igor, very nice to hear that. Thank you for testing!

Comment by Sandis Neilands (Inactive) [ 2016 Apr 22 ]

Successfully tested.

Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 25 ]

Fixed in pre-3.0.3rc1 r59665, pre-3.1.0 (trunk) r59666.

Comment by Glebs Ivanovskis (Inactive) [ 2016 Apr 27 ]

(1) Documented in known issues.

sandis.neilands CLOSED with minor changes.

Generated at Wed Apr 24 14:05:07 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.