ZABBIX BUGS AND ISSUES
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-10540

Japanese characters are changed to ?????? in "Trigger Description" have been created by the discovery

    Details

    • Type: Incident report Incident report
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.1
    • Fix Version/s: 3.0.3rc1, 3.2.0alpha1
    • Component/s: Server (S)
    • Labels:
      None
    • Environment:
      CentOS Linux release 7.2.1511 (Core)
      MySQL 5.6.27

      Description

      Japanese characters are changed to ?????? in "Trigger Description" that have been created by the discovery. (fig1.png)

      In MySQL, Japanese characters are changed to ??????. ("trigger_table_mysql.txt" and "fig2.png RED BOX")

      If edit template item , The "??????" are changed to normal (linked change. and fig1.png BLUE BOX)

      If delete the item and re-discovery, Reproduce any number of times in all of the items.

      If restart the zabbix-server (systemctl restart zabbix-server) , then operating normally.

      But It relapse after a few weeks.

      It looks like ZBX-9307

      1. trigger_table_mysql.txt
        8 kB
        takahisa
      2. ZBX-10540.patch
        1 kB
        Glebs Ivanovskis
      1. fig1.png
        36 kB
      2. fig2.png
        75 kB

        Issue Links

          Activity

          Hide
          Glebs Ivanovskis added a comment - - edited

          Did you use the correct command to create MySQL database as documented here?

          mysql> create database zabbix character set utf8 collate utf8_bin;
          

          Or maybe you have correct character set for database, but the default character set for connections is not utf8. Frontend does SET NAMES utf8; explicitly, but sever does not. So, if MySQL assumes Zabbix server connects in latin1 it will send back '?' instead of non-ASCII characters in results. Server simply takes these '???' from template and copies to discovered hosts.

          I failed to reproduce your issue by messing around with /etc/mysql/my.cnf. Could you provide more details?

          Show
          Glebs Ivanovskis added a comment - - edited Did you use the correct command to create MySQL database as documented here ? mysql> create database zabbix character set utf8 collate utf8_bin; Or maybe you have correct character set for database, but the default character set for connections is not utf8. Frontend does SET NAMES utf8; explicitly, but sever does not. So, if MySQL assumes Zabbix server connects in latin1 it will send back '?' instead of non-ASCII characters in results. Server simply takes these '???' from template and copies to discovered hosts. I failed to reproduce your issue by messing around with /etc/mysql/my.cnf. Could you provide more details?
          Hide
          Glebs Ivanovskis added a comment -

          Managed to reproduce. I believe the decisive moment was when I added this line to /etc/mysql/my.cnf:

          [mysqld]
          ...
          init-connect='SET character_set_results = latin1'
          
          Show
          Glebs Ivanovskis added a comment - Managed to reproduce. I believe the decisive moment was when I added this line to /etc/mysql/my.cnf: [mysqld] ... init-connect='SET character_set_results = latin1'
          Hide
          Glebs Ivanovskis added a comment -

          Hm, server does set names utf8 too...

          Show
          Glebs Ivanovskis added a comment - Hm, server does set names utf8 too...
          Hide
          richlv added a comment -

          ZBX-10665 could be the same

          Show
          richlv added a comment - ZBX-10665 could be the same
          Hide
          takahisa added a comment -

          ADD INFO.

          After restart the zabbix-server (systemctl restart zabbix-server) , LLD created normal entrys.
          About 1 days later(maybe) from restart, LLD created normal entrys.
          About 2 days later from restart, LLD created illegal("?????") entrys.

          I think MySQL setting is normal.
          But I tryed.

          1st settings
          character-set-server = utf8
          skip-character-set-client-handshake

          2nd settings
          character-set-server = utf8
          skip-character-set-client-handshake
          collation-server=utf8_bin

          Still relapsing....

          Show
          takahisa added a comment - ADD INFO. After restart the zabbix-server (systemctl restart zabbix-server) , LLD created normal entrys. About 1 days later(maybe) from restart, LLD created normal entrys. About 2 days later from restart, LLD created illegal("?????") entrys. I think MySQL setting is normal. But I tryed. 1st settings character-set-server = utf8 skip-character-set-client-handshake 2nd settings character-set-server = utf8 skip-character-set-client-handshake collation-server=utf8_bin Still relapsing....
          Hide
          Glebs Ivanovskis added a comment -

          Now it's clear to me. ZBX-6163 enabled auto-reconnection to MySQL, but result of set names utf8; is lost on reconnection. That's why this bug is so random. Whenever Zabbix process loses connection to database MySQL automatically reconnects and falls back to default encoding.

          Simple test with mysql client:

          mysql> select * from test_utf8;    <--- default settings
          +------+------+
          | id   | data |
          +------+------+
          |    1 | ???? |
          |    2 | ???? |
          +------+------+
          2 rows in set (0.00 sec)
          
          mysql> set names utf8;    <--- correct settings
          Query OK, 0 rows affected (0.00 sec)
          
          mysql> select * from test_utf8;
          +------+----------+
          | id   | data     |
          +------+----------+
          |    1 | фыва |
          |    2 | олдж |
          +------+----------+
          2 rows in set (0.00 sec)
          
          mysql> select * from test_utf8;
          ERROR 2006 (HY000): MySQL server has gone away    <--- restarted MySQL server in the meantime
          No connection. Trying to reconnect...
          Connection id:    37
          Current database: test_utf8
          
          +------+------+
          | id   | data |    <--- default settings again
          +------+------+
          |    1 | ???? |
          |    2 | ???? |
          +------+------+
          2 rows in set (0.00 sec)
          
          Show
          Glebs Ivanovskis added a comment - Now it's clear to me. ZBX-6163 enabled auto-reconnection to MySQL, but result of set names utf8; is lost on reconnection. That's why this bug is so random. Whenever Zabbix process loses connection to database MySQL automatically reconnects and falls back to default encoding. Simple test with mysql client: mysql> select * from test_utf8; <--- default settings +------+------+ | id | data | +------+------+ | 1 | ???? | | 2 | ???? | +------+------+ 2 rows in set (0.00 sec) mysql> set names utf8; <--- correct settings Query OK, 0 rows affected (0.00 sec) mysql> select * from test_utf8; +------+----------+ | id | data | +------+----------+ | 1 | фыва | | 2 | олдж | +------+----------+ 2 rows in set (0.00 sec) mysql> select * from test_utf8; ERROR 2006 (HY000): MySQL server has gone away <--- restarted MySQL server in the meantime No connection. Trying to reconnect... Connection id: 37 Current database: test_utf8 +------+------+ | id | data | <--- default settings again +------+------+ | 1 | ???? | | 2 | ???? | +------+------+ 2 rows in set (0.00 sec)
          Hide
          Glebs Ivanovskis added a comment -

          As a temporary workaround try to set character_set_connection, character_set_result and character_set_client to utf8 in MySQL server settings ("[client]" section).

          Show
          Glebs Ivanovskis added a comment - As a temporary workaround try to set character_set_connection , character_set_result and character_set_client to utf8 in MySQL server settings ("[client]" section).
          Hide
          takahisa added a comment -

          When the workaround has not applied,

          1.
          Connect mysql

          2.
          mysql> select * from triggers;
          <<<RETURN IS OK>>>

          3.
          <<<reboot mysql server>>>

          4.
          mysql> set names latin1;
          mysql> select * from triggers;
          <<<RETURN IS NG "??????">>>

          5.
          mysql> set names utf8;
          mysql> select * from triggers;
          <<<RETURN IS OK>>>

          I think "skip-character-set-client-handshake" is working.

          But now the workaround has applied.
          And Let's wait and see.

          Show
          takahisa added a comment - When the workaround has not applied, 1. Connect mysql 2. mysql> select * from triggers; <<<RETURN IS OK>>> 3. <<<reboot mysql server>>> 4. mysql> set names latin1; mysql> select * from triggers; <<<RETURN IS NG "??????">>> 5. mysql> set names utf8; mysql> select * from triggers; <<<RETURN IS OK>>> I think "skip-character-set-client-handshake" is working. But now the workaround has applied. And Let's wait and see.
          Hide
          takahisa added a comment -

          I think the workaround does not working

          1. Set the workaround ( set the my.cnf on the Zabbix Server = mysql client )
          2. Exec "systemctl restart zabbix-server"
          3. Reboot mysql server <----- maybe disconnect mysql connection?
          4. LLD created illegal("?????") entrys.

          Show
          takahisa added a comment - I think the workaround does not working 1. Set the workaround ( set the my.cnf on the Zabbix Server = mysql client ) 2. Exec "systemctl restart zabbix-server" 3. Reboot mysql server <----- maybe disconnect mysql connection? 4. LLD created illegal("?????") entrys.
          Hide
          Glebs Ivanovskis added a comment -

          Fix for version 3.0 is available in development branch svn://svn.zabbix.com/branches/dev/ZBX-10540 revision 59441.

          Show
          Glebs Ivanovskis added a comment - Fix for version 3.0 is available in development branch svn://svn.zabbix.com/branches/dev/ZBX-10540 revision 59441.
          Hide
          Glebs Ivanovskis added a comment -

          Patch attached. Fix is really simple. To test I had to restart MySQL server in order to cause automatic reconnects. takahisa, Igor Golubkov, your help in testing on more or less loaded system where reconnects are normal will be highly appreciated.

          Show
          Glebs Ivanovskis added a comment - Patch attached. Fix is really simple. To test I had to restart MySQL server in order to cause automatic reconnects. takahisa , Igor Golubkov , your help in testing on more or less loaded system where reconnects are normal will be highly appreciated.
          Hide
          Igor Golubkov added a comment -

          Gleb, with this patch everything seems to be normal. No more question marks spotted.

          Show
          Igor Golubkov added a comment - Gleb, with this patch everything seems to be normal. No more question marks spotted.
          Hide
          Glebs Ivanovskis added a comment -

          Igor, very nice to hear that. Thank you for testing!

          Show
          Glebs Ivanovskis added a comment - Igor, very nice to hear that. Thank you for testing!
          Hide
          Sandis Neilands (Inactive) added a comment -

          Successfully tested.

          Show
          Sandis Neilands (Inactive) added a comment - Successfully tested.
          Hide
          Glebs Ivanovskis added a comment -

          Fixed in pre-3.0.3rc1 r59665, pre-3.1.0 (trunk) r59666.

          Show
          Glebs Ivanovskis added a comment - Fixed in pre-3.0.3rc1 r59665, pre-3.1.0 (trunk) r59666.
          Hide
          Glebs Ivanovskis added a comment - - edited

          (1) Documented in known issues.

          Sandis Neilands CLOSED with minor changes.

          Show
          Glebs Ivanovskis added a comment - - edited (1) Documented in known issues . Sandis Neilands CLOSED with minor changes.

            People

            • Assignee:
              Unassigned
              Reporter:
              takahisa
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: