[ZBX-12659] Status of Zabbix Widget Not Working Created: 2017 Sep 01  Updated: 2024 Apr 10  Resolved: 2017 Oct 16

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Frontend (F), Installation (I), Server (S)
Affects Version/s: 3.4.0, 3.4.1
Fix Version/s: 3.4.3rc1, 4.0.0alpha1, 4.0 (plan)

Type: Problem report Priority: Trivial
Reporter: Jonathan W Assignee: Alexander Vladishev
Resolution: Fixed Votes: 2
Labels: frontend, regression, usability
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

CentOS 7. PHP 7.1. MariaDB 10.2.

Attachments: PNG File zabbix-status.png    
Issue Links:
Duplicate
is duplicated by ZBX-12734 Zabbix server is running "No" Closed
Team: Team B
Sprint: Sprint 16, Sprint 17, Sprint 18, Sprint 19
Story Points: 1

 Description   

No errors are reported, but the information in the status of zabbix widget is all missing and it says the zabbix server is down even though it's up and running great.

This started when 3.2 was updated to 3.4 via RPMs.

I've verified the MySQL connection info in /usr/share/zabbix/conf/zabbix.conf.php is correct.

 Comments   
Comment by Glebs Ivanovskis (Inactive) [ 2017 Sep 02 ]

Server is running: No means that frontend cannot connect to server's listening port. If you say that server is running, then you should check server host and port configuration in zabbix.conf.php and firewall rules as well. Probably StartTrappers parameter in server's configuration file too, if you are not using default one. For the rest of information in this screen frontend now relies on server running (to offload the database).

Closing as Won't Fix.

Comment by Jonathan W [ 2017 Sep 04 ]

I don't think this is a simple config issue.
The warning that is displayed at the bottom of the page if the server isn't running isn't there. It DOES appear if I manually alter the code to incorrect values for example in ''include/classes/core/CConfigFile.php''
The values in ''zabbix.conf.php'' appear to have 0 impact on anything - not even the displayed information in the widget gets updated, but again it does if I alter the values in the code.
Something is definitely wrong here and it's not a connectivity issue.

Comment by Jonathan W [ 2017 Sep 04 ]

The RPMs seem to hard code the config path to /etc/zabbix/web/zabbix.conf.php. This does indeed update the displayed connection data in the widget, but it still does not have any impact on the connection working.

The information is valid, there is no firewall blocking this local connectivity, and zabbix is listening on the port specified and confirmed with telnet.

If the Zabbix Server is ACTUALLY down then the warning at the bottom is displayed. That is not the case here, the server is up and running just fine.

Comment by Glebs Ivanovskis (Inactive) [ 2017 Sep 04 ]

What do you see in Reports -> Status of Zabbix? Can you see Administration -> Queue?

Comment by Jonathan W [ 2017 Sep 04 ]

Reports -> Status of Zabbix is identical to my screenshot.

Administration -> Queue is working properly and reporting expected data.

For whatever it's worth, the server is having no trouble keeping up with the load of data so it's not from not being able to keep up.

Comment by Glebs Ivanovskis (Inactive) [ 2017 Sep 04 ]

Hm, this is unexpected... Can you increase debug level for trappers (zabbix_server -R log_level_increase=trapper) and look for status.get?

Comment by Jonathan W [ 2017 Sep 04 ]

Yeah it's really odd. If it actually can't connect (ex. from me changing values to some that are wrong) I also get the warning at the bottom of the screen. When the values are correct I don't get that warning and everything works except this widget (and apparently Reports -> Status of Zabbix)

[root@int01-tx zabbix]# tail -f /var/log/zabbix/zabbix_server.log |grep status.get
  1090:20170904:114655.763 trapper got '{"request":"status.get","type":"ping","sid":"87adae16dafec8cb39454906e90f88f1"}'
  1069:20170904:114656.531 trapper got '{"request":"status.get","type":"ping","sid":"1eb31798f2d6ad9ca86d3099aa88d3d8"}'
  1091:20170904:114701.017 trapper got '{"request":"status.get","type":"ping","sid":"13996efc9939105036429b17d7040589"}'
  1107:20170904:114703.485 trapper got '{"request":"status.get","type":"ping","sid":"99897d01f78c6c0a5d6a11b133787d02"}'
  1075:20170904:114705.780 trapper got '{"request":"status.get","type":"ping","sid":"87adae16dafec8cb39454906e90f88f1"}'
  1088:20170904:114707.537 trapper got '{"request":"status.get","type":"ping","sid":"1eb31798f2d6ad9ca86d3099aa88d3d8"}'
  1062:20170904:114711.955 trapper got '{"request":"status.get","type":"ping","sid":"13996efc9939105036429b17d7040589"}'
Comment by Glebs Ivanovskis (Inactive) [ 2017 Sep 04 ]

Have you seen any "request":"status.get","type":"full"?

Comment by Jonathan W [ 2017 Sep 04 ]

Not a single one. Been watching the tail since I started it for the post above and they're all identical to the format in the sample above.

Comment by Jonathan W [ 2017 Sep 04 ]

Actually there were two in the past ~15 mins which I missed.

  1073:20170904:115422.633 trapper got '{"request":"status.get","type":"full","sid":"d57b8c4495f2b820675a7dec8e2737b7"}'
  1076:20170904:115639.197 trapper got '{"request":"status.get","type":"full","sid":"1eb31798f2d6ad9ca86d3099aa88d3d8"}'
Comment by Glebs Ivanovskis (Inactive) [ 2017 Sep 04 ]

Would be nice if you could capture a bit of traffic with tcpdump too.

Comment by Jonathan W [ 2017 Sep 04 ]

Anything in particular you're looking for? That's going to be a ridiculous amount of data as this box does about 1500 NVPS.

Comment by Glebs Ivanovskis (Inactive) [ 2017 Sep 04 ]

I'd like to see server's response to "request":"status.get","type":"full". Actually, you can send such request with sid using telnet.

Comment by Jonathan W [ 2017 Sep 04 ]

This contains 5 cases of that call.

[root@int01-tx zabbix]# tcpdump -nn -i lo |grep 10051
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
12:24:46.550939 IP 127.0.0.1.55734 > 127.0.0.1.10051: Flags [S], seq 1747329455, win 43690, options [mss 65495,sackOK,TS val 4167135117 ecr 0,nop,wscale 9], length 0
12:44:29.256050 IP 127.0.0.1.10051 > 127.0.0.1.55734: Flags [S.], seq 3252403672, ack 1747329456, win 43690, options [mss 65495,sackOK,TS val 4167135117 ecr 4167135117,nop,wscale 9], length 0
12:24:46.550963 IP 127.0.0.1.55734 > 127.0.0.1.10051: Flags [.], ack 1, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 0
12:24:46.551170 IP 127.0.0.1.55734 > 127.0.0.1.10051: Flags [P.], seq 1:80, ack 1, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 79
12:24:46.551179 IP 127.0.0.1.10051 > 127.0.0.1.55734: Flags [.], ack 80, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 0
12:24:46.551558 IP 127.0.0.1.10051 > 127.0.0.1.55734: Flags [P.], seq 1:33, ack 80, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 32
12:24:46.551566 IP 127.0.0.1.55734 > 127.0.0.1.10051: Flags [.], ack 33, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 0
12:24:46.551598 IP 127.0.0.1.10051 > 127.0.0.1.55734: Flags [F.], seq 33, ack 80, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 0
12:24:46.551619 IP 127.0.0.1.55734 > 127.0.0.1.10051: Flags [F.], seq 80, ack 34, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 0
12:24:46.551627 IP 127.0.0.1.10051 > 127.0.0.1.55734: Flags [.], ack 81, win 86, options [nop,nop,TS val 4167135117 ecr 4167135117], length 0
12:24:47.703536 IP 127.0.0.1.55920 > 127.0.0.1.10051: Flags [S], seq 104842301, win 43690, options [mss 65495,sackOK,TS val 4167136269 ecr 0,nop,wscale 9], length 0
14:06:57.059528 IP 127.0.0.1.10051 > 127.0.0.1.55920: Flags [S.], seq 469161959, ack 104842302, win 43690, options [mss 65495,sackOK,TS val 4167136269 ecr 4167136269,nop,wscale 9], length 0
12:24:47.703557 IP 127.0.0.1.55920 > 127.0.0.1.10051: Flags [.], ack 1, win 86, options [nop,nop,TS val 4167136269 ecr 4167136269], length 0
12:24:47.703780 IP 127.0.0.1.55920 > 127.0.0.1.10051: Flags [P.], seq 1:80, ack 1, win 86, options [nop,nop,TS val 4167136270 ecr 4167136269], length 79
12:24:47.703790 IP 127.0.0.1.10051 > 127.0.0.1.55920: Flags [.], ack 80, win 86, options [nop,nop,TS val 4167136270 ecr 4167136270], length 0
12:24:47.704120 IP 127.0.0.1.10051 > 127.0.0.1.55920: Flags [P.], seq 1:33, ack 80, win 86, options [nop,nop,TS val 4167136270 ecr 4167136270], length 32
12:24:47.704127 IP 127.0.0.1.55920 > 127.0.0.1.10051: Flags [.], ack 33, win 86, options [nop,nop,TS val 4167136270 ecr 4167136270], length 0
12:24:47.704160 IP 127.0.0.1.10051 > 127.0.0.1.55920: Flags [F.], seq 33, ack 80, win 86, options [nop,nop,TS val 4167136270 ecr 4167136270], length 0
12:24:47.704180 IP 127.0.0.1.55920 > 127.0.0.1.10051: Flags [F.], seq 80, ack 34, win 86, options [nop,nop,TS val 4167136270 ecr 4167136270], length 0
12:24:47.704188 IP 127.0.0.1.10051 > 127.0.0.1.55920: Flags [.], ack 81, win 86, options [nop,nop,TS val 4167136270 ecr 4167136270], length 0
12:24:48.423930 IP 127.0.0.1.55922 > 127.0.0.1.10051: Flags [S], seq 2532741573, win 43690, options [mss 65495,sackOK,TS val 4167136990 ecr 0,nop,wscale 9], length 0
14:58:33.731669 IP 127.0.0.1.10051 > 127.0.0.1.55922: Flags [S.], seq 2721256508, ack 2532741574, win 43690, options [mss 65495,sackOK,TS val 4167136990 ecr 4167136990,nop,wscale 9], length 0
12:24:48.423950 IP 127.0.0.1.55922 > 127.0.0.1.10051: Flags [.], ack 1, win 86, options [nop,nop,TS val 4167136990 ecr 4167136990], length 0
12:24:48.424142 IP 127.0.0.1.55922 > 127.0.0.1.10051: Flags [P.], seq 1:80, ack 1, win 86, options [nop,nop,TS val 4167136990 ecr 4167136990], length 79
12:24:48.424149 IP 127.0.0.1.10051 > 127.0.0.1.55922: Flags [.], ack 80, win 86, options [nop,nop,TS val 4167136990 ecr 4167136990], length 0
12:24:48.484984 IP 127.0.0.1.10051 > 127.0.0.1.55922: Flags [P.], seq 1:33, ack 80, win 86, options [nop,nop,TS val 4167137051 ecr 4167136990], length 32
12:24:48.484992 IP 127.0.0.1.55922 > 127.0.0.1.10051: Flags [.], ack 33, win 86, options [nop,nop,TS val 4167137051 ecr 4167137051], length 0
12:24:48.485092 IP 127.0.0.1.10051 > 127.0.0.1.55922: Flags [F.], seq 33, ack 80, win 86, options [nop,nop,TS val 4167137051 ecr 4167137051], length 0
12:24:48.485122 IP 127.0.0.1.55922 > 127.0.0.1.10051: Flags [F.], seq 80, ack 34, win 86, options [nop,nop,TS val 4167137051 ecr 4167137051], length 0
12:24:48.485132 IP 127.0.0.1.10051 > 127.0.0.1.55922: Flags [.], ack 81, win 86, options [nop,nop,TS val 4167137051 ecr 4167137051], length 0
12:24:48.886521 IP 127.0.0.1.56126 > 127.0.0.1.10051: Flags [S], seq 1642374243, win 43690, options [mss 65495,sackOK,TS val 4167137452 ecr 0,nop,wscale 9], length 0
15:31:38.007022 IP 127.0.0.1.10051 > 127.0.0.1.56126: Flags [S.], seq 484765951, ack 1642374244, win 43690, options [mss 65495,sackOK,TS val 4167137452 ecr 4167137452,nop,wscale 9], length 0
12:24:48.886550 IP 127.0.0.1.56126 > 127.0.0.1.10051: Flags [.], ack 1, win 86, options [nop,nop,TS val 4167137452 ecr 4167137452], length 0
12:24:48.886840 IP 127.0.0.1.56126 > 127.0.0.1.10051: Flags [P.], seq 1:80, ack 1, win 86, options [nop,nop,TS val 4167137453 ecr 4167137452], length 79
12:24:48.886850 IP 127.0.0.1.10051 > 127.0.0.1.56126: Flags [.], ack 80, win 86, options [nop,nop,TS val 4167137453 ecr 4167137453], length 0
12:24:48.887174 IP 127.0.0.1.10051 > 127.0.0.1.56126: Flags [P.], seq 1:33, ack 80, win 86, options [nop,nop,TS val 4167137453 ecr 4167137453], length 32
12:24:48.887181 IP 127.0.0.1.56126 > 127.0.0.1.10051: Flags [.], ack 33, win 86, options [nop,nop,TS val 4167137453 ecr 4167137453], length 0
12:24:48.887213 IP 127.0.0.1.10051 > 127.0.0.1.56126: Flags [F.], seq 33, ack 80, win 86, options [nop,nop,TS val 4167137453 ecr 4167137453], length 0
12:24:48.887234 IP 127.0.0.1.56126 > 127.0.0.1.10051: Flags [F.], seq 80, ack 34, win 86, options [nop,nop,TS val 4167137453 ecr 4167137453], length 0
12:24:48.887243 IP 127.0.0.1.10051 > 127.0.0.1.56126: Flags [.], ack 81, win 86, options [nop,nop,TS val 4167137453 ecr 4167137453], length 0
12:24:49.007026 IP 127.0.0.1.56128 > 127.0.0.1.10051: Flags [S], seq 3768803223, win 43690, options [mss 65495,sackOK,TS val 4167137573 ecr 0,nop,wscale 9], length 0
15:40:17.698185 IP 127.0.0.1.10051 > 127.0.0.1.56128: Flags [S.], seq 1749689905, ack 3768803224, win 43690, options [mss 65495,sackOK,TS val 4167137573 ecr 4167137573,nop,wscale 9], length 0
12:24:49.007056 IP 127.0.0.1.56128 > 127.0.0.1.10051: Flags [.], ack 1, win 86, options [nop,nop,TS val 4167137573 ecr 4167137573], length 0
12:24:49.007302 IP 127.0.0.1.56128 > 127.0.0.1.10051: Flags [P.], seq 1:80, ack 1, win 86, options [nop,nop,TS val 4167137573 ecr 4167137573], length 79
12:24:49.007313 IP 127.0.0.1.10051 > 127.0.0.1.56128: Flags [.], ack 80, win 86, options [nop,nop,TS val 4167137573 ecr 4167137573], length 0
12:24:49.007876 IP 127.0.0.1.10051 > 127.0.0.1.56128: Flags [P.], seq 1:33, ack 80, win 86, options [nop,nop,TS val 4167137574 ecr 4167137573], length 32
12:24:49.007885 IP 127.0.0.1.56128 > 127.0.0.1.10051: Flags [.], ack 33, win 86, options [nop,nop,TS val 4167137574 ecr 4167137574], length 0
12:24:49.007950 IP 127.0.0.1.10051 > 127.0.0.1.56128: Flags [F.], seq 33, ack 80, win 86, options [nop,nop,TS val 4167137574 ecr 4167137574], length 0
12:24:49.007973 IP 127.0.0.1.56128 > 127.0.0.1.10051: Flags [F.], seq 80, ack 34, win 86, options [nop,nop,TS val 4167137574 ecr 4167137574], length 0
12:24:49.007981 IP 127.0.0.1.10051 > 127.0.0.1.56128: Flags [.], ack 81, win 86, options [nop,nop,TS val 4167137574 ecr 4167137574], length 0
Comment by zhxiaom5 [ 2017 Sep 05 ]

I meet this too,my env is:
PHP 7.1.8 (cli) (built: Aug 22 2017 19:21:22) ( ZTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.1.0, Copyright (c) 1998-2017 Zend Technologies
with Zend OPcache v7.1.8, Copyright (c) 1999-2017, by Zend Technologies

zabbix: 3.4.1

Comment by zhxiaom5 [ 2017 Sep 05 ]

exactly the problem Jonathan W describle above

Comment by Patrik Chadima [ 2017 Sep 10 ]

Same issue like described above.

PHP Version: PHP 5.4.16 (cli) (built: Nov 6 2016 00:29:02)
MariDB Version: Ver 15.1 Distrib 5.5.52-MariaDB, for Linux (x86_64) using readline 5.1
CentOS Version: CentOS Linux release 7.3.1611 (Core)

Comment by zhxiaom5 [ 2017 Sep 11 ]

I have solved this, check you database,the session table "sessions"
I have lots of records in this table ,large than 50 bilion,the slow query is below

slow query: 20.251609 sec, "select max(lastaccess) from sessions where status=0 group by userid,status"

after I truncate session table,the status widget works well.

so,I guess maybe you should check you mysql slow query.

Comment by zhxiaom5 [ 2017 Sep 11 ]

or there is something messup with you table

Comment by Jonathan W [ 2017 Sep 11 ]

I can confirm this resolved my issue and the widget works.

MariaDB [zabbix]> select count(*) from sessions;
+----------+
| count(*) |
+----------+
|  7482801 |
+----------+
1 row in set (1.94 sec)

MariaDB [zabbix]> truncate table sessions;
Query OK, 0 rows affected (2.81 sec)
Comment by Alexander Vladishev [ 2017 Sep 11 ]

zhxiaom5, thank you! I confirm this issue with big sessions table.

32141:20170911:222000.900 slow query: 8.462081 sec, "select max(lastaccess) from sessions where status=0 group by userid,status"
Comment by Alexander Vladishev [ 2017 Oct 02 ]

Fixed in dev branch svn://svn.zabbix.com/branches/dev/ZBX-12659

Comment by Alexander Vladishev [ 2017 Oct 03 ]

(2) [D] Documentation needs to be updated

  • API: user.login method. Information should be added about need to do user.logout for prevention of generation of a large number the opened sessions.
  • Known issues: Large number the opened sessions can be created when using custom scripts with user.login method without following user.logout.

martins-v RESOLVED in:

Please review. Also, do we need to add this info for pre-3.4 versions as well?

sasha Looks good to me. Please add it to the all supported versions as well. Thanks!

martins-v Added to 2.2, 3.0, 3.2. RESOLVED

sasha Thanks! CLOSED

Comment by Miks Kronkalns [ 2017 Oct 05 ]

Frontend reviewed.

Comment by Viktors Tjarve [ 2017 Oct 06 ]

Server side successfully tested.

Comment by Alexander Vladishev [ 2017 Oct 11 ]

Fixed in 3.4.3rc1 r73431 and 4.0.0alpha1 (trunk) r73433

Generated at Fri Apr 04 13:28:59 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.