when zabbix server is working: # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 393216 2019448141 --rw------- zabbix zabbix zabbix zabbix 28 16777479 51075 51075 15:43:50 no-entry 15:43:50 m 393217 1952339277 --rw------- zabbix zabbix zabbix zabbix 28 4194554 51075 51075 15:43:50 no-entry 15:43:50 m 393218 1734235469 --rw------- zabbix zabbix zabbix zabbix 28 7130317 51075 51075 15:43:50 no-entry 15:43:50 m 393219 1935562061 --rw------- zabbix zabbix zabbix zabbix 28 1258291 51075 51075 15:43:50 no-entry 15:43:50 m 393220 1398691149 --rw------- zabbix zabbix zabbix zabbix 28 6900 51075 51075 15:43:50 no-entry 15:43:50 m 393222 1751012685 --rw------- zabbix zabbix zabbix zabbix 28 8421355 51075 51075 15:43:50 no-entry 15:43:50 m 655367 1834898279 --rw------- root wheel root wheel 0 47040 50758 0 no-entry no-entry 15:42:48 dev branch agent started: # ./zabbix_agentd -c /usr/local/etc/zabbix_agentd.conf # ps aux | grep zabbix zabbix 47513 0,0 0,1 7124 2464 ?? S 11:38 0:00,00 zabbix_agentd: main process (zabbix_agentd) zabbix 47514 0,0 0,1 7124 2564 ?? S 11:38 0:00,00 zabbix_agentd: collector [sleeping for 1 seconds] (zabbix_agentd) zabbix 47515 0,0 0,1 7124 2464 ?? S 11:38 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 47516 0,0 0,1 7124 2464 ?? S 11:38 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 47517 0,0 0,1 7124 2464 ?? S 11:38 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 51075 0,0 0,2 52932 4984 ?? I пт15 0:00,02 zabbix_server: main process (zabbix_server) zabbix 51076 0,0 0,3 52932 5564 ?? S пт15 0:41,30 zabbix_server: configuration syncer [sleeping for 25 seconds] (zabbix_server) zabbix 51077 0,0 0,2 52932 4988 ?? S пт15 0:08,93 zabbix_server: db watchdog [sleeping for 45 seconds] (zabbix_server) zabbix 51078 0,0 1,1 55108 23116 ?? S пт15 0:57,09 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 51079 0,0 1,1 55108 23104 ?? S пт15 0:57,32 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 51080 0,0 1,1 55108 23116 ?? S пт15 0:56,58 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 51081 0,0 1,1 55108 23112 ?? S пт15 0:56,64 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 51082 0,0 1,1 55108 23136 ?? S пт15 0:56,96 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 51083 0,0 0,4 55108 9240 ?? S пт15 0:13,53 zabbix_server: unreachable poller [sleeping for 4 seconds] (zabbix_server) zabbix 51084 0,0 0,8 53060 16392 ?? S пт15 0:03,80 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 51085 0,0 0,8 52932 17176 ?? S пт15 0:04,79 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 51086 0,0 0,8 53060 16720 ?? S пт15 0:03,85 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 51087 0,0 0,8 52932 17312 ?? S пт15 0:04,65 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 51088 0,0 0,8 53060 16328 ?? S пт15 0:03,82 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 51089 0,0 0,7 53316 13524 ?? S пт15 0:25,80 zabbix_server: icmp pinger [sleeping for 3 seconds] (zabbix_server) zabbix 51090 0,0 0,2 52932 5032 ?? S пт15 0:15,09 zabbix_server: alerter [sleeping for 1 seconds] (zabbix_server) zabbix 51091 0,0 0,2 52932 5032 ?? S пт15 0:18,65 zabbix_server: housekeeper [sleeping for 1216 seconds] (zabbix_server) zabbix 51092 0,0 0,3 52932 5272 ?? S пт15 0:12,78 zabbix_server: timer [sleeping for 19 seconds] (zabbix_server) zabbix 51093 0,0 0,2 52932 4988 ?? S пт15 0:17,63 zabbix_server: http poller [sleeping for 5 seconds] (zabbix_server) zabbix 51094 0,0 0,4 54980 7984 ?? S пт15 0:08,33 zabbix_server: discoverer [sleeping for 45 seconds] (zabbix_server) zabbix 51095 0,0 1,0 53060 20028 ?? S пт15 0:26,27 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 51096 0,0 1,0 53060 20212 ?? S пт15 0:26,09 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 51097 0,0 1,0 53060 20112 ?? S пт15 0:26,19 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 51098 0,0 1,0 53060 20188 ?? S пт15 0:26,07 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 51099 0,0 0,3 52932 5204 ?? S пт15 0:24,56 zabbix_server: escalator [sleeping for 1 seconds] (zabbix_server) zabbix 51100 0,0 0,2 52932 4992 ?? S пт15 0:09,73 zabbix_server: ipmi poller [sleeping for 4 seconds] (zabbix_server) zabbix 51101 0,0 0,2 52932 4988 ?? S пт15 0:09,88 zabbix_server: proxy poller [sleeping for 3 seconds] (zabbix_server) zabbix 51102 0,0 0,2 52932 4984 ?? S пт15 0:20,65 zabbix_server: self-monitoring [sleeping for 1 seconds] (zabbix_server) root 47519 0,0 0,1 3496 1260 1 S+ 11:38 0:00,00 grep zabbix ... # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 393216 2019448141 --rw------- zabbix zabbix zabbix zabbix 28 16777479 51075 51075 15:43:50 no-entry 15:43:50 m 393217 1952339277 --rw------- zabbix zabbix zabbix zabbix 28 4194554 51075 51075 15:43:50 no-entry 15:43:50 m 393218 1734235469 --rw------- zabbix zabbix zabbix zabbix 28 7130317 51075 51075 15:43:50 no-entry 15:43:50 m 393219 1935562061 --rw------- zabbix zabbix zabbix zabbix 28 1258291 51075 51075 15:43:50 no-entry 15:43:50 m 393220 1398691149 --rw------- zabbix zabbix zabbix zabbix 28 6900 51075 51075 15:43:50 no-entry 15:43:50 m 983045 1818121063 --rw------- zabbix zabbix zabbix zabbix 5 175752 47513 47513 11:38:09 11:39:00 11:38:09 m 393222 1751012685 --rw------- zabbix zabbix zabbix zabbix 28 8421355 51075 51075 15:43:50 no-entry 15:43:50 m 655367 1834898279 --rw------- root wheel root wheel 0 47040 50758 0 no-entry no-entry 15:42:48 request and agent crash: # zabbix_get -s localhost -k vfs.dev.write[ad0,ops] 47515:20121011:113942.153 Processing request. 47515:20121011:113942.253 Requested [vfs.dev.write[ad0,ops]] 47515:20121011:113942.253 In collector_diskdevice_get() devname:'ad0' 47515:20121011:113942.253 zbx_shmget() removing existing shm_id:655367 zabbix_agentd [47515]: cannot remove existing shared memory: [13] Permission denied 47515:20121011:113942.253 cannot allocate shared memory for disk statistics collector 47513:20121011:113942.254 One child process died (PID:47515,exitcode/signal:256). Exiting ... 47513:20121011:113942.254 zbx_on_exit() called 47516:20121011:113942.254 Got signal [signal:15(SIGTERM),sender_pid:47513,sender_uid:122,reason:65537]. Exiting ... 47514:20121011:113942.254 Got signal [signal:15(SIGTERM),sender_pid:47513,sender_uid:122,reason:65537]. Exiting ... 47517:20121011:113942.255 Got signal [signal:15(SIGTERM),sender_pid:47513,sender_uid:122,reason:65537]. Exiting ... 47513:20121011:113944.256 Zabbix Agent stopped. Zabbix 2.0.4rc1 (revision {ZABBIX_REVISION}). stat after crash: # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 393216 2019448141 --rw------- zabbix zabbix zabbix zabbix 28 16777479 51075 51075 15:43:50 no-entry 15:43:50 m 393217 1952339277 --rw------- zabbix zabbix zabbix zabbix 28 4194554 51075 51075 15:43:50 no-entry 15:43:50 m 393218 1734235469 --rw------- zabbix zabbix zabbix zabbix 28 7130317 51075 51075 15:43:50 no-entry 15:43:50 m 393219 1935562061 --rw------- zabbix zabbix zabbix zabbix 28 1258291 51075 51075 15:43:50 no-entry 15:43:50 m 393220 1398691149 --rw------- zabbix zabbix zabbix zabbix 28 6900 51075 51075 15:43:50 no-entry 15:43:50 m 393222 1751012685 --rw------- zabbix zabbix zabbix zabbix 28 8421355 51075 51075 15:43:50 no-entry 15:43:50 m 655367 1834898279 --rw------- root wheel root wheel 0 47040 50758 0 no-entry no-entry 15:42:48 copy config file: cp /usr/local/etc/zabbix_agentd.conf /usr/local/etc/zabbix_agentd_test.conf start with changed cnfig: # ./zabbix_agentd -c /usr/local/etc/zabbix_agentd_test.conf its processes: # ps aux | grep zabbix zabbix 47594 0,0 0,1 7124 2464 ?? S 11:43 0:00,00 zabbix_agentd: main process (zabbix_agentd) zabbix 47595 0,0 0,1 7124 2564 ?? S 11:43 0:00,00 zabbix_agentd: collector [sleeping for 1 seconds] (zabbix_agentd) zabbix 47596 0,0 0,1 7124 2464 ?? S 11:43 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 47597 0,0 0,1 7124 2464 ?? S 11:43 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 47598 0,0 0,1 7124 2464 ?? S 11:43 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) memory: # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 393216 2019448141 --rw------- zabbix zabbix zabbix zabbix 28 16777479 51075 51075 15:43:50 no-entry 15:43:50 m 393217 1952339277 --rw------- zabbix zabbix zabbix zabbix 28 4194554 51075 51075 15:43:50 no-entry 15:43:50 m 393218 1734235469 --rw------- zabbix zabbix zabbix zabbix 28 7130317 51075 51075 15:43:50 no-entry 15:43:50 m 393219 1935562061 --rw------- zabbix zabbix zabbix zabbix 28 1258291 51075 51075 15:43:50 no-entry 15:43:50 m 393220 1398691149 --rw------- zabbix zabbix zabbix zabbix 28 6900 51075 51075 15:43:50 no-entry 15:43:50 m 1048581 1818121061 --rw------- zabbix zabbix zabbix zabbix 5 175752 47594 47594 11:43:49 11:45:19 11:43:49 m 393222 1751012685 --rw------- zabbix zabbix zabbix zabbix 28 8421355 51075 51075 15:43:50 no-entry 15:43:50 m 655367 1834898279 --rw------- root wheel root wheel 0 47040 50758 0 no-entry no-entry 15:42:48 no crashes: # zabbix_get -s localhost -k vfs.dev.write[ad0,ops] 0.000000 # zabbix_get -s localhost -k vfs.dev.write[ad0,ops] 1.500000 related debuglog: 47598:20121011:114553.170 Processing request. 47598:20121011:114553.269 Requested [vfs.dev.write[ad0,ops]] 47598:20121011:114553.269 In collector_diskdevice_get() devname:'ad0' 47598:20121011:114553.269 diskstat_shm_init() allocated initial shm segment id:65544 for disk statistics collector 47598:20121011:114553.270 End of collector_diskdevice_get():0x0 47598:20121011:114553.270 In collector_diskdevice_add() devname:'ad0' 47598:20121011:114553.270 End of collector_diskdevice_add():0x2858c008 47598:20121011:114553.270 Sending back [0.000000] ... 47597:20121011:114557.978 Processing request. 47597:20121011:114558.077 Requested [vfs.dev.write[ad0,ops]] 47597:20121011:114558.077 In collector_diskdevice_get() devname:'ad0' 47597:20121011:114558.077 diskstat_shm_reattach() switched shm id from -1 to 65544 47597:20121011:114558.077 collector_diskdevice_get() device 'ad0' found 47597:20121011:114558.078 End of collector_diskdevice_get():0x2858c008 47597:20121011:114558.078 Sending back [1.500000] stat after request: # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 393216 2019448141 --rw------- zabbix zabbix zabbix zabbix 28 16777479 51075 51075 15:43:50 no-entry 15:43:50 m 393217 1952339277 --rw------- zabbix zabbix zabbix zabbix 28 4194554 51075 51075 15:43:50 no-entry 15:43:50 m 393218 1734235469 --rw------- zabbix zabbix zabbix zabbix 28 7130317 51075 51075 15:43:50 no-entry 15:43:50 m 393219 1935562061 --rw------- zabbix zabbix zabbix zabbix 28 1258291 51075 51075 15:43:50 no-entry 15:43:50 m 393220 1398691149 --rw------- zabbix zabbix zabbix zabbix 28 6900 51075 51075 15:43:50 no-entry 15:43:50 m 1048581 1818121061 --rw------- zabbix zabbix zabbix zabbix 5 175752 47594 47594 11:43:49 11:53:00 11:43:49 m 393222 1751012685 --rw------- zabbix zabbix zabbix zabbix 28 8421355 51075 51075 15:43:50 no-entry 15:43:50 m 655367 1834898279 --rw------- root wheel root wheel 0 47040 50758 0 no-entry no-entry 15:42:48 m 65544 1834898277 --rw------- zabbix zabbix zabbix zabbix 3 47040 47598 47597 11:45:58 11:53:00 11:45:53 if start to use the old path - the crash appeared again. Let's reboot this machine. after machine reboot: # ps aux | grep zabbix zabbix 1239 0,0 0,3 52932 6672 ?? I 12:12 0:00,02 zabbix_server: main process (zabbix_server) zabbix 1261 0,0 0,1 7124 2988 ?? I 12:12 0:00,00 zabbix_agentd: main process (zabbix_agentd) zabbix 1286 0,0 0,2 7124 3132 ?? S 12:12 0:00,01 zabbix_agentd: collector [sleeping for 1 seconds] (zabbix_agentd) zabbix 1287 0,0 0,1 7124 3088 ?? S 12:12 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 1288 0,0 0,2 7124 3160 ?? S 12:12 0:00,01 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 1289 0,0 0,1 7124 3020 ?? S 12:12 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 1332 0,0 0,3 52932 7140 ?? S 12:12 0:00,02 zabbix_server: configuration syncer [sleeping for 49 seconds] (zabbix_server) zabbix 1333 0,0 0,3 52932 6696 ?? S 12:12 0:00,00 zabbix_server: db watchdog [sleeping for 48 seconds] (zabbix_server) zabbix 1334 0,0 0,5 55108 10088 ?? S 12:12 0:00,11 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1335 0,0 0,5 55108 10088 ?? S 12:12 0:00,11 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1336 0,0 0,5 55108 10088 ?? S 12:12 0:00,11 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1337 0,0 0,5 55108 10084 ?? S 12:12 0:00,11 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1338 0,0 0,5 55108 10080 ?? S 12:12 0:00,11 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1339 0,0 0,5 55108 10008 ?? S 12:12 0:00,10 zabbix_server: unreachable poller [sleeping for 5 seconds] (zabbix_server) zabbix 1340 0,0 0,3 52932 6724 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1341 0,0 0,3 52932 6724 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1342 0,0 0,3 52932 7048 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1343 0,0 0,3 52932 7000 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1344 0,0 0,3 52932 6928 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1345 0,0 0,3 53316 6944 ?? S 12:12 0:00,00 zabbix_server: icmp pinger [sleeping for 3 seconds] (zabbix_server) zabbix 1346 0,0 0,3 52932 6760 ?? S 12:12 0:00,00 zabbix_server: alerter [sleeping for 5 seconds] (zabbix_server) zabbix 1347 0,0 0,3 52932 6708 ?? S 12:12 0:00,08 zabbix_server: housekeeper [sleeping for 3541 seconds] (zabbix_server) zabbix 1348 0,0 0,3 52932 7084 ?? S 12:12 0:00,00 zabbix_server: timer [sleeping for 20 seconds] (zabbix_server) zabbix 1349 0,0 0,3 52932 6700 ?? S 12:12 0:00,00 zabbix_server: http poller [sleeping for 3 seconds] (zabbix_server) zabbix 1350 0,0 0,5 54980 9736 ?? S 12:12 0:00,10 zabbix_server: discoverer [sleeping for 52 seconds] (zabbix_server) zabbix 1351 0,0 0,3 53060 7144 ?? S 12:12 0:00,01 zabbix_server: history syncer [sleeping for 2 seconds] (zabbix_server) zabbix 1352 0,0 0,3 53060 7168 ?? S 12:12 0:00,01 zabbix_server: history syncer [sleeping for 2 seconds] (zabbix_server) zabbix 1353 0,0 0,3 53060 7096 ?? S 12:12 0:00,00 zabbix_server: history syncer [sleeping for 2 seconds] (zabbix_server) zabbix 1354 0,0 0,3 53060 7096 ?? S 12:12 0:00,00 zabbix_server: history syncer [sleeping for 2 seconds] (zabbix_server) zabbix 1355 0,0 0,3 52932 6972 ?? S 12:12 0:00,01 zabbix_server: escalator [sleeping for 2 seconds] (zabbix_server) zabbix 1356 0,0 0,3 52932 6688 ?? S 12:12 0:00,00 zabbix_server: ipmi poller [sleeping for 2 seconds] (zabbix_server) zabbix 1357 0,0 0,3 52932 6684 ?? S 12:12 0:00,00 zabbix_server: proxy poller [sleeping for 2 seconds] (zabbix_server) zabbix 1358 0,0 0,3 52932 6680 ?? S 12:12 0:00,00 zabbix_server: self-monitoring [sleeping for 1 seconds] (zabbix_server) root 1479 0,0 0,1 3496 1464 0 S+ 12:14 0:00,00 grep zabbix # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 65536 1818252135 --rw------- zabbix zabbix zabbix zabbix 5 175752 1261 1261 12:12:47 12:14:23 12:12:47 m 65537 1751143757 --rw------- zabbix zabbix zabbix zabbix 28 8421355 1239 1239 12:12:53 no-entry 12:12:53 m 65538 2019579213 --rw------- zabbix zabbix zabbix zabbix 28 16777479 1239 1239 12:12:53 no-entry 12:12:53 m 65539 1952470349 --rw------- zabbix zabbix zabbix zabbix 28 4194554 1239 1239 12:12:53 no-entry 12:12:53 m 65540 1734366541 --rw------- zabbix zabbix zabbix zabbix 28 7130317 1239 1239 12:12:53 no-entry 12:12:53 m 65541 1935693133 --rw------- zabbix zabbix zabbix zabbix 28 1258291 1239 1239 12:12:53 no-entry 12:12:53 m 65542 1398822221 --rw------- zabbix zabbix zabbix zabbix 28 6900 1239 1239 12:12:53 no-entry 12:12:53 will kill it because vanilla agent has a bug and the key doesn't work. # kill 1261 # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 65537 1751143757 --rw------- zabbix zabbix zabbix zabbix 28 8421355 1239 1239 12:12:53 no-entry 12:12:53 m 65538 2019579213 --rw------- zabbix zabbix zabbix zabbix 28 16777479 1239 1239 12:12:53 no-entry 12:12:53 m 65539 1952470349 --rw------- zabbix zabbix zabbix zabbix 28 4194554 1239 1239 12:12:53 no-entry 12:12:53 m 65540 1734366541 --rw------- zabbix zabbix zabbix zabbix 28 7130317 1239 1239 12:12:53 no-entry 12:12:53 m 65541 1935693133 --rw------- zabbix zabbix zabbix zabbix 28 1258291 1239 1239 12:12:53 no-entry 12:12:53 m 65542 1398822221 --rw------- zabbix zabbix zabbix zabbix 28 6900 1239 1239 12:12:53 no-entry 12:12:53 start the dev branch agent: ./zabbix_agentd -c /usr/local/etc/zabbix_agentd.conf # ps aux | grep zabbix zabbix 1239 0,0 0,3 52932 6672 ?? I 12:12 0:00,02 zabbix_server: main process (zabbix_server) zabbix 1332 0,0 0,3 52932 7140 ?? S 12:12 0:00,03 zabbix_server: configuration syncer [sleeping for 16 seconds] (zabbix_server) zabbix 1333 0,0 0,3 52932 6696 ?? S 12:12 0:00,01 zabbix_server: db watchdog [sleeping for 15 seconds] (zabbix_server) zabbix 1334 0,0 0,5 55108 10104 ?? S 12:12 0:00,13 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1335 0,0 0,5 55108 10612 ?? S 12:12 0:00,15 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1336 0,0 0,5 55108 10612 ?? S 12:12 0:00,15 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1337 0,0 0,5 55108 10116 ?? S 12:12 0:00,13 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1338 0,0 0,5 55108 10124 ?? S 12:12 0:00,12 zabbix_server: poller [sleeping for 1 seconds] (zabbix_server) zabbix 1339 0,0 0,5 55108 10156 ?? S 12:12 0:00,11 zabbix_server: unreachable poller [sleeping for 5 seconds] (zabbix_server) zabbix 1340 0,0 0,3 52932 6724 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1341 0,0 0,3 52932 6724 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1342 0,0 0,3 52932 7060 ?? S 12:12 0:00,01 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1343 0,0 0,3 52932 7024 ?? S 12:12 0:00,01 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1344 0,0 0,3 52932 6928 ?? S 12:12 0:00,00 zabbix_server: trapper [waiting for connection] (zabbix_server) zabbix 1345 0,0 0,3 53316 6944 ?? S 12:12 0:00,02 zabbix_server: icmp pinger [sleeping for 2 seconds] (zabbix_server) zabbix 1346 0,0 0,3 52932 6760 ?? S 12:12 0:00,01 zabbix_server: alerter [sleeping for 2 seconds] (zabbix_server) zabbix 1347 0,0 0,3 52932 6708 ?? S 12:12 0:00,09 zabbix_server: housekeeper [sleeping for 3328 seconds] (zabbix_server) zabbix 1348 0,0 0,3 52932 7084 ?? S 12:12 0:00,01 zabbix_server: timer [sleeping for 18 seconds] (zabbix_server) zabbix 1349 0,0 0,3 52932 6700 ?? S 12:12 0:00,01 zabbix_server: http poller [sleeping for 5 seconds] (zabbix_server) zabbix 1350 0,0 0,5 54980 9736 ?? S 12:12 0:00,11 zabbix_server: discoverer [sleeping for 20 seconds] (zabbix_server) zabbix 1351 0,0 0,3 53060 7176 ?? S 12:12 0:00,02 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 1352 0,0 0,3 53060 7184 ?? S 12:12 0:00,02 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 1353 0,0 0,3 53060 7164 ?? S 12:12 0:00,02 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 1354 0,0 0,3 53060 7100 ?? S 12:12 0:00,01 zabbix_server: history syncer [sleeping for 5 seconds] (zabbix_server) zabbix 1355 0,0 0,3 52932 6972 ?? S 12:12 0:00,02 zabbix_server: escalator [sleeping for 3 seconds] (zabbix_server) zabbix 1356 0,0 0,3 52932 6688 ?? S 12:12 0:00,01 zabbix_server: ipmi poller [sleeping for 5 seconds] (zabbix_server) zabbix 1357 0,0 0,3 52932 6684 ?? S 12:12 0:00,01 zabbix_server: proxy poller [sleeping for 5 seconds] (zabbix_server) zabbix 1358 0,0 0,3 52932 6680 ?? S 12:12 0:00,01 zabbix_server: self-monitoring [sleeping for 1 seconds] (zabbix_server) zabbix 1550 0,0 0,2 7124 3200 ?? S 12:17 0:00,00 zabbix_agentd: main process (zabbix_agentd) zabbix 1551 0,0 0,2 7124 3296 ?? S 12:17 0:00,00 zabbix_agentd: collector [sleeping for 1 seconds] (zabbix_agentd) zabbix 1552 0,0 0,2 7124 3240 ?? S 12:17 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 1553 0,0 0,2 7124 3200 ?? S 12:17 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) zabbix 1554 0,0 0,2 7124 3200 ?? S 12:17 0:00,00 zabbix_agentd: listener [waiting for connection] (zabbix_agentd) # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 131072 1818252135 --rw------- zabbix zabbix zabbix zabbix 5 175752 1550 1550 12:17:34 12:18:23 12:17:34 m 65537 1751143757 --rw------- zabbix zabbix zabbix zabbix 28 8421355 1239 1239 12:12:53 no-entry 12:12:53 m 65538 2019579213 --rw------- zabbix zabbix zabbix zabbix 28 16777479 1239 1239 12:12:53 no-entry 12:12:53 m 65539 1952470349 --rw------- zabbix zabbix zabbix zabbix 28 4194554 1239 1239 12:12:53 no-entry 12:12:53 m 65540 1734366541 --rw------- zabbix zabbix zabbix zabbix 28 7130317 1239 1239 12:12:53 no-entry 12:12:53 m 65541 1935693133 --rw------- zabbix zabbix zabbix zabbix 28 1258291 1239 1239 12:12:53 no-entry 12:12:53 m 65542 1398822221 --rw------- zabbix zabbix zabbix zabbix 28 6900 1239 1239 12:12:53 no-entry 12:12:53 # zabbix_get -s localhost -k vfs.dev.write[ad0,ops] 0.000000 # zabbix_get -s localhost -k vfs.dev.write[ad0,ops] 0.000000 # zabbix_get -s localhost -k vfs.dev.write[ad0,ops] 34.000000 debuglog: 1552:20121011:121843.841 Processing request. 1552:20121011:121843.941 Requested [vfs.dev.write[ad0,ops]] 1552:20121011:121843.941 In collector_diskdevice_get() devname:'ad0' 1552:20121011:121843.941 diskstat_shm_init() allocated initial shm segment id:65543 for disk statistics collector 1552:20121011:121843.941 End of collector_diskdevice_get():0x0 1552:20121011:121843.941 In collector_diskdevice_add() devname:'ad0' 1552:20121011:121843.941 End of collector_diskdevice_add():0x2858c008 1552:20121011:121843.941 Sending back [0.000000] ... 1553:20121011:121846.510 Processing request. 1553:20121011:121846.610 Requested [vfs.dev.write[ad0,ops]] 1553:20121011:121846.610 In collector_diskdevice_get() devname:'ad0' 1553:20121011:121846.610 diskstat_shm_reattach() switched shm id from -1 to 65543 1553:20121011:121846.610 collector_diskdevice_get() device 'ad0' found 1553:20121011:121846.610 End of collector_diskdevice_get():0x2858c008 1553:20121011:121846.610 Sending back [0.000000] ... 1552:20121011:121850.413 Processing request. 1552:20121011:121850.513 Requested [vfs.dev.write[ad0,ops]] 1552:20121011:121850.513 In collector_diskdevice_get() devname:'ad0' 1552:20121011:121850.513 collector_diskdevice_get() device 'ad0' found 1552:20121011:121850.513 End of collector_diskdevice_get():0x2858c008 1552:20121011:121850.513 Sending back [34.000000] # ipcs -am Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME m 131072 1818252135 --rw------- zabbix zabbix zabbix zabbix 5 175752 1550 1550 12:17:34 12:19:00 12:17:34 m 65537 1751143757 --rw------- zabbix zabbix zabbix zabbix 28 8421355 1239 1239 12:12:53 no-entry 12:12:53 m 65538 2019579213 --rw------- zabbix zabbix zabbix zabbix 28 16777479 1239 1239 12:12:53 no-entry 12:12:53 m 65539 1952470349 --rw------- zabbix zabbix zabbix zabbix 28 4194554 1239 1239 12:12:53 no-entry 12:12:53 m 65540 1734366541 --rw------- zabbix zabbix zabbix zabbix 28 7130317 1239 1239 12:12:53 no-entry 12:12:53 m 65541 1935693133 --rw------- zabbix zabbix zabbix zabbix 28 1258291 1239 1239 12:12:53 no-entry 12:12:53 m 65542 1398822221 --rw------- zabbix zabbix zabbix zabbix 28 6900 1239 1239 12:12:53 no-entry 12:12:53 m 65543 1835029351 --rw------- zabbix zabbix zabbix zabbix 3 47040 1552 1553 12:18:46 12:19:00 12:18:43