[ZBX-3974] semaphore lock failed followed by a crash Created: 2011 Jul 21 Updated: 2017 Jul 25 Resolved: 2017 Jul 25 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Server (S) |
Affects Version/s: | None |
Fix Version/s: | None |
Type: | Incident report | Priority: | Critical |
Reporter: | richlv | Assignee: | Unassigned |
Resolution: | Duplicate | Votes: | 7 |
Labels: | crash, semaphores | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified | ||
Environment: |
1.8 branch, rev 20636 |
Attachments: |
![]() |
||||||||||||||||||||||||
Issue Links: |
|
Description |
4289:20110719:090517.267 server #25 started [self-monitoring] ... 4259:20110721:140615.001 History data from active proxy on [192.168.13.2] failed: proxy [Proxy 2] not found 4264:20110721:140616.016 History data from active proxy on [192.168.13.2] failed: proxy [Proxy 2] not found 4264:20110721:140617.030 History data from active proxy on [192.168.13.2] failed: proxy [Proxy 2] not found 4264:20110721:140618.043 History data from active proxy on [192.168.13.2] failed: proxy [Proxy 2] not found 4264:20110721:140619.045 History data from active proxy on [192.168.13.2] failed: proxy [Proxy 2] not found zabbix_server [4283]: [file:'selfmon.c',line:353] lock failed: [43] Identifier removed zabbix_server [4264]: [file:'selfmon.c',line:353] lock failed: [43] Identifier removed zabbix_server [4282]: [file:'selfmon.c',line:353] lock failed: [43] Identifier removed zabbix_server [4289]: [file:'selfmon.c',line:353] lock failed: [43] Identifier removed 4247:20110721:140620.439 One child process died (PID:4289,exitcode/signal:255). Exiting ... 4247:20110721:140622.442 Syncing history data... zabbix_server [4247]: [file:'dbcache.c',line:2152] lock failed: [43] Identifier removed |
Comments |
Comment by richlv [ 2011 Jul 21 ] |
same just happened in a proxy as well. 17008:20110721:140928.618 server #12 started [icmp pinger] ... 17010:20110721:140928.947 server #14 started [discoverer] 17009:20110721:140929.060 Deleted 23 records from history [0.013868 seconds] zabbix_proxy_sqlite [17008]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [17011]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [17014]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [17013]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [17012]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [16997]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [16999]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [17001]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [17000]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument zabbix_proxy_sqlite [16998]: [file:'selfmon.c',line:353] lock failed: [22] Invalid argument 16996:20110721:144014.150 One child process died (PID:17008,exitcode/signal:255). Exiting ... 16996:20110721:144016.184 Syncing history data... zabbix_proxy_sqlite [16996]: [file:'dbcache.c',line:2152] lock failed: [22] Invalid argument |
Comment by richlv [ 2011 Jul 22 ] |
full proxy log of two crashing sessions (and a bit of the next session startup) server log has been rotated over already. |
Comment by richlv [ 2011 Jul 22 ] |
potentially useful information - system was running zabbix server, two active proxies and one passive proxy |
Comment by Rudolfs Kreicbergs [ 2011 Jul 29 ] |
Changed priority to critical - could not repeat or find the cause. If this is caused by Zabbix then it does not seem like a new problem. |
Comment by Rudolfs Kreicbergs [ 2011 Aug 25 ] |
Closing as "Cannot Reproduce" - seems that the semaphores are arbitrary removed, no reason was found nor was it reproduced on any other system. Please reopen when additional case to study with LogLevel=4 is acquired. |
Comment by richlv [ 2012 May 24 ] |
something quite similar happened again. 1.8 branch, rev 27659 20098:20120524:100317.005 Zabbix agent item [agent.ping] on host [Server 4] failed: first network error, wait for 15 seconds 20100:20120524:100332.684 resuming Zabbix agent checks on host [Server 4]: connection restored zabbix_proxy [20094]: [file:'selfmon.c',line:331] lock failed: [22] Invalid argument 20091:20120524:100343.887 One child process died (PID:20094,exitcode/signal:255). Exiting ... 20091:20120524:100345.902 syncing history data... zabbix_proxy [20091]: [file:'dbcache.c',line:2197] lock failed: [22] Invalid argument |
Comment by Alexey Pustovalov [ 2012 Sep 08 ] |
Semaphore Limit. On some Linux systems, reaches a system limit on the number of semaphores in the Linux configuration. In this case, you will see a message similar to the following:
Invalid argument [22] means:
To increase the number of semaphores, run the following command as owner or root: echo 256 40000 32 32000 > /proc/sys/kernel/sem This file contains 4 numbers defining limits for System V IPC semaphores. These fields are, in order:
This modification takes effect immediately, but is reset at the next reboot. To avoid resetting the limit when rebooting, add the above command to one of your system’s startup scripts by copying the command into a file called mod_sem and then setting up the following links to execute the file each time you restart your system: cp mod_sem /etc/init.d ln -s /etc/init.d/mod_sem /etc/rc3.d/S61mod_sem ln -s /etc/init.d/mod_sem /etc/rc5.d/S61mod_sem Another method (prefered!): kernel.sem = 250 32000 100 10000 sysctl can be reloaded via the command "sysctl -p" (-p tells it to load from the standard conf file which is /etc/sysctl.conf). "sysctl -a | grep kernel.sem" can be used to check that the new values have been loaded. This can also be used after a reboot to check that the values have been permanently saved. |
Comment by Alexei Vladishev [ 2012 Sep 08 ] |
Shall we close it? <richlv> we don't know for sure what causes this - maybe zabbix passes invalid value, maybe it starts leaking semaphores, or maybe it indeed happens only when system in general is exhausted. and even in the latter case it is possible we could detect that and as a minimum print a better message, same as when we run out of config cache etc <richlv> also note that the proxies where crashes happen are not heavily loaded - they monitor some 10 hosts with 3-5 items at most with 3 nvps total... |
Comment by richlv [ 2012 Nov 08 ] |
happened again. 2.0 branch, rev 31271 zabbix_server [773]: [file:'selfmon.c',line:341] lock failed: [43] Identifier removed 758:20121108:010738.687 One child process died (PID:773,exitcode/signal:255). Exiting ... 758:20121108:010740.688 syncing history data... 758:20121108:010740.688 syncing history data done 758:20121108:010740.688 syncing trends data... zabbix_server [758]: [file:'dbcache.c',line:745] lock failed: [43] Identifier removed |
Comment by richlv [ 2012 Dec 06 ] |
and happened again. 2.0 branch, rev 31853, passive proxy zabbix_proxy [9162]: [file:'selfmon.c',line:341] lock failed: [22] Invalid argument 9156:20121206:214137.865 One child process died (PID:9162,exitcode/signal:255). Exiting ... 9156:20121206:214139.883 syncing history data... zabbix_proxy [9156]: [file:'dbcache.c',line:2011] lock failed: [22] Invalid argument |
Comment by richlv [ 2013 Jan 04 ] |
this might also be related to multiple processes attempting to use the same pidfile. just had an agent crash with : zabbix_agentd [6609]: [file:'cpustat.c',line:155] lock failed: [43] Identifier removed 6606:20130104:171246.321 One child process died (PID:6609,exitcode/signal:255). Exiting ... ...or maybe by multiple processes attempting to listen on the same port, or some combination of these |
Comment by richlv [ 2013 May 16 ] |
while seemingly different, maybe |
Comment by richlv [ 2013 Jul 31 ] |
happened again. 2.0 branch, r37403 zabbix_server [11650]: [file:'selfmon.c',line:341] lock failed: [43] Identifier removed 11635:20130731:114207.578 One child process died (PID:11650,exitcode/signal:255). Exiting ... 11635:20130731:114209.579 syncing history data... zabbix_server [11635]: [file:'dbcache.c',line:2193] lock failed: [43] Identifier removed |
Comment by Alexei Vladishev [ 2014 Mar 05 ] |
I had same crash with vanilla 2.2.2 on OpenSuSE. I suspect it is SuSE related issue since we had no reports from users of other distributions. |
Comment by richlv [ 2014 Mar 05 ] |
these seem to be triggered by running multiple zabbix daemons on the system in cases when these daemons are active and maybe started/stopped more frequently. that is probably less likely to happen during normal system operation. |
Comment by Gabriele Armao [ 2014 Mar 18 ] |
just got this (or a very similar) issue on one of my customer's server, it's a 2.0.10 Zabbix Server on a Red Hat (not sure which release). Backend database is Oracle. They were having some issues with Web scenarios (error: error doing curl_easy_perform: Couldn't resolve host name) for a few days and we were trying to debug them before restarting the server, when during the night, the zabbix server crashed with this messages in the server log file: 4528:20140316:222509.409 web scenario step "WEB2:login_check" error: error doing curl_easy_perform: Couldn't resolve host name zabbix_server [3279]: cannot open [/var/log/zabbix/zabbix_server.log]: [24] Too many open files followed, a few seconds later, by: 3114:20140316:222531.947 One child process died (PID:3279,exitcode/signal:255). Exiting ... 4505:20140316:222602.830 web scenario step "WEB:check_url" error: error doing curl_easy_perform: Couldn't resolve host name 3114:20140316:222614.540 syncing history data... 3114:20140316:222615.914 syncing history data done 3114:20140316:222615.915 syncing trends data... 4505:20140316:222802.852 web scenario step "WEB:check_url" error: error doing curl_easy_perform: Couldn't resolve host name 3114:20140316:222853.287 syncing trends data done 3114:20140316:222853.314 Zabbix Server stopped. Zabbix 2.0.10 (revision 40809). zabbix_server [4505]: [file:'selfmon.c',line:341] lock failed: [22] Invalid argument As expected, PID 4505 is an HTTP Poller: 4505:20140226:101829.725 server #857 started [http poller #3] and PID 3279 is the DB Watchdog Process: 3279:20140226:101817.264 server #2 started [db watchdog #1] So it seems in this case, they run out of system resources (file descriptors, semaphores, ...) glebs.ivanovskis [24] Too many open files does not line up with the rest of this ticket, so you may be interested in reporting it as a separate issue if it's still happening. |
Comment by richlv [ 2014 Mar 18 ] |
hmm, "Too many open files" is interesting. it might be related to the way we handle logfiles where each process opens the file individually and if the file is rotated and that process has nothing to write for it for a long time, it still holds open an fd on it. as a result, on a system with lots of processes and files being rotated relatively often we end up with lots of deleted files still being held open and on disk by those processes/fds. now that is not the case when i observed it as those usually were small instances - but running out of semaphores one way or another could be related. |
Comment by richlv [ 2015 Jan 08 ] |
had another agent crash like that. zabbix_agentd [15182]: [file:'cpustat.c',line:247] lock failed: [43] Identifier removed 15181:20150108:173956.879 One child process died (PID:15182,exitcode/signal:1). Exiting ... zabbix_agentd [15181]: Error on thread waiting. |
Comment by Valery Tishkov [ 2015 Sep 04 ] |
reproduced 5119:20150722:090923.927 Starting Zabbix Agent [_hostname_]. Zabbix 2.2.9 (revision 52686). 5119:20150722:090923.928 using configuration file: /etc/zabbix/zabbix_agentd.conf 5125:20150722:090923.932 agent #0 started [collector] 5126:20150722:090923.933 agent #1 started [listener #1] 5127:20150722:090923.933 agent #2 started [listener #2] 5128:20150722:090923.934 agent #3 started [listener #3] 5129:20150722:090923.935 agent #4 started [listener #4] 5130:20150722:090923.935 agent #5 started [listener #5] 5131:20150722:090923.936 agent #6 started [active checks #1] zabbix_agentd [5125]: [file:'cpustat.c',line:235] lock failed: [43] Identifier removed zabbix_agentd [5119]: [file:'log.c',line:236] lock failed: [43] Identifier removed zabbix_agentd [5128]: [file:'diskdevices.c',line:134] lock failed: [43] Identifier removed zabbix_agentd [5129]: [file:'diskdevices.c',line:134] lock failed: [43] Identifier removed zabbix_agentd [5127]: [file:'diskdevices.c',line:134] lock failed: [43] Identifier removed zabbix_agentd [5130]: [file:'diskdevices.c',line:134] lock failed: [43] Identifier removed zabbix_agentd [5126]: [file:'diskdevices.c',line:134] lock failed: [43] Identifier removed zabbix_agentd [5131]: [file:'log.c',line:236] lock failed: [43] Identifier removed $ sysctl -a | grep kernel.sem kernel.sem = 250 32000 32 128 $ uname -srvmpio Linux 2.6.32-504.12.2.el6.x86_64 #1 SMP Wed Mar 11 22:03:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux |
Comment by dimir [ 2015 Nov 20 ] |
As to error 43 it seems it means some external entity killed the semaphore during the work flow of Zabbix daemon: errno.h: |
Comment by Michael Quigley [ 2016 Feb 02 ] |
I've got a Zabbix server running on RedHat 7 in AWS, which has run fine without crashing for around 6 months. I updated to the latest kernel and rebooted, and now the server won't run for more than a handful of minutes without crashing like this: 2445:20160202:152910.113 server #18 started [http poller #1] 2451:20160202:152910.115 server #24 started [escalator #1] 2449:20160202:152910.115 server #22 started [history syncer #3] 2452:20160202:152910.115 server #25 started [proxy poller #1] 2450:20160202:152910.116 server #23 started [history syncer #4] 2445:20160202:153010.901 cannot process step "Login" of web scenario "pilot-stream.xxxxxxx.com" on host "Zabbix Server": response code "503" did not match any of the required status codes "200" 2433:20160202:153129.235 Zabbix agent item "system.cpu.util[,softirq]" on host "Pilot-Stream-StreamAnalyzer-i-fb53057a" failed: first network error, wait for 15 seconds 2435:20160202:153144.373 resuming Zabbix agent checks on host "Pilot-Stream-StreamAnalyzer-i-fb53057a": connection restored zabbix_server [2432]: [file:'dbconfig.c',line:5174] lock failed: [22] Invalid argument zabbix_server [2427]: [file:'log.c',line:271] lock failed: [22] Invalid argument zabbix_server [2450]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2447]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2448]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2449]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2431]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2433]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2430]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2434]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2453]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2451]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2445]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2435]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2441]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2452]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2446]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2442]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2429]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2428]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2438]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2444]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2436]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2439]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2440]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [2437]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument I've tried ensuring that the kernel semaphore configuration is adequately configured: [ec2-user@ip-172-31-3-195 logs]$ sudo sysctl -a | grep kernel.sem kernel.sem = 512 80000 512 64000 That's made no difference. Help! |
Comment by Michael Quigley [ 2016 Feb 03 ] |
This seems pretty repeatable: If I'm ssh-ed into the instance where the zabbix_server is running, tailing the log, the server seems to run. Everytime, as soon as I log out of the instance, zabbix_server crashes like this. |
Comment by Michael Quigley [ 2016 Feb 03 ] |
Keeping a nohup tail zabbix_server.log > /dev/null 2>&1 & running seems to be keeping my server alive. |
Comment by Aleksandrs Saveljevs [ 2016 Feb 03 ] |
Michael, could you please specify how exactly you tail the log and how exactly you log out of the instance to make it crash? Does it crash immediately or after some time? Note that your last command nohup tail zabbix_server.log > /dev/null 2>&1 & does not actually run in the background much, probably due to the missing -f or -F. |
Comment by Michael Quigley [ 2016 Feb 03 ] |
The good news is my zabbix_server managed to run through the night with the above workaround.Yes, the correct command I executed was nohup tail -f zabbix_server.log > /dev/null 2>&1 &. But I suspect that any command which keeps the ssh session open will keep my zabbix_server alive. The server was compiled from source and is running on an AWS instance. The server had been running 2.4.5 for 6 months reliably. In an effort to try and resolve this, I upgraded the server to 2.4.7. The server version has no effect on the problem. The zabbix_server is started using systemd with a service definition like this: [Unit] Description=Zabbix Server After=syslog.target network.target [Service] Type=forking ExecStart=/home/ec2-user/local/zabbix/sbin/zabbix_server ExecReload=/home/ec2-user/local/zabbix/sbin/zabbix_server -R config_cache_reload RemainAfterExit=yes PIDFile=/home/ec2-user/local/zabbix/logs/zabbix_server.pid User=ec2-user [Install] WantedBy=multi-user.target As soon as I kill off my background tail process, and exit my interactive ssh session with the instance, the zabbix_server process crashes. |
Comment by Michael Quigley [ 2016 Feb 03 ] |
To be even more clear... If I execute the nohup tail -f ... command, I CAN disconnect my ssh session from the instance and the zabbix_server will continue to execute. If I ssh back into the instance again, kill the tail command, and then exit my ssh session, zabbix_server will crash immediately. glebs.ivanovskis Your issue seems more related to |
Comment by Jan Klepek [ 2016 Mar 08 ] |
Hi, Why was this removed from 3.0.2rc1? Does it mean that you are not going to fix it in this version? Are you going to fix it anytime soon (like 1-2 months)? Thank you for information |
Comment by Ronny Pettersen [ 2016 Mar 08 ] |
I've have 2 test instances of Zabbix running on CentOS 7.2, and I've also noticed that both the Zabbix 2.4.7 and Zabbix 3.0.1 crashes with the same error: zabbix_server [15728]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [15724]: [file:'log.c',line:271] lock failed: [22] Invalid argument zabbix_server [15752]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument <repeat of above line 5 times> zabbix_server [15736]: zabbix_server [15738]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument[file:'selfmon.c',line:391] lock failed: [22] Invalid argument zabbix_server [15742]: [file:'selfmon.c',line:391] lock failed: [22] Invalid argument <repeat of above line 9 times> zabbix_agentd also creashes: zabbix_agentd [10796]: [file:'cpustat.c',line:252] lock failed: [22] Invalid argument zabbix_agentd [10795]: [file:'log.c',line:271] lock failed: [22] Invalid argument zabbix_agentd [10798]: [file:'log.c',line:271] lock failed: [22] Invalid argument After this, the only processes left running are: Since this was test, I've not taken much notice to when it started happening. It could possibly be it started happening after upgrade to CentOS 7.2 on December 17. In my case, it takes a while before it crashes. Not immediately after disconnect, but I start Zabbix using scripts (although without nohup, using "daemon $exec -c $conf"). |
Comment by Ronny Pettersen [ 2016 Mar 14 ] |
I've now been able to reproduce this on my system - it may not be the same behavior as for others. #!/bin/sh MYSQL="/usr/bin/mysql" CONF="--defaults-file=$HOME/.my.zabbix.cnf" SQL="select now();" /bin/echo $SQL | $MYSQL $CONF Cron: 00 15 * * * /opt/zabbix/scripts/test.sh If I run the script manually, from command line, there are no issues. |
Comment by Ronny Pettersen [ 2016 Mar 14 ] |
Also triggerting this run using "at now" gives the same result. Strace of main Zabbix process (/opt/zabbix/sbin/zabbix_server -c /opt/zabbix/etc/zabbix_server.conf): Process 13387 attached wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 1}], 0, NULL) = 13425 --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=13425, si_status=1, si_utime=0, si_stime=0} --- semop(72220682, {{0, -1, SEM_UNDO}}, 1) = -1 EINVAL (Invalid argument) write(2, "zabbix_server [13387]: ", 23) = 23 write(2, "[file:'log.c',line:271] lock fai"..., 58) = 58 write(2, "\n", 1) = 1 fcntl(3, F_SETLK, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0 close(3) = 0 munmap(0x7f4897133000, 4096) = 0 unlink("/opt/zabbix/run/zabbix_server.pid") = 0 exit_group(1) = ? +++ exited with 1 +++ And example of strace of "history syncer #1": rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART, 0x7ffcfc50e670}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 nanosleep({1, 0}, 0x7fff075c3a70) = 0 rt_sigprocmask(SIG_BLOCK, [CHLD], [], 8) = 0 rt_sigaction(SIGCHLD, NULL, {SIG_DFL, [CHLD], SA_RESTORER|SA_RESTART, 0x7ffcfc50e670}, 8) = 0 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0 nanosleep({1, 0}, 0x7fff075c3a70) = 0 times({tms_utime=0, tms_stime=0, tms_cutime=0, tms_cstime=0}) = 3128138950 semop(72351754, {{5, -1, SEM_UNDO}}, 1) = -1 EINVAL (Invalid argument) write(2, "zabbix_server [14415]: ", 23) = 23 write(2, "[file:'selfmon.c',line:391] lock"..., 62) = 62 write(2, "\n", 1) = 1 exit_group(1) = ? +++ exited with 1 +++ |
Comment by Ronny Pettersen [ 2016 Mar 14 ] |
I got the same error running any command through cron (even "at now" running date). semop(72351754, {{5, -1, SEM_UNDO}}, 1) = -1 EINVAL (Invalid argument) I see server have not been restarted after installing latest kernel, so I will attempt that later today. |
Comment by Ronny Pettersen [ 2016 Mar 14 ] |
Updating to latest kernel (3.10.0-327.10.1.el7.x86_64) + reboot solved it for me |
Comment by Sheevary [ 2016 Jun 30 ] |
I have the same problem. FreeBSD 10.1 RELEASE p35 Zabbix Server stopped. Zabbix 3.0.2 (revision 59540). How to fix it? Upd1 |
Comment by dimir [ 2016 Jun 30 ] |
Had seen the similar case when I was using more than one Zabbix server instances on the box. |
Comment by Little Martian [ 2016 Nov 22 ] |
I have a fresh install of version 3.2.1 compiled 64bit on Solaris 10. I found the same errors in the log files of both the server and the agent on the same server. zabbix_server [8967]: [file:'selfmon.c',line:380] lock failed: [22] Invalid argument zabbix_server [8961]: [file:'selfmon.c',line:380] lock failed: [22] Invalid argument zabbix_server [8965]: [file:'selfmon.c',line:380] lock failed: [22] Invalid argument zabbix_server [8951]: [file:'selfmon.c',line:380] lock failed: [22] Invalid argument zabbix_server [8952]: [file:'selfmon.c',line:380] lock failed: [22] Invalid argument I restart the server and as soon as the process is running, the agent does the same thing: zabbix_agentd [8997]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_agentd [8991]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_agentd [8998]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_agentd [8995]: [file:'log.c',line:238] lock failed: [22] Invalid argument I am running Server/Agent v 3.2.1, MySQL Ver 14.14 Distrib 5.7.16, on Solaris 5.10 Generic_150400-15, T4-2, Zabbix and MySQL compiled with gcc 5.2.0 (from OpenCSW). On the same server there is another zabbix agent running (on different port and user) version 2.x (i do not know exactly) and i just imported two server definitions from an Zabbix 2.2.5. The agents for the imported servers run on two different Solaris 5.10 Generic_150400-15, T4-2 with version: Zabbix Agent (daemon) v2.2.1 (revision 40808) (09 December 2013) Compilation time: Dec 24 2013 20:08:37 For the imported servers I also have enabled SNMP monitoring (that do not work for now, firewall issues) |
Comment by Little Martian [ 2016 Nov 22 ] |
Sorry, my mistake. The error was from the shutdown script that removed semaphores and shared memory after server stopped. If the server does not stop in 10 sec, one kill -term / sec, then a kill -kill follows and semaphores and shared memory remained orphaned and cleaned with ipcrm. I removed the semaphores cleanup and the error does not appear in the logs anymore. I have another problem with the zabbix server hanging in check_operation_conditions (the sql finishes instantly in MySQL Workbench). I will address this in another thread. 25260:20161122:145638.165 In check_operation_conditions() operationid:11 25260:20161122:145638.165 query [txnlev:0] [select conditiontype,operator,value from opconditions where operationid=11 order by conditiontype] |
Comment by Alexander Khokhlov [ 2016 Dec 09 ] |
I have this error after upgrade zabbix-server from version 3.2.1 to 3.2.2 from rpm package on CentOS 7 Linux zabbix 3.10.0-327.36.3.el7.x86_64 #1 SMP Mon Oct 24 16:09:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux [file:'log.c',line:238] lock failed: [22] 21260:20161209:230000.442 Got signal [signal:11(SIGSEGV),reason:1,refaddr:(nil)]. Crashing ... 21260:20161209:230000.442 ====== Fatal information: ====== 21260:20161209:230000.442 Program counter: 0x4328ff 21260:20161209:230000.442 === Registers: === 21260:20161209:230000.442 r8 = ffff = 65535 = 65535 21260:20161209:230000.442 r9 = f = 15 = 15 21260:20161209:230000.442 r10 = 7f8110337190 = 140192299315600 = 140192299315600 21260:20161209:230000.442 r11 = 213 = 531 = 531 21260:20161209:230000.442 r12 = 27ace40 = 41602624 = 41602624 21260:20161209:230000.443 r13 = 27acc70 = 41602160 = 41602160 21260:20161209:230000.443 r14 = 27ae4c7 = 41608391 = 41608391 21260:20161209:230000.443 r15 = 794bb0 = 7949232 = 7949232 21260:20161209:230000.443 rdi = 4edb34 = 5167924 = 5167924 21260:20161209:230000.443 rsi = 0 = 0 = 0 21260:20161209:230000.443 rdi = 4edb34 = 5167924 = 5167924 21260:20161209:230000.443 rsi = 0 = 0 = 0 21260:20161209:230000.443 rbp = 0 = 0 = 0 21260:20161209:230000.443 rbx = 7ffc3ce5ccb0 = 140721330179248 = 140721330179248 21260:20161209:230000.443 rdx = 0 = 0 = 0 21260:20161209:230000.443 rax = 0 = 0 = 0 21260:20161209:230000.443 rcx = 5 = 5 = 5 21260:20161209:230000.443 rsp = 7ffc3ce5c870 = 140721330178160 = 140721330178160 21260:20161209:230000.443 rip = 4328ff = 4401407 = 4401407 21260:20161209:230000.443 efl = 10246 = 66118 = 66118 21260:20161209:230000.443 csgsfs = 33 = 51 = 51 21260:20161209:230000.443 err = 4 = 4 = 4 21260:20161209:230000.443 trapno = e = 14 = 14 21260:20161209:230000.443 oldmask = 0 = 0 = 0 21260:20161209:230000.443 cr2 = 0 = 0 = 0 21260:20161209:230000.443 === Backtrace: === 21260:20161209:230000.444 12: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](print_fatal_info+0xae) [0x48b1ce] 21260:20161209:230000.444 11: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values]() [0x48b4a3] 21260:20161209:230000.444 10: /usr/lib64/libc.so.6(+0x35670) [0x7f8110239670] 21260:20161209:230000.444 9: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](check_vcenter_hv_status+0x5f) [0x4328ff] 21260:20161209:230000.444 8: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](get_value_simple+0x1c9) [0x4301a9] 21260:20161209:230000.444 7: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values]() [0x42e3ca] 21260:20161209:230000.444 6: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](poller_thread+0x100) [0x42e500] 21260:20161209:230000.444 5: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](zbx_thread_start+0x45) [0x48bea5] 21260:20161209:230000.444 4: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](MAIN_ZABBIX_ENTRY+0x631) [0x421ae1] 21260:20161209:230000.444 3: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](daemon_start+0x1c0) [0x48aa70] 21260:20161209:230000.444 2: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values](main+0x326) [0x419106] 21260:20161209:230000.444 1: /usr/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f8110225b15] 21260:20161209:230000.444 0: /usr/sbin/zabbix_server: poller #217 [got 0 values in 0.000000 sec, getting values]() [0x4193ad] 21260:20161209:230000.444 === Memory map: === 21260:20161209:230000.444 00400000-00593000 r-xp 00000000 fd:00 807302354 /usr/sbin/zabbix_server_mysql 21260:20161209:230000.444 00792000-00793000 r--p 00192000 fd:00 807302354 /usr/sbin/zabbix_server_mysql 21260:20161209:230000.444 00793000-0079a000 rw-p 00193000 fd:00 807302354 /usr/sbin/zabbix_server_mysql 21260:20161209:230000.444 0079a000-007a1000 rw-p 00000000 00:00 0 21260:20161209:230000.444 022a1000-022e5000 rw-p 00000000 00:00 0 [heap] 21260:20161209:230000.444 022e5000-02317000 rw-p 00000000 00:00 0 [heap] 21260:20161209:230000.444 02317000-027de000 rw-p 00000000 00:00 0 [heap] 21260:20161209:230000.444 022e5000-02317000 rw-p 00000000 00:00 0 [heap] 21260:20161209:230000.444 02317000-027de000 rw-p 00000000 00:00 0 [heap] 21260:20161209:230000.444 7f80e3d46000-7f80ea26d000 r--p 00000000 fd:00 537211368 /usr/lib/locale/locale-archive 21260:20161209:230000.445 7f80ea26d000-7f80ee26d000 rw-s 00000000 00:04 3702792 /SYSV760005c7 (deleted) 21260:20161209:230000.445 7f80ee26d000-7f80eea6d000 rw-s 00000000 00:04 3670023 /SYSV770005c7 (deleted) 21260:20161209:230000.445 7f80eea6d000-7f80f10d4000 rw-s 00000000 00:04 3604485 /SYSV730005c7 (deleted) 21260:20161209:230000.445 7f80f10d4000-7f80fea6e000 rw-s 00000000 00:04 3571716 /SYSV670005c7 (deleted) 21260:20161209:230000.445 7f80fea6e000-7f8102a6e000 rw-s 00000000 00:04 3538947 /SYSV740005c7 (deleted) 21260:20161209:230000.445 7f8102a6e000-7f8106a6e000 rw-s 00000000 00:04 3506178 /SYSV480005c7 (deleted) 21260:20161209:230000.445 7f8106a6e000-7f810aa6e000 rw-s 00000000 00:04 3473409 /SYSV680005c7 (deleted) 21260:20161209:230000.445 7f810aa6e000-7f810aa7a000 r-xp 00000000 fd:00 805308328 /usr/lib64/libnss_files-2.17.so 21260:20161209:230000.445 7f810aa7a000-7f810ac79000 ---p 0000c000 fd:00 805308328 /usr/lib64/libnss_files-2.17.so 21260:20161209:230000.445 7f810ac79000-7f810ac7a000 r--p 0000b000 fd:00 805308328 /usr/lib64/libnss_files-2.17.so 21260:20161209:230000.445 7f810ac7a000-7f810ac7b000 rw-p 0000c000 fd:00 805308328 /usr/lib64/libnss_files-2.17.so 21260:20161209:230000.445 7f810ac7b000-7f810ac81000 rw-p 00000000 00:00 0 21260:20161209:230000.445 7f810ac81000-7f810ace1000 r-xp 00000000 fd:00 805308699 /usr/lib64/libpcre.so.1.2.0 21260:20161209:230000.445 7f810ace1000-7f810aee0000 ---p 00060000 fd:00 805308699 /usr/lib64/libpcre.so.1.2.0 21260:20161209:230000.445 7f810aee0000-7f810aee1000 r--p 0005f000 fd:00 805308699 /usr/lib64/libpcre.so.1.2.0 21260:20161209:230000.445 7f810aee1000-7f810aee2000 rw-p 00060000 fd:00 805308699 /usr/lib64/libpcre.so.1.2.0 21260:20161209:230000.445 7f810aee2000-7f810aee4000 r-xp 00000000 fd:00 805307860 /usr/lib64/libfreebl3.so 21260:20161209:230000.445 7f810aee4000-7f810b0e3000 ---p 00002000 fd:00 805307860 /usr/lib64/libfreebl3.so 21260:20161209:230000.445 7f810b0e3000-7f810b0e4000 r--p 00001000 fd:00 805307860 /usr/lib64/libfreebl3.so 21260:20161209:230000.445 7f810b0e4000-7f810b0e5000 rw-p 00002000 fd:00 805307860 /usr/lib64/libfreebl3.so 21260:20161209:230000.445 7f810b0e5000-7f810b106000 r-xp 00000000 fd:00 805308741 /usr/lib64/libselinux.so.1 21260:20161209:230000.445 7f810b106000-7f810b306000 ---p 00021000 fd:00 805308741 /usr/lib64/libselinux.so.1 21260:20161209:230000.445 7f810b306000-7f810b307000 r--p 00021000 fd:00 805308741 /usr/lib64/libselinux.so.1 21260:20161209:230000.445 7f810b307000-7f810b308000 rw-p 00022000 fd:00 805308741 /usr/lib64/libselinux.so.1 21260:20161209:230000.445 7f810b308000-7f810b30a000 rw-p 00000000 00:00 0 21260:20161209:230000.445 7f810b30a000-7f810b311000 r-xp 00000000 fd:00 805308911 /usr/lib64/libffi.so.6.0.1 21260:20161209:230000.445 7f810b311000-7f810b510000 ---p 00007000 fd:00 805308911 /usr/lib64/libffi.so.6.0.1 21260:20161209:230000.445 7f810b510000-7f810b511000 r--p 00006000 fd:00 805308911 /usr/lib64/libffi.so.6.0.1 21260:20161209:230000.445 7f810b511000-7f810b512000 rw-p 00007000 fd:00 805308911 /usr/lib64/libffi.so.6.0.1 21260:20161209:230000.445 7f810b512000-7f810b519000 r-xp 00000000 fd:00 805308685 /usr/lib64/librt-2.17.so 21260:20161209:230000.445 7f810b519000-7f810b718000 ---p 00007000 fd:00 805308685 /usr/lib64/librt-2.17.so 21260:20161209:230000.445 7f810b718000-7f810b719000 r--p 00006000 fd:00 805308685 /usr/lib64/librt-2.17.so 21260:20161209:230000.445 7f810b719000-7f810b71a000 rw-p 00007000 fd:00 805308685 /usr/lib64/librt-2.17.so 21260:20161209:230000.445 7f810b71a000-7f810b722000 r-xp 00000000 fd:00 805308309 /usr/lib64/libcrypt-2.17.so 21260:20161209:230000.445 7f810b719000-7f810b71a000 rw-p 00007000 fd:00 805308685 /usr/lib64/librt-2.17.so 21260:20161209:230000.445 7f810b71a000-7f810b722000 r-xp 00000000 fd:00 805308309 /usr/lib64/libcrypt-2.17.so 21260:20161209:230000.445 7f810b722000-7f810b921000 ---p 00008000 fd:00 805308309 /usr/lib64/libcrypt-2.17.so 21260:20161209:230000.446 7f810b921000-7f810b922000 r--p 00007000 fd:00 805308309 /usr/lib64/libcrypt-2.17.so 21260:20161209:230000.446 7f810b922000-7f810b923000 rw-p 00008000 fd:00 805308309 /usr/lib64/libcrypt-2.17.so 21260:20161209:230000.446 7f810b923000-7f810b951000 rw-p 00000000 00:00 0 21260:20161209:230000.446 7f810b951000-7f810b954000 r-xp 00000000 fd:00 805309470 /usr/lib64/libkeyutils.so.1.5 21260:20161209:230000.446 7f810b954000-7f810bb53000 ---p 00003000 fd:00 805309470 /usr/lib64/libkeyutils.so.1.5 21260:20161209:230000.446 7f810bb53000-7f810bb54000 r--p 00002000 fd:00 805309470 /usr/lib64/libkeyutils.so.1.5 21260:20161209:230000.446 7f810bb54000-7f810bb55000 rw-p 00003000 fd:00 805309470 /usr/lib64/libkeyutils.so.1.5 21260:20161209:230000.446 7f810bb55000-7f810bb62000 r-xp 00000000 fd:00 805407361 /usr/lib64/libkrb5support.so.0.1 21260:20161209:230000.446 7f810bb62000-7f810bd62000 ---p 0000d000 fd:00 805407361 /usr/lib64/libkrb5support.so.0.1 21260:20161209:230000.446 7f810bd62000-7f810bd63000 r--p 0000d000 fd:00 805407361 /usr/lib64/libkrb5support.so.0.1 21260:20161209:230000.446 7f810bd63000-7f810bd64000 rw-p 0000e000 fd:00 805407361 /usr/lib64/libkrb5support.so.0.1 21260:20161209:230000.446 7f810bd64000-7f810bdda000 r-xp 00000000 fd:00 805309227 /usr/lib64/libgmp.so.10.2.0 21260:20161209:230000.446 7f810bdda000-7f810bfd9000 ---p 00076000 fd:00 805309227 /usr/lib64/libgmp.so.10.2.0 21260:20161209:230000.446 7f810bfd9000-7f810bfda000 r--p 00075000 fd:00 805309227 /usr/lib64/libgmp.so.10.2.0 21260:20161209:230000.446 7f810bfda000-7f810bfdb000 rw-p 00076000 fd:00 805309227 /usr/lib64/libgmp.so.10.2.0 21260:20161209:230000.446 7f810bfdb000-7f810c001000 r-xp 00000000 fd:00 805309523 /usr/lib64/libhogweed.so.2.5 21260:20161209:230000.446 7f810c001000-7f810c200000 ---p 00026000 fd:00 805309523 /usr/lib64/libhogweed.so.2.5 21260:20161209:230000.446 7f810c200000-7f810c201000 r--p 00025000 fd:00 805309523 /usr/lib64/libhogweed.so.2.5 21260:20161209:230000.446 7f810c201000-7f810c202000 rw-p 00026000 fd:00 805309523 /usr/lib64/libhogweed.so.2.5 21260:20161209:230000.446 7f810c202000-7f810c231000 r-xp 00000000 fd:00 805309525 /usr/lib64/libnettle.so.4.7 21260:20161209:230000.446 7f810c231000-7f810c431000 ---p 0002f000 fd:00 805309525 /usr/lib64/libnettle.so.4.7 21260:20161209:230000.446 7f810c431000-7f810c432000 r--p 0002f000 fd:00 805309525 /usr/lib64/libnettle.so.4.7 21260:20161209:230000.446 7f810c432000-7f810c433000 rw-p 00030000 fd:00 805309525 /usr/lib64/libnettle.so.4.7 21260:20161209:230000.446 7f810c433000-7f810c445000 r-xp 00000000 fd:00 805309504 /usr/lib64/libtasn1.so.6.2.3 21260:20161209:230000.446 7f810c445000-7f810c645000 ---p 00012000 fd:00 805309504 /usr/lib64/libtasn1.so.6.2.3 21260:20161209:230000.446 7f810c645000-7f810c646000 r--p 00012000 fd:00 805309504 /usr/lib64/libtasn1.so.6.2.3 21260:20161209:230000.446 7f810c646000-7f810c647000 rw-p 00013000 fd:00 805309504 /usr/lib64/libtasn1.so.6.2.3 21260:20161209:230000.446 7f810c647000-7f810c6b5000 r-xp 00000000 fd:00 805378615 /usr/lib64/libtspi.so.1.2.0 21260:20161209:230000.446 7f810c6b5000-7f810c8b4000 ---p 0006e000 fd:00 805378615 /usr/lib64/libtspi.so.1.2.0 21260:20161209:230000.446 7f810c8b4000-7f810c8b5000 r--p 0006d000 fd:00 805378615 /usr/lib64/libtspi.so.1.2.0 21260:20161209:230000.446 7f810c8b5000-7f810c8b7000 rw-p 0006e000 fd:00 805378615 /usr/lib64/libtspi.so.1.2.0 21260:20161209:230000.446 7f810c8b7000-7f810c8b8000 rw-p 00000000 00:00 0 21260:20161209:230000.446 7f810c8b8000-7f810c8f4000 r-xp 00000000 fd:00 805309056 /usr/lib64/libp11-kit.so.0.0.0 21260:20161209:230000.447 7f810c8f4000-7f810caf3000 ---p 0003c000 fd:00 805309056 /usr/lib64/libp11-kit.so.0.0.0 21260:20161209:230000.446 7f810c8b8000-7f810c8f4000 r-xp 00000000 fd:00 805309056 /usr/lib64/libp11-kit.so.0.0.0 21260:20161209:230000.447 7f810c8f4000-7f810caf3000 ---p 0003c000 fd:00 805309056 /usr/lib64/libp11-kit.so.0.0.0 21260:20161209:230000.447 7f810caf3000-7f810cafc000 r--p 0003b000 fd:00 805309056 /usr/lib64/libp11-kit.so.0.0.0 21260:20161209:230000.447 7f810cafc000-7f810cafe000 rw-p 00044000 fd:00 805309056 /usr/lib64/libp11-kit.so.0.0.0 21260:20161209:230000.447 7f810cafe000-7f810cb13000 r-xp 00000000 fd:00 805306569 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 21260:20161209:230000.447 7f810cb13000-7f810cd12000 ---p 00015000 fd:00 805306569 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 21260:20161209:230000.447 7f810cd12000-7f810cd13000 r--p 00014000 fd:00 805306569 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 21260:20161209:230000.447 7f810cd13000-7f810cd14000 rw-p 00015000 fd:00 805306569 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 21260:20161209:230000.447 7f810cd14000-7f810cd46000 r-xp 00000000 fd:00 805309212 /usr/lib64/libidn.so.11.6.11 21260:20161209:230000.447 7f810cd46000-7f810cf45000 ---p 00032000 fd:00 805309212 /usr/lib64/libidn.so.11.6.11 21260:20161209:230000.447 7f810cf45000-7f810cf46000 r--p 00031000 fd:00 805309212 /usr/lib64/libidn.so.11.6.11 21260:20161209:230000.447 7f810cf46000-7f810cf47000 rw-p 00032000 fd:00 805309212 /usr/lib64/libidn.so.11.6.11 21260:20161209:230000.447 7f810cf47000-7f810cf81000 r-xp 00000000 fd:00 805311252 /usr/lib64/libnspr4.so 21260:20161209:230000.447 7f810cf81000-7f810d180000 ---p 0003a000 fd:00 805311252 /usr/lib64/libnspr4.so 21260:20161209:230000.447 7f810d180000-7f810d181000 r--p 00039000 fd:00 805311252 /usr/lib64/libnspr4.so 21260:20161209:230000.447 7f810d181000-7f810d183000 rw-p 0003a000 fd:00 805311252 /usr/lib64/libnspr4.so 21260:20161209:230000.447 7f810d183000-7f810d185000 rw-p 00000000 00:00 0 21260:20161209:230000.447 7f810d185000-7f810d189000 r-xp 00000000 fd:00 805311253 /usr/lib64/libplc4.so 21260:20161209:230000.447 7f810d189000-7f810d388000 ---p 00004000 fd:00 805311253 /usr/lib64/libplc4.so 21260:20161209:230000.447 7f810d388000-7f810d389000 r--p 00003000 fd:00 805311253 /usr/lib64/libplc4.so 21260:20161209:230000.447 7f810d389000-7f810d38a000 rw-p 00004000 fd:00 805311253 /usr/lib64/libplc4.so 21260:20161209:230000.447 7f810d38a000-7f810d38d000 r-xp 00000000 fd:00 805311254 /usr/lib64/libplds4.so 21260:20161209:230000.447 7f810d38d000-7f810d58c000 ---p 00003000 fd:00 805311254 /usr/lib64/libplds4.so 21260:20161209:230000.447 7f810d58c000-7f810d58d000 r--p 00002000 fd:00 805311254 /usr/lib64/libplds4.so 21260:20161209:230000.447 7f810d58d000-7f810d58e000 rw-p 00003000 fd:00 805311254 /usr/lib64/libplds4.so 21260:20161209:230000.447 7f810d58e000-7f810d5b4000 r-xp 00000000 fd:00 805311255 /usr/lib64/libnssutil3.so 21260:20161209:230000.447 7f810d5b4000-7f810d7b3000 ---p 00026000 fd:00 805311255 /usr/lib64/libnssutil3.so 21260:20161209:230000.447 7f810d7b3000-7f810d7b9000 r--p 00025000 fd:00 805311255 /usr/lib64/libnssutil3.so 21260:20161209:230000.447 7f810d7b9000-7f810d7ba000 rw-p 0002b000 fd:00 805311255 /usr/lib64/libnssutil3.so 21260:20161209:230000.447 7f810d7ba000-7f810d8d8000 r-xp 00000000 fd:00 805407415 /usr/lib64/libnss3.so 21260:20161209:230000.447 7f810d8d8000-7f810dad7000 ---p 0011e000 fd:00 805407415 /usr/lib64/libnss3.so 21260:20161209:230000.447 7f810dad7000-7f810dadc000 r--p 0011d000 fd:00 805407415 /usr/lib64/libnss3.so 21260:20161209:230000.447 7f810dadc000-7f810dade000 rw-p 00122000 fd:00 805407415 /usr/lib64/libnss3.so 21260:20161209:230000.447 7f810dade000-7f810dae0000 rw-p 00000000 00:00 0 21260:20161209:230000.448 7f810dae0000-7f810db04000 r-xp 00000000 fd:00 805310904 /usr/lib64/libsmime3.so 21260:20161209:230000.448 7f810db04000-7f810dd03000 ---p 00024000 fd:00 805310904 /usr/lib64/libsmime3.so 21260:20161209:230000.448 7f810dd03000-7f810dd06000 r--p 00023000 fd:00 805310904 /usr/lib64/libsmime3.so 21260:20161209:230000.448 7f810db04000-7f810dd03000 ---p 00024000 fd:00 805310904 /usr/lib64/libsmime3.so 21260:20161209:230000.448 7f810dd03000-7f810dd06000 r--p 00023000 fd:00 805310904 /usr/lib64/libsmime3.so 21260:20161209:230000.448 7f810dd06000-7f810dd07000 rw-p 00026000 fd:00 805310904 /usr/lib64/libsmime3.so 21260:20161209:230000.448 7f810dd07000-7f810dd45000 r-xp 00000000 fd:00 805310905 /usr/lib64/libssl3.so 21260:20161209:230000.448 7f810dd45000-7f810df44000 ---p 0003e000 fd:00 805310905 /usr/lib64/libssl3.so 21260:20161209:230000.448 7f810df44000-7f810df48000 r--p 0003d000 fd:00 805310905 /usr/lib64/libssl3.so 21260:20161209:230000.448 7f810df48000-7f810df49000 rw-p 00041000 fd:00 805310905 /usr/lib64/libssl3.so 21260:20161209:230000.448 7f810df49000-7f810df4a000 rw-p 00000000 00:00 0 21260:20161209:230000.448 7f810df4a000-7f810df66000 r-xp 00000000 fd:00 805309327 /usr/lib64/libsasl2.so.3.0.0 21260:20161209:230000.448 7f810df66000-7f810e165000 ---p 0001c000 fd:00 805309327 /usr/lib64/libsasl2.so.3.0.0 21260:20161209:230000.448 7f810e165000-7f810e166000 r--p 0001b000 fd:00 805309327 /usr/lib64/libsasl2.so.3.0.0 21260:20161209:230000.448 7f810e166000-7f810e167000 rw-p 0001c000 fd:00 805309327 /usr/lib64/libsasl2.so.3.0.0 21260:20161209:230000.448 7f810e167000-7f810e196000 r-xp 00000000 fd:00 805309899 /usr/lib64/libk5crypto.so.3.1 21260:20161209:230000.448 7f810e196000-7f810e395000 ---p 0002f000 fd:00 805309899 /usr/lib64/libk5crypto.so.3.1 21260:20161209:230000.448 7f810e395000-7f810e397000 r--p 0002e000 fd:00 805309899 /usr/lib64/libk5crypto.so.3.1 21260:20161209:230000.448 7f810e397000-7f810e398000 rw-p 00030000 fd:00 805309899 /usr/lib64/libk5crypto.so.3.1 21260:20161209:230000.448 7f810e398000-7f810e399000 rw-p 00000000 00:00 0 21260:20161209:230000.448 7f810e399000-7f810e39c000 r-xp 00000000 fd:00 805308783 /usr/lib64/libcom_err.so.2.1 21260:20161209:230000.448 7f810e39c000-7f810e59b000 ---p 00003000 fd:00 805308783 /usr/lib64/libcom_err.so.2.1 21260:20161209:230000.448 7f810e59b000-7f810e59c000 r--p 00002000 fd:00 805308783 /usr/lib64/libcom_err.so.2.1 21260:20161209:230000.448 7f810e59c000-7f810e59d000 rw-p 00003000 fd:00 805308783 /usr/lib64/libcom_err.so.2.1 21260:20161209:230000.448 7f810e59d000-7f810e672000 r-xp 00000000 fd:00 805309909 /usr/lib64/libkrb5.so.3.3 21260:20161209:230000.448 7f810e672000-7f810e872000 ---p 000d5000 fd:00 805309909 /usr/lib64/libkrb5.so.3.3 21260:20161209:230000.448 7f810e872000-7f810e87f000 r--p 000d5000 fd:00 805309909 /usr/lib64/libkrb5.so.3.3 21260:20161209:230000.448 7f810e87f000-7f810e882000 rw-p 000e2000 fd:00 805309909 /usr/lib64/libkrb5.so.3.3 21260:20161209:230000.448 7f810e882000-7f810e8cb000 r-xp 00000000 fd:00 805309895 /usr/lib64/libgssapi_krb5.so.2.2 21260:20161209:230000.448 7f810e8cb000-7f810eacb000 ---p 00049000 fd:00 805309895 /usr/lib64/libgssapi_krb5.so.2.2 21260:20161209:230000.448 7f810eacb000-7f810eacc000 r--p 00049000 fd:00 805309895 /usr/lib64/libgssapi_krb5.so.2.2 21260:20161209:230000.448 7f810eacc000-7f810eace000 rw-p 0004a000 fd:00 805309895 /usr/lib64/libgssapi_krb5.so.2.2 21260:20161209:230000.448 7f810eace000-7f810ead5000 r-xp 00000000 fd:00 805309509 /usr/lib64/libgdbm.so.4.0.0 21260:20161209:230000.448 7f810ead5000-7f810ecd5000 ---p 00007000 fd:00 805309509 /usr/lib64/libgdbm.so.4.0.0 21260:20161209:230000.448 7f810ecd5000-7f810ecd6000 r--p 00007000 fd:00 805309509 /usr/lib64/libgdbm.so.4.0.0 21260:20161209:230000.448 7f810ecd6000-7f810ecd7000 rw-p 00008000 fd:00 805309509 /usr/lib64/libgdbm.so.4.0.0 21260:20161209:230000.448 7f810ecd7000-7f810ecdf000 r-xp 00000000 fd:00 805503563 /usr/lib64/libOpenIPMIutils.so.0.0.1 21260:20161209:230000.448 7f810ecdf000-7f810eede000 ---p 00008000 fd:00 805503563 /usr/lib64/libOpenIPMIutils.so.0.0.1 21260:20161209:230000.448 7f810eede000-7f810eedf000 r--p 00007000 fd:00 805503563 /usr/lib64/libOpenIPMIutils.so.0.0.1 21260:20161209:230000.449 7f810eedf000-7f810eee0000 rw-p 00008000 fd:00 805503563 /usr/lib64/libOpenIPMIutils.so.0.0.1 ... 21260:20161209:230000.451 7f8112a16000-7f8112a23000 r-xp 00000000 fd:00 805448382 /usr/lib64/libiksemel.so.3.1.1 21260:20161209:230000.451 7f8112a23000-7f8112c22000 ---p 0000d000 fd:00 805448382 /usr/lib64/libiksemel.so.3.1.1 21260:20161209:230000.451 7f8112c22000-7f8112c23000 r--p 0000c000 fd:00 805448382 /usr/lib64/libiksemel.so.3.1.1 21260:20161209:230000.452 7f8112c23000-7f8112c24000 rw-p 0000d000 fd:00 805448382 /usr/lib64/libiksemel.so.3.1.1 21260:20161209:230000.452 7f8112c24000-7f8112f43000 r-xp 00000000 fd:00 806489619 /usr/lib64/libmysqlclient.so.18.0.0 21260:20161209:230000.452 7f8112f43000-7f8113143000 ---p 0031f000 fd:00 806489619 /usr/lib64/libmysqlclient.so.18.0.0 21260:20161209:230000.452 7f8113143000-7f811314b000 r--p 0031f000 fd:00 806489619 /usr/lib64/libmysqlclient.so.18.0.0 21260:20161209:230000.452 7f811314b000-7f81131be000 rw-p 00327000 fd:00 806489619 /usr/lib64/libmysqlclient.so.18.0.0 21260:20161209:230000.452 7f81131be000-7f81131c8000 rw-p 00000000 00:00 0 21260:20161209:230000.452 7f81131c8000-7f81131e9000 r-xp 00000000 fd:00 805307863 /usr/lib64/ld-2.17.so 21260:20161209:230000.452 7f81133a0000-7f81133a2000 rw-p 00000000 00:00 0 21260:20161209:230000.452 7f81133a2000-7f81133ca000 rw-s 00000000 00:04 3637254 /SYSV530005c7 (deleted) 21260:20161209:230000.452 7f81133ca000-7f81133cb000 rw-p 00000000 00:00 0 21260:20161209:230000.452 7f81133cb000-7f81133e9000 rw-p 00000000 00:00 0 21260:20161209:230000.452 7f81133e9000-7f81133ea000 r--p 00021000 fd:00 805307863 /usr/lib64/ld-2.17.so 21260:20161209:230000.452 7f81133ea000-7f81133eb000 rw-p 00022000 fd:00 805307863 /usr/lib64/ld-2.17.so 21260:20161209:230000.452 7f81133eb000-7f81133ec000 rw-p 00000000 00:00 0 21260:20161209:230000.452 7ffc3ce31000-7ffc3d0f3000 rw-p 00000000 00:00 0 [stack] 21260:20161209:230000.452 7ffc3d14d000-7ffc3d14f000 r-xp 00000000 00:00 0 [vdso] 21260:20161209:230000.452 ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] 21260:20161209:230000.452 ================================ 21260:20161209:230000.452 Please consider attaching a disassembly listing to your bug report. 21260:20161209:230000.452 This listing can be produced with, e.g., objdump -DSswx zabbix_server. 21260:20161209:230000.452 ================================ 21041:20161209:230000.455 One child process died (PID:21260,exitcode/signal:1). Exiting ... 21041:20161209:230002.905 Zabbix Server stopped. Zabbix 3.2.2 (revision 64245). 22283:20161209:230013.237 Starting Zabbix Server. Zabbix 3.2.2 (revision 64245). |
Comment by Oleksii Zagorskyi [ 2016 Dec 10 ] |
Previous comment looks like duplicate of |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Feb 03 ] |
So, anyone who is experiencing this problem, how do you launch Zabbix? I mean from which directory you run a launch command, where is configuration file situated, how many daemons run at the same time, do you launch them with independent configuration files or do you edit one file between launch commands, maybe you generate configuration file on the fly and remove it once daemon has started. More details is better! |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 03 ] |
We are facing this problem in our environment. [Unit] Description = Zabbix Server After = syslog.target network.target mysqld.service [Service] Type = oneshot ExecStart = / usr / local / sbin / zabbix_server -c /usr/local/etc/zabbix_server.conf ExecReload = / usr / local / sbin / zabbix_server -R config_cache_reload RemainAfterExit = yes PIDFile = / tmp / zabbix_server.pid [Install] WantedBy = multi-user.target errors in zabbix_server, in zabbix_proxy and zabbix_agent
glebs.ivanovskis Errm... Running zabbix_server with -R but without -c if you started it with -c is not a good idea. Run time control relies on pid of main process stored in pid file and path to pid file is taken from PidFile parameter in configuration file. Without -c Zabbix will search for default configuration file in default path... |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 03 ] |
Hummm, thanks for the tip... but, this conf or parameter can create these errors? |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Feb 03 ] |
Maybe. I'm trying to figure it out right now. Configuration file path is involved in generation of the key for semaphore array (this is the first column of ipcs -s output). If the path does not exist or we don't have enough permissions to use it, we generate key from '.' as a backup plan. In this case there should be this error in the log: cannot create IPC key for path '</path/to/config>', try to create for path '.': <error description> Generating keys from '.' isn't really good because we will end up with key conflict and all Zabbix daemons will compete for the same semaphores. |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 03 ] |
hummm, but, this parameter, is only in Reload state: ExecReload = / usr / local / sbin / zabbix_server -R config_cache_reload Even then, may be generating these errors? Even without reloading services? The Parameter that I should use then, can be: ExecReload = / usr / local / sbin / zabbix_server -R config_cache_reload -c /usr/local/etc/zabbix_server.conf Because, the path /usr/local/etc/zabbix_server.conf, is the default path!!! It does not make sense, repeat the path to the configuration file. |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 03 ] |
Feedback: I removed parameter -c and the command line reload in the systemctl script, so far, no errors appeared. The script looks like this: [Unit] Description = Zabbix Server After = syslog.target network.target mysqld.service [Service] Type = oneshot ExecStart = / usr / local / sbin / zabbix_server RemainAfterExit = yes PIDFile = / tmp / zabbix_server.pid [Install] WantedBy = multi-user.target I can not say that it has completely resolved, but I will monitor if it happens again. |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Feb 05 ] |
I think it is a bit bad luck on top of misuse of ftok() from our side that causes the crash. If you compare the first column (semaphore key) of output of this command: $ ipcs -s | grep zabbix with the output of $ printf '%0.2x%0.4x\n' $(($(stat -c '%d' /path/to/config_file) & 0xff)) $(($(stat -c '%i' /path/to/config_file) & 0xffff)) you will see that the only difference is 0x7a in the beginning. This is ASCII code of 'z' and is common for all Zabbix daemons. If you have multiple Zabbix daemons on one system with configuration files on the same drive and if configuration file inode numbers share 16 lower bits - you are in trouble - your Zabbix daemons will compete for the same semaphore set. Problem has nothing to do with system-wide limits on semaphore set number and semaphore number in semaphore set mentioned above. |
Comment by richlv [ 2017 Feb 06 ] |
as far as i recall, most of these were observed during the zabbix training sessions when trainees were starting up 3 zabbix proxies on the same machine - and yes, running them with the wrong -c parameter was very likely to happen at some time. |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Feb 06 ] |
Wrong config file usually result in a more obvious error. Bind failure due to port conflict, pid file conflict or something like that. Key conflict may happen even if daemons have separate config files. If you are interested, probability of this happening is given by this formula (assuming random inode numbers): 65536 - 1 65536 - 2 65536 - (N - 1) P = 1 - --------- * --------- * ... * --------------- 65536 65536 65536 where N is a number of Zabbix daemons. With N = 2 it is negligible 1/65536 but with N = 10 it comes close to a quite noticeable 0.1%. Plus you need to multiply it by number of students |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 06 ] |
The problem came back again... glebs.ivanovskis Can you please check config file inode numbers (for all Zabbix daemons you have on the system) as I described above to prove or disprove my hypothesis? |
Comment by richlv [ 2017 Feb 06 ] |
ah, thanks a lot for the formula i now recall some users in their prod environments seeing this issue, and they were running 2 or 3 agents on some boxes. i guess restarting agents every now and then makes sure it will happen at some point. |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 10 ] |
@Glebs Ivanovskis Can you please check config file inode numbers (for all Zabbix daemons you have on the system) as I described above to prove or disprove my hypothesis? Sorry for the delay, i have been sick last week, this is the result: [zabbix@srvdfzbxpdf01 ~]$ ipcs -s | grep zabbix 0x7a00e54d 294917 zabbix 600 13 0x7a00f0a5 327686 zabbix 600 13 [zabbix@srvdfzbxpdf01 ~]$ printf '%0.2x%0.4x\n' $(($(stat -c '%d' /usr/local/etc/zabbix_server.conf) & 0xff)) $(($(stat -c '%i' /usr/local/etc/zabbix_server.conf) & 0xffff)) 00f0a5 I think you right have any way to fix this? glebs.ivanovskis Please show printf ... for zabbix_agentd.conf and zabbix_proxy.conf. So far we can't decide. |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 10 ] |
glebs.ivanovskis Thank you, for all your patience and help here, we have been working in 2 servers First Server: We have - zabbix_server and zabbix_agent Second Server, we have only zabbix_proxy First server: [root@srvdfzbxpdf02 zabbix]# ipcs -s | grep zabbix 0x7a00e55e 491525 zabbix 600 13 0x7a00e54d 524294 zabbix 600 13 zabbix_server.conf [root@srvdfzbxpdf02 zabbix]# printf '%0.2x%0.4x\n' $(($(stat -c '%d' /usr/local/etc/zabbix_server.conf) & 0xff)) $(($(stat -c '%i' /usr/local/etc/zabbix_server.conf) & 0xffff)) 00196a zabbix_agentd.conf [root@srvdfzbxpdf02 zabbix]# printf '%0.2x%0.4x\n' $(($(stat -c '%d' /usr/local/etc/zabbix_agentd.conf) & 0xff)) $(($(stat -c '%i' /usr/local/etc/zabbix_agentd.conf) & 0xffff)) 00e54d Second Server: [root@srvpxzbxpdf04 zabbix]# ipcs -s | grep zabbix 0x7a003415 98304 zabbix 600 13 [root@srvpxzbxpdf04 zabbix]# printf '%0.2x%0.4x\n' $(($(stat -c '%d' /usr/local/etc/zabbix_proxy.conf) & 0xff)) $(($(stat -c '%i' /usr/local/etc/zabbix_proxy.conf) & 0xffff)) 003415 |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Feb 10 ] |
Looks good, no conflicting inode numbers. Don't worry that printf ... matches ipcs ... output, that's how Zabbix works currently. I guess everything is working fine for you now, isn't it? |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 10 ] |
No, in both Servers, in log files, i have this errors: 4039:20170210:101250.615 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4053:20170210:101250.615 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4054:20170210:101250.616 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4055:20170210:101250.617 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4056:20170210:101250.618 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4058:20170210:101250.619 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4057:20170210:101250.619 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4059:20170210:101250.620 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4060:20170210:101250.621 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4062:20170210:101250.621 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4061:20170210:101250.622 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4064:20170210:101250.622 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4063:20170210:101250.623 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4066:20170210:101250.623 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4065:20170210:101250.624 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4068:20170210:101250.624 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4067:20170210:101250.625 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4069:20170210:101250.625 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4070:20170210:101250.626 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4071:20170210:101250.626 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4074:20170210:101250.626 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4072:20170210:101250.627 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4075:20170210:101250.627 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4073:20170210:101250.628 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4076:20170210:101250.628 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4077:20170210:101250.629 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4080:20170210:101250.629 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4078:20170210:101250.630 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4082:20170210:101250.630 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4079:20170210:101250.631 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4083:20170210:101250.631 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4081:20170210:101250.632 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4084:20170210:101250.632 Got signal [signal:15(SIGTERM),sender_pid:1,sender_uid:0,reason:0]. Exiting ... 4039:20170210:101252.623 syncing history data... 4039:20170210:101252.623 syncing history data done 4039:20170210:101252.623 syncing trend data... 4039:20170210:101252.637 syncing trend data done 4039:20170210:101252.637 Zabbix Server stopped. Zabbix 3.2.3 (revision 64610). glebs.ivanovskis This looks like normal Zabbix termination. No lock failed ... errors? Théo: Sorry, i Posted wrong log. |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 23 ] |
I deleted the whole environment. I have not changed any server configuration, Red Hat Rhel 7, and even then, the error persists. I do not know what to do anymore. 1303:20170222:173942.979 housekeeper [deleted 267404 hist/trends, 0 items, 0 events, 3 problems, 0 sessions, 0 alarms, 0 audit items in 13.121322 sec, idle for 1 hour(s)] zabbix_server [1322]: [file:'selfmon.c',line:380] lock failed: [22] Invalid argument zabbix_server [1213]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1297]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1296]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1299]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1298]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1303]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1300]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1312]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1307]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1316]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1311]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1290]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1315]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1294]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1295]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1323]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1289]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1293]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1291]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1306]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1308]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1302]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1317]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1292]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1288]: [file:'log.c',line:238] lock failed: [22] Invalid argument zabbix_server [1318]: [file:'log.c',line:238] lock failed: [22] Invalid argument Could it be some error related to Rhel 7? |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Feb 23 ] |
Dear theocastelo, have you checked config file inodes after re-installation? Since you mentioned RHEL 6/7, your issue may be more related to |
Comment by Théo Castelo N. de Araújo [ 2017 Feb 23 ] |
glebs.ivanovskis I think this is it. after changing the REMOVE_IPC parameter, now working fine... Probably the problem was caused by the application user be the same used for ssh access, when we log off, the error happened. Thank you so much |
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jul 25 ] |
Situation must be much better in 3.4 after implementation of |