[ZBX-12466] Zabbix Server 3.2.6 crash Created: 2017 Aug 04  Updated: 2024 Apr 10  Resolved: 2018 Aug 10

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Server (S)
Affects Version/s: 3.2.6
Fix Version/s: None

Type: Problem report Priority: Critical
Reporter: seregasoft Assignee: Unassigned
Resolution: Cannot Reproduce Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

FreeBSD 11.0-RELEASE-p1


Attachments: File checks_ipmi_ZBX-12466.c     File config.log     File zabbix_server.log_1     File zabbix_server1.log    
Team: Team A
Sprint: Sprint 14, Sprint 15, Sprint 16, Sprint 17, Sprint 18, Sprint 19, Sprint 21, Sprint 22, Sprint 23, Sprint 24, Sprint 25, Sprint 26
Story Points: 0.5

 Description   

After update from v 2.4.8 to 3.2.6 server works unstably. Crashing within 1-3 hours after start.

 1802:20170802:145825.054 Got signal [signal:11(SIGSEGV),reason:1,refaddr:0x9f5e69380]. Crashing ...
  1802:20170802:145825.054 ====== Fatal information: ======
  1802:20170802:145825.054 program counter not available for this architecture
  1802:20170802:145825.054 === Registers: ===
  1802:20170802:145825.055 register dump not available for this architecture
  1802:20170802:145825.055 === Backtrace: ===
  1802:20170802:145825.056 3: 0x48d426 <print_fatal_info+0x86> at /usr/local/sbin/zabbix_server
  1802:20170802:145825.056 2: 0x48d8ce <zbx_set_common_signal_handlers+0x32e> at /usr/local/sbin/zabbix_server
  1802:20170802:145825.056 1: 0x803e3379d <pthread_sigmask+0x50d> at /lib/libthr.so.3
  1802:20170802:145825.056 0: 0x803e32d6f <pthread_getspecific+0xe9f> at /lib/libthr.so.3
  1802:20170802:145825.056 === Memory map: ===
  1802:20170802:145825.056 memory map not available for this platform
  1802:20170802:145825.056 ================================
   705:20170802:145825.058 One child process died (PID:1802,exitcode/signal:1). Exiting ...
   705:20170802:145827.122 syncing history data...
   705:20170802:145827.122 syncing history data done
   705:20170802:145827.122 syncing trend data...
   705:20170802:145833.821 syncing trend data done
   705:20170802:145833.821 Zabbix Server stopped. Zabbix 3.2.6 (revision 67849).

Install from /usr/ports/net-mgmt/zabbix32-server



 Comments   
Comment by Vladislavs Sokurenko [ 2017 Aug 04 ]

1802, which process is it ? It should print info in the start.
once you find out process then increase log level of that process and attach log please.
Do you use unixODBC ?

Comment by Glebs Ivanovskis (Inactive) [ 2017 Aug 04 ]

...or NetSNMP?

Comment by seregasoft [ 2017 Aug 04 ]

Looks like problem in "unreachable poller", please see log.
I'm use MySQL for Zabbix server and SNMP, IPMI items.

Comment by Andris Mednis [ 2017 Aug 04 ]

Seems like ZBX-10940, ZBX-11697 but on FreeBSD.
Can you attach full log from process 85691 ? (E.g. "grep 85691 zabbix_server.log > zabbix_server85691.log") ?
Disabling of IPMI items will prevent this crash (just a workaround).
This is a tough bug, it was never reproduced here. Can You compile server from source code, if I send special patches to study this crash ?

Comment by seregasoft [ 2017 Aug 07 ]

"grep 85691" added https://support.zabbix.com/secure/attachment/52452/zabbix_server.log_2017_08_04_26
Please send a patch with the compile instructions.
I have not turned off IPMI itesm yet. I made a script that checks the work of the zabbx_server, if there are no processes, it starts zabbx_server

Comment by Andris Mednis [ 2017 Aug 07 ]

Thanks for log ! Looking into it. It would be easier to analyze it if hosts were obfuscated like SOME_HOST1, SOME_HOST2 etc. so we can see them different, not just SOME_HOST everywhere.

Comment by seregasoft [ 2017 Aug 08 ]

ok, i'm attach https://support.zabbix.com/secure/attachment/52484/zabbix_server.log_2017_08_04_HOSTS with different HOSTs

Comment by Andris Mednis [ 2017 Aug 08 ]

Thanks, much better. Got a copy, now you can remove both large attachments.

Comment by Andris Mednis [ 2017 Aug 18 ]

Unreachable poller log file supplied by seregasoft shows that:
1) Connections to IPMI hosts come up and go down, OpenIPMI library invokes our callback function zbx_connection_change_cb() every time.
2) If a connection goes down, sometimes zbx_connection_change_cb() succeeds, sometimes it fails with "[60] Operation timed out", sometimes it fails with "[16777411] Unknown error: 16777411".
3) If zbx_connection_change_cb() succeeds or fails with "[60] Operation timed out", then things go well - ipmi_domain_close() is called, sensors and entities are deleted and domain is closed.
4) Problems arise if zbx_connection_change_cb() fails with "[16777411] Unknown error: 16777411". For unknown reasons zbx_connection_change_cb() is occasionally called with a valid 'cb_data' argument value, but invalid (from Zabbix point of view) 'domain' argument value. This invalid 'domain' value is then passed by zbx_connection_change_cb() into ipmi_domain_close() and a crash may follow.

Attached file 'checks_ipmi_ZBX-12466.c' is intended to replace the original Zabbix 3.2.6 src/zabbix_server/poller/checks_ipmi.c file. The attached file contains additional debug logging and an experimental protection. The protection is simple: when a new domain is created by OpenIPMI library, it is assigned a new pointer to domain. This new pointer is stored in Zabbix agent in zbx_ipmi_host_t structure as original domain pointer issued by ipmi_open_domain(). When connection goes down, our zbx_connection_change_cb() checks is its 'domain' argument value equal to the original pointer. If values match, domain closing proceeds. If values differ, then ipmi_domain_close() is not called to (hopefully) avoid crash.

Comment by Andris Mednis [ 2017 Aug 18 ]

Example how to compile Zabbix 3.2.6 with modified checks_ipmi.c

Choose your work directory and go to it:

$ cd your_work_directory

Check out source code:

$ svn co svn://svn.zabbix.com/tags/3.2.6 zabbix_326
$ cd zabbix_326

Make a backup of checks_ipmi.c.

$ cp src/zabbix_server/poller/checks_ipmi.c src/zabbix_server/poller/checks_ipmi.c.ORG

Replace 'checks_ipmi.c' file with a modified one from Jira attachment:

$ wget -O src/zabbix_server/poller/checks_ipmi.c https://support.zabbix.com/secure/attachment/52714/checks_ipmi_ZBX-12466.c

Compile (adjust configure options as necessary):

$ ./bootstrap.sh
$ CFLAGS="-g -O2 -D_FORTIFY_SOURCE=2" ./configure --enable-server --enable-proxy --enable-agent --enable-ipv6 \
      --with-net-snmp --with-unixodbc --with-libxml2 --with-libcurl --with-ssh2 \
      --with-openipmi --with-ldap --with-postgresql --with-openssl --prefix=`pwd` 2>&1 | tee my_configure.out
$ make dbschema
$ make install 2>&1 | tee my_make_install.out

Start Zabbix server, for example:

$ sbin/zabbix_server -c /path_to_your_config_file/zabbix_server.conf
Comment by Andris Mednis [ 2017 Aug 18 ]

Hi, Sergey!
A brief instruction how to compile a modified Zabbix is in the previous comment. Let me know if something is missing.

Comment by seregasoft [ 2017 Aug 31 ]

Sorry for long response.
After run

CFLAGS="-g -O2 -D_FORTIFY_SOURCE=2" ./configure --enable-server --enable-proxy --enable-agent --enable-ipv6 \
      --with-net-snmp --with-unixodbc --with-libxml2 --with-libcurl --with-ssh2 \
      --with-openipmi --with-ldap --with-mysql --with-fping --with-javagw --with-openssl --prefix=`pwd` 2>&1 | tee my_configure.out

i get:

checking for ICONV support... configure: error: Unable to use iconv (libiconv check failed)
Comment by Andris Mednis [ 2017 Sep 04 ]

Are 'iconv' header files installed ? On my FreeBSD 11 machine:

$ find /usr/include/ -name '*iconv*'
/usr/include/iconv.h
/usr/include/sys/iconv.h

and libraries:

$ find /lib -name '*iconv*'
/lib/libkiconv.so.4
$ find /usr/lib -name '*iconv*'
/usr/lib/libkiconv.so
/usr/lib/i18n/libiconv_std.so
/usr/lib/i18n/libiconv_std.so.4
/usr/lib/i18n/libiconv_none.so
/usr/lib/i18n/libiconv_none.so.4
/usr/lib/libkiconv.a
/usr/lib/libkiconv_p.a
Comment by seregasoft [ 2017 Sep 05 ]

Yes,

find /usr/include/ -name '*iconv*'
/usr/include/iconv.h
/usr/include/sys/iconv.h

find /lib -name '*iconv*'
/lib/libkiconv.so.4

find /usr/lib -name '*iconv*'
/usr/lib/i18n/libiconv_none.so.4
/usr/lib/i18n/libiconv_none.so
/usr/lib/i18n/libiconv_std.so.4
/usr/lib/i18n/libiconv_std.so
/usr/lib/libkiconv_p.a
/usr/lib/libkiconv.so
/usr/lib/libkiconv.a
Comment by Andris Mednis [ 2017 Sep 05 ]

Can you put in attachment 'config.log' file - it is produced while running "./configure" script, so we can look why 'iconv' is not found ?

Comment by seregasoft [ 2017 Sep 06 ]

I attached config.log file

Comment by Andris Mednis [ 2017 Sep 06 ]

Thanks for log, I need to investigate it.

Comment by Andris Mednis [ 2017 Sep 07 ]

Do you have pkgconf or pkg-config package installed ?
On my machine:

$ pkg info pkgconf
pkgconf-1.3.7,1
Name           : pkgconf
Version        : 1.3.7,1
Installed on   : Mon Sep  4 19:27:19 2017 EEST
Origin         : devel/pkgconf
Architecture   : FreeBSD:11:amd64
Prefix         : /usr/local
Categories     : devel
Licenses       : ISCL
Maintainer     : [email protected]
WWW            : https://github.com/pkgconf/pkgconf
Comment        : Utility to help to configure compiler and linker flags
Shared Libs provided:
	libpkgconf.so.2
Annotations    :
	repo_type      : binary
	repository     : FreeBSD
Flat size      : 205KiB
Description    :
pkgconf is a program which helps to configure compiler and linker flags for
development frameworks. It is similar to pkg-config, but was written from
scratch in Summer of 2011 to replace pkg-config, which now needs itself to
build itself.
....
Comment by seregasoft [ 2017 Sep 08 ]

Yes, pkgconf is installed

$ pkg info pkgconf
pkgconf-1.3.7,1
Name           : pkgconf
Version        : 1.3.7,1
Installed on   : Wed Aug  2 12:11:30 2017 MSK
Origin         : devel/pkgconf
Architecture   : FreeBSD:11:amd64
Prefix         : /usr/local
Categories     : devel
Licenses       : ISCL
Maintainer     : [email protected]
WWW            : https://github.com/pkgconf/pkgconf
Comment        : Utility to help to configure compiler and linker flags
Shared Libs provided:
        libpkgconf.so.2
Annotations    :
Flat size      : 205KiB
Description    :
pkgconf is a program which helps to configure compiler and linker flags for
development frameworks. It is similar to pkg-config, but was written from
scratch in Summer of 2011 to replace pkg-config, which now needs itself to
build itself.

WWW: https://github.com/pkgconf/pkgconf
Comment by Andris Mednis [ 2017 Sep 14 ]

Can you upload attachment with list of installed packages - "pkg info" output ?

Comment by seregasoft [ 2017 Sep 14 ]
pkg info
alsa-lib-1.1.2                 ALSA compatibility library
apache24-2.4.27                Version 2.4.x of Apache web server
apcupsd-3.14.14_2              Set of programs for controlling APC UPS
apr-1.6.2.1.6.0                Apache Portability Library
autoconf-2.69_1                Automatically configure source code on many Un*x platforms
autoconf-wrapper-20131203      Wrapper script for GNU autoconf
automake-1.15.1                GNU Standards-compliant Makefile generator
automake-wrapper-20131203      Wrapper script for GNU automake
avahi-app-0.6.31_5             Service discovery on a local network
bash-4.4.12_2                  GNU Project's Bourne Again SHell
bigreqsproto-1.1.2             BigReqs extension headers
binutils-2.28,1                GNU binary tools
bison-3.0.4,1                  Parser generator from FSF, (mostly) compatible with Yacc
bootstrap-openjdk-r351880_1    Oracle\'s Java 6 virtual machine release under the GPL v2
ca_root_nss-3.32               Root certificate bundle from the Mozilla Project
cairo-1.14.8_1,2               Vector graphics library with cross-device output support
check-0.11.0                   Unit test framework for C
cmake-3.8.2                    Cross-platform Makefile generator
cmake-modules-3.8.2            Modules and Templates for CMake
compositeproto-0.4.2           Composite extension headers
convmv-2.03                    Convert filenames from one encoding to another
cups-2.2.3                     Common UNIX Printing System
curl-7.54.1                    Command line tool and library for transferring data with URLs
cvs-1.20120905_4               Version control system
damageproto-1.2.1              Damage extension headers
db5-5.3.28_6                   Oracle Berkeley DB, revision 5.3
dbus-1.10.16_1                 Message bus system for inter-application communication
dbus-glib-0.108                GLib bindings for the D-BUS messaging system
dejavu-2.37                    Bitstream Vera Fonts clone with a wider range of characters
dhcping-1.2                    Send DHCP request to DHCP server for monitoring purposes
dialog4ports-0.1.6             Console Interface to configure ports
dmidecode-3.1_1                Tool for dumping DMI (SMBIOS) contents in human-readable format
docbook-1.5                    Meta-port for the different versions of the DocBook DTD
docbook-sgml-4.5_1             DocBook SGML DTD
docbook-xml-5.0_3              DocBook XML DTD
docbook-xsl-1.76.1_3           XSL DocBook stylesheets
dri2proto-2.8                  DRI2 prototype headers
dri3proto-1.0                  DRI3 prototype headers
expat-2.2.1                    XML 1.0 parser written in C
fixesproto-5.0                 Fixes extension headers
font-util-1.3.1                Create an index of X font files in a directory
fontconfig-2.12.1,1            XML-based font configuration API for X Windows
fontsproto-2.1.3,1             Fonts extension headers
fping-3.16                     Quickly ping multiple hosts without flooding the network
freetype2-2.8                  Free and portable TrueType font rendering engine
gdbm-1.13_1                    GNU database manager
gettext-runtime-0.19.8.1_1     GNU gettext runtime libraries and programs
gettext-tools-0.19.8.1         GNU gettext development and translation tools
glib-2.50.2_4,1                Some useful routines of C programming (current stable version)
glproto-1.4.17                 GLX extension headers
gmake-4.2.1_1                  GNU version of 'make' utility
gmp-6.1.2                      Free library for arbitrary precision arithmetic
gnome_subr-1.0                 Common startup and shutdown subroutines used by GNOME scripts
gnutls-3.5.13                  GNU Transport Layer Security library
gobject-introspection-1.50.0,1 Generate interface introspection data for GObject libraries
help2man-1.47.4                Automatically generating simple manual pages from program output
html2text-1.3.2a               Converts HTML documents into plain text
indexinfo-0.2.6                Utility to regenerate the GNU info page index
inputproto-2.3.2               Input extension headers
intltool-0.51.0_1              Tools to internationalize various kinds of data files
ipmitool-1.8.18_1              CLI to manage IPMI systems
iso8879-1986_3                 Character entity sets from ISO 8879:1986 (SGML)
java-zoneinfo-2017.b           Updated Java timezone definitions
javavmwrapper-2.5_2            Wrapper script for various Java Virtual Machines
jpeg-turbo-1.5.1               SIMD-accelerated JPEG codec which replaces libjpeg
jsoncpp-1.8.1                  JSON reader and writer library for C++
kbproto-1.0.7                  KB extension headers
ldb-1.1.29_1                   LDAP-like embedded database
libICE-1.0.9_1,1               Inter Client Exchange library for X11
libSM-1.2.2_3,1                Session Management library for X11
libX11-1.6.5,1                 X11 library
libXau-1.0.8_3                 Authentication Protocol library for X11
libXaw-1.0.13,2                X Athena Widgets library
libXdamage-1.1.4_3             X Damage extension library
libXdmcp-1.1.2                 X Display Manager Control Protocol library
libXext-1.3.3_1,1              X11 Extension library
libXfixes-5.0.3                X Fixes extension library
libXfont-1.5.2,2               X font library
libXfont2-2.0.1                X font library
libXi-1.7.9,1                  X Input extension library
libXmu-1.1.2_3,1               X Miscellaneous Utilities libraries
libXp-1.0.3,1                  X print library
libXpm-3.5.12                  X Pixmap library
libXrender-0.9.10              X Render extension library
libXt-1.1.5,1                  X Toolkit library
libXtst-1.2.3                  X Test extension
libXv-1.0.11,1                 X Video Extension library
libXvMC-1.0.10                 X Video Extension Motion Compensation library
libXxf86vm-1.1.4_1             X Vidmode Extension
libarchive-3.3.1,1             Library to create and read several streaming archive formats
libclc-0.3.0.20170225          Required library functions for OpenCL C programming language
libdaemon-0.14_1               Lightweight C library that eases the writing of UNIX daemons
libdrm-2.4.82,1                Userspace interface to kernel Direct Rendering Module services
libedit-3.1.20170329_2,1       Command line editor library
libepoxy-1.4.3                 Library to handle OpenGL function pointer management
libevent-2.1.8                 API for executing callback functions on events or timeouts
libffi-3.2.1                   Foreign Function Interface
libfontenc-1.1.3_1             The fontenc Library
libgcrypt-1.8.0                General purpose crypto library based on code used in GnuPG
libgpg-error-1.27              Common error values for all GnuPG components
libiconv-1.14_10               Character set conversion library
libidn2-2.0.3                  Implementation of IDNA2008 internationalized domain names
libinotify-20170201            Kevent based inotify compatible library
liblz4-1.7.5,1                 LZ4 compression library, lossless and very fast
libnghttp2-1.24.0              HTTP/2.0 C Library
libpaper-1.1.24.4              Library providing routines for paper size management
libpciaccess-0.13.5            Generic PCI access library
libpthread-stubs-0.4           This library provides weak aliases for pthread functions
libssh2-1.8.0,3                Library implementing the SSH2 protocol
libsunacl-1.0                  Wrapper providing SunOS NFSv4 ACL API
libtasn1-4.12                  ASN.1 structure parser library
libtool-2.4.6                  Generic shared library support script
libunistring-0.9.7             Unicode string library
libunwind-20170113_1           Generic stack unwinding library
libuv-1.13.1                   Multi-platform support library with a focus on asynchronous I/O
libxcb-1.12_2                  The X protocol C-language Binding (XCB) library
libxkbfile-1.0.9               XKB file library
libxml2-2.9.4                  XML parser library for GNOME
libxshmfence-1.2_2             Shared memory 'SyncFence' synchronization primitive
libxslt-1.1.29_1               The XSLT C library for GNOME
libyaml-0.1.6_2                YAML 1.1 parser and emitter written in C
llvm40-4.0.1                   LLVM and Clang
lua52-5.2.4                    Small, compilable scripting language providing easy access to C code
lzo2-2.10_1                    Portable speedy, lossless data compression library
m4-1.4.18,1                    GNU M4
makedepend-1.0.5,1             Dependency generator for makefiles
mesa-dri-17.1.5                OpenGL hardware acceleration drivers for DRI2+
mesa-libs-17.1.5               OpenGL libraries that support GLX and EGL clients
meson-0.41.1                   High performance build system
minixmlto-0.0.2_1              Minimalistic alternative to xmlto
mkfontdir-1.0.7                Create an index of X font files in a directory
mkfontscale-1.1.2              Creates an index of scalable font files for X
mod_php56-5.6.31               PHP Scripting Language
mpfr-3.1.5_1                   Library for multiple-precision floating-point computations
mysql56-client-5.6.37          Multithreaded SQL database (client)
mysql56-server-5.6.37          Multithreaded SQL database (server)
nasm-2.11.08_1,1               General-purpose multi-platform x86 and amd64 assembler
net-snmp-5.7.3_17              Extendable SNMP implementation
nettle-3.3                     Low-level cryptographic library
ninja-1.7.2,2                  Ninja is a small build system closest in spirit to Make
oniguruma5-5.9.6_1             BSDL Regular Expressions library compatible with POSIX/GNU/Perl
openipmi-2.0.21_1              Complex IPMI management software
openjdk-7.141.02,1             Java Development Kit 7
openldap-client-2.4.45         Open source LDAP client implementation
p11-kit-0.23.7                 Library for loading and enumerating of PKCS#11 modules
p5-Locale-gettext-1.07         Message handling functions
p5-Parse-Yapp-1.05_2           Perl extension for generating and using LALR parsers
p5-XML-Parser-2.44             Perl extension interface to James Clark's XML parser, expat
pbzip2-1.1.13                  Parallel BZIP2
pciids-20170701                Database of all known IDs used in PCI devices
pcre-8.40_1                    Perl Compatible Regular Expressions library
perl5-5.24.2                   Practical Extraction and Report Language
php56-5.6.31                   PHP Scripting Language
php56-bcmath-5.6.31            The bcmath shared extension for php
php56-ctype-5.6.31             The ctype shared extension for php
php56-dom-5.6.31               The dom shared extension for php
php56-extensions-1.0           "meta-port" to install PHP extensions
php56-filter-5.6.31            The filter shared extension for php
php56-gd-5.6.31                The gd shared extension for php
php56-gettext-5.6.31           The gettext shared extension for php
php56-hash-5.6.31              The hash shared extension for php
php56-iconv-5.6.31             The iconv shared extension for php
php56-json-5.6.31              The json shared extension for php
php56-ldap-5.6.31              The ldap shared extension for php
php56-mbstring-5.6.31          The mbstring shared extension for php
php56-mysql-5.6.31             The mysql shared extension for php
php56-mysqli-5.6.31            The mysqli shared extension for php
php56-opcache-5.6.31           The opcache shared extension for php
php56-pdo-5.6.31               The pdo shared extension for php
php56-pdo_sqlite-5.6.31        The pdo_sqlite shared extension for php
php56-phar-5.6.31              The phar shared extension for php
php56-posix-5.6.31             The posix shared extension for php
php56-session-5.6.31           The session shared extension for php
php56-simplexml-5.6.31         The simplexml shared extension for php
php56-snmp-5.6.31              The snmp shared extension for php
php56-sockets-5.6.31           The sockets shared extension for php
php56-sqlite3-5.6.31           The sqlite3 shared extension for php
php56-tokenizer-5.6.31         The tokenizer shared extension for php
php56-xml-5.6.31               The xml shared extension for php
php56-xmlreader-5.6.31         The xmlreader shared extension for php
php56-xmlwriter-5.6.31         The xmlwriter shared extension for php
pixman-0.34.0                  Low-level pixel manipulation library
pkg-1.10.1                     Package manager
pkgconf-1.3.7,1                Utility to help to configure compiler and linker flags
png-1.6.29                     Library for manipulating PNG images
popt-1.16_2                    Getopt(3) like library with a number of enhancements, from Redhat
portmaster-3.17.10             Manage your ports without external databases or languages
presentproto-1.1               Present protocol specification
printproto-1.0.5               Print extension headers
py27-Babel-2.3.4               Collection of tools for internationalizing Python applications
py27-Jinja2-2.9.5              Fast and easy to use stand-alone template engine
py27-MarkupSafe-1.0            Implements XML/HTML/XHTML Markup safe string for Python
py27-alabaster-0.7.6           Modified Kr Sphinx theme
py27-dnspython-1.15.0          DNS toolkit for Python
py27-docutils-0.13.1           Python Documentation Utilities
py27-enum34-1.1.6              Python 3.4 Enum backported to 3.3, 3.2, 3.1, 2.7
py27-imagesize-0.7.1           Python image size library
py27-iso8601-0.1.11            Simple module to parse ISO 8601 dates
py27-pip-9.0.1                 Tool for installing and managing Python packages
py27-pygments-2.2.0            Syntax highlighter written in Python
py27-pystemmer-1.3.0_1         Snowball Stemming Algorithms for Information Retrieval
py27-pytz-2016.10,1            World Timezone Definitions for Python
py27-setuptools-36.0.1         Python packages installer
py27-six-1.10.0                Python 2 and 3 compatibility utilities
py27-snowballstemmer-1.2.0_1   Snowball stemming library collection for Python
py27-sphinx-1.4.8_1,1          Python documentation generator
py27-sphinx_rtd_theme-0.2.4    Mobile-friendly py-sphinx theme
py36-setuptools-36.0.1         Python packages installer
python2-2_3                    The "meta-port" for version 2 of the Python interpreter
python27-2.7.13_6              Interpreted object-oriented programming language
python36-3.6.2                 Interpreted object-oriented programming language
randrproto-1.5.0               Randr extension headers
rdesktop-1.8.3                 RDP client for Windows NT/2000/2003 Terminal Server
readline-7.0.3                 Library for editing command lines as they are typed
recordproto-1.14.2             RECORD extension headers
renderproto-0.11.1             RenderProto protocol headers
resourceproto-1.2.0            Resource extension headers
rhash-1.3.4                    Utility and library for computing and checking of file hashes
rsync-3.1.2_7                  Network file distribution/synchronization utility
ruby-2.3.4_2,1                 Object-oriented interpreted scripting language
ruby23-gems-2.6.12             Package management framework for the Ruby language
rubygem-hpricot-0.8.6          Fast, Enjoyable HTML Parser for Ruby
rubygem-mustache-1.0.5         Framework-agnostic way to render logic-free views
rubygem-rdiscount-2.2.0.1      Fast Markdown converter for ruby based on discount
rubygem-ronn-0.7.3             Builds manuals
s2tc-1.0+20151228              Subset of a well-known texture compression scheme
samba46-4.6.6                  Free SMB/CIFS and AD/DC server and client for Unix
scons-2.5.1_1                  Build tool alternative to make
screen-4.5.0_3                 Multi-screen window manager
scrnsaverproto-1.2.2           ScrnSaver extension headers
sdocbook-xml-1.1_2,2           "Simplified" DocBook XML DTD
serf-1.3.9_1                   Serf HTTP client library
sqlite3-3.19.3_1               SQL database engine in a C library
subversion-1.9.7               Version control system
sudo-1.8.20p2_2                Allow others to run commands as root
swig13-1.3.40_1                Simplified Wrapper and Interface Generator
swig30-3.0.12                  Generate wrappers for calling C/C++ code from other languages
t1lib-5.1.2_4,1                Type 1 font rasterization library for Unix/X11
talloc-2.1.9                   Hierarchical pool based memory allocator
tdb-1.3.12,1                   Trivial Database
tevent-0.9.31                  Talloc based event loop library
texinfo-6.4_1,1                Typeset documentation system with multiple format output
tpm-emulator-0.7.4_2           Trusted Platform Module (TPM) emulator
trousers-0.3.14_1              Open-source TCG Software Stack
tshark-2.4.0                   Powerful network analyzer/capture tool
unixODBC-2.3.4                 ODBC library suite for Unix
unzip-6.0_7                    List, test, and extract compressed files from a ZIP archive
videoproto-2.3.3               Video extension headers
vim-lite-8.0.0809              Improved version of the vi editor (lite package)
wget-1.19.1_1                  Retrieve files from the Net via HTTP(S) and FTP
xcb-proto-1.12                 The X protocol C-language Binding (XCB) protocol
xcb-util-0.4.0_2,1             Module with libxcb/libX11 extension/replacement libraries
xcb-util-renderutil-0.3.9_1    Convenience functions for the Render extension
xcmiscproto-1.2.2              XCMisc extension headers
xextproto-7.3.0                XExt extension headers
xf86bigfontproto-1.2.0         XFree86-Bigfont extension headers
xf86driproto-2.1.1             XFree86-DRI extension headers
xf86vidmodeproto-2.3.1         XFree86-VidModeExtension extension headers
xineramaproto-1.2.1            Xinerama extension headers
xkbcomp-1.4.0                  Compile XKB keyboard description
xkeyboard-config-2.21          X Keyboard Configuration Database
xmlcatmgr-2.2_2                SGML and XML catalog manager
xmlcharent-0.3_2               XML character entities
xorg-macros-1.19.1             X.Org development aclocal macros
xorg-vfbserver-1.19.1_3,1      X virtual framebuffer server from X.Org
xproto-7.0.31                  X11 protocol headers
xtrans-1.3.5                   Abstract network code for X
zabbix32-agent-3.2.6           Enterprise-class open source distributed monitoring (agent)
zabbix32-frontend-3.2.6        Enterprise-class open source distributed monitoring (frontend)
zabbix32-server-3.2.6          Enterprise-class open source distributed monitoring (server)
zip-3.0_1                      Create/update ZIP files compatible with PKZIP
zoneinfo-2017.b_1              Updated timezone definitions
Comment by Andris Mednis [ 2017 Sep 14 ]

My pkg info is much simpler:

apr-1.5.2.1.5.4_2              Apache Portability Library
autoconf-2.69_1                Automatically configure source code on many Un*x platforms
autoconf-wrapper-20131203      Wrapper script for GNU autoconf
autoconf213-2.13.000227_6      Automatically configure source code on many Un*x platforms (legacy 2.13)
automake-1.15.1                GNU Standards-compliant Makefile generator
automake-wrapper-20131203      Wrapper script for GNU automake
autotools-20130627             Autotools meta-port
bash-4.4.12_2                  GNU Project's Bourne Again SHell
binutils-2.28,1                GNU binary tools
db5-5.3.28_6                   Oracle Berkeley DB, revision 5.3
expat-2.2.1                    XML 1.0 parser written in C
gcc-ecj-4.5                    Eclipse Java Compiler used to build GCC Java
gdbm-1.13_1                    GNU database manager
gettext-runtime-0.19.8.1_1     GNU gettext runtime libraries and programs
glib-2.50.2_3,1                Some useful routines of C programming (current stable version)
gmp-6.1.2                      Free library for arbitrary precision arithmetic
indexinfo-0.2.6                Utility to regenerate the GNU info page index
kbproto-1.0.7                  KB extension headers
libX11-1.6.5,1                 X11 library
libXau-1.0.8_3                 Authentication Protocol library for X11
libXdmcp-1.1.2                 X Display Manager Control Protocol library
libXext-1.3.3_1,1              X11 Extension library
libevent-2.1.8                 API for executing callback functions on events or timeouts
libffi-3.2.1                   Foreign Function Interface
libiconv-1.14_10               Character set conversion library
libpthread-stubs-0.4           This library provides weak aliases for pthread functions
libslang2-2.3.1                Routines for rapid alpha-numeric terminal applications development
libssh2-1.8.0,3                Library implementing the SSH2 protocol
libtool-2.4.6                  Generic shared library support script
libxcb-1.12_2                  The X protocol C-language Binding (XCB) library
libxml2-2.9.4                  XML parser library for GNOME
m4-1.4.18,1                    GNU M4
mc-4.8.19_3                    Midnight Commander, a free Norton Commander Clone
mpc-1.0.3                      Library of complex numbers with arbitrarily high precision
mpfr-3.1.5_1                   Library for multiple-precision floating-point computations
openipmi-2.0.21_1              Complex IPMI management software
pcre-8.40_1                    Perl Compatible Regular Expressions library
perl5-5.24.1_1                 Practical Extraction and Report Language
pkg-1.10.1                     Package manager
pkgconf-1.3.7,1                Utility to help to configure compiler and linker flags
png-1.6.29                     Library for manipulating PNG images
popt-1.16_2                    Getopt(3) like library with a number of enhancements, from Redhat
postgresql96-client-9.6.4_2    PostgreSQL database (client)
python27-2.7.13_7              Interpreted object-oriented programming language
readline-7.0.3                 Library for editing command lines as they are typed
serf-1.3.9_1                   Serf HTTP client library
sqlite3-3.20.0_1               SQL database engine in a C library
subversion-1.9.7               Version control system
sudo-1.8.20p2_3                Allow others to run commands as root
xextproto-7.3.0                XExt extension headers
xproto-7.0.31                  X11 protocol headers
Comment by Andris Mednis [ 2017 Sep 19 ]

Reproduced bug with checking for ICONV support... configure: error: Unable to use iconv (libiconv check failed).
The bug occurs if --with-libcurl option is used and libiconv package is installed.
If --with-libcurl option is not used or libiconv package is removed then configure script succeeds.
The same ICONV detection bug was observed in ZBX-9655.
Workaround - add --with-iconv=/usr/local to configure options.

Created a new bugreport ZBX-12756.

Comment by Andris Mednis [ 2017 Sep 20 ]

Hi, Sergey!

The ICONV error can be worked around with adding -with-iconv=/usr/local to configure options, like:

$ CFLAGS="-g -O2 -D_FORTIFY_SOURCE=2" ./configure --enable-server --enable-proxy \
--enable-agent --enable-ipv6 --with-net-snmp --with-unixodbc --with-libxml2 --with-libcurl \
--with-ssh2 --with-openipmi --with-ldap --with-postgresql --with-openssl \
--with-iconv=/usr/local --prefix=`pwd` 2>&1 | tee my_configure.out

Before running

$ make dbschema

I had to edit create/bin/gen_data.pl and create/bin/gen_schema.pl files - replace the 1st line #!/usr/bin/perl -w with #!/usr/local/bin/perl -w.

Comment by seregasoft [ 2017 Sep 21 ]

Thanks! zabbix_server was compiled, started, waiting for crash...

Comment by seregasoft [ 2017 Sep 22 ]

zabbix_server.log_1

Comment by Andris Mednis [ 2017 Sep 22 ]

Thanks, Sergey, for log!
It shows that protection by ignoring an unexpected pointer helped to delay crash by 13 hours...

Comment by Andris Mednis [ 2017 Sep 22 ]

Hi, Sergey!
Can You try to run Zabbix server under Valgrind for some time ?
(I had to do "# pkg install valgrind")
For example, start it like

$ valgrind -v --tool=memcheck --leak-check=full --trace-children=yes --track-origins=yes \
              --read-var-info=yes --time-stamp=yes --log-file=/home/zabbix/valgrind/zabbix_server_%p \
              --leak-resolution=high --show-leak-kinds=all \
                sbin/zabbix_server -c /home/zabbix/zabbix_server.conf

(adjust options to your environment).
Valgrind will catch many errors, but Zabbix server will run slower under Valgrind. Depends on your environment, if it is possible for You. If You could try it maybe for half an hour or longer if possible (until those"Please report this to Zabbix developers" messages appear), then compress and upload Valgrind result files zabbix_server_NNN.

Comment by seregasoft [ 2017 Oct 06 ]

I run valgrind. After several minutes i'll get the message "12345:***Please report this to Zabbix developers" in zabbix log. Trying to stop zabbix "sudo kill -u zabbix" so that valgrind can write logs of problem process ID 12345. But nothing happing, zabbix_server proccess don't whant to stop. If make "sudo kill -9 -u zabbix" zabbix stopped, but i have not valgrind log files.

Comment by Andris Mednis [ 2017 Oct 06 ]

Thanks, Sergey!
Here is an example how I do it on Linux:
For example, I start server with:

$ valgrind -v --tool=memcheck --leak-check=full --trace-children=yes --track-origins=yes \
        --read-var-info=yes --time-stamp=yes  --log-file=/home/zabbix40/zabbix_server_%p \
        --leak-resolution=high --show-leak-kinds=all sbin/zabbix_server -c ~/zabbix_server.conf

When I want to stop it, I do 'ps':

$ ps -fu zabbix40
UID        PID  PPID  C STIME TTY          TIME CMD
zabbix40  5938  5937  0 11:34 pts/1    00:00:00 -su
zabbix40 10513     1 18 14:11 ?        00:00:01 /usr/bin/valgrind.bin -v --tool=memcheck --leak-check=full --trace-children=ye
zabbix40 10517 10513  0 14:11 ?        00:00:00 /usr/bin/valgrind.bin -v --tool=memcheck --leak-check=full --trace-children=ye
zabbix40 10518 10513  0 14:11 ?        00:00:00 /usr/bin/valgrind.bin -v --tool=memcheck --leak-check=full --trace-children=ye
zabbix40 10519 10513  0 14:11 ?        00:00:00 /usr/bin/valgrind.bin -v --tool=memcheck --leak-check=full --trace-children=ye
zabbix40 10520 10513  0 14:11 ?        00:00:00 /usr/bin/valgrind.bin -v --tool=memcheck --leak-check=full --trace-children=ye
zabbix40 10521 10513  0 14:11 ?        00:00:00 /usr/bin/valgrind.bin -v --tool=memcheck --leak-check=full --trace-children=ye
zabbix40 10522 10513  2 14:11 ?        00:00:00 /usr/bin/valgrind.bin -v --tool=memcheck --leak-check=full --trace-children=ye
....

Look for process with PPID = 1. It is process with PID 10513. Then:

$ kill 10513

Now there are files /home/zabbix40/zabbix_server_* - you can tar and gzip them and upload.

Comment by seregasoft [ 2018 Feb 27 ]

Unfortunately i could not successfully stop valgrind to get logs. But after upgrade FreeBSD to 11.1-RELEASE-p4 #0: Tue Nov 14 06:12:40 UTC 2017 and Zabbix Server to zabbix34-server-3.4.6 problem disappeared, sever two weeks working without crash.

Comment by Andris Mednis [ 2018 Feb 28 ]

Thanks, Sergey, for your effort !

Generated at Fri May 02 07:03:27 EEST 2025 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.