server:
zabbix-server-2.0.5-1.el6.x86_64

$ zabbix_agentd --version
Zabbix Agent (daemon) v2.0.5 (revision 33558) (12 February 2013)
Compilation time: Feb 14 2013 10:58:53

The use of the netlink interface has been implemented at svn://svn.zabbix.com/branches/dev/ZBX-8545.

This change applies only to 1) systems using a Linux kernel starting from version 2.6.14 and 2) the net.tcp.listen item.

The netlink subprotocol required for this type of diagnostic (NETLINK_INET_DIAG) was added only in the 2.6.14 kernel and unfortunately, there's no clear way of retrieving UDP protocol socket information. Even the previously mentioned ss(1) utility from the iproute2 package resorts to reading from /proc/net/udp when it comes to UDP sockets.

The main issue in implementing anything netlink related is the severe lack of documentation regarding requests and responses, although documentation on the delivery mechanism and its operation is abundant.

Also, an issue that we're currently facing is determining the effectiveness of this change as detailed execution time advantages might only become apparent on high connectivity systems. If anyone has a system where the shortcomings of the previous implementations of the net.tcp.listen item were obviously too slow, it would be nice to hear some feedback on performance changes after applying this patch.

Successfully tested, please review my code changes in r48800, r48854

Changes merged in 2.5.0 (trunk) at r48983.

(1) Compiler gives a warning regarding the new code:

$ make
...
net.c: In function ‘NET_TCP_LISTEN’:
net.c:570:44: warning: unused variable ‘found’ [-Wunused-variable]
  int  ret = SYSINFO_RET_FAIL, n, buffer_alloc = 64 * ZBX_KIBIBYTE, found = 0;
                                            ^
...

jurism RESOLVED.

asaveljevs CLOSED

(2) The following change is suggested:

$ svn di
Index: src/libs/zbxsysinfo/linux/net.c
===================================================================
--- src/libs/zbxsysinfo/linux/net.c     (revision 48989)
+++ src/libs/zbxsysinfo/linux/net.c     (working copy)
@@ -71,7 +71,7 @@
        NLERR_UNKNOWNMSGTYPE
 };
 
-int    nlerr;
+static int     nlerr;
 
 static int     find_tcp_port_by_state_nl(unsigned short port, int state, int *found)
 {
@@ -593,7 +593,7 @@
        {
                char    *error = NULL;
 
-               switch(nlerr)
+               switch (nlerr)
                {
                        case NLERR_UNKNOWN:
                                error = zbx_strdup(error, "unrecognized netlink error occurred");

jurism RESOLVED.

asaveljevs CLOSED

(3) docs :

• whatsnew
• upgrade notes
• describe somewhere the mechanism that is used to obtain information

asaveljevs ChangeLog says that "Old method of information retrieval also improved". However, it nowhere says how exactly.

jurism What's new has been updated, Upgrade notes contain a terse description with a backlink to the what's new page containing the details of the change. RESOLVED.

asaveljevs Pages in question are:

• https://www.zabbix.com/documentation/3.0/manual/introduction/whatsnew300
• https://www.zabbix.com/documentation/3.0/manual/installation/upgrade_notes_300

The first one looks good, but the following change is proposed for the second:

net.tcp.listen on Linux kernels 2.6.14 and above, when detecting NETLINK capabilities, now tries to obtain socket information on sockets in the LISTEN state from the kernel.

to

net.tcp.listen on Linux kernels 2.6.14 and above, now tries to obtain information on sockets in the LISTEN state from the kernel's NETLINK interface.

Also, richlv's third suggestion was not addressed and it might be useful to add it to net.tcp.listen[] item at https://www.zabbix.com/documentation/3.0/manual/config/items/itemtypes/zabbix_agent . It may be done it a way similar to sensor[] item. REOPENED.

jurism
I disagree. The system doesn't do so, unless it actually detects that the kernel provides netlink facilities. They are not guaranteed, and in the event of NOT being able to employ the interface, which isn't impossible, it falls back on reading from the kernels procfs interface. Why remove this piece of information?

Also, seeing as the code interfacing with the kernel isn't based on a standard or official documentation, we cannot guarantee the correctness and conformance of the code. This warrants internal documentation at best.

wiper I think it would be better not to oversaturate the https://www.zabbix.com/documentation/3.0/manual/config/items/itemtypes/zabbix_agent with information, but to have separate pages with detailed description. The same goes for sensor item. However that would take a lot of work.

asaveljevs It might be that a couple of sentences like "On Linux 2.6.14 and above information is obtained using the kernel's NETLINK interface, if possible. If not, information is read from /proc/net.tcp." would not be that much of an oversaturation.

jurism Added comment about NETLINK to the net.tcp.listen item in 3.0 documentation.

asaveljevs Your changes above also touched upgrade notes. I have fixed a typo at https://www.zabbix.com/documentation/3.0/manual/installation/upgrade_notes_300?&#item_changes . Please review. RESOLVED.

jurism Everything looks fine. CLOSED.

Fixes in code have been commited to trunk at r49015.
Documentation is pending.

jurism Documentation has been updated.

Please revert trunk changes and create a proper development branch.

jurism Changes have been reverted. RESOLVED.

asaveljevs The reverting commit was r49052. CLOSED.

(4) As sasha mentioned in (2) in ZBX-8367, functions should always set output parameters and not rely on the variable being initialized prior to the call. In this case, function find_tcp_port_by_state_nl() relies on "found" variable being initialized outside of the function.

jurism RESOLVED.

asaveljevs It seems variable "found" could have been left uninitialized in case we get NLMSG_DONE. I have fixed that and removed unnecessary initializations and string allocations in r49081. Please take a look. RESOLVED.

jurism Changes look fine. CLOSED.

(5) I have a Linux kernel above 2.6.14, but by default trunk compiles without Netlink support on my machine. It should be documented what needs to be done to compile with Netlink.

asaveljevs Seems to be an error on my part - did not rerun ./bootstrap.sh. WON'T FIX.

The necessary changes have been commited to the development branch at its original location.

Change merged into 2.5.0 (trunk) at r49187.

subissues still open : (3)