[ZBXNEXT-786] Key net.tcp.service[ntp] does not work with UDP Created: 2011 May 16  Updated: 2018 Jul 05  Resolved: 2015 Jun 18

Status: Closed
Project: ZABBIX FEATURE REQUESTS
Component/s: Agent (G), Documentation (D)
Affects Version/s: None
Fix Version/s: 2.0.15, 2.2.10, 2.4.6, 2.5.0

Type: Change Request Priority: Minor
Reporter: Oleksii Zagorskyi Assignee: Unassigned
Resolution: Fixed Votes: 12
Labels: item, ntp, udp
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: XML File zbx_export_hosts (8).xml    
Issue Links:
Duplicate
is duplicated by ZBX-9401 Zabbix server crashed with wrong simp... Closed

 Description   

Generally NTP uses only UDP as transport. TCP almost not used at all. So, key "net.tcp.service[ntp]" which tries to use TCP as transport is useless.

I see a 3 different way to solve:
1. Remove support of NTP as parameter to key net.tcp.service[] to not mislead users anymore.
2. Make a note to documentation about NTP ant TCP and their useless.
3. Introduce new key net.udp.service[ntp] and remove NTP from net.tcp.service[].



 Comments   
Comment by Oleksii Zagorskyi [ 2011 May 16 ]

bahh, again ZBXNEXT. In fact i wanted to create ZBX.

Comment by Aleksandrs Saveljevs [ 2011 May 16 ]

Checking NTP over UDP is a new feature, so ZBXNEXT is OK.

Comment by nelsonab [ 2011 May 16 ]

I think you may want to edit your description if you can. The three ways you talk about solving the problem are not different, but rather the three steps needed to fix the issue.

Comment by Oleksii Zagorskyi [ 2011 May 16 ]

Nelsonab, as you can see the way #3 includes way #1. If way #3 will be accepted, then step #2 should be discarded (i hope it's obvious). But i'm not sure that developers want to select way #3.
At least way #1 or #2 should be performed. So, i think you are not absolutely right

Of course, the way #3 is the best and based on this Aleksandrs is absolutely right .

Comment by BW57899 [ 2014 Aug 08 ]

Seems this issue is still opened, I found a workaround way to check ntp server.

You can use net.udp.listen[123] to detect ntp service running or not.

net.udp.listen[port]

Checks if this UDP port is in LISTEN state.
0 - it is not
1 - it is in LISTEN state
port - UDP port number
Example: net.udp.listen[68]

On Linux supported since Zabbix agent version 1.8.4

Comment by Peter Daniel [ 2014 Sep 01 ]

What version of Zabbix Server are you running? I can't locate the net.udp.listen[port] you mentioned. thanks

Comment by Durval Menezes [ 2014 Sep 29 ]

@Peter Daniel: you have to change the Item type from "Simple test" to "Zabbix agent" in order to make it available.

@BW57899: Did it, and now my Zabbix server is logging the following:
item "testhost:net.udp.listen[ntp]" became not supported: Not supported by Zabbix Agent
But the host is Solaris10, and it is marked here as supported: https://www.zabbix.com/documentation/2.2/manual/appendix/items/supported_by_platform
So, what gives?

Comment by BW57899 [ 2014 Sep 30 ]

@Durval Menezes

I did the test on version 2.2

could you please try net.udp.listen[123], not net.udp.listen[ntp]

Comment by richlv [ 2015 Mar 17 ]

looks like tcp is completely wrong, too - rfc does not even say tcp can be used.
from https://www.ietf.org/rfc/rfc5905.txt : "The NTP packet is a UDP datagram"

now it sounds more like a bug to me

Comment by Aleksandrs Saveljevs [ 2015 Mar 17 ]

Note that our implementation does not use the full NTP protocol, but rather SNTP. Useful reading regading that:

Comment by Aleksandrs Saveljevs [ 2015 Mar 27 ]

This comment shall describe changes to the old implementation and the reasoning behind them:

  • "data->mode" was set to 1 (which corresponds to NTP_ACTIVE), but it is now set to NTP_CLIENT (3) based on http://tools.ietf.org/html/rfc4330#page-10 :

    Mode: This is a three-bit number indicating the protocol mode. The
    values are defined as follows:

    Mode Meaning
    ------------------------------------
    0 reserved
    1 symmetric active
    2 symmetric passive
    3 client
    4 server
    5 broadcast
    6 reserved for NTP control message
    7 reserved for private use

    In unicast and manycast modes, the client sets this field to 3
    (client) in the request, and the server sets it to 4 (server) in the
    reply. In broadcast mode, the server sets this field to 5
    (broadcast). The other modes are not used by SNTP servers and
    clients.

  • a lot of "ntp_data" fields were set in the request, but their values actually matter only in server messages, therefore, these fields are now zeroed; the only fields that have to be set seem to be the following:

    A unicast or manycast client initializes the NTP message header,
    sends the request to the server, and strips the time of day from the
    Transmit Timestamp field of the reply. For this purpose, all the NTP
    header fields shown above are set to 0, except the Mode, VN, and
    optional Transmit Timestamp fields.

  • response packet is now validated according to http://tools.ietf.org/html/rfc4330#section-5 .
Comment by Aleksandrs Saveljevs [ 2015 Mar 31 ]

The plan is as follows. We shall fix net.tcp.service[ntp] to work in 2.0, 2.2, 2.4 and trunk (pre-3.0). After that, we shall rename net.tcp.service[ntp] to net.udp.service[ntp] in trunk only. This is because such a change affects documentation, templates, help items, etc., so this allows us to reduce the number of changes we have to make. Finally, we may even add NTP protocol as an option to network discovery in a separate ZBXNEXT.

asaveljevs That ZBXNEXT is ZBXNEXT-2781.

Comment by Aleksandrs Saveljevs [ 2015 Mar 31 ]

Item net.tcp.service[ntp] has been fixed for 2.0 in development branch svn://svn.zabbix.com/branches/dev/ZBX-9401 (note the branch name).

The new NTP code seems to work and is simpler than the previous implementation. However, the UDP socket implementation is half-baked and remains to be discussed with the reviewer.

Comment by Aleksandrs Saveljevs [ 2015 Apr 10 ]

The UDP socket implementation has been simplified using connect() and can be reviewed in the same development branch, r53049. It also changes zbx_set_tcp_strerror() to zabbix_log() in zbx_tcp_timeout_set(), otherwise errors (if any) are never seen.

Once reviewed and tested, the next step is to create a separate development branch for 2.4, because socket code was changed in 2.4 a bit under ZBXNEXT-2203.

Once 2.4 is ready, in trunk we should probably rename socket functions like ZBX_TCP_START() and zbx_set_tcp_strerror() to ZBX_SOCK_START() and zbx_set_sock_strerror(). We should also consider whether we need ZBX_SOCK_START() in send() and recv() functions.

Comment by dimir [ 2015 Apr 14 ]

Successfully tested NTP, as well as other UDP and TCP connections with IPv4 and IPv6 addresses.

Comment by Aleksandrs Saveljevs [ 2015 Apr 15 ]

Fix for 2.4 is available in development branch svn://svn.zabbix.com/branches/dev/ZBX-9401-2.4 .

Comment by dimir [ 2015 Apr 15 ]

Fix for 2.4 tested.

Comment by richlv [ 2015 Apr 21 ]

(1) documentation - whatsnew, item pages and also https://www.zabbix.com/documentation/3.0/manual/appendix/items/net.tcp.service_details for all versions

asaveljevs Changes in pre-3.0 versions are documented at the following locations:

The linked page about "net.tcp.service" details is handled in ZBX-7760 (5).

asaveljevs RESOLVED for pre-3.0 versions.

asaveljevs Additionally, removed a note about using "service.ntp" before 1.8.3 on the following pages:

asaveljevs RESOLVED

<dimir> Looks fine, CLOSED

Comment by Aleksandrs Saveljevs [ 2015 Apr 22 ]

Changes that make "net.tcp.service[ntp]" work properly over UDP are available in pre-2.0.15 r53295, pre-2.2.10 r53296, pre-2.4.6 r53299, pre-2.5.0 (trunk) r53300.

Comment by Aleksandrs Saveljevs [ 2015 Apr 23 ]

The next step is making socket code pretty in trunk. Development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-786 renames functions that deal with both TCP and UDP socket to "zbx_socket_*". Function names that are specific to TCP start with "zbx_tcp_*" and those specific to UDP start with "zbx_udp_*". Please review. During review, it is advisable to go commit by commit, as each commit contains an isolated change.

Comment by dimir [ 2015 Apr 29 ]

Please review my small changes in r53433 and r53435.

asaveljevs Looks good. CLOSED.

Comment by Aleksandrs Saveljevs [ 2015 Apr 30 ]

Socket function renaming changes are available in pre-2.5.0 (trunk) r53451.

Comment by Aleksandrs Saveljevs [ 2015 Apr 30 ]

The final step is to rename "net.tcp.service[ntp]" and "net.tcp.service.perf[ntp]" to "net.udp.service[ntp]" and "net.udp.service.perf[ntp]" in trunk.

Server side and a bit of PHP is available in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-786 . It can be reviewed and tested. On PHP side, it remains to do the automatic key conversion during XML import, which is queued under (4).

Based on richlv's suggestion, item key helper was not updated and left for ZBXNEXT-2791.

Comment by Aleksandrs Saveljevs [ 2015 May 05 ]

(2) Fixed net.tcp.service[] item key, which was lacking a comma, at the following locations:

<dimir> CLOSED

Comment by Aleksandrs Saveljevs [ 2015 May 05 ]

(3) Documentation regarding net.udp.service[ntp] updated on the following pages:

<dimir> Specified:

Please check.

asaveljevs The distinction between TCP and UDP is nice, but since ZBXNEXT-2791 requires some extra work: agent item descriptions should be synced from the manual. In order to reduce work for translators, I have copied the first two columns for service items from the agent page to the simple check page. Then, I have updated CHelpItems.php with your new descriptions, and fixed a bug of net.udp.service not existing in CHelpItems.php for simple checks. Please review r53535 and (5). RESOLVED.

<dimir> This is great! CLOSED

Comment by Aleksandrs Saveljevs [ 2015 May 05 ]

(4) [F] XML import should convert net.tcp.service[ntp]-like keys automatically, but it might be better done after ZBX-7654.

oleg.egorov RESOLVED IN r53696

sasha

  • possible "Undefined offset: 0" in C20ItemKeyConverter.php:38 when converting key without parameters

REOPENED

oleg.egorov RESOLVED IN r53700

sasha CLOSED

Comment by Aleksandrs Saveljevs [ 2015 May 11 ]

(5) Translation strings:

removed

  • Checks performance of service. Returns 0 - service is down; seconds - the number of seconds spent while connecting to the service
  • Checks performance of service. Returns 0 - service is down; seconds - the number of seconds spent waiting for response from the service
  • Check if service is available. 0 - service is down, 1 - service is running. If <ip> is missing, IP or DNS name is taken from host definition. If <port> is missing, default service port is used.
  • Check performance of service. 0 - service is down, sec - number of seconds spent on connection to the service. If <ip> is missing, IP or DNS name is taken from host definition. If <port> is missing, default service port is used.

added

  • Checks performance of TCP service. Returns 0 - service is down; seconds - the number of seconds spent while connecting to the service
  • Checks performance of UDP service. Returns 0 - service is down; seconds - the number of seconds spent waiting for response from the service

<dimir> Wasn't also

  • Checks if service is running and accepting TCP connections. Returns 0 - service is down; 1 - service is running
  • Checks if service is running and responding to UDP requests. Returns 0 - service is down; 1 - service is running

added?

asaveljevs They were added for simple checks, but these strings already existed for agent checks.

<dimir> That's right. CLOSED.

Comment by Alexander Vladishev [ 2015 May 21 ]

(6) [XML] C20ItemKeyConverter:

  • the input key can be invalid (validation should be added);
  • the first parameter should not be trimmed;
  • double-quoting of the parameters should not be removed
    • incorrect key can be generated (net.tcp.service[ntp,"\"string"] => net.udp.service[ntp,"string])
  • spacing between parameters should not be removed

RESOLVED in r53702

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(7) [XML] "Undefined property: CParserResult::$expression" in include/classes/import/converters/C20TriggerConverter.php:94 while convert trigger expression with LLD macros

sasha RESOLVED in r53710.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(8) [XML] "Undefined offset: 0" in include/classes/import/converters/C20TriggerConverter.php:70 while convert an incorrect trigger expression.

{host:item.last()

See (10).

sasha RESOLVED in r53709.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(9) [XML] The C20TriggerConverter works incorrectly:

Input:
    {host:item.last()} > 0

Output
    {host:item.last()}

See (10).

sasha RESOLVED in r53709.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(10) [XML] This code must be removed

         public function __construct() {
                 $this->functionMacroParser = new CFunctionMacroParser();
                 $this->lldMacroParser = new CMacroParser('#');
                 $this->itemKeyConverter = new C20ItemKeyConverter();
-                $this->triggerExpressionParser = new CTriggerExpression();
         }

         /**
          * Convert a trigger expression from 2.2 format to 2.4.
          *
          * The method will replace old operators with their analogues: "&" with "and", "|" - "or" and "#" - "<>".
          *
          * @param string $expression
          *
          * @return string
          */
        public function convert($expression) {
-                // don't try to parse the expression if there's nothing to replace
-                if (strpos($expression, '#') === false
-                                && strpos($expression, '&') === false
-                                && strpos($expression, '|') === false) {
-
-                        $this->triggerExpressionParser->parse($expression);
-
-                        $expression = '{'.
-                                $this->triggerExpressionParser->expressions[0]['host'].':'.
-                                $this->itemKeyConverter->convert($this->triggerExpressionParser->expressions[0]['item']).'.'.
-                                $this->triggerExpressionParser->expressions[0]['function'].
-                        '}';
-
-                        return $expression;
-                }
-
                 // find all the operators that need to be replaced
                 $found_operators = [];
                 $pos = 0;
                 while (isset($expression[$pos])) {

The issues (8) and (9) will solved after removing this code.

sasha RESOLVED in r53709.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(9) [XML] The C20TriggerConverter works incorrectly with localized characters:

Input:
    {host:log["/вар/лог/заббикс/заббикс_сервер.лог"].regexp("\<системная ошибка\>")}#0
Output:
    {host:log["/вар/лог/заббикс/заббикс_сервер.лог"].regexp("\<системная ошибка\>")}#0

sasha RESOLVED in r53711.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(10) [XML] The C20TriggerConverter works incorrectly without spaces after function macro:

Input:
    {host:item.last()}#{host:item.last()}&{host:item.last()}|{host:item.last()}
Output:
    {host:item.last()}#{host:item.last()}&{host:item.last()}|{host:item.last()}

sasha RESOLVED in r53711.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(11) [XML] Unit tests and minor changes

sasha RESOLVED in r53711 and r53712.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(12) [XML] item keys in templates should be also converted

sasha RESOLVED in r53723.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(13) Take a look at my changes in r53720:53722 (code refactoring).

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(14) [XML] item prototype keys in hosts and templates are not converted

sasha RESOLVED in r53725.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 21 ]

(15) [XML] graph_item/item/key are not converted for graphs and graph prototypes

sasha RESOLVED in r53727.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 22 ]

(16) [XML] graph and graph prototype min/max axis items are not converted

sasha RESOLVED in r53728.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 22 ]

(17) [XML] screen_item/resource/key are not converted for screens and screen prototypes

sasha RESOLVED in r53728 and r53729.

oleg.egorov CLOSED

Comment by Oleg Egorov (Inactive) [ 2015 May 26 ]

(18) [XML] File: zbx_export_hosts (8).xml

No "evaltype" given for the filter of discovery rule "9".

sasha RESOLVED in r53804.

oleg.egorov CLOSED

Comment by Alexander Vladishev [ 2015 May 27 ]

Item keys net.tcp.service[ntp] and net.tcp.service.perf[ntp] were renamed to net.udp.service[ntp] and net.udp.service.perf[ntp] in pre-2.5.0 r53819.

Comment by Oleg Egorov (Inactive) [ 2015 May 28 ]

(19) Automatic database upgrade

 12877:20150528:122944.319 starting automatic database upgrade
 12877:20150528:122944.319 query [txnlev:1] [begin;]
 12877:20150528:122944.320 query [txnlev:1] [select itemid,key_ from items where type in (0,3,7) and key_ like 'net.tcp.service%[%ntp%']
 12877:20150528:122944.327 query [txnlev:1] [update items set key_='net.udp.service[ntp]' where itemid=23650]
 12877:20150528:122944.328 query [txnlev:1] [update items set key_='net.udp.service[ntp]' where itemid=25414]
 12877:20150528:122944.329 query [txnlev:1] [update items set key_='net.udp.service[ntp,1]' where itemid=29842]
 12877:20150528:122944.330 [Z3005] query failed: [1062] Duplicate entry '10362-net.udp.service[ntp,1]' for key 2 [update items set key_='net.udp.service[ntp,1]' where itemid=29842]
 12877:20150528:122944.330 query [update items set key_='net.udp.service[ntp,1]' where itemid=29842] failed, setting transaction as failed
 12877:20150528:122944.330 query [txnlev:1] [rollback;]
 12877:20150528:122944.331 database upgrade failed
 12877:20150528:122944.331 End of DBcheck_version():FAIL

asaveljevs Without any additional information, this looks correct. In case there are two items that get transformed into the same key during upgrade (e.g., "net.tcp.service[ntp]" and "net.tcp.service[service.ntp]"), the upgrade should fail.

asaveljevs What is it that you find wrong?

asaveljevs After discussion it was concluded that the expected behavior for the server is to print a warning that it could not convert the item key and continue with the upgrade.

asaveljevs RESOLVED in r53844.

<dimir> CLOSED

Comment by Aleksandrs Saveljevs [ 2015 Jun 04 ]

Fix for (19) available in pre-2.5.0 (trunk) r53948.

Comment by richlv [ 2016 Mar 12 ]

the template changes seem to be missing from the upgrade notes & the new version of the template is not available in the template download page - instead of handling that in this issue, created ZBX-10528 to handle several templates like that

Generated at Thu Apr 25 04:18:16 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.