[ZBXNEXT-786] Key net.tcp.service[ntp] does not work with UDP Created: 2011 May 16 Updated: 2018 Jul 05 Resolved: 2015 Jun 18 |
|
Status: | Closed |
Project: | ZABBIX FEATURE REQUESTS |
Component/s: | Agent (G), Documentation (D) |
Affects Version/s: | None |
Fix Version/s: | 2.0.15, 2.2.10, 2.4.6, 2.5.0 |
Type: | Change Request | Priority: | Minor |
Reporter: | Oleksii Zagorskyi | Assignee: | Unassigned |
Resolution: | Fixed | Votes: | 12 |
Labels: | item, ntp, udp | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Attachments: | zbx_export_hosts (8).xml | ||||||||
Issue Links: |
|
Description |
Generally NTP uses only UDP as transport. TCP almost not used at all. So, key "net.tcp.service[ntp]" which tries to use TCP as transport is useless. I see a 3 different way to solve: |
Comments |
Comment by Oleksii Zagorskyi [ 2011 May 16 ] |
bahh, again ZBXNEXT. In fact i wanted to create ZBX. |
Comment by Aleksandrs Saveljevs [ 2011 May 16 ] |
Checking NTP over UDP is a new feature, so ZBXNEXT is OK. |
Comment by nelsonab [ 2011 May 16 ] |
I think you may want to edit your description if you can. The three ways you talk about solving the problem are not different, but rather the three steps needed to fix the issue. |
Comment by Oleksii Zagorskyi [ 2011 May 16 ] |
Nelsonab, as you can see the way #3 includes way #1. If way #3 will be accepted, then step #2 should be discarded (i hope it's obvious). But i'm not sure that developers want to select way #3. Of course, the way #3 is the best and based on this Aleksandrs is absolutely right . |
Comment by BW57899 [ 2014 Aug 08 ] |
Seems this issue is still opened, I found a workaround way to check ntp server. You can use net.udp.listen[123] to detect ntp service running or not. net.udp.listen[port] Checks if this UDP port is in LISTEN state. On Linux supported since Zabbix agent version 1.8.4 |
Comment by Peter Daniel [ 2014 Sep 01 ] |
What version of Zabbix Server are you running? I can't locate the net.udp.listen[port] you mentioned. thanks |
Comment by Durval Menezes [ 2014 Sep 29 ] |
@Peter Daniel: you have to change the Item type from "Simple test" to "Zabbix agent" in order to make it available. @BW57899: Did it, and now my Zabbix server is logging the following: |
Comment by BW57899 [ 2014 Sep 30 ] |
@Durval Menezes I did the test on version 2.2 could you please try net.udp.listen[123], not net.udp.listen[ntp] |
Comment by richlv [ 2015 Mar 17 ] |
looks like tcp is completely wrong, too - rfc does not even say tcp can be used. now it sounds more like a bug to me |
Comment by Aleksandrs Saveljevs [ 2015 Mar 17 ] |
Note that our implementation does not use the full NTP protocol, but rather SNTP. Useful reading regading that: |
Comment by Aleksandrs Saveljevs [ 2015 Mar 27 ] |
This comment shall describe changes to the old implementation and the reasoning behind them:
|
Comment by Aleksandrs Saveljevs [ 2015 Mar 31 ] |
The plan is as follows. We shall fix net.tcp.service[ntp] to work in 2.0, 2.2, 2.4 and trunk (pre-3.0). After that, we shall rename net.tcp.service[ntp] to net.udp.service[ntp] in trunk only. This is because such a change affects documentation, templates, help items, etc., so this allows us to reduce the number of changes we have to make. Finally, we may even add NTP protocol as an option to network discovery in a separate ZBXNEXT. asaveljevs That ZBXNEXT is ZBXNEXT-2781. |
Comment by Aleksandrs Saveljevs [ 2015 Mar 31 ] |
Item net.tcp.service[ntp] has been fixed for 2.0 in development branch svn://svn.zabbix.com/branches/dev/ZBX-9401 (note the branch name). The new NTP code seems to work and is simpler than the previous implementation. However, the UDP socket implementation is half-baked and remains to be discussed with the reviewer. |
Comment by Aleksandrs Saveljevs [ 2015 Apr 10 ] |
The UDP socket implementation has been simplified using connect() and can be reviewed in the same development branch, r53049. It also changes zbx_set_tcp_strerror() to zabbix_log() in zbx_tcp_timeout_set(), otherwise errors (if any) are never seen. Once reviewed and tested, the next step is to create a separate development branch for 2.4, because socket code was changed in 2.4 a bit under Once 2.4 is ready, in trunk we should probably rename socket functions like ZBX_TCP_START() and zbx_set_tcp_strerror() to ZBX_SOCK_START() and zbx_set_sock_strerror(). We should also consider whether we need ZBX_SOCK_START() in send() and recv() functions. |
Comment by dimir [ 2015 Apr 14 ] |
Successfully tested NTP, as well as other UDP and TCP connections with IPv4 and IPv6 addresses. |
Comment by Aleksandrs Saveljevs [ 2015 Apr 15 ] |
Fix for 2.4 is available in development branch svn://svn.zabbix.com/branches/dev/ZBX-9401-2.4 . |
Comment by dimir [ 2015 Apr 15 ] |
Fix for 2.4 tested. |
Comment by richlv [ 2015 Apr 21 ] |
(1) documentation - whatsnew, item pages and also https://www.zabbix.com/documentation/3.0/manual/appendix/items/net.tcp.service_details for all versions asaveljevs Changes in pre-3.0 versions are documented at the following locations:
The linked page about "net.tcp.service" details is handled in asaveljevs RESOLVED for pre-3.0 versions. asaveljevs Additionally, removed a note about using "service.ntp" before 1.8.3 on the following pages:
asaveljevs RESOLVED <dimir> Looks fine, CLOSED |
Comment by Aleksandrs Saveljevs [ 2015 Apr 22 ] |
Changes that make "net.tcp.service[ntp]" work properly over UDP are available in pre-2.0.15 r53295, pre-2.2.10 r53296, pre-2.4.6 r53299, pre-2.5.0 (trunk) r53300. |
Comment by Aleksandrs Saveljevs [ 2015 Apr 23 ] |
The next step is making socket code pretty in trunk. Development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-786 renames functions that deal with both TCP and UDP socket to "zbx_socket_*". Function names that are specific to TCP start with "zbx_tcp_*" and those specific to UDP start with "zbx_udp_*". Please review. During review, it is advisable to go commit by commit, as each commit contains an isolated change. |
Comment by dimir [ 2015 Apr 29 ] |
Please review my small changes in r53433 and r53435. asaveljevs Looks good. CLOSED. |
Comment by Aleksandrs Saveljevs [ 2015 Apr 30 ] |
Socket function renaming changes are available in pre-2.5.0 (trunk) r53451. |
Comment by Aleksandrs Saveljevs [ 2015 Apr 30 ] |
The final step is to rename "net.tcp.service[ntp]" and "net.tcp.service.perf[ntp]" to "net.udp.service[ntp]" and "net.udp.service.perf[ntp]" in trunk. Server side and a bit of PHP is available in development branch svn://svn.zabbix.com/branches/dev/ZBXNEXT-786 . It can be reviewed and tested. On PHP side, it remains to do the automatic key conversion during XML import, which is queued under (4). Based on richlv's suggestion, item key helper was not updated and left for |
Comment by Aleksandrs Saveljevs [ 2015 May 05 ] |
(2) Fixed net.tcp.service[] item key, which was lacking a comma, at the following locations:
<dimir> CLOSED |
Comment by Aleksandrs Saveljevs [ 2015 May 05 ] |
(3) Documentation regarding net.udp.service[ntp] updated on the following pages:
<dimir> Specified:
Please check. asaveljevs The distinction between TCP and UDP is nice, but since <dimir> This is great! CLOSED |
Comment by Aleksandrs Saveljevs [ 2015 May 05 ] |
(4) [F] XML import should convert net.tcp.service[ntp]-like keys automatically, but it might be better done after oleg.egorov RESOLVED IN r53696
REOPENED oleg.egorov RESOLVED IN r53700 sasha CLOSED |
Comment by Aleksandrs Saveljevs [ 2015 May 11 ] |
(5) Translation strings: removed
added
<dimir> Wasn't also
added? asaveljevs They were added for simple checks, but these strings already existed for agent checks. <dimir> That's right. CLOSED. |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(6) [XML] C20ItemKeyConverter:
RESOLVED in r53702 oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(7) [XML] "Undefined property: CParserResult::$expression" in include/classes/import/converters/C20TriggerConverter.php:94 while convert trigger expression with LLD macros sasha RESOLVED in r53710. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(8) [XML] "Undefined offset: 0" in include/classes/import/converters/C20TriggerConverter.php:70 while convert an incorrect trigger expression. {host:item.last() See (10). sasha RESOLVED in r53709. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(9) [XML] The C20TriggerConverter works incorrectly: Input: {host:item.last()} > 0 Output {host:item.last()} See (10). sasha RESOLVED in r53709. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(10) [XML] This code must be removed public function __construct() { $this->functionMacroParser = new CFunctionMacroParser(); $this->lldMacroParser = new CMacroParser('#'); $this->itemKeyConverter = new C20ItemKeyConverter(); - $this->triggerExpressionParser = new CTriggerExpression(); } /** * Convert a trigger expression from 2.2 format to 2.4. * * The method will replace old operators with their analogues: "&" with "and", "|" - "or" and "#" - "<>". * * @param string $expression * * @return string */ public function convert($expression) { - // don't try to parse the expression if there's nothing to replace - if (strpos($expression, '#') === false - && strpos($expression, '&') === false - && strpos($expression, '|') === false) { - - $this->triggerExpressionParser->parse($expression); - - $expression = '{'. - $this->triggerExpressionParser->expressions[0]['host'].':'. - $this->itemKeyConverter->convert($this->triggerExpressionParser->expressions[0]['item']).'.'. - $this->triggerExpressionParser->expressions[0]['function']. - '}'; - - return $expression; - } - // find all the operators that need to be replaced $found_operators = []; $pos = 0; while (isset($expression[$pos])) { The issues (8) and (9) will solved after removing this code. sasha RESOLVED in r53709. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(9) [XML] The C20TriggerConverter works incorrectly with localized characters: Input: {host:log["/вар/лог/заббикс/заббикс_сервер.лог"].regexp("\<системная ошибка\>")}#0 Output: {host:log["/вар/лог/заббикс/заббикс_сервер.лог"].regexp("\<системная ошибка\>")}#0 sasha RESOLVED in r53711. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(10) [XML] The C20TriggerConverter works incorrectly without spaces after function macro: Input: {host:item.last()}#{host:item.last()}&{host:item.last()}|{host:item.last()} Output: {host:item.last()}#{host:item.last()}&{host:item.last()}|{host:item.last()} sasha RESOLVED in r53711. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(11) [XML] Unit tests and minor changes sasha RESOLVED in r53711 and r53712. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(12) [XML] item keys in templates should be also converted sasha RESOLVED in r53723. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(13) Take a look at my changes in r53720:53722 (code refactoring). oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(14) [XML] item prototype keys in hosts and templates are not converted sasha RESOLVED in r53725. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 21 ] |
(15) [XML] graph_item/item/key are not converted for graphs and graph prototypes sasha RESOLVED in r53727. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 22 ] |
(16) [XML] graph and graph prototype min/max axis items are not converted sasha RESOLVED in r53728. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 22 ] |
(17) [XML] screen_item/resource/key are not converted for screens and screen prototypes sasha RESOLVED in r53728 and r53729. oleg.egorov CLOSED |
Comment by Oleg Egorov (Inactive) [ 2015 May 26 ] |
(18) [XML] File: zbx_export_hosts (8).xml No "evaltype" given for the filter of discovery rule "9". sasha RESOLVED in r53804. oleg.egorov CLOSED |
Comment by Alexander Vladishev [ 2015 May 27 ] |
Item keys net.tcp.service[ntp] and net.tcp.service.perf[ntp] were renamed to net.udp.service[ntp] and net.udp.service.perf[ntp] in pre-2.5.0 r53819. |
Comment by Oleg Egorov (Inactive) [ 2015 May 28 ] |
(19) Automatic database upgrade 12877:20150528:122944.319 starting automatic database upgrade 12877:20150528:122944.319 query [txnlev:1] [begin;] 12877:20150528:122944.320 query [txnlev:1] [select itemid,key_ from items where type in (0,3,7) and key_ like 'net.tcp.service%[%ntp%'] 12877:20150528:122944.327 query [txnlev:1] [update items set key_='net.udp.service[ntp]' where itemid=23650] 12877:20150528:122944.328 query [txnlev:1] [update items set key_='net.udp.service[ntp]' where itemid=25414] 12877:20150528:122944.329 query [txnlev:1] [update items set key_='net.udp.service[ntp,1]' where itemid=29842] 12877:20150528:122944.330 [Z3005] query failed: [1062] Duplicate entry '10362-net.udp.service[ntp,1]' for key 2 [update items set key_='net.udp.service[ntp,1]' where itemid=29842] 12877:20150528:122944.330 query [update items set key_='net.udp.service[ntp,1]' where itemid=29842] failed, setting transaction as failed 12877:20150528:122944.330 query [txnlev:1] [rollback;] 12877:20150528:122944.331 database upgrade failed 12877:20150528:122944.331 End of DBcheck_version():FAIL asaveljevs Without any additional information, this looks correct. In case there are two items that get transformed into the same key during upgrade (e.g., "net.tcp.service[ntp]" and "net.tcp.service[service.ntp]"), the upgrade should fail. asaveljevs What is it that you find wrong? asaveljevs After discussion it was concluded that the expected behavior for the server is to print a warning that it could not convert the item key and continue with the upgrade. asaveljevs RESOLVED in r53844. <dimir> CLOSED |
Comment by Aleksandrs Saveljevs [ 2015 Jun 04 ] |
Fix for (19) available in pre-2.5.0 (trunk) r53948. |
Comment by richlv [ 2016 Mar 12 ] |
the template changes seem to be missing from the upgrade notes & the new version of the template is not available in the template download page - instead of handling that in this issue, created |