[ZBXNEXT-2732] Network Discovery: Load balancing of the discovery process Created: 2015 Mar 03  Updated: 2024 Apr 10  Resolved: 2023 Jun 20

Status: Closed
Project: ZABBIX FEATURE REQUESTS
Component/s: Proxy (P), Server (S)
Affects Version/s: None
Fix Version/s: 7.0.0alpha2, 7.0 (plan)

Type: New Feature Request Priority: Trivial
Reporter: Kazuo Ito Assignee: Aleksejs Sestakovs
Resolution: Fixed Votes: 10
Labels: discoveryrule, networkdiscovery, performance
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Attachments: PNG File disabled_discoverers.png     PNG File image-2023-06-01-12-02-37-268.png     PNG File screenshot-1.png     PNG File screenshot-10.png     PNG File screenshot-11.png     PNG File screenshot-12.png     PNG File screenshot-13.png     PNG File screenshot-14.png     PNG File screenshot-15.png     PNG File screenshot-16.png     PNG File screenshot-17.png     PNG File screenshot-18.png     PNG File screenshot-19.png     PNG File screenshot-2.png     PNG File screenshot-20.png     PNG File screenshot-21.png     PNG File screenshot-3.png     PNG File screenshot-4.png     PNG File screenshot-5.png     PNG File screenshot-6.png     PNG File screenshot-7.png     PNG File screenshot-8.png     PNG File screenshot-9.png     File snmpv3_thread.patch     Text File splite_drule.py     PNG File stats_key_error.png     PNG File stats_key_working.png     HTML File zabbix_proxy_40547     HTML File zabbix_server_45325    
Issue Links:
Causes
causes ZBXNEXT-8827 Add UI error to Network Discovery rule Closed
causes ZBX-23075 Crash with frequent network discovery... Closed
causes ZBX-22362 Pád servera v 6.4.0rc1 Confirmed
caused by ZBXNEXT-8270 Replace family of zbx_socket_*() fun... Closed
Duplicate
is duplicated by ZBXNEXT-5950 Suggestion on "Autodiscovery cycle" Closed
is duplicated by ZBX-11420 Zabbix 3.0 Network Discovery delay no... Closed
is duplicated by ZBX-12310 Network discovery rules execution sta... Closed
Sub-task
depends on ZBX-22336 Log level control for threads Closed
part of ZBXNEXT-8447 Update Zabbix health templates Closed
Sub-Tasks:
Key
Summary
Type
Status
Assignee
ZBXNEXT-8327 Network Discovery: capacity field for... Specification change (Sub-task) Closed Diana Dauksa  
ZBXNEXT-8447 Update Zabbix health templates Specification change (Sub-task) Closed Denis Rasikhov  
ZBXNEXT-8670 Add support of async snmp to network ... Change Request (Sub-task) Closed Michael Veksler  
Epic Link: Zabbix 7.0
Team: Team B
Sprint: Sprint 95 (Dec 2022), Sprint 96 (Jan 2023), Sprint 97 (Feb 2023), Sprint 98 (Mar 2023), Sprint 99 (Apr 2023), Sprint 100 (May 2023), Sprint 101 (Jun 2023)
Story Points: 10

 Description   

When we put wide range in the discovery settings, load is concentrated on one of the discoverer process.(For example, /20)

Is it possible to handle it in multiple discoverer process?



 Comments   
Comment by Oleksii Zagorskyi [ 2015 Mar 05 ]

Documentation https://www.zabbix.com/documentation/2.4/manual/discovery/network_discovery states:

Note that one discovery rule will always be processed by a single discoverer process. The IP range will not be split between multiple discoverer processes.

I don't think anything can be changed in this regard.

Especially taken into account that recently we extended syntax of supported IP ranges - now it may be too complex to be split on parts:
https://www.zabbix.com/documentation/2.4/manual/introduction/whatsnew244#extended_ip_range_option_in_network_discovery

I believe that ZBXNEXT-2721 should resolve current troubles.

Comment by Harsimran Singh [ 2016 Nov 10 ]

how it work for multiple discovery rules ? would all other rules also processed by same single discoverer?

Comment by richlv [ 2016 Nov 10 ]

rubal033, while the documentation does not explicitly say that, if there are multiple discoverer processes, they will split the rules between themselves (but a single rule will never be split).

this feature request asks to split rules among multiple discoverer processes.
potentially, discoverers could also work on multiple rules in parallel, but that will depend on the implementation detail.

Comment by Kazuo Ito [ 2016 Nov 14 ]

I thought about how to divide the range with Zabbix API and register it.

python splite_drule.py <url> <user> <password> <iprange> <split> <json file>

<url>       : http://192.xxx.xx.xxx/zabbix/api_jsonrpc.php
<user>      : admin
<password>  : zabbix
<iprange>   : 192.168.0.0/16
<split>     : 19
<json file> : /tmp/drule.json

drule.json

{
    "jsonrpc": "2.0",
    "method": "drule.create",
    "params": {
        "name": "test",
        "aiprange": "iprange",
        "delay": "86400",
        "dchecks": [
            {
                "type": "12",
                "uniq": "0"
            },
            {
                "type": "11",
                "key_": "1.3.6.1.2.1.1.1",
                "ports": "161",
                "snmp_community": "public",
                "uniq": "0"
            }
        ],
        "status" :"1"
    },
    "auth": "auth",
    "id": 1
}
Comment by Mike Yurlov [ 2020 May 14 ]

If Zabbix have ability add address list in one rule, then Zabbix must handle this list as fast as usual "one subnet" rule.  Otherwise the ability to create lists is simply cosmetic, but not usable in production.

At least the list should be splitted by regexp by comma/space and distributed to all discoveres. Best solution: calculate list address space size and split subnets to all discoverers. Medium: split big subnets to /24 and distribute

Comment by Aleksejs Sestakovs [ 2023 Jun 08 ]

Available in versions:

Comment by Martins Valkovskis [ 2023 Jun 15 ]

Updated documentation:

Generated at Fri Apr 26 14:12:49 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.