The current implementation of Distributed Monitoring (DM / Proxy) is not very optimal.
- Proxy: The primary Zabbix server is the SPOF. If it fails, the proxy cannot send data to the master and does not have the ability to notify anyone about problems
- DM: They individually check & report when there are issues, which means this can lead to duplicated reports or false positives/negatives when one of the servers detects a failure (which might be due to network issues on the Zabbix server's end)
It would be better if the DM setup would be more like a voting system where a master could be selected which takes care of reporting and the nodes only report the status.
When the primary elected node fails, the voting system kicks in and picks a different server.
This would make the Zabbix monitoring infrastructure more resilient and would only have one server responsible for sending reports at any given time to prevent duplicated reports.
MongoDB uses a similar approach to elect a new master, this might be a valuable improvement to Zabbix' distributed monitoring.