ExportXMLWordPrintable

    • Type: New Feature Request
    • Resolution: Unresolved
    • Priority: Trivial
    • None
    • Affects Version/s: 8.0.0rc3
    • Component/s: Server (S)
    • None

      CEP cause-symptom windows always mark the first event in the window as the cause. Every event that enters the window later becomes a symptom, and can never become the cause, even when it is the actual root cause.

      As a result, cause/symptom correctness depends entirely on which problem is detected first, not on which problem caused the others. For example:

      1. A CEP rule correlates datacenter events, grouped by host group DC-Primary, window 10m.
      2. A distribution switch fails. Its "Unavailable by ICMP" trigger uses max(/host/icmpping,#3)=0 with a 1m interval, so the problem opens about 3 minutes after the failure.
      3. The servers and applications behind that switch are monitored with faster-reacting checks: web scenarios every 1m, agent availability, application health items. Their problems open within the first minute.
      4. The first of those downstream problems opens the window and becomes the cause. The distribution switch problem arrives about 2 minutes later and becomes a symptom of one of its own symptoms.

      To make the switch the cause, its trigger would have to fire before every downstream trigger, in every case. In practice that means aligning update intervals and trigger expressions across hundreds or thousands of trigger definitions and templates, and keeping them aligned whenever any template or item interval changes. Even then, polling jitter, proxy delays and queue load can reverse the order.

      Rule conditions and group-by cannot solve this. They control which events enter the window and how windows are split, but not which event in the window is the cause. The first event wins.

      Requested functionality

      Allow a later event in a cause-symptom window to become the cause.

      When an event enters a cause-symptom window, the rule should be able to:

      • evaluate cause conditions (event name, tag/value, host, host group, severity) against the incoming event;
      • if they match, make the incoming event the cause of the window;
      • re-link the previous cause and its symptoms in the window as symptoms of the new cause.

      For the example above, a cause condition on role=distribution would make the switch problem the cause when it arrives, and the earlier server and web scenario problems would become its symptoms.

      Re-linking existing events already exists for manual cause/symptom changes (Mark as cause, Mark selected as symptoms), so CEP would apply the same rank change automatically.

      Expected result

      The cause in a cause-symptom window is determined by rule configuration, not by detection speed. Upstream problems become the cause even when they are detected after their symptoms, without tuning update intervals and trigger expressions across the whole monitoring configuration.

            Assignee:
            Andris Zeila
            Reporter:
            Christos Diamantis
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: