ExportXMLWordPrintable

    • Type: Problem report
    • Resolution: Unresolved
    • Priority: Trivial
    • None
    • Affects Version/s: 8.0.0
    • Component/s: Server (S)
    • None

      Problem

      Two triggers on the same item, WARN (last>80) depending on CRIT (last>90). In 8.0.0 the dependency is not reliably applied to the value that changes the parent's state:

      • Dependent created first (WARN has the lower triggerid): a single value above both thresholds raises both WARN and CRIT, i.e. the dependency does not suppress WARN. Reproduced on every run (4/4).
      • Parent created first (CRIT has the lower triggerid): results vary between runs on the same setup. Observed: correct suppression on the jump but WARN raised one value late when CRIT recovers into the WARN band (2 runs), and the duplicate WARN + CRIT on the jump (1 run).

      The variation suggests a race between the evaluation of the parent and of the dependent for the same value, which the dependent loses when it has the lower triggerid.

      In 7.4.16 the same steps gave the expected result in both creation orders (one run per order).

      Impact: template import assigns triggerids in YAML order, so the common layout "WARNING trigger listed before the CRITICAL trigger it depends on" gives the dependent the lower id. Every jump straight past the critical threshold then creates a duplicate WARNING problem next to the HIGH one, and actions run for it.

      Steps to reproduce

      1. Create host dep-order-probe with a Zabbix trapper item x (Numeric unsigned, Allowed hosts: the address zabbix_sender connects from; the 8.0 default {$TRAPPER.ALLOWED_HOSTS} is localhost only).
      2. Create trigger WARN: last(/dep-order-probe/x)>80, severity Warning.
      3. Create trigger CRIT: last(/dep-order-probe/x)>90, severity High.
      4. Edit WARN: add dependency on CRIT. (WARN now has the lower triggerid: "dependent first".)
      5. Send values, a few seconds apart, and check the trigger values after each:
        zabbix_sender -z <server> -s dep-order-probe -k x -o 10
        zabbix_sender -z <server> -s dep-order-probe -k x -o 95
        zabbix_sender -z <server> -s dep-order-probe -k x -o 85
        zabbix_sender -z <server> -s dep-order-probe -k x -o 85
        zabbix_sender -z <server> -s dep-order-probe -k x -o 10
        
      6. Repeat with CRIT created first and WARN (depending on CRIT) second ("parent first"), several times.

      The attached probe.py does this through the API and the trapper protocol and prints both trigger values after each value: python3 probe.py http://<frontend> <server> 10051 child-first|parent-first.

      Result (8.0.0)

      Dependent first, every run:

      Value CRIT WARN Expected WARN
      10 0 0 0
      95 1 1 0
      85 0 1 1
      10 0 0 0

      Parent first, typical run (WARN one value late):

      Value CRIT WARN Expected WARN
      10 0 0 0
      95 1 0 0
      85 0 0 1
      85 0 1 1
      10 0 0 0

      Parent first, other runs: the dependent-first result (WARN=1 together with CRIT=1 on 95).

      Expected

      As observed in 7.4.16: the dependency is evaluated against the parent's state for the same value, independent of triggerid order. A value that makes both CRIT and WARN true raises only CRIT; a value that recovers CRIT into the WARN band raises WARN immediately.

      Environment

      • Zabbix server and frontend 8.0.0, official Docker images zabbix/zabbix-server-pgsql:alpine-8.0.0 and zabbix/zabbix-web-nginx-pgsql:alpine-8.0.0, default server configuration, PostgreSQL 18.6.
      • Comparison: same setup with the alpine-7.4.16 images.
      • Values sent with the trapper protocol, one value per request, 4 s apart.

      Analysis (8.0.0 source, tag 8.0.0 / c81e276e505)

      In 7.4.16, process_triggers() (src/zabbix_server/cachehistory/cachehistory_server.c:252) sorted the batch with zbx_trigger_topoindex_compare before process_trigger(), so a dependent was always processed after its parent and saw the parent's new value.

      The sort was removed by commit 9b297e3d89a ("ZBXNEXT-10353 moved event processing to a separate event processor", 2026-02-28), which replaced the per-trigger process_trigger() with batch assessment by CEP. In 8.0.0 the batch stays in triggerid order (ZBX_DEFAULT_UINT64_PTR_COMPARE_FUNC, cachehistory_server.c:461), and dependencies are decided by CEP (zbx_cep_assess_trigger_events(), cachehistory_server.c:166), which does not see the parent's new value from the same batch:

      1. cep_check_trigger_dependency() (src/zabbix_server/cep/cep.c:1133) returns DENY as soon as the parent has an open problem, before checking whether the parent is in the current batch (which would DEFER). When CRIT recovers into the WARN band, CRIT's OK value is in the same batch, but its problem is still open, so WARN's PROBLEM is denied: WARN one value late.
      2. A dependent whose parent is in the batch is DEFERred and re-checked when its event is opened, in cep_open_trigger_event() (cep.c:1355), with cep_check_trigger_dependency(cep, NULL, ...), which only stops on DENY. If the parent's own problem is still pending (queued, not yet opened), the check returns DEFER and the dependent's problem is opened: the duplicate. Events are queued in triggerid order, so a dependent with a lower triggerid nearly always gets there first.
      3. Events are opened by CEP_WORKER processes (10 by default, src/zabbix_server/server.c:300) from a shared queue, so with the parent first the order between parent and dependent is a race: the run-to-run variation.

      Possible fixes: check "parent in the current batch" before "parent has open problems" in cep_check_trigger_dependency(); in cep_open_trigger_event() (and cep_close_trigger_events()), do not open a dependent's event while the parent has pending events (re-queue it after the parent's), or process a batch's events in topological order (topoindex) as 7.4 did.

      Related

      ZBX-27068 (7.4.2, Won't fix) is a different case: parent and dependent on different items, dependent not re-evaluated until its own item gets a new value.

            Assignee:
            Zabbix Development Team
            Reporter:
            francismh
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: