Trend cache of deleted items is flushed to DB after housekeeper cleanup, orphan trends for deleted items persist forever

ExportXMLWordPrintable

    • Sprint candidates

      When a host/item is deleted, historical trends are removed via the housekeeper table. However, if the item still has accumulated data in the trend cache, that data can be flushed to trends/trends_uint after the corresponding housekeeper rows have already been processed and removed.
      Official docs state trends are accumulated in trend cache and flushed on hour change/server stop:
      https://www.zabbix.com/documentation/6.0/en/manual/config/items/history_and_trends

      As a result:
      if trend cache flushes data for a deleted item after the corresponding housekeeper rows were removed, the inserted trend rows become permanent orphans.
      There is no later automatic path that removes them. This looks like a bug/design gap.

      Related but not the same: ZBX-7006 (deleted/disabled item data remaining in trend cache; focused on cache cleanup).

      Steps to reproduce:
      1. Zabbix 6.0.x, internal housekeeping enabled for history/trends.
      2. Collect numeric values so the current hour is accumulating in trend cache.

      - Before delete:
+-----------------+--------+
| table        | count   |
+-----------------+--------+
| trends_uint     |   1075 |
| history_uint    | 106689 |

      3. Delete the host mid-hour.

      Operation:
- Delete hostid=10379
- Delete time:
  2026-07-14 14:30

      4. Confirm housekeeper contains trends_uint tasks for the deleted itemids.

      - Housekeeper rows after delete:
mysql> SELECT * FROM housekeeper;
+---------------+--------------+-----------+-------+
| housekeeperid | tablename    | field     | value |
+---------------+--------------+-----------+-------+
...
| 46926 | trends_uint  | itemid | 37109 |
| 47102 | trends_uint  | itemid | 37143 |
...

      5. Wait for the next automatic housekeeper cycle(or forct it with zabbix_server -R): trends_uint becomes 0 for those itemids; trends_uint tasks disappear from housekeeper.

      - Log:
  3075:20260714:150317.984 executing housekeeper
  3075:20260714:150326.676 housekeeper [deleted 480 hist/trends, 98346 items/triggers, ...]

- DB result after 1st cycle:
+-----------------+--------+
| table        | count   |
+-----------------+--------+
...
| trends_uint     |      0 |
...

- Housekeeper no longer contains trends_uint tasks:
mysql> SELECT * FROM housekeeper;
+---------------+--------------+--------+-------+
| housekeeperid | tablename    | field  | value |
+---------------+--------------+--------+-------+
|         46922 | history_uint | itemid | 37109 |
|         47098 | history_uint | itemid | 37143 |
+---------------+--------------+--------+-------+
2 rows in set (0.00 sec)

      6. Wait for trend flush of that hour (next hour boundary).

      - Next relevant housekeeper cycle happens after hour boundary:
  3075:20260714:160326.775 executing housekeeper

      7. Wait for /Run housekeeper again and check trends_uint for the deleted itemids.

      - Log:
  3075:20260714:160330.233 housekeeper [deleted 408 hist/trends, 10000 items/triggers, ...]

- DB result after 2nd cycle:
+-----------------+--------+
| table        | count   |
+-----------------+--------+
...
| trends_uint     |      2 |
...

- Remaining housekeeper rows still do not include trends_uint:
mysql> SELECT * FROM housekeeper;
+---------------+--------------+--------+-------+
| housekeeperid | tablename    | field  | value |
+---------------+--------------+--------+-------+
|         46922 | history_uint | itemid | 37109 |
|         47098 | history_uint | itemid | 37143 |
+---------------+--------------+--------+-------+
2 rows in set (0.00 sec)

Observed orphan rows
- Final check:
mysql> SELECT  *,FROM_UNIXTIME(clock)  FROM trends_uint WHERE itemid<37187 and itemid>37108;
+--------+------------+-----+-----------+-----------+-----------+----------------------+
| itemid | clock      | num | value_min | value_avg | value_max | FROM_UNIXTIME(clock) |
+--------+------------+-----+-----------+-----------+-----------+----------------------+
|  37109 | 1784005200 |  68 |         0 |         0 |         0 | 2026-07-14 14:00:00  |
|  37143 | 1784005200 |  68 |        98 |        98 |        98 | 2026-07-14 14:00:00  |
+--------+------------+-----+-----------+-----------+-----------+----------------------+
2 rows in set (0.00 sec)

      Result:
      New trends_uint rows appear for deleted itemids.
      Later housekeeper cycles do not remove them.

      Expected:
      Deleted-item trend data must not remain permanently.

            Assignee:
            Romans Trizna
            Reporter:
            Syusei YIn
            Team C
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: