Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-23173

memory leak in Agent2 for perf_counter key, when "negative denominator" error happens

XMLWordPrintable

    • Sprint 103 (Aug 2023), Sprint 104 (Sep 2023), Sprint 105 (Oct 2023)
    • 1

      There were a few tickets when this error has been discussed:  ZBX-20269, ZBX-20356, ZBX-21661

      Key point of this report - memory leak on zabbix_agent2.exe process.

      Attaching a few graphs, where agent restarts happened.
      Timestamps are taken from agent2 log.

      2023/07/25 07:21:48.214367 Starting Zabbix Agent 2 (6.4.2)
2023/07/25 12:39:38.655118 Starting Zabbix Agent 2 (6.4.2)
2023/07/25 13:00:46.133411 Starting Zabbix Agent 2 (6.4.2)

      When restarting agent at 13:00:46 it started with DebugLevel=4, log available if needed.

      The "negative denominator" error has been logged only twice, but the memory leak on the graph is constant - increasing linearly.
      As agent2's log does not have PIDs in log headers, it's hard to track log messages in Debug, so here are just those lines:

      2023/07/25 13:13:28.006407 Detected performance counter with negative denominator, retrying in 1 second
...
2023/07/25 13:13:29.006953 Detected performance counter with negative denominator the second time after retry, giving up...
2023/07/25 13:13:29.009218 [Cpu] cannot obtain CPU#1 utilization counter value: Es wurde ein Leistungsindikator mit einem negativen Nenner gefunden.

       

        1. agent2log.txt
          5 kB
        2. def.png
          def.png
          34 kB
        3. history-25-07.png
          history-25-07.png
          44 kB
        4. leak1_fix.diff
          0.7 kB
        5. reproduce_leak1.diff
          1 kB
        6. trends.png
          trends.png
          43 kB

            MVekslers Michael Veksler
            zalex_ua Oleksii Zagorskyi
            Team B
            Votes:
            1 Vote for this issue
            Watchers:
            11 Start watching this issue

              Created:
              Updated:
              Resolved: