Uploaded image for project: 'ZABBIX BUGS AND ISSUES'
  1. ZABBIX BUGS AND ISSUES
  2. ZBX-12919

Preprocessing Manager - extreme memory usage

    Details

    • Type: Incident report
    • Status: Closed
    • Priority: Critical
    • Resolution: Workaround proposed
    • Affects Version/s: 3.4.2, 3.4.3
    • Fix Version/s: None
    • Component/s: Server (S)
    • Environment:

      Description

      Hi all,

      since we updated our Zabbix Server to version 3.4 we are having a lot of trouble regarding the memory usage of preprocessing manager. The memory usage of this process sporadically raises up to 90% of total server memory and results in OOM process kills on OS level -> In most cases the server itself or the database are beeing killed with a loss of data in result.

      Strange fact: This mostly happens on saturdays (I already checked cron-/anacrontab but couldn't find anything).

      Steps to reproduce:
      -

      Result:

      1. High CPU and memory usage of preprocessing manager (see "atop_preprocessingmem.PNG" and "atop_proprocessingcpu.PNG")
      2. OOM Killer killing processes (see "oom_report.png")
      3. Loss of data within zabbix (see "Dataloss.png")

      Seems like I am not the only one with this issue: https://www.zabbix.com/forum/showthread.php?p=203975#post203975

      Tomorrow I will raise the debug level for preprocessing process as mentioned in ZBX-12791

      Thanks in advance!

        Attachments

        1. atop_preprocessingcpu.PNG
          atop_preprocessingcpu.PNG
          75 kB
        2. atop_preprocessingmem.PNG
          atop_preprocessingmem.PNG
          101 kB
        3. Dataloss.png
          Dataloss.png
          134 kB
        4. oom_report.png
          oom_report.png
          28 kB
        5. solved_internal_processes.PNG
          solved_internal_processes.PNG
          56 kB
        6. solved_preprocessing_queue.PNG
          solved_preprocessing_queue.PNG
          15 kB
        7. Status_of_zabbix.png
          Status_of_zabbix.png
          11 kB
        8. Zabbix_environment.PNG
          Zabbix_environment.PNG
          13 kB
        9. zabbix_server.conf.png
          zabbix_server.conf.png
          22 kB

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              abiesenb Andreas Biesenbach
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: