-
Type:
Problem report
-
Resolution: Unresolved
-
Priority:
Trivial
-
None
-
Affects Version/s: 8.0.0
-
Component/s: Frontend (F), Installation (I)
-
None
Steps to reproduce:
- Run the official web image zabbix/zabbix-web-apache-mysql:ubuntu-8.0.0 (the same happens with zabbix-web-nginx-pgsql:ubuntu-8.0.0) with the default PHP_FPM_PM_* settings.
- Open Monitoring -> Hosts -> Graphs for a host with several graphs and leave the page open with auto-refresh. You can also request chart.php / chart2.php in a loop.
- Watch the RSS of the php-fpm: pool zabbix workers, or the memory of the web container.
Result:
- The RSS of the php-fpm workers keeps growing and is never released.
- In production, with about 3,500 graph requests per hour per frontend node, the web container grew by about 95 MiB/h: from 150 MiB to 3 GiB in 2.5 days, until RAM and swap ran out.
- Each worker reached 240+ MB after about 19,000 requests. All workers were still alive since the container start.
- Almost all of this memory is in [heap] (glibc malloc), not in the Zend memory manager. The php-fpm status page shows "last request memory" of only 4-8 MB.
- The heap is full of small 32-80 byte blocks with glyph data.
Root cause:
- The leak is in FreeType 2.14.2, the Ubuntu 26.04 package libfreetype6 2.14.2+dfsg-1ubuntu0.2. libgd uses it in imagettftext(), which draws all text on graphs.
- It leaks about 46 bytes for every glyph drawn, whatever the font: NotoSansCJKjp-Regular.otf (the 8.0 graph font) and DejaVuSans.ttf give the same result.
- Isolated test: PHP CLI inside the official images, 20,000 imagettftext() calls with the 9-character string "12.5 MB/s". The leak per call is the RSS growth from call 5,000 to call 25,000, divided by 20,000.
| Image / library set | Leak per call |
|---|---|
| ubuntu-7.0.12 (Ubuntu 24.04, PHP 8.3.6, libgd 2.3.3-9ubuntu5, FreeType 2.13.2) | ~12 B (no leak) |
| ubuntu-8.0.0 (Ubuntu 26.04, PHP 8.5.4, libgd 2.3.3-13ubuntu2, FreeType 2.14.2+dfsg-1ubuntu0.2) | 413 B |
| alpine-8.0.0 (PHP 8.5.13, bundled gd, FreeType 2.14.3) | ~14 B (no leak) |
| ubuntu-8.0.0 + libgd 2.3.3-9ubuntu5 from Ubuntu 24.04 | 413 B (libgd is not the cause) |
| ubuntu-8.0.0 + FreeType 2.13.2 (via LD_LIBRARY_PATH) | 11 B |
| ubuntu-8.0.0 + FreeType 2.14.2+dfsg-1 (Ubuntu release build, without the security patches) | 416 B |
| ubuntu-8.0.0 + FreeType 2.15.0+dfsg-1 (Debian sid) | 11 B |
Why it accumulates:
- The web images default to PHP_FPM_PM_MAX_REQUESTS=0, so php-fpm workers never restart.
- The leaked memory therefore stays allocated for the whole life of the container.
- Ubuntu 26.04 has no newer libfreetype6 yet, and the ubuntu-8.0-latest tag has the same package.
Workaround (verified):
PHP_FPM_PM_MAX_REQUESTS=500 for the web container. On a test stand (MySQL 8.4 + web ubuntu-8.0.0, a page that draws 60 graph labels per request, 6 workers):
| Requests | php-fpm total RSS, default | php-fpm total RSS, MAX_REQUESTS=500 |
|---|---|---|
| 2,000 | 148 MiB | 148 MiB |
| 10,000 | 250 MiB | 131 MiB |
Without the setting, memory grows in a straight line (+12.8 MiB per 1,000 requests). With it, memory stays flat. In production, both frontend nodes now run with this setting.
Expected:
- The ubuntu web images are rebuilt with a FreeType that does not leak, once Ubuntu ships one.
- Until then, a non-zero default for PHP_FPM_PM_MAX_REQUESTS in the web images, or an entry in "Known issues -> Graphs".
ZBX-25191got such an entry for a libgd problem.
Reproduction script (run with php inside the ubuntu-8.0.0 web image):
<?php function rss() { preg_match('/VmRSS:\s+(\d+)/', file_get_contents('/proc/self/status'), $m); return (int)$m[1]; } $font = '/usr/share/zabbix/assets/fonts/NotoSansCJKjp-Regular.otf'; for ($i = 1; $i <= 25000; $i++) { $im = imagecreatetruecolor(800, 200); imagettftext($im, 7, 0, 10, 20, 0x333333, $font, '12.5 MB/s'); imagedestroy($im); if ($i == 5000) $a = rss(); } printf("leak = %.0f B per call\n", (rss() - $a) * 1024 / 20000);
Swap only the library to compare, for example: LD_LIBRARY_PATH=/path/to/freetype-2.13.2 php leak.php.
See also ZBX-25191 (an earlier graph problem caused by a system graphics library, documented in Known issues).