[ZBX-10264] Document about vm.memory.size inconsistence on linux Created: 2016 Jan 14  Updated: 2017 May 30  Resolved: 2016 Jan 19

Status: Closed
Project: ZABBIX BUGS AND ISSUES
Component/s: Documentation (D)
Affects Version/s: 2.4.7
Fix Version/s: None

Type: Incident report Priority: Trivial
Reporter: Javier Barroso Assignee: Unassigned
Resolution: Won't fix Votes: 0
Labels: memory
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File free.txt     Text File meminfo.txt     Text File zabbix-available.txt     Text File zabbix-pavailable.txt     Text File zabbix-pused.txt     Text File zabbix-used.txt    

 Description   

Hello,

192.168.20.139 is a linux with zabbix agent 2.4.7:

/opt/zabbix-2.4.8rc1/bin/zabbix_get -s 192.168.20.139 -k vm.memory.size[pused]
53.858698
/opt/zabbix-2.4.8rc1/bin/zabbix_get -s 192.168.20.139 -k vm.memory.size[pavailable]
85.354566

At doc https://www.zabbix.com/documentation/2.0/manual/appendix/items/vm.memory.size_params , I can read:

The sum of vm.memory.size[used] and vm.memory.size[available] does not necessarily equal total. For instance, on FreeBSD ...

I would like to add what is happening on linux world and make clear that pused + pavailable is not 100%

Thank you !



 Comments   
Comment by Ingus Vilnis [ 2016 Jan 14 ]

Hi,

You are using the documentation link for Zabbix 2.0.
In documentation for 2.4 few lines are added: https://www.zabbix.com/documentation/2.4/manual/appendix/items/vm.memory.size_params

on Linux available is free + buffers + cached
on Linux pavailable is free + buffers + cached in relation to total

Is that what you were looking for?

Comment by Aleksandrs Saveljevs [ 2016 Jan 14 ]

FreeBSD is only given as an example. It seems clear to me that the rule is implied for all platforms. martins-v, please see whether there is anything to be done here.

Comment by Javier Barroso [ 2016 Jan 14 ]

I would change:

The sum of vm.memory.size[used] and vm.memory.size[available] does not necessarily equal total.

to

The sum of vm.memory.size[used] and vm.memory.size[available] does not equal total, so the sum of vm.memory.size[pused] and vm.memory.size[pavailable] will not be 100.

Thank you very much

Comment by Aleksandrs Saveljevs [ 2016 Jan 14 ]

On Windows, vm.memory.size[used] plus vm.memory.size[available] does equal total, so that would not be correct.

Comment by Javier Barroso [ 2016 Jan 14 ]

Sorry, I did not test on windows. So only clarifies too that it is not true on linux.

Thank you

Comment by Javier Barroso [ 2016 Jan 18 ]

Hello,

See free command :

$ free -m
total used free shared buffers cached
Mem: 3042 2964 78 0 260 1453
-/+ buffers/cache: 1249 1792
Swap: 2047 0 2047

You can use the line -/+ buffers/cache, which give you the used/available memory (1792+.1249 == 3041 ~= total)

[ sorry I did not check the code before I wrote the next parragraph, zabbix already read MemAvailable from /proc/meminfo if it exist ]
On modern linux (kernel > 3.14), available memory can be read from /proc/meminfo directly, so memory used should be total - available. free output can vary from my example on such systems.

On linux zabbix seems to read from /proc/meminfo, the fields are:

I thinik available memory should be calculated, with the sum of:

MemFree: xx kB
Buffers: yy kB
Cached: zz kB

And the used memory should be total - available.

Looking at memory.c, I'm not sure , but I think zabbix is not summing cached memory to available memory (the same for pavailable):

static int      VM_MEMORY_AVAILABLE(AGENT_RESULT *result)
{
        struct sysinfo  info;
        AGENT_RESULT    result_tmp;
        int             ret = SYSINFO_RET_FAIL;

        if (0 != sysinfo(&info))
        {
                SET_MSG_RESULT(result, zbx_dsprintf(NULL, "Cannot obtain system information: %s", zbx_strerror(errno)));
                return SYSINFO_RET_FAIL;
        }

        init_result(&result_tmp);

        ret = VM_MEMORY_CACHED(&result_tmp);

        if (SYSINFO_RET_OK == ret)
                SET_UI64_RESULT(result, (zbx_uint64_t)(info.freeram + info.bufferram) * info.mem_unit + result_tmp.ui64);
        else
                SET_MSG_RESULT(result, zbx_strdup(NULL, result_tmp.msg));

        free_result(&result_tmp);

        return ret;
}

So , I think this is a bug on the code, modifying the documentation would be only a workaround.

Thank you very much

Comment by Aleksandrs Saveljevs [ 2016 Jan 19 ]

Available memory is calculated as "free" + "buffer" + "cached" in the following lines:

        ret = VM_MEMORY_CACHED(&result_tmp);

        if (SYSINFO_RET_OK == ret)
                SET_UI64_RESULT(result, (zbx_uint64_t)(info.freeram + info.bufferram) * info.mem_unit + result_tmp.ui64);
Comment by Javier Barroso [ 2016 Jan 19 ]

Files:

zabbix-xxx.txt (zabbix_get ... -k vm.memory.size[x]

meminfo.txt (cat /proc/meminfo)
free.txt ( free -b)

The files are created more or less in the same instant (second)

Thank you

Comment by Aleksandrs Saveljevs [ 2016 Jan 19 ]

Pasting them here for convenience:

$ cat free.txt
             total       used       free     shared    buffers     cached
Mem:    3190132736 3172110336   18022400          0  256385024 1596010496
-/+ buffers/cache: 1319714816 1870417920
Swap:   2147475456      98304 2147377152

$ cat meminfo.txt
MemTotal:      3115364 kB
MemFree:         17600 kB
Buffers:        250376 kB
Cached:        1558604 kB
SwapCached:          0 kB
Active:        1542560 kB
Inactive:      1483292 kB
HighTotal:     2228208 kB
HighFree:         7960 kB
LowTotal:       887156 kB
LowFree:          9640 kB
SwapTotal:     2097144 kB
SwapFree:      2097048 kB
Dirty:             228 kB
Writeback:           4 kB
AnonPages:     1216836 kB
Mapped:          58888 kB
Slab:            57892 kB
PageTables:       2796 kB
NFS_Unstable:        0 kB
Bounce:              4 kB
CommitLimit:   3654824 kB
Committed_AS:  1782556 kB
VmallocTotal:   114680 kB
VmallocUsed:      4044 kB
VmallocChunk:   109796 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     4096 kB

$ cat zabbix-available.txt
1870417920

$ cat zabbix-pavailable.txt
58.631351

$ cat zabbix-pused.txt
99.435058

$ cat zabbix-used.txt
3172110336
Comment by Aleksandrs Saveljevs [ 2016 Jan 19 ]

Everything looks correct:

$ echo '(17600+250376+1558604)*1024' | bc
1870417920
$ echo 'scale=10; 1870417920/3190132736' | bc
.5863135094
$ echo 'scale=10; 3172110336/3190132736' | bc
.9943505799

What exactly is your point?

Comment by Javier Barroso [ 2016 Jan 19 ]

Hello, you are right, available value is calculated on the right way. I think that used is not well calculated

For completion, I'm attaching a complete example. Maybe only specific host have this problem?

The incorrect value returned by zabbix is "used" and consecuently "pused".

$ uname -r -v -m -p -o -i -s
Linux 2.6.18-371.4.1.el5 #1 SMP Thu Jan 30 06:09:24 EST 2014 i686 athlon i386 GNU/Linux

$ cat /etc/redhat-release 
CentOS release 5.10 (Final)

$ /opt/zabbix-agent/sbin/zabbix_agentd --version
Zabbix Agent (daemon) v2.4.7 (revision 56694) (12 November 2015)
Compilation time: Nov 16 2015 18:05:05

 $awk '/^Cached|^Buffers|MemFree/ {tot+=$2*1024} END {print tot}' meminfo.txt  
1870417920
 $awk '/^-\/+/ {print $4}' free.txt
1870417920
 $cat zabbix-available.txt 
1870417920
 $awk ' /MemTotal/ {total=$2*1024;}
/^Cached|^Buffers|MemFree/ { available+=$2*1024 }
END {
print "Used ("total" - "available"): "total - available
}' meminfo.txt
Used (3190132736 - 1870417920): 1319714816
 $cat zabbix-used.txt 
3172110336

Thank you very much

Comment by Aleksandrs Saveljevs [ 2016 Jan 19 ]

Item "vm.memory.size[used]" returns exactly the same value as the "free" command.

Regarding the semantics of "vm.memory.size[used]", quoting the documentation at https://www.zabbix.com/documentation/2.4/manual/appendix/items/vm.memory.size_params:

The sum of vm.memory.size[used] and vm.memory.size[available] does not necessarily equal total. For instance, on FreeBSD active, inactive, wired, cached memories are considered used, because they store some useful information.
At the same time inactive, cached, free memories are considered available, because these kinds of memories can be given instantly to processes that request more memory.
So inactive memory is both used and available simultaneously. Because of this, item vm.memory.size[used] is designed for informational purposes only, while item vm.memory.size[available] is designed to be used in triggers.

Comment by Javier Barroso [ 2016 Jan 19 ]

Hello

Everything looks correct:

$ echo '(17600+250376+1558604)*1024' | bc
1870417920
$ echo 'scale=10; 1870417920/3190132736' | bc
.5863135094
$ echo 'scale=10; 3172110336/3190132736' | bc
.9943505799

What exactly is your point?

Used memory + Available memory should be the total memory.

May be should be 2 different used items:

used_with_cache_and_buffers [ and free memory ]
used_without_cache_and_buffers [ and available memory ]

I'm not sure how should be called such items

I hope I explained well now. Sorry for my english

Thank you
update:

I propose, a new option:

  • "noavailable", which would be total - available, so we can monitor pavailable+pnoavailable and have a decent graph on memory.
Comment by Aleksandrs Saveljevs [ 2016 Jan 19 ]

Items "vm.memory.size[used]" and "vm.memory.size[available]" carry different semantics in Zabbix, as explained in the documentation. The sum of these two items is not meant to equal total. This is intended.

If you wish to have an item with your semantics, create a calculated item that would compute the difference between "vm.memory.size[total]" and "vm.memory.size[available]".

Generated at Thu Apr 25 10:40:40 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.