-
Type:
Problem report
-
Resolution: Unresolved
-
Priority:
Trivial
-
None
-
Affects Version/s: 7.0.29
-
Component/s: Templates (T)
-
None
-
Environment:Oracle AI Database 26ai / Oracle 23.26.x, Multitenant RAC, 4 vCPU per node, Zabbix Agent 2 Oracle plugin, godror.
Summary:
Oracle Agent 2 oracle.ts.stats – CDB_FREE_SPACE query causes excessive PX workload on Multitenant RAC
Description:
We are experiencing significant Oracle database load caused by the tablespace statistics query used by Zabbix Agent 2 (oracle.ts.stats) on an Oracle Multitenant RAC environment.
The issue is reproducible and appears to be caused mainly by the way CDB_FREE_SPACE is aggregated in the current query.
Environment:
- Oracle AI Database 26ai
- Oracle Multitenant
- Oracle RAC
- 4 vCPU per RAC node
- Zabbix Agent 2 Oracle monitoring
- Oracle connection module: godror v0.48.0 + ODPI-C 5.5.1
The production Zabbix SQL observed in Oracle has SQL_ID:
a198c2at3jq30
The expensive part of the current query is:
SELECT TRUNC(SUM(bytes)) AS free,
file_id
FROM cdb_free_space
GROUP BY file_id
This aggregates CDB_FREE_SPACE without first restricting the operation to the PDB/tablespace requested by the Zabbix item.
On our Oracle Multitenant environment, the CDB views are internally transformed into CONTAINERS FULL operations and Parallel Query is used.
Under concurrent Zabbix polling, multiple Query Coordinators (QCs) can coexist and each execution can use PX servers. On RAC, this produces a significant amount of additional CPU activity.
We performed controlled tests to isolate the SQL cost.
1. ORIGINAL QUERY
Test target:
PDB: PSPRE
Tablespace: DATA_VIGIADM_1
Controlled-test SQL_ID:
3d9jak4mg9ugk
Plan hash:
1475281604
SQL*Plus elapsed time:
4.25 seconds
PX child statistics:
CPU time: 2.4241 s
Elapsed time: 4.1615 s
Buffer gets: 258,515
Disk reads: 38,665
The execution plan shows:
PX COORDINATOR
CONTAINERS FULL DBA_TABLESPACES
CONTAINERS FULL DBA_DATA_FILES
CONTAINERS FULL DBA_FREE_SPACE
The DBA_FREE_SPACE branch processed approximately:
A-Rows: ~165,000
Buffers: ~257,000
Disk reads: 38,665
A-Time: ~4.12 s
2. PROPOSED QUERY REWRITE
We changed the free-space aggregation so that CDB_FREE_SPACE is restricted to the requested tablespace/PDB before aggregation.
The relevant part of the proposed query is:
SELECT fs.con_id,
fs.file_id,
TRUNC(SUM(fs.bytes)) AS free_bytes
FROM cdb_free_space fs
JOIN (
SELECT DISTINCT con_id
FROM cdb_tablespaces
WHERE con$name = :pdb_name
) c
ON c.con_id = fs.con_id
WHERE fs.tablespace_name = :tbs_name
GROUP BY fs.con_id,
fs.file_id
The result is then joined using both CON_ID and FILE_ID:
ON fs.con_id = df.con_id
AND fs.file_id = df.file_id
3. CONTROLLED TEST OF THE OPTIMIZED QUERY
Controlled-test SQL_ID:
1rbv1dmm5bsrk
Plan hash:
3566185902
SQL*Plus elapsed time:
0.09 seconds
PX child statistics:
CPU time: 0.0496 s
Elapsed time: 0.0627 s
Buffer gets: 2,099
Disk reads: 0
DBA_FREE_SPACE branch:
A-Rows: 4,819
Buffers: 892
A-Time: ~0.02 s
The optimized query returned exactly the same JSON result as the original query for the tested PDB/tablespace:
[\{"DATA_VIGIADM_1":{"contents":"PERMANENT","file_bytes":1521703960576,"max_bytes":1614859452416,"free_bytes":350385405952,"used_bytes":1171318554624,"used_pct_max":94.23,"used_file_pct":76.97,"used_from_max_pct":72.53,"status":1}}]
4. PERFORMANCE COMPARISON
Current query Proposed query
SQL*Plus elapsed 4.25 s 0.09 s
PX child CPU 2.4241 s 0.0496 s
PX child buffer gets 258,515 2,099
PX child disk reads 38,665 0
DBA_FREE_SPACE A-Rows ~165K 4,819
DBA_FREE_SPACE buffers ~257K 892
JSON result Same Same
In this controlled execution, this represents approximately:
- 97.9% lower SQL*Plus elapsed time
- 98.0% lower PX child CPU time
- 99.2% fewer buffer gets
The physical-read reduction should be interpreted with caution because the second execution may benefit from cached blocks. However, the reduction in logical I/O/buffer gets demonstrates that substantially less database work is being performed.
5. NO_PARALLEL TEST
We also tested the proposed query both with and without NO_PARALLEL hints.
With NO_PARALLEL:
SQL_ID: cs5hxpgu51nnd
Plan hash: 3566185902
Buffer gets: 2,099
Disk reads: 0
SQL*Plus elapsed: 0.09 s
Without NO_PARALLEL:
SQL_ID: 1rbv1dmm5bsrk
Plan hash: 3566185902
Buffer gets: 2,099
Disk reads: 0
SQL*Plus elapsed: 0.09 s
Both variants produced the same plan hash and essentially identical execution statistics.
Furthermore, PX COORDINATOR and CONTAINERS FULL operations are still present in the execution plan.
Therefore, the performance improvement does not appear to come from NO_PARALLEL.
The important change is reducing the amount of data processed by CDB_FREE_SPACE before aggregation.
6. IMPACT OBSERVED WITH ZABBIX ENABLED/DISABLED
We also performed an operational A/B comparison with Zabbix enabled and disabled during equivalent 09:00-10:30 time windows.
ASH ON CPU samples:
PSPRE:
Zabbix ON: 1,527
Zabbix OFF: 340
Reduction: 77.7%
NEOPRE:
Zabbix ON: 1,753
Zabbix OFF: 224
Reduction: 87.2%
Total:
Zabbix ON: 3,280
Zabbix OFF: 564
Reduction: 82.8%
These values are ASH sample counts and should not be interpreted as literal CPU seconds.
When Zabbix was enabled, multiple godror sessions and PX processes associated with the tablespace query were observed. This activity disappeared when the agent was stopped.
7. RELATED ZABBIX ISSUE
We found ZBX-24269:
"Oracle tablespace query sometimes failed by timeout"
That issue describes the Oracle tablespace query as non-optimized.
According to the issue history, the implemented resolution increased the item timeout in Zabbix 7.0.22 / 7.4.6 and later.
Increasing the timeout may prevent collection failures, but it does not reduce the Oracle workload generated by the query itself.
Our tests suggest that filtering CDB_FREE_SPACE before aggregation can substantially reduce the database workload while preserving the expected JSON result.
REQUEST
Could the Zabbix team please review the SQL implementation used by oracle.ts.stats and consider changing the CDB_FREE_SPACE aggregation so that the requested PDB/tablespace is filtered before the aggregation?
We can provide:
- Complete current/original SQL
- Complete proposed SQL
- DBMS_XPLAN output with ALLSTATS LAST +PARALLEL +PREDICATE
- GV$SQL statistics
- RAC/PX session evidence
- Additional tests if required
Thank you.