[ZBXNEXT-3205] Add support for wildcards and / or global regular expressions in context macros Created: 2016 Mar 21 Updated: 2024 Apr 10 Resolved: 2020 Jun 11 |
|
Status: | Closed |
Project: | ZABBIX FEATURE REQUESTS |
Component/s: | Frontend (F) |
Affects Version/s: | 3.0.1, 4.0.14, 4.4.1, 5.0.0alpha1 |
Fix Version/s: | 5.0.2rc1, 5.2.0alpha1, 5.2 (plan) |
Type: | Change Request | Priority: | Major |
Reporter: | Ingus Vilnis | Assignee: | Andris Zeila |
Resolution: | Fixed | Votes: | 21 |
Labels: | context, macro, macrocontext, regexps | ||
Σ Remaining Estimate: | Not Specified | Remaining Estimate: | Not Specified |
Σ Time Spent: | Not Specified | Time Spent: | Not Specified |
Σ Original Estimate: | Not Specified | Original Estimate: | Not Specified |
Attachments: | macros.png | ||||||||||||||||
Issue Links: |
|
||||||||||||||||
Sub-Tasks: |
|
||||||||||||||||
Team: | Team A | ||||||||||||||||
Sprint: | Sprint 13, Sprint 64 (May 2020), Sprint 65 (Jun 2020) | ||||||||||||||||
Story Points: | 2 |
Description |
Context macros now work great if the context is a static string. It has some limitations though, e.g. if one would like to set a different trigger threshold for all LLD disk mounts with string home in it. It is impractical to have a hardcoded context macro for every possible mount point which may happen in large organizations. Here are few possible examples for this request: {$LOW_SPACE_LIMIT:/*/home*} or even better: {$LOW_SPACE_LIMIT:@Global regexp} |
Comments |
Comment by richlv [ 2016 Mar 21 ] | ||||||
good idea. for consistency, should probably support regexps + global regexps, thus the first example would look like this : {$LOW_SPACE_LIMIT:/.*/home/.*} using curly braces likely would require quoting :
{$LOW_SPACE_LIMIT:"/home/[a-z]{3}"}
| ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jan 30 ] | ||||||
I suspect the main source of such variable contexts is LLD. The syntax then could look like {$LOW_SPACE_LIMIT:"{{#FSNAME}.regsub(\"(boot|home|root)\", \1)}"}. The remaining part is defining {$LOW_SPACE_LIMIT:boot}, {$LOW_SPACE_LIMIT:home}, {$LOW_SPACE_LIMIT:root} and default {$LOW_SPACE_LIMIT}. This uses existing syntactic features - macro functions - and goes in line with other requests that could be implemented in a form of macro functions for LLD macro. But it has few problems. From user perspective quoting and double quote escaping will very soon get totally unbearable and from developer's perspective parsing such "Russian doll macros" gets exponentially harder with every new nested layer. | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jan 30 ] | ||||||
With regexp directly in macro context it is a bit ambiguous, suppose we have: {$M:[a-z]} => 0 {$M:[0-9a-f]} => 1 What would be the value of {$M:a}? Is "a" a letter or a hexadecimal digit? | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jul 28 ] | ||||||
Today I wouldn't suggest to implement this feature as I proposed 7 months ago. Today we have item preprocessing ( As we discussed, after offloading LLD processing from data gathering processes ( Existing LLD filters can be naturally implemented as another LLD JSON preprocessing (like custom multipliers and deltas became item preprocessing options in ZNXNEXT-1443): "for every row in LLD JSON take {#THIS} macro and if it (not) matches @regexp delete row". This is nice because it reduces complexity of Zabbix. However, such implementation path affects frontend, API, XML import, server and requires DB upgrade. And, as said, it depends on | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jul 28 ] | ||||||
If by "old proposal" you mean issue description or this comment - it is simply out of question for a simple reason: ambiguity. Let's say, I've created two macros:
Here "foo" and "bar" are regular expressions, not just strings. Now Zabbix encounters macro {$M:foobar}. It's context matches both "foo" and "bar", which value should Zabbix choose? Additionally, there is a cost of regexp compilation and matching times the number of different contexts defined for {$M}, context macro substitution will be really slow in this case. If by "old proposal" you mean my initial suggestion I'm not sure it would be cheap either. It does not require changing frontend, API or XML import, it does not require patching DB, but server changes are very substantial. Due to complicated syntax (for example: {$LOW_SPACE_LIMIT:"{{#FSNAME}.regsub(\"(boot|home|root)\", \1)}"}) user friendliness is almost zero. And performance wouldn't be ideal either. | ||||||
Comment by Dimitri Bellini [ 2017 Jul 31 ] | ||||||
Hi Glebs, | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jul 31 ] | ||||||
Dear dimitri.bellini, please check my comment above. | ||||||
Comment by Dimitri Bellini [ 2017 Jul 31 ] | ||||||
Hi Glen, {ISP - Template VCS OS Linux:vfs.fs.size[{#FSNAME},pfree].last(0)}<{$LOW_SPACE_LIMIT:"{#FSNAME}"} and {{#FSNAME}.regexp(@MyExcludePath)}=0 Sorry for so many question but i think "Macro context" is very useful but it lack of flexibility requested for big installations. | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jul 31 ] | ||||||
I don't get the idea of this trigger expression. How is it supposed to work? | ||||||
Comment by Dimitri Bellini [ 2017 Jul 31 ] | ||||||
Sorry for the bad explaination, i will try to clarify my idea: {#FSNAME} LLD macro of a mountpoint{{#FSNAME} .regexp(MyRegExp)} => An Item that explode the content of the last value of {#FSNAME}and try to mach with a desired RegExp. 0 not matched | 1 matched I hope you could understand my intention. | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jul 31 ] | ||||||
In my understanding if you have trigger prototype {ISP - Template VCS OS Linux:vfs.fs.size[{#FSNAME},pfree].last(0)}<{$LOW_SPACE_LIMIT:"{#FSNAME}"} and {{#FSNAME}.regexp(@MyExcludePath)}=0 you will end up with two types of discovered triggers, either {ISP - Template VCS OS Linux:vfs.fs.size[<somemountpoint>,pfree].last(0)}<{$LOW_SPACE_LIMIT:"<somemountpoint>"} and 0=0 or {ISP - Template VCS OS Linux:vfs.fs.size[<somemountpoint>,pfree].last(0)}<{$LOW_SPACE_LIMIT:"<somemountpoint>"} and 1=0 depending on whether "<somemountpoint>" matching @MyExcludePath or not. And the latter trigger does not make much sense because it will never fire. So how is this approach better than having two discovery rules with different filters? | ||||||
Comment by Dimitri Bellini [ 2017 Jul 31 ] | ||||||
Ok, i will try simplify I also explore the simple idea of two "Filesystem Discovery" but i could not have same prototype keys on the same host. | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Jul 31 ] | ||||||
That was exactly my suggestion. There is an ugly workaround of putting spaces in prototype key that won't change the meaning of the key, but will make keys different. It is doable in current Zabbix, but it is messy because you will end up with multiple discovery rules and multiple prototype sets to manage. | ||||||
Comment by Dimitri Bellini [ 2017 Jul 31 ] | ||||||
Ok, no other ways with not so huge development work? | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Aug 01 ] | ||||||
Just to clarify. For all mount points you want an item and a trigger, but for those like "%datafile%" you just want an item (or maybe a trigger that never fires), right? If you insist on writing triggers like this one {ISP - Template VCS OS Linux:vfs.fs.size[{#FSNAME},pfree].last(0)}<{$LOW_SPACE_LIMIT:"{#FSNAME}"} and {{#FSNAME}.regexp(@MyExcludePath)}=0 it looks more like a request to apply trigger functions to non-item objects in trigger expression. Or like a request to save text {#FSNAME} in "calculated" item to use it then in trigger expression. For your specific case I can suggest a custom vfs.fs.discovery (maybe implemented as a module) which would enrich LLD JSON with custom LLD macros based on existing LLD macro values and regexps. Like so: vfs.fs.discovery[{#FSCATEGORY}={#FSNAME}.regsub(.*(datafile).*, \1)] It is basically the same idea as LLD preprocessing, filtering and enrichment, but on agent level instead of server and for individual key. Or one more option is making the "Create enabled/Create disabled" checkbox of every prototype dependent on a condition... | ||||||
Comment by Dimitri Bellini [ 2017 Aug 01 ] | ||||||
Hi Glebs, I did not like so much the item proliferation (option 2) because in huge Zabbix installations can impact on performance but the option 1 seems good also. | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Aug 01 ] | ||||||
I think we need to discuss possible solutions internally. There are so many options... I understand that the best solution in my opinion is a "step too big", but small step in wrong direction is not ideal either. | ||||||
Comment by Dimitri Bellini [ 2017 Aug 01 ] | ||||||
Yes i understand your position and i deal with you, i will wait for a final possible solution. | ||||||
Comment by Glebs Ivanovskis (Inactive) [ 2017 Aug 07 ] | ||||||
Just letting you know, that it seems that discussion and decision regarding this development is postponed. | ||||||
Comment by Dimitri Bellini [ 2017 Aug 07 ] | ||||||
No to so good news | ||||||
Comment by Christoph Loesch [ 2017 Oct 01 ] | ||||||
slightly offtopic but this comment got my attention: Glebs Ivanovskis added a comment - 2017 Aug 01 08:53 it would be awesome if this could be implemented! i have set up prototype items and triggers for network interface discovery to get notified when there is a higher throughput and for this i get many items and triggers created for every single interface per device, because the monitored devices, in my case routers, have many interfaces. i "handled" that by limiting global regular expressions, but that is not optimal this way. hope this example gives an idea as to why it would be great to have an option for conditions here. | ||||||
Comment by Евланов Александр Викторович [ 2017 Dec 11 ] | ||||||
Syntax like this will be enough for solving problem of context part multi-matching: If one regexp matches several macro values - pretend that user knows what she do and use last or first match (this behaviour must be documented of course). {$FS_PFREE_AVG:regexp(/db/vol[0-9]+)}=1
| ||||||
Comment by Vladislavs Sokurenko [ 2020 Apr 23 ] | ||||||
Please also have a look at regsub it could be used to solve the issue but probably not so user friendly customername_1 regsub to extract what is needed {$MACRO:"{{#IFALIAS}.regsub(\"(.*)_([0-9]+)\", \1)}"} Context macro with required info
{$MACRO:"customername"}
| ||||||
Comment by Roberts Lataria (Inactive) [ 2020 Jun 11 ] | ||||||
Implemented in:
Updated documentation: |