[ZBX-9347] Information about quoting incoming file for zabbix_sender utility Created: 2015 Feb 27 Updated: 2017 May 30 Resolved: 2017 Feb 14 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Documentation (D) |
Affects Version/s: | 2.2.8, 2.4.4 |
Fix Version/s: | 2.2.11rc1, 2.4.7rc1, 3.0.0alpha4 |
Type: | Incident report | Priority: | Blocker |
Reporter: | Alexey Pustovalov | Assignee: | Unassigned |
Resolution: | Fixed | Votes: | 0 |
Labels: | sender | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Issue Links: |
|
||||
Team: | Team A | ||||
Sprint: | Sprint 1 |
Description |
If objects have whitespaces, these objects must be quoted using double quotes. |
Comments |
Comment by Alexander Vladishev [ 2015 Mar 04 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Related issue: | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by Alexander Vladishev [ 2015 Mar 04 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
zabbix_sender input file format: ,------------, -->+->| value |->+--> ^ '------------' | | ,------------, | '--| whitespace |<-' '------------' value ,---------------, -->+->| quoted string |->+--> | '---------------' ^ | ,---------------, | '->| string |--' '---------------' whitespace ,---------------------, V ,---------, | -->+->+->| <SPACE> |->+->+--> | '---------' ^ | ,---------, | '->| <TAB> |--' '---------' quoted string ,------------------------------------, | ,------------------------------, | | | ,------------------, | | ,---, | V | Any UNICODE | | V ,---, -->| " |->+->+->+->| character except |->+->+->+->| " |--> '---' | | (") and (<LF>) | ^ '---' | '------------------' | | ,---, ,---, | +----->| \ |--| " |------+ | '---' '---' | | ,---, ,---, | '----->| \ |--| n |------' '---' '---' string ,-----------------------------, | ,-----------------------, | V | Any UNICODE | | -->+->| character except |->+--> | (<SPACE>) and (<TAB>) | '-----------------------' | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by Martins Valkovskis [ 2015 Mar 31 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(1) Documented in:
<richlv> the current note seems to be a generic one, it does not seem to be clear that doublequoting is supposed to be used in the input file - maybe we can clarify that a bit martins-v It's true, but we are not mentioning an "input file" on that page before. So I wonder if it is ok to start talking about it there. <richlv> well, it's mentioned in a couple of notes like "If sending many values from an input file..." and "All entries from an input file...". but yes, we don't even mention other things about the input file or even which flag to use for it. maybe it's better to leave all that for the manpage then, and just link to it (like we do already) ? <dimir> Here's what I did for 2.2 . When/If you are happy with that I could add that info to 2.4 and 3.0 also.
Changed pages:
What else could be done:
martins-v I would move the first sentence to man page, and leave the second in the manual because it's like a manual-style background information. Overall, nice to see that much more information is made available to users. Great work! <dimir> Moved UTF-8 to the man page. I agree on keeping timestamp adjustment in the manual. One more thing, currently we have a note about sending up to 250 entries in one go in both the man page and online manual. What do you see as best place to keep it? <Rich> manpage <dimir> So, in addition:
RESOLVED in r55882 (note, due to the branch recreation you'd have to remove the old checkout and pull the new one) <richlv>
<dimir> Thanks, great suggestions. Regarding the last one, I really like the example to be right after simple description and after that list all the special things. Please see how I did it, also added a couple of other changes. RESOLVED in r56225 <richlv> ok, looks real good now, just last minor nitpick <dimir> I'm sure it'd be quicker if you'd do all the changes yourself and I just review, your language skills are much better than mine. Just in case there are more things to improve. RESOLVED in r56230 <richlv> we were so close already, and by now it all looks good to me - thank you | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by richlv [ 2015 Mar 31 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(2) should this be mentioned in the manpage ? it currently talks about doublequoting empty values in the example section <dimir> After discussion with martins-v and richlv we decided it's best to describe in detail the input file format in the man page, generate html version from it and then just link to it from anywhere. Development branch for 2.0: svn://svn.zabbix.com/branches/dev/ZBX-9347 RESOLVED in r53223 a) in "to use hostname from configuration file" let's be specific that it's "agent hostname" <dimir> Based on more discussion on the subject I've added the following changes:
Please review. RESOLVED in r53267 <richlv> discussed in more detail in other subissues, this subissue | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by dimir [ 2015 Apr 20 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
richlv, would it be possible for you to review the (2)? The changes are: -i, --input-file <inputfile> Load values from input file. Specify - to read values from standard input. Each line of the input file must contain 3 whitespace delimited entries: <hostname> <key> <value>. Specify - as <hostname> to use hostname from configuration file (specified with --config) or --host argument. All entries are sent in a sequential order top-down. An example of a line of the input file: host1 item1 sometext If an entry contains whitespace it must be double-quoted (escaping a whitespace will not work): "Zabbix server" item1 "some text" If an entry contains special characters, they must be escaped with a backslash: "Zabbix server" item1 "sending \"some text\"" A backslash itself must not be escaped: "Zabbix server" item1 "C:\Documents\My Folder" "Zabbix server" item1 C:\Documents In case the last character is a backslash double-quotes cannot be used: "Zabbix server" item1 C:\ "Zabbix server" item1 "C:\Documents\" <-- this will not work! -T, --with-timestamps This option can be only used with --input-file option. Each line of the input file must contain 4 whitespace delimited entries: <hostname> <key> <timestamp> <value>. Timestamp should be specified in Unix timestamp format. If target item has triggers referencing it, all timestamps must be in an increasing order, otherwise event calculation will not be correct. An example of a line of the input file: host1 item1 1429533600 sometext For more examples on the input file format please see option --input-file. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by dimir [ 2015 Apr 21 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Let's try to list all the possible ways we could send data to server (tested with latest 2.0 (2.0.14)):
So based on above I came up with the following use cases:
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by dimir [ 2015 Apr 21 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
So, if we separate the above into rules, use cases and notes for the man page: The rules
Examples
Notes Single backslash characters can be used without escaping under certain conditions in quoted entries but please do not rely on it. Support for that might be removed in the future.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by Martins Valkovskis [ 2015 Apr 22 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(3) Maybe worth changing "quoting character is double-quote" into "double-quote is the quoting character" (seems less ambiguous to me). Other than that, the above sounds good to me. <dimir> Thanks, fixed in r53283. CLOSED | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by richlv [ 2015 Aug 28 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(4) "Entries with spaces must be quoted." - what about tab characters ? would it be safer to use "whitespace" here, like we do when describing the line format ? <dimir> Good point, as TAB char actually is preserved. RESOLVED in r55448 <richlv> "Entries with whitespace must be quoted" would sound better - changed in r55450 (and i'm slightly sorry about the typo in the commit message )), still RESOLVED <dimir> Nice :-D CLOSED | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by richlv [ 2015 Aug 28 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(5) rules and notes currently duplicate this information :
i would suggest to remove the second entry <dimir> RESOLVED in r55447 <richlv> currently it says :
assuming that this is good, as "\n" is mentioned the first time we use "linefeed" in this manpage, but not in the second | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by richlv [ 2015 Aug 28 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(6) the last table in the "-i" section is a good thing to have in this issue or maybe in some wiki article, but it seems to document practice that is not really supported and suggested to be used - i would suggest to remove it and place the newline trimming examples in the first table <dimir> RESOLVED in r55447 <richlv> looks like the decision was to remove those examples completely, as they are considered to document an unsupported behaviour - fine by me | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by richlv [ 2015 Sep 08 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(7) while we are at this, maybe it's a good idea to also mention the things referenced in (1) ? namely :
if done so, i'd also suggest removing them from the specific manual page as we have manpages online and maintaining duplicate information has always ended up badly <dimir> Fixed in r55652, r55654. Updated https://www.zabbix.com/documentation/3.0/manual/concepts/sender Rich, could you try to generate the web page from the manual to see how it works? The manual now has some fancy things like tables which might not work. <richlv> indeed, the conversion process seems to get confused by the new bullet list. the table is there, but some fields have incorrect newlines and one seems to have incorrect contents. will investigate. <dimir> Fixes man page format to be acceptable by man2html that comes with standard Debian installation. RESOLVED in r55690 <richlv> i'd suggest removing this part :
the sentence before that does explain the concept well enough, and any changes to the output will require manpage changes <dimir> RESOLVED in r55788 <richlv> thanks, looks good to me martins-v It doesn't look as though we have updated online copies of the manpages here:
In the meantime, users are complaining about the parts that were removed from the main manual, in ZBX-10026; also translators of the Russian manual for 2.4 are mentioning parts "lost" between 2.2 and 2.4 sender documentation. REOPENED <dimir> Correct me if I'm wrong, but it looks like 3.0 manual is up-to-date, the problem is with 2.2 and 2.4? As for removed information, do you suggest we keep 2 copies of the same info one in the manual and one in online manpages? martins-v Apologies, it looks like indeed the problem is with 2.2. and 2.4 (not 3.0). I've fixed the corresponding links above. Ignore my other comments. As long as we have the manpages online, we should not duplicate. <dimir> OK, great, updated the 2.2 and 2.4 online manpages. RESOLVED martins-v Thanks, CLOSED. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by richlv [ 2015 Sep 08 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(8) [D] when we are done, all the great work has to be mentioned in whatsnew <dimir> Documentation updated: martins-v I extended the sentences slightly, purely for consistent sentence structure. Please close the subissue, if satisfied. <dimir> Looks great, thanks! CLOSED | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by dimir [ 2015 Sep 30 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Recreated branch against 2.2, fixed in svn://svn.zabbix.com/branches/dev/ZBX-9347 (r55858). | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Comment by dimir [ 2015 Oct 19 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Fixed in pre-2.2.11rc1 (r56246), pre-2.4.7rc1 (r56247), pre-3.0.0alpha4 (r56250). |