Hello Ramiro!

can you please upload some screenshots of the triggers, which cause this. Also, screenshots from Problems page, when you click on the actions icon, you should be able to see, how many actions where taken:

Also, is there anything about this in the log?

If not, can you increase the log level for alerter and see if that shows anything useful?

zabbix_server -R log_level_increase=alerter

Hello Edgar!

I've wanted to be the more concise I could in the bug report. May be this additional information could help:

• The duplicated emails happen for all the triggers we have.
• I've tried creating an action from scratch, and the mails also is sent twice.
• If I add a second media to the same action (let's say a mattermost message), the mail is sent twice but the other media (mattermost message) is sent once.
• If I create an action from scratch for the mattermos message without mail, the mattermost message is sent once as expected.
• The same system before the update sent just one mail for the same problems/triggers.

I'm attaching next the screenshots you requested.

Problems view:

Actions for the problem:

Mails that I received:

This messages are logged in zabbix_server.log:

810:20181018:115519.976 End of zbx_ipc_socket_read():SUCCEED
810:20181018:115519.977 In send_email() smtp_server:'XXX.XXX.XXX.XXX' smtp_port:25 smtp_security:0 smtp_authentication:0
810:20181018:115521.762 End of send_email():SUCCEED
810:20181018:115521.762 In zbx_ipc_socket_write()
810:20181018:115521.762 End of zbx_ipc_socket_write():SUCCEED
810:20181018:115521.762 __zbx_zbx_setproctitle() title:'alerter #1 [sent 0, failed 0 alerts, idle 8584.726845 sec during 8586.511962 sec]'
810:20181018:115521.762 In zbx_ipc_socket_read()

And in the mail server I can see that the destination address is repeated, when it should be once:

Oct 18 11:56:18 mail qpsmtpd @400000005bc89f1b3447992c 32534 (queue) logging::logterse: ` <zabbix ip>        <zabbixhostname>      <domain> <zabbixfromaddres>   <destination_address>,<destination_address> queued          <>

Do I understand correctly, that only one zabbix user receives mail? From the screenshot, looks like 2 other have errors and did not receive email?

Can you look at the media settings for the user, that receives double emails?

And maybe also run this in db:

select * from media where mediatypeid = 1;

Yes, that's correct. The users with errors don't have media defined.

I've already checked the media for the user, it has the email address where the mails are delivered twice:

Action definition:

The users with errors are in the Zabbix Administrators Group and have no media defined. That why they error.

Database content (I have to masquerade the email and IP adresses):

zabbix=# select * from users where userid=1;
[ RECORD 1 ]+--------------------------------
userid         | 1
alias          | Admin
name           | Zabbix
surname        | Administrator
passwd         | 5fce1b3e34b520afeffb37ce08c7cd66
{{url            | }}
autologin      | 1
autologout     | 0
lang           | en_GB
refresh        | 30s
type           | 3
theme          | default
attempt_failed | 2
attempt_ip     | XXX.XXX.XXX.XXX
attempt_clock  | 1528466907
rows_per_page  | 50

zabbix=# select * from media where mediatypeid = 1 and userid=1;
[ RECORD 1 ]-----------------------------
mediaid     | 1
userid      | 1
mediatypeid | 1
sendto      | <destination_address@ourdomain>
active      | 0
severity    | 63
period      | 1-7,00:00-24:00

This looks very strange, I was unable to reproduce the problem. Not on a clean 4.0 install, and not even after upgrade from version 3.4. I always get just one e-mail. So this is either some local misconfiguration, or I don't have enough info.

I will think about what further steps can we take, to find the cause of the problem.

In the mean time, do you mind to make some very simple tests?

Try to create another user with an e-mail address (or add an address to the existing ones) and see if they also receive two messages.

Also, have you tried to delete the email address from your user and then recreate it? If not, can you please do that?

Please be so kind and provide curl version that is used with a server it might not be reproducible on some other curl versions.

I've found at least another report of this in the forums:
https://www.zabbix.com/forum/zabbix-troubleshooting-and-problems/366972-double-alert-emails-from-zabbix-4-0

I've tried before setting the email address of one the other users of the administrators group and that user also gets the emails duplicated. I've just tried deleting the email address, and adding it back and still get 2 emails.

I was checking the last commits of zabbix-server on github and a works has been done to support multiple email addres, maybe a bug was introduced there o the database migrations has some bug. I'm just guessing though.

We have 3 zabbix servers, we are having this issue with two of them but one doesn't do this. Comparing the servers the only difference we found is that the one that doesn't send duplicated emails uses MySQL as the DB, and the two that send duplicated emails use PSQL. This could support the idea that something is wrong in the db migrations.

Please let me know if I can help with anything else. This servers are for IT use only, so we can test things without too much trouble.

Thank you for your help!

Hello Vladislavs!

Both servers with the problem run on up-to-date Ubuntu 16.04 and PSQL. Also, both have the same curl version:

{{curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets }}

And the one that doesn't send duplicated emails is running on Ubuntu 14.04.5, MySQL and following curl version:

curl 7.35.0 (x86_64-pc-linux-gnu) libcurl/7.35.0 OpenSSL/1.0.1f zlib/1.2.8 libidn/1.28 librtmp/2.3
{{Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smtp smtps telnet tftp }}
Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz TLS-SRP

Strange, I have the exact same setup, but all works as expected for me.
Just wild guess - can you look at Administration -> Media types -> <your email config> -> Options -> Attempts
If the number there is 2, maybe you can try and change it?
As the result of action states, that sending was successful, this probably won't help, but until we can't offer any better ideas...

Hello Edgards,

Atemps was set to 3, setting it to 1 or even 2, doesn't make any difference sadly.

I'm hoping that is something realted to curl as Vladislavs said.

I tested on ubuntu appliance, but the curl version is the same 7.47.0. Ubuntu 16.04.5
The only difference that I can see is postgresql against mysql.
If you have the time to spare, maybe you can try to test this on a clean install?

If you can reproduce the issue then you can try increasing log level of alerted process and attaching here, there might be something that will shed the light on current situation.

Hello Guys!

"Good news". I made a clean install of Zabbix 3.4 on Ubuntu 16.04 with PostgreSQL, added 2 hosts and all good; e-mail alerts are sent one time as expected.

Then I updated to Zabbix 4.0 following the Wiki guide and mail alerts started to be sent twice just like with my other servers.

See the mail's time of arrival in the following capture:

The "zabbix server has just been restarted" alert is when I updated to 4.0, note that this alert is already duplicated, then I unplugged an IP phone to trigger the "Operational status was changed" alert, and got it twice when before updating this alert arrived once (17:01 PROBLEM and 17:02 OK).

This confirms that is something going on whit the alerts when zabbix is using PostgreSQL as DB backend. Please let me know how can I help, what logs do you want or what test do you want me to do.

Thank you again for your support.

Best regards,

Ramiro.

You could try following patch if that's an option, it will show if there were multiple recipients or if it's something with curl

Index: src/libs/zbxmedia/email.c
===================================================================
--- src/libs/zbxmedia/email.c	(revision 86064)
+++ src/libs/zbxmedia/email.c	(working copy)
@@ -707,8 +707,10 @@
 	}
 
 	for (i = 0; i < to_mails->values_num; i++)
+	{
+		zabbix_log(LOG_LEVEL_INFORMATION, "recipient '%s'", ((zbx_mailaddr_t *)to_mails->values[i])->addr);
 		recipients = curl_slist_append(recipients, ((zbx_mailaddr_t *)to_mails->values[i])->addr);
-
+	}
 	if (CURLE_OK != (err = curl_easy_setopt(easyhandle, CURLOPT_MAIL_RCPT, recipients)))
 		goto error;
 
@@ -805,7 +807,7 @@
 	int			ret = FAIL;
 	zbx_vector_ptr_t	from_mails, to_mails;
 
-	zabbix_log(LOG_LEVEL_DEBUG, "In %s() smtp_server:'%s' smtp_port:%hu smtp_security:%d smtp_authentication:%d",
+	zabbix_log(LOG_LEVEL_INFORMATION, "In %s() smtp_server:'%s' smtp_port:%hu smtp_security:%d smtp_authentication:%d",
 			__function_name, smtp_server, smtp_port, (int)smtp_security, (int)smtp_authentication);
 
 	*error = '\0';

Sure thing. Could you point me to some instructions? I suppose I'll have to compile from source zabbix-server with that patch applied.
vso yes, you can try https://www.zabbix.com/documentation/3.0/manual/installation/install

I still consider this might be something specific to your environment. I tried to reproduce the exact steps (ubuntu 16.04, zabbix 3.4, postgresql), emails come in 1. Upgraded to 4.0, still receive just one email. The only thing that differs is, that I only have 1 host (default zabbix server), but I don't think this makes a difference.
Is it possible to try and define a different mailserver? Just for a test, I have no idea about your infrastructure, but a wild guess - is there a spam/antivirus relay in front of the mailserver?
If possible, you should still try to apply the patch, that vso supplied. That could give us much more detailed info about what's happening.

Nothing special on our email infrastructure. Will try with other mail server. I'm compiling from source right now to test the patch. I'll let you know how that goes

I have some news..

If I configure the email server parameters without authentication, the emails are sent twice; but it I configure the same email server but with authentication the emails are sent just once.

I couldn't replicate this with other server, so is something specific between zabbix 4.0 and our SMTP server without authentication. The odd thing is that with zabbix < 4.0 we didn't have this issue. I even got to reproduce the behaviour with the clean installation (zabbix 3.4 1 copy of the email and zabbix 4.0 two copies).

I've compiled from source applying the patch and found nothing remarkable in the logs.

For example, I get this in the zabbix_server.log:
4315:20181029:173820.732 In send_email() smtp_server:'<our_smtp_ip>' smtp_port:25 smtp_security:0 smtp_authentication:0

And 2 copies of the email:

If I use another mail server, like a simple python smtp server I get one copy (notice that there's 1 problem and 1 OK):

$:~/zabbix-4.0.0$ python -m smtpd -n -c DebuggingServer 0.0.0.0:1025
---------- MESSAGE FOLLOWS ----------
From: <zabbix@*****.com.ar>
To: <ralgozino@*****.com.ar>
Date: Mon, 29 Oct 2018 15:10:10 -0300
Subject: PROBLEM: ramironotebook is unavailable by ICMP
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
X-Peer: 127.0.0.1

VHJpZ2dlcjogcmFtaXJvbm90ZWJvb2sgaXMgdW5hdmFpbGFibGUgYnkgSUNNUA0KVHJpZ2dlciBz
dGF0dXM6IFBST0JMRU0NClRyaWdnZXIgc2V2ZXJpdHk6IEF2ZXJhZ2UNClRyaWdnZXIgVVJMOiAN
Cg0KSXRlbSB2YWx1ZXM6DQoNCjEuIElDTVAgcGluZyAocmFtaXJvbm90ZWJvb2s6aWNtcHBpbmcp
OiBEb3duICgwKQ0KMi4gKlVOS05PV04qICgqVU5LTk9XTio6KlVOS05PV04qKTogKlVOS05PV04q
DQozLiAqVU5LTk9XTiogKCpVTktOT1dOKjoqVU5LTk9XTiopOiAqVU5LTk9XTioNCg0KT3JpZ2lu
YWwgZXZlbnQgSUQ6IDEzMDU3
------------ END MESSAGE ------------
---------- MESSAGE FOLLOWS ----------
From: <zabbix@*****.com.ar>
To: <ralgozino@*****.com.ar>
Date: Mon, 29 Oct 2018 15:16:19 -0300
Subject: OK: ramironotebook is unavailable by ICMP
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: base64
X-Peer: 127.0.0.1

VHJpZ2dlcjogcmFtaXJvbm90ZWJvb2sgaXMgdW5hdmFpbGFibGUgYnkgSUNNUA0KVHJpZ2dlciBz
dGF0dXM6IE9LDQpUcmlnZ2VyIHNldmVyaXR5OiBBdmVyYWdlDQpUcmlnZ2VyIFVSTDogDQoNCkl0
ZW0gdmFsdWVzOg0KDQoxLiBJQ01QIHBpbmcgKHJhbWlyb25vdGVib29rOmljbXBwaW5nKTogVXAg
KDEpDQoyLiAqVU5LTk9XTiogKCpVTktOT1dOKjoqVU5LTk9XTiopOiAqVU5LTk9XTioNCjMuICpV
TktOT1dOKiAoKlVOS05PV04qOipVTktOT1dOKik6ICpVTktOT1dOKg0KDQpPcmlnaW5hbCBldmVu
dCBJRDogMTMwNTc=
------------ END MESSAGE ------------

I think that something has changed in zabbix 4.0 that's triggering this odd behaviour. But I can't confirm.

It looks like there is duplicated code to send mail, following patch should fix the issue.

--- src/libs/zbxmedia/email.c	(revision 86070)
+++ src/libs/zbxmedia/email.c	(working copy)
@@ -522,25 +522,6 @@
 			zbx_snprintf(error, max_error_len, "wrong answer on RCPT TO \"%s\"", response);
 			goto close;
 		}
-
-		if (-1 == write(s.socket, cmd, strlen(cmd)))
-		{
-			zbx_snprintf(error, max_error_len, "error sending RCPT TO to mailserver: %s", zbx_strerror(errno));
-			goto close;
-		}
-
-		if (FAIL == smtp_readln(&s, &response))
-		{
-			zbx_snprintf(error, max_error_len, "error receiving answer on RCPT TO request: %s", zbx_strerror(errno));
-			goto close;
-		}
-
-		/* May return 251 as well: User not local; will forward to <forward-path>. See RFC825. */
-		if (0 != strncmp(response, OK_250, strlen(OK_250)) && 0 != strncmp(response, OK_251, strlen(OK_251)))
-		{
-			zbx_snprintf(error, max_error_len, "wrong answer on RCPT TO \"%s\"", response);
-			goto close;
-		}
 	}
 
 	/* send DATA */

Hello Vladislav,

The last patch solves the issue, I no longer get the emails duplicated using our server without authentication.

Fixed in:

• pre-4.0.2rc1 r86390
• pre-4.2.0alpha1 (trunk) r86391

Hi!

I have the same problem about Double email, what are the solutions?

Best!

Hi! Which version are you using ?

Hi !

I have the same problem, zabbix is version 4.0.5. I reboot the server and i'm waiting for new alerts.

Thanks

EDIT : Issue still there after reboot

Is there something in the action log ?

My Zabbix version: 4.0.3

Pleae show action log and media configuration, are there concurrent sessions configured ?

Sorry, I check action log, it was a misconfiguration.

Thanks

Enrique, Could you tell me what was your misconfigurations? maybe it's similar to my mistake

Thank!

Hi,

I set two actions, action A send two triggers A and B to user group A with host group A
and action B send same triggers to user group A with host group B

I simply forget to add host group in each action so we receive email alerts twice.