Description
 
We have an issue upgrading our existing Windows Zabbix Agent from 6.2.3 to 6.2.5. All installations (+-600 servers) fails with the following error:

zabbix_agent2 : ERROR: failed to stop Zabbix Agent 2 service: failed to send stop request to service: The service has not been started. 

 
We use msiexec.exe via CMD to install/upgrade our Windows Zabbix agent together with the zabbix_agent2-6.2.5-windows-amd64-openssl.msi.
Below procedure was used in previous upgrades 6.2.0 -> 6.2.3 and proven successful.
 
Steps to reproduce
1. Windows Server 2016/2019/2022 with Windows Zabbix Agent 6.2.3 installed. (Basic config is sufficient)
2. Download the zabbix_agent2-6.2.5-windows-amd64-openssl.msi from the official Zabbix Repo. (Location of choice, example: C:\programdata\Zabbixinstall\ )
3. Open a CMD prompt with administrator privileges
4. Execute following command:

C:\\Windows\system32\msiexec.exe /i "C:\programdata\Zabbixinstall\zabbix_agent2-6.2.5-windows-amd64-openssl.msi" /L*V "C:\ProgramData\Zabbixinstall\log.txt" /qn /norestart 

5. Wait 1-3 minutes
6. The upgrade failed and Windows Zabbix Agent 6.2.3 is removed from the server.
 
Expected

• Windows Zabbix Agent 6.2.3 upgrades to 6.2.5 and starts with the same config as before the upgrade.
   
  Troubleshooting
   
  We tried different upgrading methods/paths and activated extensive logging:
   
  1. Upgrading Windows Zabbix agent from 6.2.3 to 6.2.5 via GUI.
• Execute command in elevated CMD: 

  C:\\Windows\system32\msiexec.exe /i "C:\programdata\Zabbixinstall\zabbix_agent2-6.2.5-windows-amd64-openssl.msi" /L*V "C:\ProgramData\Zabbixinstall\log.txt"

• Run through the installation wizard without changing settings.
• The upgrades finish successfully
  Log can be found in the attachment : log-6.2.3--6.2.5-GUI.txt
   
  2. Upgrading Windows Zabbix agent from 6.2.0 to 6.2.3 via GUI.
• Execute command in elevated CMD: 

  C:\\Windows\system32\msiexec.exe /i "C:\programdata\Zabbixinstall\zabbix_agent2-6.2.5-windows-amd64-openssl.msi" /L*V "C:\ProgramData\Zabbixinstall\log.txt"

• Run through the installation wizard without changing settings.
• The upgrades finish successfully
  Log can be found in the attachment : log-6.2.0--6.2.3-GUI.txt
   
  3. Upgrading Windows Zabbix agent from 6.2.3 to 6.2.5 via CMD.
• Execute command in elevated CMD: 

  C:\\Windows\system32\msiexec.exe /i "C:\programdata\Zabbixinstall\zabbix_agent2-6.2.5-windows-amd64-openssl.msi" /L*V "C:\ProgramData\Zabbixinstall\log.txt"

• The upgrades finish in ERROR
  Log can be found in the attachment : log-6.2.3--6.2.5-CMD.txt
   
  4. Upgrading Windows Zabbix agent from 6.2.0 to 6.2.3 via GUI.
• Execute command in elevated CMD: 

  C:\\Windows\system32\msiexec.exe /i "C:\programdata\Zabbixinstall\zabbix_agent2-6.2.5-windows-amd64-openssl.msi" /L*V "C:\ProgramData\Zabbixinstall\log.txt"

• The upgrades finish successfully
  Log can be found in the attachment : log-6.2.0--6.2.3-CMD.txt
   
  Log Digging
  In all successful logs (log-6.2.3-6.2.5-GUI.txt , log-6.2.06.2.3-GUI.txt , log-6.2.0-6.2.3-CMD.txt) we see the following:
   
   

  MSI (s) (5C:68) [14:06:33:754]: Executing op: CustomActionSchedule(Action=AgentService_Stop,ActionType=3137,Source=BinaryData,Target=WixQuietExec64,CustomActionData="C:\Program Files\Zabbix Agent 2\zabbix_agent2.exe" --config "C:\Program Files\Zabbix Agent 2\zabbix_agent2.conf" --stop) MSI (s) (5C:34) [14:06:33:754]: Invoking remote custom action. DLL: C:\Windows\Installer\MSIA18B.tmp, Entrypoint: WixQuietExec64 MSI (s) (5C:24) [14:06:33:754]: Generating random cookie. MSI (s) (5C:24) [14:06:33:754]: Created Custom Action Server with PID 5316 (0x14C4). MSI (s) (5C:A8) [14:06:33:785]: Running as a service. MSI (s) (5C:A8) [14:06:33:801]: Hello, I'm your 32bit Elevated Non-remapped custom action server. WixQuietExec64:  zabbix_agent2 : ERROR: failed to stop Zabbix Agent 2 service: failed to send stop request to service: The service has not been started. WixQuietExec64:  Error 0x80070001: Command line returned an error. WixQuietExec64:  Error 0x80070001: QuietExec64 Failed WixQuietExec64:  Error 0x80070001: Failed in ExecCommon method MSI (s) (5C:68) [14:06:33:871]: Executing op: ActionStart(Name=AgentService_RegisterRollback,,) CustomAction AgentService_Stop returned actual error code 1603 but will be translated to success due to continue marking

   
   
  The executed action "C:\Program Files\Zabbix Agent 2\zabbix_agent2.exe" --config "C:\Program Files\Zabbix Agent 2\zabbix_agent2.conf" --stop fails, but is ignored due to a "continue marking".
  The "--stop" happens once during the upgrade.
   
  We see the same in log-6.2.3--6.2.5-CMD.txt, but a bit later the following happens.
   
   

  MSI (s) (5C:A4) [14:03:53:973]: Executing op: CustomActionSchedule(Action=NonMsiAgentService_Stop,ActionType=3073,Source=BinaryData,Target=WixQuietExec64,CustomActionData="C:\Program Files\Zabbix Agent 2\zabbix_agent2.exe" -c "C:\Program Files\Zabbix Agent 2\zabbix_agent2.conf" -f=false --stop) MSI (s) (5C:3C) [14:03:53:973]: Invoking remote custom action. DLL: C:\Windows\Installer\MSI3131.tmp, Entrypoint: WixQuietExec64 WixQuietExec64:  Error 0x80070002: Command failed to execute. WixQuietExec64:  Error 0x80070002: QuietExec64 Failed WixQuietExec64:  Error 0x80070002: Failed in ExecCommon method CustomAction NonMsiAgentService_Stop returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)

   
  An extra stop is executed, resulting in a failed without a "continue marking".
   
  Extra info
  We also found out that this issue also occurs when upgrade 6.2.3 -> 6.2.4.
  We tried upgrading 6.2.0 -> 6.2.4 with no luck either.
   
  There is also a similar question on the Zabbix forum without any response.
  https://www.zabbix.com/forum/zabbix-troubleshooting-and-problems/453894-windows-zabbix-agent-2-6-2-4-fails-to-install-on-top-of-6-2-3