On Windows I've found a way : make a powershell script stored on each server at the same location, and add a UserParameter in the zabbix_agentd.conf.
Then on the Zabbix Server I've created a global macro "Target Zabbix Version", a trigger if the host agent version is < the "Target version" then an action that launch the scritp.

On Linux I've tried to make a script that runs apt update and apt install zabbix-agent but it doesn't work well since the script is called by zabbix and it can't be upgraded while running.

It would be nice to have a way to upgrade the agent directly in Zabbix without having to connect on each server to manually upgrade the agent, since it can be time consumming when the number of servers increase.