zabbix-agentd is listening on :: (all interfaces, ipv6-system). If a connection comes in as ipv4 it gets masked as ::ffff:192.168.0.1, this will not be recognized by zabbix-agentd if the allowed server is only a hostname or the ipv4-ip. So in this scenario these entries don't work:
But this one does:
Also giving a IPv4 listening-address solves the problem.