Agent2 is missing a feature to start it with passive checks disabled.
If only active checks are used, then for security reasons it's reasonable to just disable listening for incoming connections.
https://www.zabbix.com/documentation/6.4/en/manual/concepts/agent2