Hello team

We used the Zabbix Helm Chart to deploy zabbix_agent and kube-state-metrics in our cluster. It works well.

However, since we already have 2 Zabbix Proxy servers in the subnet, we don't want to deploy ZabbixProxy inside the cluster and want to re-use our already working Zabbix Proxy server.

How the Kubernetes: Get state metrics template is currently working (3 steps)

Template : https://git.zabbix.com/projects/ZBX/repos/zabbix/browse/templates/app/kubernetes_http/kubernetes_state_http/template_kubernetes_state.yaml

1. apiRequest() function to check api-server API endpoint is working with the provided Auth Token.
2. getMetricsEndpoint() do a GET request to /api/v1/endpoints and fetch the IP / Port of the kube-state-metrics Pod. (ex: 172.16.0.57:8080)
3. getStateMetrics() do a GET request to [http://\|http:]{{ IP }}:{{ PORT}}/metrics from previous values returned by getMetricsEndpoint()

It will ONLY work when Zabbix-Proxy is deployed inside the cluster, because /api/v1/endpoints only return private IPs (containers subnet).

Since we can disable the deployments of ZabbixProxy from the HelmChart, we should be allowed to use an exposed kube-state-metrics endpoint through a LoadBalancer or NodePort service and then configure a Template variable to fetch metrics from that custom endpoint.

As a workaround, i've removed the ApiRequest() and getMetricsEndpoint() functions and hardcoded the URL variable in getMetricsEndpoint() to match the a new zabbix template variable :  {$KUBE.STATE.ENDPOINT.URL} 

var response,
  request = new HttpRequest(),
  url = Kube.params.state_endpoint_url;

with Template variable

{$KUBE.STATE.ENDPOINT.URL} = "https://kube-state-metrics.private.k8s.local:8080/metrics"

I'm not comfortable with Zabbix, so not sure if this is the best solution to make the template compatible with internal or external exposition of kube-state-metrics. 

Another solution could be to request /api/v1/services instead of /api/v1/endpoints to fetch the external IP?

Kindly,

Benjamin