[ZBX-25272] zabbix-agent2 web.certificate.get certificate validation bug Created: 2024 Sep 20 Updated: 2025 Mar 18 Resolved: 2025 Feb 08 |
|
Status: | Closed |
Project: | ZABBIX BUGS AND ISSUES |
Component/s: | Agent2 plugin (G) |
Affects Version/s: | 6.0.32 |
Fix Version/s: | 6.0.39rc1, 7.0.10rc1, 7.2.4rc1, 7.4.0alpha1 |
Type: | Documentation task | Priority: | Major |
Reporter: | Evren Yurtesen | Assignee: | Andrii Fediuk |
Resolution: | Fixed | Votes: | 2 |
Labels: | agent2, certificate, validation, x509 | ||
Remaining Estimate: | Not Specified | ||
Time Spent: | Not Specified | ||
Original Estimate: | Not Specified |
Attachments: |
![]() ![]() |
||||
Issue Links: |
|
||||
Team: | |||||
Sprint: | S24-W44/45, S24-W46/47, DOC S25-W2/3 | ||||
Story Points: | 2 |
Description |
Steps to reproduce:
Result: "result":{"value":"invalid","message":"failed to verify certificate: x509: certificate signed by unknown authority"} Expected:I I expected result to be "valid-but-self-signed"
The problem is that the if statement used to detect validity is expecting `subject == issuer` However this is often not the case. The certificate can still be self-signed without such requirement. Often the subject and issuer won't match on self-signed certificates.
I believe this requirement should be removed from the checks. Not matching subject and issuer does not make a certificate invalid and has little to do with certificate being self signed or not.
A self-signed certificate without, subject == issuer, is still a valid certificate. Also, is it the plugins task to verify such things? Isn't comparing two strings match or not type check what the template triggers should do? Because then one can setup a macro to say "it is ok if subject != issuer". Now we have no control over what the plugin thinks valid/invalid etc. |
Comments |
Comment by Alex Kalimulin [ 2024 Nov 15 ] |
I believe there is some confusion in terminology here. Or, perhaps, a gap in the documentation. Zabbix may return the error valid-but-self-signed for end entity (EE) self-signed certificates, i.e. those EE certificates that have issuer == subject and basic constraint CA flag == TRUE. In this case, there is no CA to validate and no chain of trust. We make this exception because such certificates are very common for testing, internal ad-hoc deployments etc. You are referring to another popular configuration - when an EE certificate is signed with a self-signed CA. In this case, issuer != subject for EE, and, more importantly, there is a chain of trust, which needs to be verified. From the user standpoint, there is not much difference because in both cases EE and CA certificates are controlled by the same user or organization. However, I don't see a way to distinguish between the case when you create your own chain of trust with self-signed CAs and then verify it, and when you verify a public chain of trust that may be broken. In other words, how do you tell the difference between a self-signed CA not in a trust store because you created it and a real CA that is not trusted anymore? Perhaps, there are some workarounds, but whether these scale to all possible situations is unclear. As for not having control over the result - the plugin just gives an opinion along with cert fields. You are free to ignore this opinion. web.certificate.get only fails on connection errors, in other cases it returns JSON with both validation result and certificate details that you can still process. You can ignore $.result.value if you need to. In this case, for example, you can check $.result.message for an unknown authority error and not fire the trigger. Another way to deal with this is to add your CA to OS trust store. |
Comment by Evren Yurtesen [ 2024 Nov 16 ] |
My actual problem is described in the last paragraph (and related to triggers in certificate template). But perhaps something in the code can be improved also. how do you tell the difference between a self-signed CA not in a trust store because you created it and a real CA that is not trusted anymore? What does it help to know the difference? What problem does it solve? Wouldn't it be much more elegant to check if the certificate is from a trusted source or not? I can also ask this in inverse. You said: Another way to deal with this is to add your CA to OS trust store. If you add the CA to OS trust store, how do you tell that it is self-signed/self-issued or not? If you look at the code below, wouldn't it just return valid cert for self-signed? I believe it would not detect the self-signed nature of the cert at all.
func getValidationResult(leaf *x509.Certificate, opts x509.VerifyOptions, subject, issuer string) ValidationResult { var out ValidationResult if _, err := leaf.Verify(opts); err != nil { if errors.As(err, &x509.UnknownAuthorityError{}) && subject == issuer { out = ValidationResult{ "valid-but-self-signed", "certificate verified successfully, but determined to be self signed", } } else { out = ValidationResult{"invalid", fmt.Sprintf("failed to verify certificate: %s", err.Error())} } } else { out = ValidationResult{"valid", "certificate verified successfully"} } return out }
Also, I believe that the code is checking `self-issued` and not `self-signed`. According to the RFC, not all `self-signed` certificates are `self-issued`. RFC 5280 defines self-signed certificates as "self-issued certificates where the digital signature may be verified by the public key bound into the certificate" [7] whereas a self-issued certificate is a certificate "in which the issuer and subject are the same entity".
The real problem I am having is: The plugin might be giving an opinion. But the zabbix web certificate template triggers an error saying certificate is invalid. But certificate is perfectly valid but self-signed. Also the "SSL certificate expires soon" trigger depends on the "SSL certificate is invalid" trigger. Therefore we won't be notified of expired, self-signed (but not self-issued) certs.
|
Comment by Alex Kalimulin [ 2024 Nov 19 ] |
The problem of correct results. We cannot report invalid certificates as "valid-but-...", this will create a great deal of confusion.
If the certificate is not found in a trust store the problem does not go away. If you (or an OS vendor) remove something from a trust store because the CA became untrusted you don't want such certificates be reported as "valid-but-self-signed", because plain and simple - such certificates are not valid.
You don't. By adding the CA to a trust store you declare this CA trusted so it is "valid" without any buts.
RFC 5280 says: "Self-signed certificates are self-issued certificates where the digital signature may be verified by the public key bound into the certificate." This means that all self-signed certificates must be self-issued AND verifiable by a key in this certificate. But it's important to note that RFC 5280 does not define what is a self-signed or self-issued EE certificate, all these definitions only apply to CAs. So, even by the letter of this RFC, web.certificate.get ticks all the right boxes when reporting valid-but-self-signed: it checks self-issued, verifies by the public key and does this for CA, because in the case of single certificate EE is also CA.
It is perfectly valid for you because you trust your CA. As I mentioned before, Zabbix doesn't know about your trust unless you assert it by adding your CA to a trust store. There may be various takes on what is right and wrong but we are not swimming against the current here. curl does exactly the same: it only reports self-signed certificates with no chain of trust (EE is CA), and wrong issuer in other cases. |
Comment by Evren Yurtesen [ 2024 Nov 19 ] |
Hi Alex, Yes, you seem to be right. Thank you for your time. For my problem host, I will just disable the "SSL certificate is invalid" trigger so the "SSL certificate expires soon" (which depends on it) will work. |
Comment by Alex Kalimulin [ 2024 Nov 19 ] |
yurtesen, thank you too for bringing this up anyway, as it helps to improve the documentation. |
Comment by Ryan Squires [ 2025 Jan 08 ] |
Hopefully it's OK to hijack this post. Is there anyway for agent2 (6.4.19) to use a different trust store than the system trust store (RHEL 8, trust anchors / update-ca-trust)? It is not desirable to update all hosts in various environments with different internal dev / test CAs, the trust is only required from the agent, not across the whole system. Something like a path to a directory with pem files, a pkcs12 file, a file with a list of trusted ca pem certs concatenated, etc.? |
Comment by Alex Kalimulin [ 2025 Jan 09 ] |
[email protected], try settings SSL_CERT_DIR and/or SSL_CERT_FILE for the agent: https://pkg.go.dev/crypto/x509#SystemCertPool
|
Comment by Ryan Squires [ 2025 Jan 09 ] |
Kalimulin, that worked, thanks! For clarity for anyone looking for the same solution: export SSL_CERT_DIR=/path/to/pem/encoded/ca/certs/directory or in my systemd service file:
Environment="SSL_CERT_DIR=/path/to/pem/encoded/ca/certs/directory"
Would be great if the documentation was updated to include the SSL_CERT_FILE and SSL_CERT_DIR options! Thanks Again! |
Comment by Andrii Fediuk [ 2025 Feb 07 ] |
Updated documentation: |