[ZBXNEXT-1660] Wallet for application credentials Created: 2013 Mar 11  Updated: 2024 Apr 10  Resolved: 2020 Oct 09

Status: Closed
Project: ZABBIX FEATURE REQUESTS
Component/s: Java gateway (J), Proxy (P), Server (S)
Affects Version/s: 2.0.5
Fix Version/s: 5.2.0beta1, 5.2 (plan)

Type: New Feature Request Priority: Major
Reporter: Marc Assignee: Vladislavs Sokurenko
Resolution: Fixed Votes: 66
Labels: credentials, encryption, macros, security, wallet
Σ Remaining Estimate: Not Specified Remaining Estimate: Not Specified
Σ Time Spent: Not Specified Time Spent: Not Specified
Σ Original Estimate: Not Specified Original Estimate: Not Specified

Attachments: PNG File Selection_199.png     PNG File Selection_200.png     PNG File ZbxNext1660.png     PNG File image-2020-09-11-13-39-45-786.png     PNG File image-2020-09-11-13-40-09-476.png    
Issue Links:
Causes
Duplicate
is duplicated by ZBXNEXT-7806 Zabbix Server DB Credentials from vailt Closed
is duplicated by ZBXNEXT-3953 Allow frontend script command to be h... Open
is duplicated by ZBXNEXT-4214 Encrypted Macro support Closed
is duplicated by ZBXNEXT-5001 Macro type password Closed
is duplicated by ZBX-15767 Clear text bind password in ldap auth... Closed
Sub-task
Sub-Tasks:
Key
Summary
Type
Status
Assignee
ZBXNEXT-6107 Frontend: Wallet for application cred... Specification change (Sub-task) Closed Miks Kronkalns  
ZBXNEXT-6152 Encrypt secrets in memory Specification change (Sub-task) Closed Andrejs Sitals  
Team: Team A
Sprint: Sprint 66 (Jul 2020), Sprint 67 (Aug 2020), Sprint 68 (Sep 2020), Sprint 69 (Oct 2020)
Story Points: 7

 Description   

Would be/become mission-critical for companies dealing with sensitive data to have no passwords stored in clear-text. Neither in database nor on file system.
To me the only way to achieve this in a comfortable manner (beside using Hardware Security Modules) would be to have something like a wallet, keystore, truststore or whatever one wants to name it.

This could be an encrypted database which stores all credentials and their usage used by or for Zabbix securely on file system.
The database is opened on startup by passing a pass-phrase or key encryption key.
Credentials which are used for items could be provided by macros and are bound to pre-defined criteria like host-names, ip-addresses, host goups or user groups -but never substituted anywhere except for the case they are intended for.

A standard what requires such kind of security is PCI DSS (See: https://www.pcisecuritystandards.org/security_standards/documents.php )



 Comments   
Comment by richlv [ 2013 Mar 11 ]

also, when dealing with proxies, such data would be passed to proxies, stored in ram, but never written to the proxy db

Comment by Oleksii Zagorskyi [ 2013 Mar 12 ]

It's very serious request about macros substitution limitation.

At the moment we have several places where we can try to resolve for example a global usermacro even when I have just user-level account in zabbix frontend.

Just to keep things linked - macros supporting accumulated in ZBXNEXT-353.

Comment by Marc [ 2013 Mar 12 ]

There may be now way to gain access to passwords. Neither by using macros (if macros would be used for this issue) in different context (e.g. email body), nor by items or hosts what are not intended to be used by them.

If macros would be the way for referencing, then it might be useful to add an additional dedicated macro class.

@richlv
Exactly, all clear-text passwords may only reside in RAM or have to be rendered securely.
This would even apply to database passwords used by Zabbix processes. I know this sounds extreme but I really have had a lot of discussions about that with auditors.

For example:
There are thousands of applications running which needs access to their databases. No one can expect that application administrators jump on every application server to enter passwords or key-encryption keys. Any procedure to protect a password has to be accessible by the application itself to make sens and thus actual isn't a protection.

However, the only way that was tolerated is to implement some kind of encoding with the token for decoding be available somewhere hard to find.
The intention according to the auditor is not to make the passwords absolutely secure, but to avoid that if someone gets unauthorized access to the system can't find a clear-text password where expected and thus won't be able to use this by another application or from another system.

I think for Zabbix it's sufficient having an Admin to logon to the Zabbix-Server and passing the wallet passphrase or key.

Comment by Marc [ 2013 Aug 24 ]

This is a nice post which describes the mentioned dilemma.

Comment by Oleksii Zagorskyi [ 2016 Nov 04 ]

Cross references of related things: ZBXNEXT-2957, ZBXNEXT-2461

Comment by Marc [ 2016 Dec 21 ]

I could also think of an implementation without a "wallet". As usual, it's not well thought out and might not consider important aspects. So, any comments are welcome

Example scenario

  • Zabbix server reads a Key-Encryption-Key (KEK) into RAM from a file defined in its configuration file during startup. If Zabbix server is started by "root", it should create the KEK file descriptor and drop privileges, resp. switch user context to "zabbix" afterwards.
  • Zabbix server reads the KEK encrypted Data-Encryption-Key (DEK) from the Zabbix database and decrypts it with the KEK. Zabbix has now the unencrypted DEK available in RAM!
  • User macro configuration in Zabbix frontend gets a toggle button for encryption - per User macro. Once a user macro (resp. its host/template) gets saved with enabled encryption, the respective button(s) are disabled for affected User macro(s). That's to say, the action is not reversible once enabled.
  • In case of encryption, the Zabbix API makes an encryption request to the Zabbix server over a secure channel. The Zabbix server responds with the (DEK) encrypted value(s), which are finally stored as macro value by the API method. User macros have a flag in the database indicating whether being encrypted or not.
  • Zabbix server decrypts User macro values on configuration synchronization, if necessary. Hence the configuration cache has always clear text User macro values.
  • Configuration synchronization with Zabbix proxies remain untouched and can be secured by existing TLS encryption.
  • Zabbix proxies may be configured with their own KEK and DEK similar to Zabbix server.
  • Zabbix frontend/API cannot decrypt data. If encrypted User macros are referenced for displaying or queried by the Zabbix API, then they get resolved to an appropriate error string.

Key management

  • The KEK is stored in a dedicated file. Under normal circumstances, Zabbix started with user "root", owned/accessable by "root" only.
  • The Zabbix server configuration has an option to reference the KEK file.
  • The DEK is automatically rendered and stored securely in the Zabbix database, if it does not exist yet and a usable KEK is available.
  • A new Zabbix server runtime configuration may trigger generation of a new DEK. This includes (in one transaction):
    1. re-reading the KEK from the still open file descriptor.
    2. Implicitely triggering the configuration synchronization.
    3. encrypting and storing of the new generated DEK in the Zabbix database.
    4. encrypting and storing as encrypted flagged macro values from the configuration cache in the Zabbix database.
  • Zabbix server cannot start, if KEK is configured but KEK or DEK are not valid/usable.

Optionally

  • If the KEK option of the Zabbix server configuration file is set to e.g. "stdin", the KEK is read from stdin, resp. it is prompted for it in interactive shells.
  • API method to export and import the encrypted DEK.

Open questions

  • Where should decryption of User macro values be supported?
    1. Item password fields?
    2. Item key parameters?
    3. Web monitoring?
    4. Remote commands?
  • Should decryption of cryptograms be supported via runtime control?
  • If KEK is not configured, how are encrypted values resolved - to an appropriate error string?
  • How to secure the channel between Zabbix frontend/API and Zabbix server for sending the clear text data to encrypt?
    1. Same as for other Zabbix protocols (PSK, Certificate)?
Comment by Duncan Mountford [ 2017 Nov 06 ]

I would love to see this in a future release.

We have many situations where storing passwords in Zabbix would be a great help, however due to security constraints we simply cannot store them in a database as plain text.

This certainly gets my vote!

Comment by Jonathan Boucher [ 2017 Nov 17 ]

I would also love to see this in a future release.

Zabbix could fit all our needs but, our company policies don't allow us to store password in plain text.

This feature is for me a high priority.

Comment by Gatis Rumbens [ 2018 Oct 11 ]

btw simple solution for DB...if we just talk about sensitive data in DB/tables is implemented in MySQL 5.7 and 8.0 . RDBMS supports table space encryption as out-of-the box. Just enable keyring plugin and alter table

ALTER TABLE t1 ENCRYPTION='Y';
Comment by Marc [ 2018 Oct 15 ]

radix, thanks for sharing this. That's an interesting option, although I have my doubts that good encryption can just be a matter of switching it on - without further proper processes However, While encryption on lower levels like database or file system promises some increased security, it still remains transparent to people/systems having access to the data interfaces (e.g. database instance or file system access).

My intention behind this feature request is not only ensuring data gets not stored in plain text on persistent storage but also limiting the risk of revealing sensitive data to database or even Zabbix admins - without having access to the cryptograms and knowing the encrpytion key. The principle idea behind is that only the Zabbix processs may decrypt sensitive data and that the information required for decryption is not found at the same place, resp. it is stored in different locations (DEK in database and KEK on Zabbix server (or elsewhere).

Comment by Francine SAUVAGE [ 2019 Sep 25 ]

Hello,

Finally, Is there a way to encrypt password (in a MACRO {$PASSWORD} in my use case) ?

Regards,

Francine

Comment by Raymond Kuiper [ 2019 Sep 25 ]

Not currently, no.

Please vote on the issue to let Zabbix devteam know that you would like this issue deveolped and possibly send [email protected] a request for a (co-)sponsorship qoute if you are willing to contribute to have the feature developed.

 

 

Comment by Francine SAUVAGE [ 2019 Sep 25 ]

I have already voted. It is critical !!

Comment by Alexei Vladishev [ 2020 Feb 12 ]

Zabbix 5.0 is coming with support of masking for user macros, see ZBXNEXT-2957. It has nothing to do with encryption (yet) but it will provide a nice way to hide secrets in Zabbix UI.

Comment by Francine SAUVAGE [ 2020 Jul 31 ]

Security by obstruction is one of the worst way (not) to do.

Comment by Rostislav Palivoda [ 2020 Aug 11 ]

Acceptance criteria v1.1

  1. Zabbix must be extended to support storage of secrets in external HashiCorp's Vault application
    1. We assume that secrets will be managed by someone else, therefore Zabbix will provide only read-only access to secrets
  2. A new type 'Vault' of user macros must be supported
    1. Value of the user macro will be visible and contain a reference (key) to the secret in the vault
    2. Instead of using macros value, Zabbix must connect to the Vault and retrieve secret value
      1. It must be done on every refresh on configuration data
        1. If Zabbix is unable to retrieve value due to unrecoverable error then use of such macro should lead to item change to not supported item with appropriate error message
      2. New command line option must be supported to trigger refresh of secrets from the Vault
    3. Similarly to Secret macros, there will be no way of seeing macros value by Zabbix user. This information must be protected by any means
  3. Proxy must keep secrets (values of Secret and values of retrieved Vault macros) in memory only, this information must not be stored in the database
    1. It means that Zabbix Proxy cannot start data collection after restart until it receives configuration data update from Zabbix Server for the first time
  4. Secrets stored in memory of Zabbix Server and Proxies must be encrypted with some randomly generated key (symmetric encryption) in order to make memory scans and analysis of core dumps more difficult
  5. A warning message must be written in Zabbix Server log file on startup if we use Secret or Vault macros but there no encryption enabled between Zabbix Server and one of Proxies
  6. Zabbix must support optional storage of database password used for Zabbix Server, Proxies and UI in the Vault
    1. Zabbix UI installation wizard must be extended to enable entering of Vault related parameters
    2. Zabbix UI must be able to cache database password retrieved from the Vault for better performance
      1. Zabbix UI must not lose ability to perform load balancing!
  7. Zabbix Server and Proxy must support the following ways of providing secret token used for authentication with the Vault:
    1. By specifying it in configuration file
    2. By retrieving it from an environment variable
Comment by Vladislavs Sokurenko [ 2020 Aug 25 ]

Implemented in development branch feature/ZBXNEXT-1660-5.1

Comment by Miks Kronkalns [ 2020 Sep 25 ]

Implemented in:

Comment by Miks Kronkalns [ 2020 Oct 02 ]

Updated Zabbix manual:

Updated Zabbix API documentation:

Comment by Alexander Romanov [ 2020 Oct 10 ]

w00t!
It's been just 7 years

Comment by Yurii Polenok [ 2021 Feb 12 ]

Is it possible to add support for KV Secrets Engine - Version 1?
Our company is using v1 and I need to implement Vault support in our Zabbix installation to get rid of a lot of custom scripts.

Generated at Thu Apr 25 00:04:52 EEST 2024 using Jira 9.12.4#9120004-sha1:625303b708afdb767e17cb2838290c41888e9ff0.