Keep your secrets at a safe place: HashiCorp Vault explained by our expert

Gertjan Roggemans has been a DevOps and System Engineer at Axxes for over five years now. He is also a Competence Coach, which means that he trains colleagues involved in open source infrastructure. He does this using, among other things, keynotes, such as the lecture on HashiCorp Vault he presented at Haxx Digital Edition.

For anyone who has never heard of HashiCorp: this American company creates tools to manage modern applications that preferably run in the cloud. To a lesser extent, they also focus on legacy applications or systems operating on-premise.

HashiCorp has four renowned products that help companies automate their cloud infrastructure. Workload orchestrator Nomad can be compared to Kubernetes and serves to let applications operate. Consul ensures streamlined communication between applications. Terraform is an open-source infrastructure as code software tool and is used to manage and deploy infrastructure. And not to forget: Vault, used for security and secrets management. Secrets management can also be described as ‘storing your sensitive information like passwords and data in a secure and controlled manner’. Just like a physical vault in a bank!

Which Vault do you need?

Just as no vault is alike, there are various versions of the HashiCorp Vault. They have all been programmed in Go, an open-source programming language developed at Google. This ensures that applications can be compiled easily for numerous operating systems.

There are three versions of HashiCorp Vault: a free open source version, an enterprise platform, and enterprise modules. These last two are used in more complex organisations by one or more teams. Whichever version you use: HashiCorp offers its system in a static binary: a single file you place on your system to run Vault. There are no dependencies or libraries you will need to look for or install.

This binary is subsequently used to operate a server with a web interface. Because Vault is fully API-first, you can easily integrate the tool into other applications.

Opening and closing the vault

If you have ever kept someone at a bank, you know that a vault often has a large, imposing door. This is also the case at Vault: this ‘door’ will need to be unsealed the first time you want to open the server. This ‘unsealing’ is done by multiple persons. This is necessary because there are multiple keys – this principle works on the basis of shared shamir keys. These are combined into a single key used to unseal and decrypt the storage.

This may sound like the plot of an exciting film, but you do not actually need multiple people to be able to unseal your Vault. There are already numerous systems for storing data in a secure manner in the cloud. Just like HSM systems, these can be used on-premise to automatically unseal the vault.

Owning a key is one thing, but you will also need to identify yourself. You do this by logging in to Vault using tokens. No need to worry: you will not need to learn the typical string of letters and figures by heart, but can also log in using other authentication systems.

Who may enter?

The benefit of a vault is that not everyone can simply enter or leave it. In Vault, too, you can assign specific policies to certain identifies or applications. You can indicate who can, and who cannot, see what, for example. You can define these policies per API path. You can compare it to Google Docs: your security team can decide who can only read data, who can use things, and who cannot do anything at all.

In the Vault, there are numerous places to store your secrets: the so-called Secrets Engines. There are various versions:

• Generic: Key Value, certificates, SSH keys...
• Cloud: AWS, Azure, GCP...
• Infra: databases, other HashiCorp products, such as Nomad and Consul

We define Key Value secrets as user names and passwords, among other things. An additional useful feature is the option of requesting these in JSON, which is fully in line with the philosophy of HashiCorp Vault to be API-first. Your applications can perfectly read a secret in JSON format using a curl call. If the request complies with the policy (= may this person access the vault?), the required reply will be returned to the application. In this case, your application will also receive metadata. When the secret was created, or when it was removed, for example.

The oddballs

Old and legacy applications can also be provided with secrets using an alternative approach. The Vault Agent is used for this, a small application you launch using specific options. It is essentially a sort of link. Anyone who wants to use it must write a template that sets out the secrets needed by the application. The agent subsequently requests these in Vault, enters them into the template, and renders the template, after which it can be used as a configuration file by your application.

This concludes our vault metaphor. HashiCorp goes one step further with Vault. When you store a secret, it may be useful to know who was able to access or use what. After all, you do not want somebody who is leaving your company to maintain access, but you also do not want to change everything in this situation.

HashiCorp has found a solution for this: Dynamic Secrets. These enable you to configure a specific period for the matters requested from your Vault. After this ‘lease period’, anything that was removed from the vault will simply be deleted. This ensures that your security team always knows who has access to your sensitive information and with which rights.

Vault also has another significant use. Because we increasingly process data in the cloud, there are more and more questions about privacy and the GDPR. HashiCorp addresses these by means of Encryption as a Service. The idea? Vault will encrypt our data.

This takes place at a central level, which means that you will need to separately implement encryption in each application. The application sends the data to Vault and the data is returned in an encrypted format, after which it can store it in the database in this form. It can subsequently be decrypted using the Vault API. This allows you to configure who has access to the unencrypted form through Vault.