Injecting Secrets into Kubernetes Pods via Vault Agent Containers💣
The BigBang vault package supports the Vault’s agent injector service to bind vault secrets to running pods.
For a detailed description and walk-through, see Injecting Secrets into Kubernetes Pods via Vault Agent Containers
There are 3 main parts to getting secrets to Kubernetes pods:
Vault Server Configuration💣
When autoInit is enabled, the Vault helm chart will enable the Kubernetes integration after the server is initialized. See init.sh defined in configmap-for-vault-init.yaml
The command that configure Kubernetes:
vault auth enable kubernetes
vault write auth/kubernetes/config \
kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
issuer="https://kubernetes.default.svc.cluster.local"
Creating A Vault KV Secret💣
You can create a vault secret using Vault’s web interface, the vault command line interface, or the Vault API.
Creating a Vault Keystore And Secret In The BigBang Vault Server POD💣
This shows creating a vault secret using the vault command line interface which can be found in a running bigbang installation
In order to talk to the vault server. we need the admin secret which can be found with this command:
kubectl get secrets -n vault vault-token -o json | jq -r '.data["key"]' | base64 -d
We’ll shell into the pod and configure a secret called testsecret stored under bigbang/gitlab/. The secret will store two encrypted keys: username and password. We will then create a policy called internal-app for the secret and bind the policy to a Kubernetes service account called internal-app and the mynamespace namespace.
kubectl exec -n vault -it vault-vault-0 -- /bin/sh
#inside the running pod
export VAULT_TOKEN=<SECRET_TOKEN>
vault secrets enable -path=bigbang kv-v2
vault kv put bigbang/gitlab/testsecret username="bbuser1" password="password1"
vault kv get bigbang/gitlab/testsecret
# create a policy for a secret
vault policy write internal-app - <<EOF
path "bigbang/data/gitlab/testsecret" {
capabilities = ["read"]
}
EOF
# bind the policy to a k8s service account and namespace
vault write auth/kubernetes/role/internal-app \
bound_service_account_names=internal-app \
bound_service_account_namespaces=mynamespace \
policies=internal-app \
ttl=24h
You should see the following messages as you run the commands:
...
Success! Uploaded policy: internal-app
...
Success! Data written to: auth/kubernetes/role/internal-app
Configure A Deployment For Injection💣
For the injection to work, we’ll create the internal-app service account we mapped earlier.
For this example, you will also need a secret to pull the example image from ironbank:
kubectl create ns mynamespace
kubectl -n mynamespace create sa internal-app
kubectl apply -n mynamespace -f <some_path>/private-registry-secret.yaml
We need to add the following annotations and service account to our deployment definition:
---
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/role: "internal-app"
vault.hashicorp.com/agent-inject-secret-testsecret: "bigbang/gitlab/testsecret"
---
spec:
serviceAccountName: internal-app
A complete deployment (this will also require the imagePullSecrets named private-registry installed to the namespace :
The vault-ingress: true label is required
apiVersion: apps/v1
kind: Deployment
metadata:
name: mshell
labels:
app: mshell
spec:
selector:
matchLabels:
app: mshell
template:
metadata:
labels:
app: mshell
vault-ingress: "true"
annotations:
vault.hashicorp.com/agent-inject: "true"
vault.hashicorp.com/agent-init-first: "true"
vault.hashicorp.com/role: "internal-app"
vault.hashicorp.com/agent-inject-secret-testsecret: "bigbang/data/gitlab/testsecret"
spec:
serviceAccountName: internal-app
imagePullSecrets:
- name: private-registry
containers:
- name: mshell
image: registry1.dso.mil/ironbank/redhat/ubi/ubi8-minimal:8.4
imagePullPolicy: IfNotPresent
command: ["bash"]
args: ["-c", "sleep 3600"]
Saving the full deployment above to /tmp/y.yaml, we can install it:
kubectl apply -n mynamespace -f /tmp/t.yaml
We should see the pod was deployed with additional vault-agent containers
To test that the secret was injected, shell into the mshell pod and run
cat /vault/secrets/testsecret
which produces:
data: map[password:password1 username:bbuser1]
metadata: map[created_time:2021-12-10T14:37:52.7041051Z deletion_time: destroyed:false version:1]