Encrypt Kubernetes PVCs with Vault in airgapped bare metal

Portworx Encrypted Volumes

Portworx has two different kinds of encrypted volumes:

• Encrypted Volumes

  Encrypted volumes are regular volumes which can be accessed from only one node.

• Encrypted Sharedv4 Volumes

  Encrypted sharedv4 volume allows access to the same encrypted volume from multiple nodes.

There are two ways in which Portworx volumes can be encrypted and are dependent on how the Vault secret key is provided to Portworx.

Encryption using Storage Class

In this method, Portworx uses the cluster wide secret key to encrypt PVCs.

• By default, Portworx looks for the secret in path secret/<secret-name> and gets the first value under the path. Users can modify the path with BACKEND_PATH or BASE_PATH, depending on the vault's version.
• The cluster wide secret is the global secret-name for the cluster.
• secret-name can be modified with annotations. See the next section for an example.

From the diagram:

• The top-most secret can be accessed with default BACKEND_PATH secret/ and secret-name px-vault.
• The middle secret can be accessed with default BACKEND_PATH secret/ and secret-name custom/name.
• The bottom secret can be accessed with BACKEND_PATH aws-secret/ and secret-name aws-vault.

Step 1: Store the secret in Vault

vault kv put secret/px-vault foo=bar 

Step 2: Set a cluster wide secret

A cluster wide secret key is a common key that can be used to encrypt all your volumes. This common key needs to be pre-created in your KMS provider. You can set the cluster secret key using the following command:

pxctl secrets set-cluster-key

Enter cluster wide secret key: *****
Successfully set cluster secret key!

In the above prompt you need to enter the secret key that you created in your KMS. This command needs to be run just once for the cluster.

note

To use the secret we just created, enter px-vault for the the cluster wide sceret.

If you are using Vault Namespaces use the following command to set the cluster-wide secret key in a specific vault namespace:

pxctl secrets set-cluster-key --secret_options=vault-namespace=<name of vault-namespace>

Step 3: Create a StorageClass

Create a storage class with the secure parameter set to true.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: px-secure-sc
provisioner: pxd.portworx.com
parameters:
  secure: "true"
  repl: "3"

To create a sharedv4 encrypted volume set the sharedv4 parameter to true as well.

Step 4: Create Persistent Volume Claim

Create a PVC that uses the above px-secure-sc storage class.

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: secure-pvc
spec:
  storageClassName: px-secure-sc
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

Step 5: Verify the volume

Once the PVC has been created, verify the volume created in Portworx is encrypted:

PX_POD=$(kubectl get pods -l name=portworx -n <px-namespace> -o jsonpath='{.items[0].metadata.name}')
kubectl exec $PX_POD -n <px-namespace> -- /opt/pwx/bin/pxctl volume list

ID                 NAME                                      ...  ENCRYPTED  ...
10852605918962284  pvc-xxxxxxxx-xxxx-xxxx-xxxx-080027ee1df7  ...  yes        ...

Encryption using PVC annotations

In this method, each PVC can be encrypted with its own secret key.

Step 1: Create a Storage Class

Create a storage class with the secure parameter set to true.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: px-secure-sc
provisioner: pxd.portworx.com
parameters:
  secure: "true"
  repl: "3"

To create a sharedv4 encrypted volume set the sharedv4 parameter to true as well.

Step 2: Create a PVC with annotations

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: secure-mysql-pvc
  annotations:
    px/secret-name: your-secret-name
spec:
  storageClassName: px-secure-sc
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

important

Make sure secret your-secret-name exists in Vault.

Encryption using PVC annotations with Vault Namespaces

If you have Vault Namespaces enabled and your secret resides inside a specific namespace, you must provide the name of that namespace and the secret key to Portworx.

Step 1: Create a Storage Class

Create a storage class with the secure parameter set to true.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: px-secure-sc
provisioner: pxd.portworx.com
parameters:
  secure: "true"
  repl: "3"

To create a sharedv4 encrypted volume set the sharedv4 parameter to true as well.

Step 2: Create a PVC with annotations

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: secure-mysql-pvc
  annotations:
    px/secret-name: <your-secret-name>
    px/vault-namespace: <your-vault-namesapce>
spec:
  storageClassName: px-secure-sc
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 2Gi

The PVC requires an extra annotation px/vault-namespace to indicate the Vault namespace where the secret key resides. If your key resides in the global vault namespace set in Portworx using the parameter VAULT_NAMESPACE, you don't need to specify this annotation. However if the key resides in any other namespace then this annotation is required.

important

Make sure the secret your-secret-key exists in the namespace your-vault-namespace in Vault.