Create and Manage Volumes
To create and manage volumes, use pxctl volume. You can use the created volumes directly with Docker with the -v option.
/opt/pwx/bin/pxctl volume help
NAME:
pxctl volume - Manage volumes
USAGE:
pxctl volume [global options] command [command options] [arguments...]
COMMANDS:
create, c Create a volume
list, l List volumes in the cluster
update Update volume settings
ha-update, u Update volume HA level
snap-interval-update, snap Update volume configuration
inspect, i Inspect a volume
requests Show all pending requests
delete, d Delete a volume
stats, st Volume Statistics
alerts, a Show volume related alerts
import Import data into a volume
clone, cl Create a clone volume
restore, r Restore volume from snapshot
snapshot, s Manage volume snapshots
GLOBAL OPTIONS:
--help, -h show helpCreate volumes
Portworx creates volumes from the global capacity of a cluster. You can expand capacity and throughput by adding a node to the cluster. Portworx protects storage volumes from hardware and node failures through automatic replication.
- Durability: Set replication through policy, using the High Availability setting.
- Each write is synchronously replicated to a quorum set of nodes.
- Any hardware failure means that the replicated volume has the latest acknowledged writes.
- Elastic: Add capacity and throughput at each layer, at any time.
- Volumes are thinly provisioned, only using capacity as needed by the container.
- You can expand and contract the volume’s maximum size, even after data has been written to the volume.
A volume can be created before use by its container or by the container directly at runtime. Creating a volume returns the volume’s ID. This same volume ID is returned in Docker commands (such as Docker volume ls) as is shown in pxctl commands.
Example of creating a volume through pxctl, where the volume ID is returned:
pxctl volume create foobar
3903386035533561360Throughput is controlled per container and can be shared. Volumes have fine-grained control, set through policy.
- Throughput is set by the IO Priority setting. Throughput capacity is pooled.
- Adding a node to the cluster expands the available throughput for reads and writes.
- The best node is selected to service reads, whether that read is from a local storage devices or another node’s storage devices.
- Read throughput is aggregated, where multiple nodes can service one read request in parallel streams.
- Fine-grained controls: Policies are specified per volume and give full control to storage.
- Policies enforce how the volume is replicated across the cluster, IOPs priority, filesystem, blocksize, and additional parameters described below.
- Policies are specified at create time and can be applied to existing volumes.
Set policies on a volume through the options parameter. These options can also be passed in through the scheduler or using the inline volume spec. See the section Inline volume spec below for more details.
Show the available options through the –help command, as shown below:
/opt/pwx/bin/pxctl volume create -h
NAME:
pxctl volume create - Create a volume
USAGE:
pxctl volume create [command options] volume-name
OPTIONS:
--shared make this a globally shared namespace volume
--secure encrypt this volume using AES-256
--secret_key value secret_key to use to fetch secret_data for the PBKDF2 function
--use_cluster_secret Use cluster wide secret key to fetch secret_data
--label pairs, -l pairs list of comma-separated name=value pairs
--size value, -s value volume size in GB (default: 1)
--fs value filesystem to be laid out: none|xfs|ext4 (default: "ext4")
--block_size size, -b size block size in Kbytes (default: 32)
--repl factor, -r factor replication factor [1..3] (default: 1)
--scale value, --sc value auto scale to max number [1..1024] (default: 1)
--io_priority value, --iop value IO Priority: [high|medium|low] (default: "low")
--journal Journal data for this volume
--io_profile value, --prof value IO Profile: [sequential|random|db|db_remote] (default: "sequential")
--sticky sticky volumes cannot be deleted until the flag is disabled [on | off]
--aggregation_level level, -a level aggregation level: [1..3 or auto] (default: "1")
--nodes value comma-separated Node Ids
--zones value comma-separated Zone names
--racks value comma-separated Rack names
--group value, -g value group
--enforce_cg, --fg enforce group during provision
--periodic mins,k, -p mins,k periodic snapshot interval in mins,k (keeps 5 by default), 0 disables all schedule snapshots
--daily hh:mm,k, -d hh:mm,k daily snapshot at specified hh:mm,k (keeps 7 by default)
--weekly weekday@hh:mm,k, -w weekday@hh:mm,k weekly snapshot at specified weekday@hh:mm,k (keeps 5 by default)
--monthly day@hh:mm,k, -m day@hh:mm,k monthly snapshot at specified day@hh:mm,k (keeps 12 by default)
--policy value, --sp value policy names separated by commaCreate with Docker
All docker volume commands are reflected into Portworx storage. For example, a docker volume createcommand provisions a storage volume in a Portworx storage cluster.
docker volume create -d pxd --name <volume_name>As part of the docker volume command, you can add optional parameters through the --opt flag. The option parameters are the same, whether you use Portworx storage through the Docker volume or the pxctlcommands.
Example of options for selecting the container’s filesystem and volume size:
docker volume create -d pxd --name <volume_name> --opt fs=ext4 --opt size=10GInline volume spec
PX supports passing the volume spec inline along with the volume name. This is useful when creating a volume with your scheduler application template inline and you do not want to create volumes before hand.
For example, a PX inline spec can be specified as the following:
docker volume create -d pxd io_priority=high,size=10G,repl=3,snap_schedule="periodic=60#4;daily=12:00#3",name=demovolumeThis is useful when you need to create a volume dynamically while using docker run. For example, the following command will create a volume and launch the container dynamically:
docker run --volume-driver pxd -it -v io_priority=high,size=10G,repl=3,snap_schedule="periodic=60#4;daily=12:00#3",name=demovolume:/data busybox shThe above command will create a volume called demovolume with an initial size of 10G, HA factor of 3, snap scheudle with periodic and daily snapshot creation and a IO priority level of high and start the busybox container.
Each spec key must be comma separated. The following are supported key value pairs:
IO priority - io_priority=[high|medium|low]
Volume size - size=[1..9][G|M|T]
HA factor - repl=[1,2,3]
Block size - bs=[4096...]
Shared volume - shared=true
File System - fs=[xfs|ext4]
Encryption - passphrase=secret
snap_schedule - "periodic=mins#k;daily=hh:mm#k;weekly=weekday@hh:mm#k;monthly=day@hh:mm#k" where k is the number of snapshots to retain.These inline specs can be passed in through the scheduler application template. For example, below is a snippet from a marathon configuration file:
"parameters": [
{
"key": "volume-driver",
"value": "pxd"
},
{
"key": "volume",
"value": "size=100G,repl=3,io_priority=high,name=mysql_vol:/var/lib/mysql"
}],Global Namespace (Shared Volumes)
To use Portworx volumes across nodes and multiple containers, click here.
Inspect volumes
Volumes can be inspected for their settings and usage using the pxctl volume inspect sub menu.
/opt/pwx/bin/pxctl volume inspect clitest
Volume : 970758537931791410
Name : clitest
Size : 1.0 GiB
Format : ext4
HA : 1
IO Priority : LOW
Shared : no
Status : up
State : detached
Reads : 0
Reads MS : 0
Bytes Read : 0
Writes : 0
Writes MS : 0
Bytes Written : 0
IOs in progress : 0
Bytes used : 33 MiB
Replica sets on nodes:
Set 0
Node : 10.99.117.133You can also inspect multiple volumes in one command.
To inspect the volume in json format, use the -j flag. Following is a sample output of:
Following is a sample output of the json volume inspect.
/opt/pwx/bin/pxctl -j v i 486256711004992211
[{
"id": "486256711004992211",
"source": {
"parent": "",
"seed": ""
},
"readonly": false,
"locator": {
"name": "pvc-2910e5ab-1b5e-11e8-97a3-0269077ba1bd",
"volume_labels": {
"namespace": "default",
"pvc": "px-nginx-shared-pvc"
}
},
"ctime": "2018-02-27T01:33:16Z",
"spec": {
"ephemeral": false,
"size": "1073741824",
"format": "ext4",
"block_size": "65536",
"ha_level": "2",
"cos": "low",
"io_profile": "sequential",
"dedupe": false,
"snapshot_interval": 0,
"volume_labels": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"PersistentVolumeClaim\",\"metadata\":{\"annotations\":{\"volume.beta.kubernetes.io/storage-class\":\"px-nginx-shared-sc\"},\"name\":\"px-nginx-shared-pvc\",\"namespace\":\"default\"},\"spec\":{\"accessModes\":[\"ReadWriteOnce\"],\"resources\":{\"requests\":{\"storage\":\"1Gi\"}}}}\n",
"repl": "2",
"shared": "true",
"volume.beta.kubernetes.io/storage-class": "px-nginx-shared-sc",
"volume.beta.kubernetes.io/storage-provisioner": "kubernetes.io/portworx-volume"
},
"shared": true,
"aggregation_level": 1,
"encrypted": false,
"passphrase": "",
"snapshot_schedule": "",
"scale": 0,
"sticky": false,
"group_enforced": false,
"compressed": false,
"cascaded": false,
"journal": false,
"nfs": false
},
"usage": "33964032",
"last_scan": "2018-02-27T01:33:16Z",
"format": "ext4",
"status": "up",
"state": "detached",
"attached_on": "",
"attached_state": "ATTACH_STATE_INTERNAL_SWITCH",
"device_path": "",
"secure_device_path": "",
"replica_sets": [
{
"nodes": [
"k2n3",
"k2n1"
]
}
],
"runtime_state": [
{
"runtime_state": {
"FullResyncBlocks": "[{0 0} {1 0} {-1 0} {-1 0} {-1 0}]",
"ID": "0",
"ReadQuorum": "1",
"ReadSet": "[0 1]",
"ReplNodePools": "1,1",
"ReplRemoveMids": "",
"ReplicaSetCurr": "[0 1]",
"ReplicaSetCurrMid": "k2n3,k2n1",
"ReplicaSetNext": "[0 1]",
"ReplicaSetNextMid": "k2n3,k2n1",
"ResyncBlocks": "[{0 0} {1 0} {-1 0} {-1 0} {-1 0}]",
"RuntimeState": "clean",
"TimestampBlocksPerNode": "[0 0 0 0 0]",
"TimestampBlocksTotal": "0",
"WriteQuorum": "2",
"WriteSet": "[0 1]"
}
}
],
"error": ""
}]Volume snapshots
You can take snapshots of PX volumes. Snapshots are thin and do not take additional space. PX snapshots use branch-on-write so that there is no additional copy when a snapshot is written to. This is done through B+ Trees.
PX version 1.3 and higher
pxctl volume snapshot
NAME:
pxctl volume snapshot - Manage volume snapshots
USAGE:
pxctl volume snapshot command [command options] [arguments...]
COMMANDS:
create, c Create a volume snapshot
OPTIONS:
--help, -h show helpSnapshots are read-only. To restore a volume from a snapshot, use the pxctl volume restore command.
PX version 1.2
pxctl snap --help
NAME:
pxctl snap - Manage volume snapshots
USAGE:
pxctl snap command [command options] [arguments...]
COMMANDS:
create, c Create a volume snapshot
list, l List volume snapshots in the cluster
delete, d Delete a volume snapshot
OPTIONS:
--help, -h show helpSnapshot volumes can be used as any other regular volume. For example, they can be passed into docker run -v snapshot:/mount_path
Volume Clone
PX version 1.3 and higher
In order to create a volume clone from volume/snapshot, Use pxctl volume clone command.
NAME:
pxctl volume clone - Create a clone volume
USAGE:
pxctl volume clone [command options] volume-name-or-ID
OPTIONS:
--name value user friendly name
--label pairs, -l pairs list of comma-separated name=value pairsIn the below example, myvol_clone is the clone from the parent volume myvol
pxctl volume clone -name myvol_clone myvol
Volume clone successful: 55898055774694370Volume Restore
PX version 1.3 and higher
In order to restore a volume from snapshot, Use pxctl volume restore command.
NAME:
pxctl volume restore - Restore volume from snapshot
USAGE:
pxctl volume restore [command options] volume-name-or-ID
OPTIONS:
--snapshot value, -s value snapshot-name-or-IDIn the below example parent volume myvol is restored from its snapshot mysnap. Make sure volume is detached in order to restore from the snapshot.
pxctl volume restore --snapshot mysnap myvol
Successfully started restoring volume myvol from mysnap.