diff --git a/content/docs/1.4.0/references/storage-class-parameters.md b/content/docs/1.4.0/references/storage-class-parameters.md new file mode 100644 index 000000000..497b1b4c6 --- /dev/null +++ b/content/docs/1.4.0/references/storage-class-parameters.md @@ -0,0 +1,197 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.4.1/references/storage-class-parameters.md b/content/docs/1.4.1/references/storage-class-parameters.md new file mode 100644 index 000000000..497b1b4c6 --- /dev/null +++ b/content/docs/1.4.1/references/storage-class-parameters.md @@ -0,0 +1,197 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.4.2/references/storage-class-parameters.md b/content/docs/1.4.2/references/storage-class-parameters.md new file mode 100644 index 000000000..497b1b4c6 --- /dev/null +++ b/content/docs/1.4.2/references/storage-class-parameters.md @@ -0,0 +1,197 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.4.3/references/storage-class-parameters.md b/content/docs/1.4.3/references/storage-class-parameters.md new file mode 100644 index 000000000..497b1b4c6 --- /dev/null +++ b/content/docs/1.4.3/references/storage-class-parameters.md @@ -0,0 +1,197 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.4.4/references/storage-class-parameters.md b/content/docs/1.4.4/references/storage-class-parameters.md new file mode 100644 index 000000000..497b1b4c6 --- /dev/null +++ b/content/docs/1.4.4/references/storage-class-parameters.md @@ -0,0 +1,197 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.5.0/references/storage-class-parameters.md b/content/docs/1.5.0/references/storage-class-parameters.md new file mode 100644 index 000000000..11ba63995 --- /dev/null +++ b/content/docs/1.5.0/references/storage-class-parameters.md @@ -0,0 +1,206 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +# backendStoreDriver: "" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +#### Backend Store Driver *(field: `parameters.backendStoreDriver`)* +> Default: `"v1"` + + - Use "v2" to run the v2 SPDK-based engine (experimental in 1.5.) If empty, "v1" is assumed. + +> Global setting: [V2 Data Engine](../settings#v2-data-engine). +> More details in [V2 Data Engine Quick Start](../../spdk/quick-start#create-a-storageclass). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.5.1/references/storage-class-parameters.md b/content/docs/1.5.1/references/storage-class-parameters.md new file mode 100644 index 000000000..11ba63995 --- /dev/null +++ b/content/docs/1.5.1/references/storage-class-parameters.md @@ -0,0 +1,206 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +# backendStoreDriver: "" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +#### Backend Store Driver *(field: `parameters.backendStoreDriver`)* +> Default: `"v1"` + + - Use "v2" to run the v2 SPDK-based engine (experimental in 1.5.) If empty, "v1" is assumed. + +> Global setting: [V2 Data Engine](../settings#v2-data-engine). +> More details in [V2 Data Engine Quick Start](../../spdk/quick-start#create-a-storageclass). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.5.2/references/storage-class-parameters.md b/content/docs/1.5.2/references/storage-class-parameters.md new file mode 100644 index 000000000..11ba63995 --- /dev/null +++ b/content/docs/1.5.2/references/storage-class-parameters.md @@ -0,0 +1,206 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +# backendStoreDriver: "" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones and nodes in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +#### Backend Store Driver *(field: `parameters.backendStoreDriver`)* +> Default: `"v1"` + + - Use "v2" to run the v2 SPDK-based engine (experimental in 1.5.) If empty, "v1" is assumed. + +> Global setting: [V2 Data Engine](../settings#v2-data-engine). +> More details in [V2 Data Engine Quick Start](../../spdk/quick-start#create-a-storageclass). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.6.0/advanced-resources/deploy/customizing-default-settings.md b/content/docs/1.6.0/advanced-resources/deploy/customizing-default-settings.md index 12fc1510e..c916dcc1e 100644 --- a/content/docs/1.6.0/advanced-resources/deploy/customizing-default-settings.md +++ b/content/docs/1.6.0/advanced-resources/deploy/customizing-default-settings.md @@ -73,6 +73,7 @@ From the project view in Rancher, go to **Apps && Marketplace > Longhorn > Insta #auto-delete-pod-when-volume-detached-unexpectedly: #disable-scheduling-on-cordoned-node: #replica-zone-soft-anti-affinity: + #replica-disk-soft-anti-affinity: #node-down-pod-deletion-policy: #node-drain-policy: #replica-replenishment-wait-interval: @@ -145,6 +146,7 @@ You can also provide a copy of the `values.yaml` file with the default settings autoSalvage: false disableSchedulingOnCordonedNode: false replicaZoneSoftAntiAffinity: false + replicaDiskSoftAntiAffinity: false volumeAttachmentRecoveryPolicy: never nodeDownPodDeletionPolicy: do-nothing guaranteedInstanceManagerCpu: 15 diff --git a/content/docs/1.6.0/advanced-resources/orphaned-data-cleanup.md b/content/docs/1.6.0/advanced-resources/orphaned-data-cleanup.md index 33fa8513c..f1a17dd5c 100644 --- a/content/docs/1.6.0/advanced-resources/orphaned-data-cleanup.md +++ b/content/docs/1.6.0/advanced-resources/orphaned-data-cleanup.md @@ -147,4 +147,4 @@ Longhorn will not create an `orphan` resource for an orphaned directory when - The volume volume.meta file is missing. - The orphaned replica directory is on an evicted node. - The orphaned replica directory is in an evicted disk. -- The orphaned data cleanup mechanism does not clean up a stable replica, also known as an error replica. Instead, the stale replica is cleaned up according to the [staleReplicaTimeout](../../volumes-and-nodes/create-volumes/#creating-longhorn-volumes-with-kubectl) setting. \ No newline at end of file +- The orphaned data cleanup mechanism does not clean up a stale replica, also known as an error replica. Instead, the stale replica is cleaned up according to the [staleReplicaTimeout](../../volumes-and-nodes/create-volumes/#creating-longhorn-volumes-with-kubectl) setting. diff --git a/content/docs/1.6.0/references/settings.md b/content/docs/1.6.0/references/settings.md index 45b97fe1f..12a87fc59 100644 --- a/content/docs/1.6.0/references/settings.md +++ b/content/docs/1.6.0/references/settings.md @@ -60,6 +60,7 @@ weight: 1 - [Disable Scheduling On Cordoned Node](#disable-scheduling-on-cordoned-node) - [Replica Node Level Soft Anti-Affinity](#replica-node-level-soft-anti-affinity) - [Replica Zone Level Soft Anti-Affinity](#replica-zone-level-soft-anti-affinity) + - [Replica Disk Level Soft Anti-Affinity](#replica-disk-level-soft-anti-affinity) - [Replica Auto Balance](#replica-auto-balance) - [Storage Minimal Available Percentage](#storage-minimal-available-percentage) - [Storage Over Provisioning Percentage](#storage-over-provisioning-percentage) @@ -297,6 +298,7 @@ This information will help us gain insights how Longhorn is being used, which wi - Replica Replenishment Wait Interval - Replica Soft Anti Affinity - Replica Zone Soft Anti Affinity + - Replica Disk Soft Anti Affinity - Restore Concurrent Limit - Restore Volume Recurring Jobs - Snapshot Data Integrity @@ -575,7 +577,10 @@ When this setting is un-checked, the Longhorn Manager will schedule replicas on When this setting is checked, the Longhorn Manager will allow scheduling on nodes with existing healthy replicas of the same volume. -When this setting is un-checked, the Longhorn Manager will not allow scheduling on nodes with existing healthy replicas of the same volume. +When this setting is un-checked, Longhorn Manager will forbid scheduling on nodes with existing healthy replicas of the same volume. + +> **Note:** +> - This setting is superseded if replicas are forbidden to share a zone by the Replica Zone Level Anti-Affinity setting. #### Replica Zone Level Soft Anti-Affinity @@ -583,12 +588,24 @@ When this setting is un-checked, the Longhorn Manager will not allow scheduling When this setting is checked, the Longhorn Manager will allow scheduling new replicas of a volume to the nodes in the same zone as existing healthy replicas. -When this setting is un-checked, Longhorn Manager will not allow scheduling new replicas of a volume to the nodes in the same zone as existing healthy replicas. +When this setting is un-checked, Longhorn Manager will forbid scheduling new replicas of a volume to the nodes in the same zone as existing healthy replicas. > **Note:** > - Nodes that don't belong to any zone will be treated as if they belong to the same zone. > - Longhorn relies on label `topology.kubernetes.io/zone=` in the Kubernetes node object to identify the zone. +#### Replica Disk Level Soft Anti-Affinity + +> Default: `true` + +When this setting is checked, the Longhorn Manager will allow scheduling new replicas of a volume to the same disks as existing healthy replicas. + +When this setting is un-checked, Longhorn Manager will forbid scheduling new replicas of a volume to the same disks as existing healthy replicas. + +> **Note:** +> - Even if the setting is "true" and disk sharing is allowed, Longhorn will seek to use a different disk if possible, even if on the same node. +> - This setting is superseded if replicas are forbidden to share a zone or a node by either of the other Soft Anti-Affinity settings. + #### Replica Auto Balance > Default: `disabled` diff --git a/content/docs/1.6.0/references/storage-class-parameters.md b/content/docs/1.6.0/references/storage-class-parameters.md new file mode 100644 index 000000000..3a6b63dba --- /dev/null +++ b/content/docs/1.6.0/references/storage-class-parameters.md @@ -0,0 +1,216 @@ +--- +title: Storage Class Parameters +weight: 1 +--- + +## Overview + +Storage Class as a resource object has a number of settable parameters. Here's a sample YAML: +```yaml +kind: StorageClass +apiVersion: storage.k8s.io/v1 +metadata: + name: longhorn-test +provisioner: driver.longhorn.io +allowVolumeExpansion: true +reclaimPolicy: Delete +volumeBindingMode: Immediate +parameters: + numberOfReplicas: "3" + staleReplicaTimeout: "2880" + fromBackup: "" + fsType: "ext4" +# mkfsParams: "" +# migratable: false +# encrypted: false +# dataLocality: "disabled" +# replicatAutoBalance: "ignored" +# diskSelector: "ssd,fast" +# nodeSelector: "storage,fast" +# recurringJobSelector: '[{"name":"snap-group", "isGroup":true}, +# {"name":"backup", "isGroup":false}]' +# backingImageName: "" +# backingImageChecksum: "" +# backingImageDataSourceType: "" +# backingImageDataSourceParameters: "" +# unmapMarkSnapChainRemoved: "ignored" +# disableRevisionCounter: false +# replicaSoftAntiAffinity: "ignored" +# replicaZoneSoftAntiAffinity: "ignored" +# replicaDiskSoftAntiAffinity: "ignored" +# backendStoreDriver: "" +``` + +## Built-in Fields +Some fields are common to all Kubernetes storage classes. +See also [Kubernetes Storage Class](https://kubernetes.io/docs/concepts/storage/storage-classes). + +#### Provisioner *(field: `provisioner`)* +Specifies the plugin that will be used for dynamic creation of persistent volumes. For Longhorn, that is always "driver.longhorn.io". +> See [Kubernetes Storage Class: Provisioner](https://kubernetes.io/docs/concepts/storage/storage-classes/#provisioner). + +#### Allow Volume Expansion *(field: `allowVolumeExpansion`)* +> Default: `true` +> See [Kubernetes Storage Class: Allow Volume Expansion](https://kubernetes.io/docs/concepts/storage/storage-classes/#allow-volume-expansion). + +#### Reclaim Policy *(field: `reclaimPolicy`)* +> Default: `Delete` +> See [Kubernetes Storage Class: Reclaim Policy](https://kubernetes.io/docs/concepts/storage/storage-classes/#reclaim-policy). + +#### Volume Binding Mode *(field: `volumeBindingMode`)* +> Default `Immediate` +> See [Kubernetes Storage Class: Volume Binding Mode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode). + +## Longhorn-specific Parameters +Note that some of these parameters also exist and may be specified in global settings. When a volume is provisioned with Kubernetes against a particular StorageClass, StorageClass parameters override the global settings. +These fields will be applied for new volume creation only. If a StorageClass is modified, neither Longhorn nor Kubernetes is responsible for propagating changes to its parameters back to volumes previously created with it. + +#### Number Of Replicas *(field: `parameters.numberOfReplicas`)* +> Default: `3` + +The desired number of copies (replicas) for redundancy. + - Must be between 1 and 20. + - Replicas will be placed across the widest possible set of zones, nodes, and disks in a cluster, subject to other constraints, such as NodeSelector. + +> Global setting: [Default Replica Count](../settings#default-replica-count). + +#### Stale Replica Timeout *(field: `parameters.staleReplicaTimeout`)* +> Default: `30` + +Minutes after a replica is marked unhealthy before it is deemed useless for rebuilds and is just deleted. + +#### From Backup *(field: `parameters.fromBackup`)* +> Default: `""` +> Example: `"s3://backupbucket@us-east-1?volume=minio-vol01&backup=backup-eeb2782d5b2f42bb"` + +URL of a backup to be restored from. + +#### FS Type *(field: `parameters.fsType`)* +> Default: `ext4` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Mkfs Params *(field: `parameters.mkfsParams`)* +> Default: `""` +> For more details, see [Creating Longhorn Volumes with Kubernetes](../../volumes-and-nodes/create-volumes#creating-longhorn-volumes-with-kubectl) + +#### Migratable *(field: `parameters.migratable`)* +> Default: `false` + +Allows for a Longhorn volume to be live migrated from one node to another. Useful for volumes used by Harvester. + +#### Encrypted *(field: `parameters.encrypted`)* +> Default: `false` +> More details in [Encrypted Volumes](../../advanced-resources/security/volume-encryption) + +#### Data Locality *(field: `parameters.dataLocality`)* +> Default: `disabled` + +If enabled, try to keep the data on the same node as the workload for better performance. + - For "best-effort", a replica will be co-located if possible, but is permitted to find another node if not. + - For "strict-local" the Replica count should be 1, or volume creation will fail with a parameter validation error. + - If "strict-local" is not possible for whatever other reason, volume creation will be failed. A "strict-local" replica that becomes displaced from its workload will be marked as "Stopped". + +> Global setting: [Default Data Locality](../settings#default-data-locality) +> More defails in [Data Locality](../../high-availability/data-locality). + +#### Replica Auto-Balance *(field: `parameters.replicaAutoBalance`)* +> Default: `ignored` + +If enabled, move replicas to more lightly-loaded nodes. + - "ignored" means use the global setting. + - Other options are "disabled", "least-effort", "best-effort". + +> Global setting: [Replica Auto Balance](../settings#replica-auto-balance) +> More details in [Auto Balance Replicas](../../high-availability/auto-balance-replicas). + +#### Disk Selector *(field: `parameters.diskSelector`)* +> Default: `""` +> Example: `"ssd,fast"` + +A list of tags to select which disks are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Node Selector *(field: `parameters.nodeSelector`)* +> Default: `""` +> Example: `"storage,fast"` + +A list of tags to select which nodes are candidates for replica placement. +> More details in [Storage Tags](../../volumes-and-nodes/storage-tags) + +#### Recurring Jobs Selector *(field: `parameters.recurringJobsSelector`)* +> Default: `""` +> Example: `[{"name":"backup", "isGroup":true}]` + +A list of recurring jobs that are to be run on a volume. +> More details in [Recurring Snapshots and Backups](../../snapshots-and-backups/scheduling-backups-and-snapshots) + +#### Backing Image Name *(field: `parameters.backingImageName`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Checksum *(field: `parameters.backingImageChecksum`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Type *(field: `parameters.backingImageDataSourceType`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Backing Image Data Source Parameters *(field: `parameters.backingImageDataSourceParameters`)* +> Default: `""` +> See [Backing Image](../../advanced-resources/backing-image#create-and-use-a-backing-image-via-storageclass-and-pvc) + +#### Unmap Mark Snap Chain Removed *(field: `parameters.unmapMarkSnapChainRemoved`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Remove Snapshots During Filesystem Trim](../settings#remove-snapshots-during-filesystem-trim). +> More details in [Trim Filesystem](../../volumes-and-nodes/trim-filesystem). + +#### Disable Revision Counter *(field: `parameters.disableRevisionCounter`)* +> Default: `false` + +> Global setting: [Disable Revision Counter](../settings#disable-revision-counter). +> More details in [Revision Counter](../../advanced-resources/deploy/revision_counter). + +#### Replica Soft Anti-Affinity *(field: `parameters.replicaSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Node Level Soft Anti-Affinity](../settings#replica-node-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling) and [Best Practices](../../best-practices#replica-node-level-soft-anti-affinity). + +#### Replica Zone Soft Anti-Affinity *(field: `parameters.replicaZoneSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Zone Level Soft Anti-Affinity](../settings#replica-zone-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +#### Replica Disk Soft Anti-Affinity *(field: `parameters.replicaDiskSoftAntiAffinity`)* +> Default: `ignored` + + - "ignored" means use the global setting. + - Other values are "enabled" and "disabled". + +> Global setting: [Replica Disk Level Soft Anti-Affinity](../settings#replica-disk-level-soft-anti-affinity). +> More details in [Scheduling](../../volumes-and-nodes/scheduling). + +#### Backend Store Driver *(field: `parameters.backendStoreDriver`)* +> Default: `"v1"` + + - Use "v2" to run the v2 SPDK-based engine (experimental in 1.6.) If empty, "v1" is assumed. + +> Global setting: [V2 Data Engine](../settings#v2-data-engine). +> More details in [V2 Data Engine Quick Start](../../spdk/quick-start#create-a-storageclass). + +## Helm Installs + +If Longhorn is installed via Helm, values in the default storage class can be set by editing the corresponding item in [values.yaml](https://github.com/longhorn/longhorn/blob/v{{< current-version >}}/chart/values.yaml). All of the Storage Class parameters have a prefix of "persistence". For example, `persistence.defaultNodeSelector`. + diff --git a/content/docs/1.6.0/volumes-and-nodes/scheduling.md b/content/docs/1.6.0/volumes-and-nodes/scheduling.md index 7de0bc4b2..f86b404bd 100644 --- a/content/docs/1.6.0/volumes-and-nodes/scheduling.md +++ b/content/docs/1.6.0/volumes-and-nodes/scheduling.md @@ -12,7 +12,7 @@ If any tag has been set in order to be selected for scheduling, the node tag and The first stage is the **node and zone selection stage.** Longhorn will filter the node and zone based on the `Replica Node Level Soft Anti-Affinity` and `Replica Zone Level Soft Anti-Affinity` settings. -The second stage is the **disk selection stage.** Longhorn will filter the disks that satisfy the first stage based on the `Storage Minimal Available Percentage`, `Storage Over Provisioning Percentage`, and other disk-related factors like requested disk space. +The second stage is the **disk selection stage.** Longhorn will filter the disks that satisfy the first stage based on the `Replica Disk Level Soft Anti-Affinity`, `Storage Minimal Available Percentage`, `Storage Over Provisioning Percentage`, and other disk-related factors like requested disk space. #### The Node and Zone Selection Stage @@ -28,11 +28,13 @@ Last, Longhorn will look for an existing node with an existing zone to schedule #### Disk Selection Stage -Once the node and zone stage is satisfied, Longhorn will decide if it can schedule the replica on the disk of the node. Longhorn will check the available disks on the selected node with the matching tag, the total disk space, and the available disk space. +Once the node and zone stage is satisfied, Longhorn will decide whether it can schedule the replica on any disk of the node. Longhorn will check the available disks on the selected node with the matching tag, the total disk space, and the available disk space. It will also check whether another replica already exists and whether anti-affinity is set to be "hard" (no sharing) or "soft" (prefer not to share.) For example, after the node and zone stage, Longhorn finds `Node A` satisfies the requirements for scheduling a replica to the node. Longhorn will check all the available disks on this node. -Assume this node has two disks: `Disk X` with available space 1 GB, and `Disk Y` with available space 2 GB. And the replica Longhorn going to schedule needs 1 GB. With default `Storage Minimal Available Percentage` 25, Longhorn can only schedule the replica on `Disk Y` if this `Disk Y` matches the disk tag, otherwise Longhorn will return failure on this replica selection. But if the `Storage Minimal Available Percentage` is set to 0, and `Disk X` also matches the disk tag, Longhorn can schedule the replica on `Disk X`. +Assume this node has two disks and neither one has another replica: `Disk X` with available space 1 GB, and `Disk Y` with available space 2 GB. And the replica Longhorn going to schedule needs 1 GB. With default `Storage Minimal Available Percentage` 25, Longhorn can only schedule the replica on `Disk Y` if this `Disk Y` matches the disk tag, otherwise Longhorn will return failure on this replica selection. But if the `Storage Minimal Available Percentage` is set to 0, and `Disk X` also matches the disk tag, Longhorn can schedule the replica on `Disk X`. + +Now suppose one of the potential candidate disks has an existing replica and `Replica Disk Soft Anti-Affinity" is set to true. In principle, Longhorn would be allowed to choose either disk, but in practice, it will avoid the existing replica and place the new replica on another disk, even if it is an otherwise inferior choice. ### Settings @@ -41,6 +43,7 @@ For more information on settings that are relevant to scheduling replicas on nod - [Disable Scheduling On Cordoned Node](../../references/settings/#disable-scheduling-on-cordoned-node) - [Replica Soft Anti-Affinity](../../references/settings/#replica-node-level-soft-anti-affinity) (also called Replica Node Level Soft Anti-Affinity) - [Replica Zone Level Soft Anti-Affinity](../../references/settings/#replica-zone-level-soft-anti-affinity) +- [Replica Disk Level Soft Anti-Affinity](../../references/settings/#replica-disk-level-soft-anti-affinity) - [Storage Minimal Available Percentage](../../references/settings/#storage-minimal-available-percentage) - [Storage Over Provisioning Percentage](../../references/settings/#storage-over-provisioning-percentage) - [Allow Empty Node Selector Volume](../../references/settings/#allow-empty-node-selector-volume)