Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions modules/persistent-storage-byok.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
[id="byok_{context}"]
= User-managed encryption

[role="_abstract"]
The user-managed encryption feature allows you to provide keys during installation that encrypt {product-title} node root volumes, and enables all managed storage classes to use these keys to encrypt provisioned storage volumes. You must specify the custom key in the `platform.<cloud_type>.defaultMachinePlatform` field in the install-config YAML file.

This features supports the following storage types:
Expand All @@ -17,5 +18,7 @@ This features supports the following storage types:
* Microsoft Azure Disk storage

* Google Cloud Platform (GCP) persistent disk (PD) storage
+
For information about installing with user-managed encryption for GCP PD, see _Installation configuration parameters_ under _Additional resources_.

* IBM Virtual Private Cloud (VPC) Block storage
2 changes: 2 additions & 0 deletions modules/persistent-storage-csi-gcp-hyperdisk-ha-overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@
= Hyperdisk-balanced high availability disks overview

[role="_abstract"]
You can improve application resilience against zone failures by using Hyperdisk Balanced High Availability volumes that synchronously replicate data across two zones in the same region.

{product-title} supports Hyperdisk Balanced High Availability volumes.

Hyperdisk Balanced High Availability volumes are useful for:
Expand Down
7 changes: 5 additions & 2 deletions modules/persistent-storage-csi-gcp-hyperdisk-limitations.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,9 @@
[id="persistent-storage-csi-gcp-hyperdisk-limitations_{context}"]
= C3 and N4 instance type limitations

[role="_abstract"]
Before deploying hyperdisk-balanced disks on C3 bare metal or N4 machine series instances, review the volume size, cloning, resizing, and storage class requirements to ensure successful configuration.<

The GCP PD CSI driver support for the C3 instance type for bare metal and N4 machine series have the following limitations:

* You must set the volume size to at least 4Gi when you create hyperdisk-balanced disks. {product-title} does not round up to the minimum size, so you must specify the correct size yourself.
Expand All @@ -25,7 +28,7 @@ For information about creating the storage class, see Step 2 in Section _Setting
ifndef::openshift-dedicated[]
* Clusters with mixed virtual machines (VMs) that use different storage types, for example, N2 and N4, are not supported. This is due to hyperdisks-balanced disks not being usable on most legacy VMs. Similarly, regular persistent disks are not usable on N4/C3 VMs.

* A GCP cluster with c3-standard-2, c3-standard-4, n4-standard-2, and n4-standard-4 nodes can erroneously exceed the maximum attachable disk number, which should be 16 (link:https://issues.redhat.com/browse/OCPBUGS-39258[JIRA link]).
* A GCP cluster with c3-standard-2, c3-standard-4, n4-standard-2, and n4-standard-4 nodes can erroneously exceed the maximum attachable disk number, which should be 16 (for more information, see _JIRA link_ under _Additional resources_).

* link:https://cloud.google.com/compute/docs/disks/hyperdisks#limitations[Additional limitations].
* For more limitations, see _Additional limitation_ under _Additional resources_.
endif::openshift-dedicated[]
10 changes: 10 additions & 0 deletions modules/persistent-storage-csi-gcp-hyperdisk-overview.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
// Module included in the following assemblies:
//
// * storage/container_storage_interface/persistent-storage-csi-gcp-pd.adoc

:_mod-docs-content-type: CONCEPT
[id="persistent-storage-csi-gcp-hyperdisk-overview_{context}"]
= C3 instance type for bare metal and N4 machines series

[role="_abstract"]
You can use hyperdisk-balanced storage on Google Cloud Platform (GCP) C3 bare metal and N4 machine series instances to achieve high performance.
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,7 @@
[id="persistent-storage-csi-gcp-hyperdisk-storage-pools-overview_{context}"]
= Storage pools for hyperdisk-balanced disks overview

[role="_abstract"]
You can simplify storage management and reduce costs by using hyperdisk storage pools to aggregate capacity, throughput, and IOPS into a single pool instead of managing individual disks.

Hyperdisk storage pools can be used with Compute Engine for large-scale storage. A hyperdisk storage pool is a purchased collection of capacity, throughput, and IOPS, which you can then provision for your applications as needed. You can use hyperdisk storage pools to create and manage disks in pools and use the disks across multiple workloads. By managing disks in aggregate, you can save costs while achieving expected capacity and performance growth. By using only the storage that you need in hyperdisk storage pools, you reduce the complexity of forecasting capacity and reduce management by going from managing hundreds of disks to managing a single storage pool.
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,17 @@
[id="persistent-storage-csi-gcp-hyperdisk-storage-pools-procedure_{context}"]
= Setting up hyperdisk-balanced disks

[role="_abstract"]
To provision high-performance hyperdisk-balanced storage volumes, configure a storage class, create persistent volume claims, and deploy applications that use the hyperdisk storage

.Prerequisites
* Access to the cluster with administrative privileges

.Procedure
Complete the following steps to set up hyperdisk-balanced disks:

ifdef::openshift-dedicated[]
. Create an {product-title} cluster on {GCP} with attached disks provisioned with hyperdisk-balanced disks. This can be achieved by provisioning the cluster with compute node types that support hyperdisk-balanced disks, such as the C3 and N4 machine series from {GCP}.
. Once the OSD cluster is ready, navigate to the **OpenShift console** for Storage Class creation.
. After the OSD cluster is ready, navigate to the **OpenShift console** for Storage Class creation.
Within the console, navigate to the **Storage** section to create a Storage Class specifying the hyperdisk-balanced disk:
+
.Example StorageClass YAML file
Expand All @@ -23,26 +25,32 @@ Within the console, navigate to the **Storage** section to create a Storage Clas
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: hyperdisk-sc <1>
name: hyperdisk-sc
annotations:
storageclass.kubernetes.io/is-default-class: 'true'
provisioner: pd.csi.storage.gke.io <2>
provisioner: pd.csi.storage.gke.io
parameters:
replication-type: none
storage-pools: projects/myproject/zones/us-east1-c/storagePools/hyperdisk-storagepool <3>
type: hyperdisk-balanced <4>
storage-pools: projects/myproject/zones/us-east1-c/storagePools/hyperdisk-storagepool
type: hyperdisk-balanced
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: Immediate
----
<1> Specify the name for your storage class. In this example, the name is `hyperdisk-sc`.
<2> Specify the GCP CSI provisioner as `pd.csi.storage.gke.io`.
<3> If using storage pools, specify a list of specific storage pools that you want to use in the following format: `projects/PROJECT_ID/zones/ZONE/storagePools/STORAGE_POOL_NAME`.
<4> Specify the disk type as `hyperdisk-balanced`.
+
Where:
+
* `metadata.name`: Specify the name for your storage class. In this example, the name is `hyperdisk-sc`.

* `provisioner`: Specify the GCP CSI provisioner as `pd.csi.storage.gke.io`.

* `parameters.storage-pools`: If using storage pools, specify a list of specific storage pools that you want to use in the following format: `projects/PROJECT_ID/zones/ZONE/storagePools/STORAGE_POOL_NAME`.

* `parameters.type`: Specify the disk type as `hyperdisk-balanced`.
+
[NOTE]
====
If you use storage pools, you must first create a Hyperdisk Storage Pool of the type "Hyperdisk Balanced" in the Google Cloud console prior to referencing it in the OpenShift Storage Class. The Hyperdisk Storage Pool must be created in the same zone as the compute node supporting Hyperdisk is installed in the cluster configuration. For more information about creating a Hyperdisk Storage Pool, see link:https://cloud.google.com/compute/docs/disks/create-storage-pools#create-pool[Create a Hyperdisk Storage Pool] in the Google Cloud documentation.
If you use storage pools, you must first create a Hyperdisk Storage Pool of the type "Hyperdisk Balanced" in the Google Cloud console prior to referencing it in the OpenShift Storage Class. The Hyperdisk Storage Pool must be created in the same zone as the compute node supporting Hyperdisk is installed in the cluster configuration. For more information about creating a Hyperdisk Storage Pool, see _Create a Hyperdisk Storage Pool_ in the Google Cloud documentation under _Additional resources_.
====

endif::openshift-dedicated[]
Expand Down Expand Up @@ -79,9 +87,9 @@ controlPlane:
name: master
platform:
gcp:
type: n4-standard-4 <1>
type: n4-standard-4
osDisk:
diskType: hyperdisk-balanced <2>
diskType: hyperdisk-balanced
diskSizeGB: 200
replicas: 3
compute:
Expand All @@ -90,12 +98,16 @@ compute:
replicas: 3
platform:
gcp:
type: n4-standard-4 <1>
type: n4-standard-4
osDisk:
diskType: hyperdisk-balanced <2>
diskType: hyperdisk-balanced
----
<1> Specifies the node type as n4-standard-4.
<2> Specifies the node has the root disk backed by hyperdisk-balanced disk type. All nodes in the cluster should use the same disk type, either hyperdisks-balanced or pd-*.
+
Where:
+
* `controlPlane.platform.gcp.type` and `compute.platform.gcp.type`: Specifies the node type as n4-standard-4.

* `controlPlane.platform.gcp.osDisk.diskType` and `compute.platform.osDisk.diskType`: Specifies the node has the root disk backed by hyperdisk-balanced disk type. All nodes in the cluster should use the same disk type, either hyperdisks-balanced or pd-*.
+
[NOTE]
====
Expand All @@ -109,7 +121,6 @@ ifndef::openshift-dedicated[]
* cluster_csi_driver.yaml - specifies opting out of the default storage class creation
* storageclass.yaml - creates a hyperdisk-specific storage class
+
--
.Example cluster CSI driver YAML file
[source, yaml]
----
Expand All @@ -123,44 +134,52 @@ spec:
operatorLogLevel: Normal
storageClassState: Unmanaged <1>
----
<1> Specifies disabling creation of the default {product-title} storage classes.
--
+
--
Where `spec.storageClassState` specifies disabling creation of the default {product-title} storage classes.
+
.Example storage class YAML file
[source, yaml]
----
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: hyperdisk-sc <1>
name: hyperdisk-sc
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: pd.csi.storage.gke.io <2>
provisioner: pd.csi.storage.gke.io
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete
parameters:
type: hyperdisk-balanced <3>
type: hyperdisk-balanced
replication-type: none
provisioned-throughput-on-create: "140Mi" <4>
provisioned-iops-on-create: "3000" <5>
storage-pools: projects/my-project/zones/us-east4-c/storagePools/pool-us-east4-c <6>
allowedTopologies: <7>
provisioned-throughput-on-create: "140Mi"
provisioned-iops-on-create: "3000"
storage-pools: projects/my-project/zones/us-east4-c/storagePools/pool-us-east4-c
allowedTopologies:
- matchLabelExpressions:
- key: topology.kubernetes.io/zone
values:
- us-east4-c
...
----
<1> Specify the name for your storage class. In this example, it is `hyperdisk-sc`.
<2> `pd.csi.storage.gke.io` specifies GCP CSI provisioner.
<3> Specifies using hyperdisk-balanced disks. To specify high availability hyperdisk-balanced disk, set the value to `hyperdisk-balanced-high-availability`.
<4> Specifies the throughput value in MiBps using the "Mi" qualifier. For example, if your required throughput is 250 MiBps, specify "250Mi". If you do not specify a value, the capacity is based upon the disk type default.
<5> Specifies the IOPS value without any qualifiers. For example, if you require 7,000 IOPS, specify "7000". If you do not specify a value, the capacity is based upon the disk type default.
<6> If using storage pools, specify a list of specific storage pools that you want to use in the format: projects/PROJECT_ID/zones/ZONE/storagePools/STORAGE_POOL_NAME.
<7> If using storage pools, set `allowedTopologies` to restrict the topology of provisioned volumes to where the storage pool exists. In this example, `us-east4-c`.
--
+
Where:
+
* `metadata`.name: Specify the name for your storage class. In this example, it is `hyperdisk-sc`.

* `provisioner`: `pd.csi.storage.gke.io` specifies GCP CSI provisioner.

* `parameters.type`: Specifies using hyperdisk-balanced disks. To specify high availability hyperdisk-balanced disk, set the value to `hyperdisk-balanced-high-availability`.

* `parameters.provisioned-throughput-on-create`: Specifies the throughput value in MiBps using the "Mi" qualifier. For example, if your required throughput is 250 MiBps, specify "250Mi". If you do not specify a value, the capacity is based upon the disk type default.

* `parameters.provisioned-iops-on-create`: Specifies the IOPS value without any qualifiers. For example, if you require 7,000 IOPS, specify "7000". If you do not specify a value, the capacity is based upon the disk type default.

* `parameters.storage-pools`: If using storage pools, specify a list of specific storage pools that you want to use in the format: projects/PROJECT_ID/zones/ZONE/storagePools/STORAGE_POOL_NAME.

* `parameters.allowedTopologies`: If using storage pools, set `allowedTopologies` to restrict the topology of provisioned volumes to where the storage pool exists. In this example, `us-east4-c`.

endif::openshift-dedicated[]

. Create a persistent volume claim (PVC) that uses the hyperdisk-specific storage class using the following example YAML file:
Expand All @@ -173,15 +192,19 @@ kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
storageClassName: hyperdisk-sc <1>
storageClassName: hyperdisk-sc
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 2048Gi <2>
storage: 2048Gi
----
<1> PVC references the storage pool-specific storage class. In this example, `hyperdisk-sc`.
<2> Target storage capacity of the hyperdisk-balanced volume. In this example, `2048Gi`.
+
Where:

* `spec.storageClassName`: PVC references the storage pool-specific storage class. In this example, `hyperdisk-sc`.

* `spec.resources.requests.storage`: Target storage capacity of the hyperdisk-balanced volume. In this example, `2048Gi`.

. Create a deployment that uses the PVC that you just created. Using a deployment helps ensure that your application has access to the persistent storage even after the pod restarts and rescheduling:

Expand All @@ -206,7 +229,7 @@ spec:
app: postgres
spec:
nodeSelector:
cloud.google.com/machine-family: n4 <1>
cloud.google.com/machine-family: n4
containers:
- name: postgres
image: postgres:14-alpine
Expand All @@ -217,10 +240,14 @@ spec:
volumes:
- name: sdk-volume
persistentVolumeClaim:
claimName: my-pvc <2>
claimName: my-pvc
----
<1> Specifies the machine family. In this example, it is `n4`.
<2> Specifies the name of the PVC created in the preceding step. In this example, it is `my-pfc`.
+
Where:
+
* `spec.template.spec.nodeSelector`: Specifies the machine family. In this example, it is `n4`.

* `spec.template.spec.volumes.persistentVolumeClaim.claimName`: Specifies the name of the PVC created in the preceding step. In this example, it is `my-pfc`.

.. Confirm that the deployment was successfully created by running the following command:
+
Expand All @@ -246,7 +273,6 @@ $ oc get pvc my-pvc
----
+
.Example output
+
[source, terminal]
----
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
Expand All @@ -261,15 +287,14 @@ $ gcloud compute disks list
----
+
.Example output
+
[source, terminal]
----
NAME LOCATION LOCATION_SCOPE SIZE_GB TYPE STATUS
instance-20240914-173145-boot us-central1-a zone 150 pd-standard READY
instance-20240914-173145-data-workspace us-central1-a zone 100 pd-balanced READY
c4a-rhel-vm us-central1-a zone 50 hyperdisk-balanced READY <1>
c4a-rhel-vm us-central1-a zone 50 hyperdisk-balanced READY
----
<1> Hyperdisk-balanced disk.
Where `c4a-rhel.vm` is a hyperdisk-balanced disk.

.. If using storage pools, check that the volume is provisioned as specified in your storage class and PVC by running the following command:
+
Expand All @@ -279,7 +304,6 @@ $ gcloud compute storage-pools list-disks pool-us-east4-c --zone=us-east4-c
----
+
.Example output
+
[source, terminal]
----
NAME STATUS PROVISIONED_IOPS PROVISIONED_THROUGHPUT SIZE_GB
Expand Down
Loading