Skip to content

Commit 3c33e0c

Browse files
allmightyspiffallmightyspiff
authored
Merge pull request #704 from softlayer/nvme
NVMe Ordering example
2 parents 5f0ef58 + 9762274 commit 3c33e0c

File tree

1 file changed

+235
-0
lines changed

1 file changed

+235
-0
lines changed

‎content/article/nvmeOrdering.md‎

Lines changed: 235 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,235 @@
1+
---
2+
title: "Mirrored M.2 Ordering Examples"
3+
description: "An explanation of the required settings for ordering a Mirrored M.2 server from the API."
4+
date: "2025-06-10"
5+
tags:
6+
- "article"
7+
- "sldn"
8+
- "order"
9+
- "package"
10+
- "slcli"
11+
---
12+
13+
*NOTE*: This article is for [slcli v6.2.7](https://github.com/softlayer/softlayer-python/releases/tag/v6.2.7) and [ibmcloud sl v1.5.7](https://github.com/softlayer/softlayer-cli/releases/tag/v1.5.7)
14+
15+
16+
# Basics
17+
18+
Ordering a server with a mirrored NVMe M.2 boot drive is a little tricky. There are 2 main difference between this setup and a normal order.
19+
20+
1. `storageGroups` need to be defined with the NVMe array first
21+
2. NVMe array needs to set `hardDriveCategoryCodes` and `diskControllerIndex: 1`
22+
23+
> The `0th` index storageGroup will be what the server boots from. It will not match the order the raid cards are ordered in.
24+
25+
3. You will need to have DISK_CONTROLLER_RAID (`diskControllerIndex: 0`) in your order, before DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 (`diskControllerIndex: 1`). DISK_CONTROLLER_RAID uses the `disk_controller` category price, and DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 uses `disk_controller1` category price when both are present in an order.
26+
27+
This ordering is important because otherwise the SLCLI will pick the incorrect price id, and you will get one of the errors listed below.
28+
29+
4. A basic order will look like this:
30+
31+
```bash
32+
./slcli -v order place --verify 2U_DUAL_INTEL_XEON_SAPPHIRE_RAPIDS_FAMILY_16_DRIVES DALLAS13 \
33+
--complex-type SoftLayer_Container_Product_Order_Hardware_Server --billing monthly \
34+
--extras '{"hardware":[{"hostname":"test1nvme","domain":"ibm.com"}],"storageGroups":[{"arraySize":480,"arrayTypeId":2, "diskControllerIndex":1,"hardDriveCategoryCodes":["secondary_pcie_slot0","secondary_pcie_slot1"]}, {"arraySize":1920,"arrayTypeId":2,"diskControllerIndex": 0,"hardDrives":[0,1]}]}' \
35+
AUTOMATED_NOTIFICATION MONITORING_HOST_PING NOTIFICATION_EMAIL_AND_TICKET REBOOT_KVM_OVER_IP \
36+
UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT BANDWIDTH_0_GB 1_IP_ADDRESS \
37+
1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS DISK_CONTROLLER_RAID \
38+
DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 \
39+
MIRRORED_NVME_M_2_OS_BOOT_480GB 256_GB_RAM_DDR5 OS_CENTOS_STREAM_9_X_64_BIT \
40+
INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70 REDUNDANT_POWER_SUPPLY HARD_DRIVE_1_9_TB_SSD \
41+
HARD_DRIVE_1_9_TB_SSD
42+
```
43+
44+
```bash
45+
Calling: SoftLayer_Product_Order::verifyOrder(id=None, mask='', filter='None', args=({'orderContainers': [{
46+
'hardware': [{'hostname': 'test1nvme', 'domain': 'ibm.com'}],
47+
'storageGroups': [
48+
{
49+
'arraySize': 480,'arrayTypeId': 2, 'diskControllerIndex': 1,
50+
'hardDriveCategoryCodes': ['secondary_pcie_slot0', 'secondary_pcie_slot1']
51+
},{
52+
'arraySize': 1920, 'arrayTypeId': 2, 'diskControllerIndex': 0, 'hardDrives': [0, 1]
53+
}],
54+
'packageId': 3137,
55+
'quantity': 1,
56+
'location': 1854895,
57+
'useHourlyPricing': False,
58+
'complexType': 'SoftLayer_Container_Product_Order_Hardware_Server',
59+
'prices': [
60+
{'id': 58, 'categories': [{'categoryCode': 'response'}], 'item': {'keyName': 'AUTOMATED_NOTIFICATION'}},
61+
{'id': 55, 'categories': [{'categoryCode': 'monitoring'}], 'item': {'keyName': 'MONITORING_HOST_PING'}},
62+
{'id': 57, 'categories': [{'categoryCode': 'notification'}], 'item': {'keyName': 'NOTIFICATION_EMAIL_AND_TICKET'}},
63+
{'id': 906, 'categories': [{'categoryCode': 'remote_management'}], 'item': {'keyName': 'REBOOT_KVM_OVER_IP'}},
64+
{'id': 420, 'categories': [{'categoryCode': 'vpn_management'}], 'item': {'keyName': 'UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT'}},
65+
{'id': 22505, 'categories': [{'categoryCode': 'bandwidth'}], 'item': {'keyName': 'BANDWIDTH_0_GB'}},
66+
{'id': 21, 'categories': [{'categoryCode': 'pri_ip_addresses'}], 'item': {'keyName': '1_IP_ADDRESS'}},
67+
{'id': 14023, 'categories': [{'categoryCode': 'port_speed'}], 'item': {'keyName': '1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS'}},
68+
{'id': 22484, 'categories': [{'categoryCode': 'disk_controller'}], 'item': {'keyName': 'DISK_CONTROLLER_RAID'}},
69+
{'id': 306531, 'categories': [{'categoryCode': 'disk_controller1'}], 'item': {'keyName': 'DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2'}},
70+
{'id': 311834, 'categories': [{'categoryCode': 'pcie_slot0'}], 'item': {'keyName': 'MIRRORED_NVME_M_2_OS_BOOT_480GB'}},
71+
{'id': 301502, 'categories': [{'categoryCode': 'ram'}], 'item': {'keyName': '256_GB_RAM_DDR5'}},
72+
{'id': 307539, 'categories': [{'categoryCode': 'os'}], 'item': {'keyName': 'OS_CENTOS_STREAM_9_X_64_BIT'}},
73+
{'id': 313326, 'categories': [{'categoryCode': 'server'}], 'item': {'keyName': 'INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70'}},
74+
{'id': 50221, 'categories': [{'categoryCode': 'power_supply'}], 'item': {'keyName': 'REDUNDANT_POWER_SUPPLY'}},
75+
{'id': 300316, 'categories': [{'categoryCode': 'disk0'}], 'item': {'keyName': 'HARD_DRIVE_1_9_TB_SSD'}},
76+
{'id': 300316, 'categories': [{'categoryCode': 'disk0'}], 'item': {'keyName': 'HARD_DRIVE_1_9_TB_SSD'}}
77+
]
78+
}]}) limit=None, offset=None)
79+
80+
┌─────────────────────────────────────────────────────┬────────────────────────────────────────────────┬──────┐
81+
│ keyName │ description │ cost │
82+
├─────────────────────────────────────────────────────┼────────────────────────────────────────────────┼──────┤
83+
│ AUTOMATED_NOTIFICATION │ Automated Notification │ 0 │
84+
│ MONITORING_HOST_PING │ Host Ping │ 0 │
85+
│ NOTIFICATION_EMAIL_AND_TICKET │ Email and Ticket │ 0 │
86+
│ REBOOT_KVM_OVER_IP │ Reboot / KVM over IP │ 0 │
87+
│ UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT │ Unlimited SSL VPN Users │ 0 │
88+
│ BANDWIDTH_0_GB │ 0 GB Bandwidth Allotment │ 0 │
89+
│ 1_IP_ADDRESS │ 1 IP Address │ 0 │
90+
│ 1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS │ 1 Gbps Redundant Private Network Uplinks │ 0 │
91+
│ DISK_CONTROLLER_RAID │ RAID │ 0 │
92+
│ DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 │ RAID 1 (Mirrored NVMe M.2) │ 0 │
93+
│ MIRRORED_NVME_M_2_OS_BOOT_480GB │ Mirrored NVMe M.2 for OS Boot (480GB) │ 0 │
94+
│ 256_GB_RAM_DDR5 │ 256 GB RAM │ 0 │
95+
│ OS_CENTOS_STREAM_9_X_64_BIT │ CentOS Stream 9.x (64 bit) │ 0 │
96+
│ INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70 │ Dual Intel Xeon Gold 6416H (24 Cores, 2.7 GHz) │ 0 │
97+
│ REDUNDANT_POWER_SUPPLY │ Redundant Power Supply │ 0 │
98+
│ HARD_DRIVE_1_9_TB_SSD │ 1.9TB SSD │ 0 │
99+
│ HARD_DRIVE_1_9_TB_SSD │ 1.9TB SSD │ 0 │
100+
└─────────────────────────────────────────────────────┴────────────────────────────────────────────────┴──────┘
101+
```
102+
103+
From there you can customize your server how you like.
104+
105+
# Extras Explained
106+
107+
Here I have 2 raid arrays.
108+
109+
0. Is the NVMe array in RAID1. I have to specify `secondary_pcie_slot0` / `secondary_pcie_slot1` to let the API know what drives to attach to this card. `diskControllerIndex` is 1, because I want it to use the second raid card on my order, `DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2` in this case.
110+
1. This is for my other drives I'm adding, again in a RAID1. `diskControllerIndex` is 0, because I want it to use the first raid card on my order, `DISK_CONTROLLER_RAID` in this case.
111+
2. [Advanced Bare Metal Ordering](https://sldn.softlayer.com/python/orderBareMetal/) has some more example of how to setup raid arrays.
112+
113+
114+
```json
115+
{
116+
"hardware": [
117+
{
118+
"hostname": "test1nvme",
119+
"domain": "ibm.com"
120+
}
121+
],
122+
"storageGroups": [
123+
{
124+
"arraySize": 480,
125+
"arrayTypeId": 2,
126+
"diskControllerIndex": 1,
127+
"hardDriveCategoryCodes": [
128+
"secondary_pcie_slot0",
129+
"secondary_pcie_slot1"
130+
]
131+
},
132+
{
133+
"arraySize": 1920,
134+
"arrayTypeId": 2,
135+
"diskControllerIndex": 0,
136+
"hardDrives": [0,1]
137+
}
138+
]
139+
}
140+
```
141+
142+
143+
# Common Errors
144+
145+
## Only 1 RAID card
146+
147+
This order works, but since it only specifies 1 raid card, the other drives will not get attached properly, and will lilely require Datacenter manual intervention to fix this order. Or the drives will be in a JBOD array.
148+
149+
150+
```bash
151+
slcli -v order place --verify 2U_DUAL_INTEL_XEON_SAPPHIRE_RAPIDS_FAMILY_16_DRIVES DALLAS13 \
152+
--complex-type SoftLayer_Container_Product_Order_Hardware_Server --billing monthly \
153+
--extras '{"hardware":[{"hostname":"test1nvme","domain":"ibm.com"}],"storageGroups":[{"arraySize":480,"arrayTypeId":2,"diskControllerIndex":0,"hardDriveCategoryCodes":["secondary_pcie_slot0","secondary_pcie_slot1"]},{"arraySize":1920,"arrayTypeId":2, "hardDrives":[0,1]}]}' \
154+
AUTOMATED_NOTIFICATION MONITORING_HOST_PING NOTIFICATION_EMAIL_AND_TICKET REBOOT_KVM_OVER_IP \
155+
UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT BANDWIDTH_0_GB 1_IP_ADDRESS \
156+
1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 \
157+
MIRRORED_NVME_M_2_OS_BOOT_480GB 256_GB_RAM_DDR5 OS_CENTOS_STREAM_9_X_64_BIT \
158+
INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70 REDUNDANT_POWER_SUPPLY HARD_DRIVE_1_9_TB_SSD \
159+
HARD_DRIVE_1_9_TB_SSD
160+
```
161+
```
162+
┌─────────────────────────────────────────────────────┬────────────────────────────────────────────────┬──────┐
163+
│ keyName │ description │ cost │
164+
├─────────────────────────────────────────────────────┼────────────────────────────────────────────────┼──────┤
165+
│ AUTOMATED_NOTIFICATION │ Automated Notification │ 0 │
166+
│ MONITORING_HOST_PING │ Host Ping │ 0 │
167+
│ NOTIFICATION_EMAIL_AND_TICKET │ Email and Ticket │ 0 │
168+
│ REBOOT_KVM_OVER_IP │ Reboot / KVM over IP │ 0 │
169+
│ UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT │ Unlimited SSL VPN Users │ 0 │
170+
│ BANDWIDTH_0_GB │ 0 GB Bandwidth Allotment │ 0 │
171+
│ 1_IP_ADDRESS │ 1 IP Address │ 0 │
172+
│ 1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS │ 1 Gbps Redundant Private Network Uplinks │ 0 │
173+
│ DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 │ RAID 1 (Mirrored NVMe M.2) │ 0 │
174+
│ MIRRORED_NVME_M_2_OS_BOOT_480GB │ Mirrored NVMe M.2 for OS Boot (480GB) │ 0 │
175+
│ 256_GB_RAM_DDR5 │ 256 GB RAM │ 0 │
176+
│ OS_CENTOS_STREAM_9_X_64_BIT │ CentOS Stream 9.x (64 bit) │ 0 │
177+
│ INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70 │ Dual Intel Xeon Gold 6416H (24 Cores, 2.7 GHz) │ 0 │
178+
│ REDUNDANT_POWER_SUPPLY │ Redundant Power Supply │ 0 │
179+
│ HARD_DRIVE_1_9_TB_SSD │ 1.9TB SSD │ 0 │
180+
│ HARD_DRIVE_1_9_TB_SSD │ 1.9TB SSD │ 0 │
181+
└─────────────────────────────────────────────────────┴────────────────────────────────────────────────┴──────┘
182+
```
183+
## Disk controller #1 is used by storage groups but it does not have a price.
184+
185+
`DISK_CONTROLLER_RAID` is missing from the order
186+
187+
```bash
188+
slcli -v order place --verify 2U_DUAL_INTEL_XEON_SAPPHIRE_RAPIDS_FAMILY_16_DRIVES DALLAS13 \
189+
--complex-type SoftLayer_Container_Product_Order_Hardware_Server --billing monthly \
190+
--extras '{"hardware":[{"hostname":"test1nvme","domain":"ibm.com"}],"storageGroups":[{"arraySize":1920,"arrayTypeId":2,"diskControllerIndex": 0,"hardDrives":[0,1]},{"arraySize":480,"arrayTypeId":2,"diskControllerIndex":1,"hardDriveCategoryCodes":["secondary_pcie_slot0","secondary_pcie_slot1"]}]}' \
191+
AUTOMATED_NOTIFICATION MONITORING_HOST_PING NOTIFICATION_EMAIL_AND_TICKET REBOOT_KVM_OVER_IP \
192+
UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT BANDWIDTH_0_GB 1_IP_ADDRESS \
193+
1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 \
194+
MIRRORED_NVME_M_2_OS_BOOT_480GB 256_GB_RAM_DDR5 OS_CENTOS_STREAM_9_X_64_BIT \
195+
INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70 REDUNDANT_POWER_SUPPLY HARD_DRIVE_1_9_TB_SSD \
196+
HARD_DRIVE_1_9_TB_SSD
197+
```
198+
SoftLayerAPIError(500): Disk controller #1 is used by storage groups but it does not have a price.
199+
200+
201+
## Unable to add RAID because a Disk Controller price has already been added.
202+
203+
This is caused because older versions of the `slcli`/`ibmcloud sl` are not able to lookup the correct price if there are 2 raid cards in an order.
204+
205+
```bash
206+
./slcli -v order place --verify 2U_DUAL_INTEL_XEON_SAPPHIRE_RAPIDS_FAMILY_16_DRIVES DALLAS13 \
207+
--complex-type SoftLayer_Container_Product_Order_Hardware_Server --billing monthly \
208+
--extras '{"hardware":[{"hostname":"test1nvme","domain":"ibm.com"}],"storageGroups":[{"arraySize":1920,"arrayTypeId":2,"diskControllerIndex": 0,"hardDrives":[0,1]},{"arraySize":480,"arrayTypeId":2,"diskControllerIndex":1,"hardDriveCategoryCodes":["secondary_pcie_slot0","secondary_pcie_slot1"]}]}' \
209+
AUTOMATED_NOTIFICATION MONITORING_HOST_PING NOTIFICATION_EMAIL_AND_TICKET REBOOT_KVM_OVER_IP \
210+
UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT BANDWIDTH_0_GB 1_IP_ADDRESS \
211+
1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS DISK_CONTROLLER_RAID \
212+
DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 \
213+
MIRRORED_NVME_M_2_OS_BOOT_480GB 256_GB_RAM_DDR5 OS_CENTOS_STREAM_9_X_64_BIT \
214+
INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70 REDUNDANT_POWER_SUPPLY HARD_DRIVE_1_9_TB_SSD \
215+
HARD_DRIVE_1_9_TB_SSD
216+
```
217+
SoftLayerAPIError(500): Unable to add RAID because a Disk Controller price has already been added.
218+
219+
220+
## SoftLayerAPIError(500): A storage group with M.2 drive(s) must be the primary storage group.
221+
222+
Storage groups are in the wrong order.
223+
```bash
224+
slcli -v order place --verify 2U_DUAL_INTEL_XEON_SAPPHIRE_RAPIDS_FAMILY_16_DRIVES DALLAS13 \
225+
--complex-type SoftLayer_Container_Product_Order_Hardware_Server --billing monthly \
226+
--extras '{"hardware":[{"hostname":"test1nvme","domain":"ibm.com"}],"storageGroups":[{"arraySize":1920,"arrayTypeId":2,"diskControllerIndex": 0,"hardDrives":[0,1]},{"arraySize":480,"arrayTypeId":2,"diskControllerIndex":1,"hardDriveCategoryCodes":["secondary_pcie_slot0","secondary_pcie_slot1"]}]}' \
227+
AUTOMATED_NOTIFICATION MONITORING_HOST_PING NOTIFICATION_EMAIL_AND_TICKET REBOOT_KVM_OVER_IP \
228+
UNLIMITED_SSL_VPN_USERS_1_PPTP_VPN_USER_PER_ACCOUNT BANDWIDTH_0_GB 1_IP_ADDRESS \
229+
1_GBPS_REDUNDANT_PRIVATE_NETWORK_UPLINKS DISK_CONTROLLER_RAID \
230+
DISK_CONTROLLER_RAID_1_MIRRORED_NMVE_M_2 \
231+
MIRRORED_NVME_M_2_OS_BOOT_480GB 256_GB_RAM_DDR5 OS_CENTOS_STREAM_9_X_64_BIT \
232+
INTEL_INTEL_XEON_6416_UEFI_BOOT_MODE_2_70 REDUNDANT_POWER_SUPPLY HARD_DRIVE_1_9_TB_SSD \
233+
HARD_DRIVE_1_9_TB_SSD
234+
```
235+
SoftLayerAPIError(500): A storage group with M.2 drive(s) must be the primary storage group.

0 commit comments

Comments
 (0)