The maintenance mode is very important in virtualization technology to do any maintenance / troubleshooting activity on hardware / software to resolve the issue. Nutanix HCI also provide the maintenance mode to enable / enter / exit / remove the Nutanix AHV, Hyper-V hypervisor and Controller-VM CVM into / from maintenance mode via acli / ncli command line to using simple steps.
Right Procedure to enable / enter / exit into the maintenance mode on Nutanix AHV, Hyper-v host / node and Controller-VM CVM is given below:
Step 1: First enable the maintenance mode on Nutanix AHV hypervisor
Step 2: Enable the maintenance mode on Nutanix CVM
Procedure to Exit / Remove / Disable Nutanix AHV and CVM Maintenance mode :
Step 1: Exit Nutanix Controller-VM CVM from maintenance mode
Step 2: Exit Nutanix AHV and Hyper-v hypervisor from maintenance mode
Now read out the complete Nutanix AHV and CVM maintenance mode guide below:
Nuanix AHV / Hyper-v Maintenance mode
Before enter Nutanix AHV and Hyper-v hypervisor into maintenance mode to make sure you are connected to correct CVM by checking its name and IP address. Follow steps from KB 2852 / Blog to verify cluster health. Do not proceed if cluster cannot tolerate failure of at least 1 node. Put the Nutanix AHV node / host and Nutanix Controller-VM CVM in the maintenance mode:
STEP 1: Connect to Nutanix CVM via SSH
STEP 2: Check the availability of changing Nutanix AHV host maintenance mode of target hosts:
nutanix@cvm$ acli host.enter_maintenance_mode_check <host ip>
STEP 3: Put the Nutanix node / host into maintenance mode – this will live migrate running VMs to other hosts:
Nutanix AHV Node
nutanix@cvm$ acli host.enter_maintenance_mode <host ip>
Nutanix Hyper-v Node
To run following command to enter Hyper-V host into maintenance mode
On Hyper-V hosts, check if the node is in maintenance mode by running the ncli host ls command on the CVM. Verify if the parameter Under Maintenance Mode is set to False for the node where the services are down. If the parameter Under Maintenance Mode is set to True, remove the node from maintenance mode by running the following command:
nutanix@cvm$ ncli host edit id=<host id> enable-maintenance-mode=true
Note: To check node id run following command
nutanix@cvm$ ncli host ls
After execution of above Nutanix AHV maintenance command will take couple of minutes depends on the number of running VMs on the AHV host. Need to wait until command executes successfully.
Exit Nutanix Node Maintenance Mode
If you want to exit the Nutanix AHV node / host from maintenance mode then you have to run following command:
STEP 1: Connect to Nutanix CVM via SSH
STEP 2: Exit the Nutanix node / host from maintenance mode – this will live restore the owned VMs from other hosts to serve data locality :
Nutanix AHV Node
nutanix@cvm$ acli host.exit_maintenance_mode <host ip>
Nutanix Hyper-v Node
nutanix@cvm$ ncli host edit id=<host id> enable-maintenance-mode=false
Note: To check node id run following command
nutanix@cvm$ ncli host ls
Nutanix CVM Maintenance Mode
Enable maintenance mode for the Nutanix Controller-VM CVM on the target host. This step is for preventing CVM services from being impacted by possible connectivity issue. You may skip this step if the Nutanix CVM services are not running, or cluster has stopped.
STEP 1: Connect to Nutanix CVM via SSH
STEP 2: Find Nutanix AHV host ID of targeted running CVM needs to be enter into maintenance mode. To execute following command:
nutanix@cvm$ ncli host list
Command output would be similar like following
Note: You can find <host ID> in output of “ncli host ls” command:
Id : 00058977-c18c-af17-0000-004384300006f19::9812 <--- "9812" is host
ID Uuid : ddc9d93b-68e0-4220-85f9-63b7kjslf34349sdfj
STEP 3: Enter Nutanix Controller-VM CVM into maintenance mode via command:
nutanix@cvm$ ncli host edit id=<host ID> enable-maintenance-mode=true
Command Example : nutanix@cvm$ ncli host edit id=00058977-c18c-af17-0000-004384300006f19 enable-maintenance-mode=true
OR
Command Example : nutanix@cvm$ ncli host edit id=9812 enable-maintenance-mode=true
STEP 4: Check Nutanix CVM maintenance mode status to execute following command
nutanix@cvm$ ncli host list
Command output:
Id : 00058977-c18c-af17-0000-004384300006f19::9812
Uuid : ddc9d93b-68e0-4220-85f9-63b7kjslf34349sdfj
Name : NTNX-SERIAL-X
IPMI Address : x.x.x.x
Controller VM Address :x.x.x.x
Hypervisor Address : x.x.x.x
Host Status : NORMAL
Oplog Disk Size : 428.02 GiB (459,587,190,986 bytes) (0.7%)
Under Maintenance Mode : True
Metadata store status : Metadata store enabled on the node
Exit Nutanix CVM from Maintenance Mode
If all tests are successfully completed exit / remove Nutanix Controller-VM CVM from maintenance mode:
STEP 1: Connect to Nutanix CVM via SSH
STEP 2: From one of the other CVMs run following command to exit affected Nutanix CVM from maintenance mode:
nutanix@cvm$ ncli host edit id=<host ID> enable-maintenance-mode=false
Read also: How to shutdown Nutanix Acropolis Cluster
Conclusion
To enter / enable the maintenance mode on Nutanix AHV and Controller-VM CVM is preferred in best practice to do before any software / hardware firmware activity it does automatically by the Nutanix automation system. but sometime needs to be done by manually. Nutanix offer maintenance mode feature through command line only but in future releases it would be available in Prism UI to avoid the command line interface.
Hopefully you enjoyed the Nutanix AHV and Controller-VM CVM maintenance mode procedure.
Thanks to being with HyperHCI Tech Blog.