Enable Nutanix CVM, AHV Maintenance mode

Enable Nutanix Controller-VM CVM, AHV Maintenance Mode

The maintenance mode is very important in virtualization technology to do any maintenance / troubleshooting activity on hardware / software to resolve the issue. Nutanix HCI also provide the maintenance mode to enable / enter / exit / remove the Nutanix AHV, Hyper-V hypervisor and Controller-VM CVM into / from maintenance mode via acli / ncli command line to using simple steps.

Right Procedure to enable / enter / exit into the maintenance mode on Nutanix AHV, Hyper-v host / node and Controller-VM CVM is given below:

Step 1: First enable the maintenance mode on Nutanix AHV hypervisor
Step 2: Enable the maintenance mode on Nutanix CVM

Procedure to Exit / Remove / Disable Nutanix AHV and CVM Maintenance mode :

Step 1: Exit Nutanix Controller-VM CVM from maintenance mode
Step 2: Exit Nutanix AHV and Hyper-v hypervisor from maintenance mode

Now read out the complete Nutanix AHV and CVM maintenance mode guide below:

Nuanix AHV / Hyper-v Maintenance mode

Before enter Nutanix AHV and Hyper-v hypervisor into maintenance mode to make sure you are connected to correct CVM by checking its name and IP address. Follow steps from KB 2852 / Blog to verify cluster health. Do not proceed if cluster cannot tolerate failure of at least 1 node. Put the Nutanix AHV node / host and Nutanix Controller-VM CVM in the maintenance mode:

STEP 1: Connect to Nutanix CVM via SSH

STEP 2: Check the availability of changing Nutanix AHV host maintenance mode of target hosts:

nutanix@cvm$ acli host.enter_maintenance_mode_check <host ip>

STEP 3: Put the Nutanix node / host into maintenance mode – this will live migrate running VMs to other hosts:

Nutanix AHV Node

nutanix@cvm$ acli host.enter_maintenance_mode <host ip>

Nutanix Hyper-v Node

To run following command to enter Hyper-V host into maintenance mode

On Hyper-V hosts, check if the node is in maintenance mode by running the ncli host ls command on the CVM. Verify if the parameter Under Maintenance Mode is set to False for the node where the services are down. If the parameter Under Maintenance Mode is set to True, remove the node from maintenance mode by running the following command:

nutanix@cvm$ ncli host edit id=<host id> enable-maintenance-mode=true

Note: To check node id run following command

nutanix@cvm$ ncli host ls

After execution of above Nutanix AHV maintenance command will take couple of minutes depends on the number of running VMs on the AHV host. Need to wait until command executes successfully.

Exit Nutanix Node Maintenance Mode

If you want to exit the Nutanix AHV node / host from maintenance mode then you have to run following command:

STEP 1: Connect to Nutanix CVM via SSH

STEP 2: Exit the Nutanix node / host from maintenance mode – this will live restore the owned VMs from other hosts to serve data locality :

Nutanix AHV Node

nutanix@cvm$ acli host.exit_maintenance_mode <host ip>

Nutanix Hyper-v Node

nutanix@cvm$ ncli host edit id=<host id> enable-maintenance-mode=false

Note: To check node id run following command

nutanix@cvm$ ncli host ls

Nutanix CVM Maintenance Mode

Enable maintenance mode for the Nutanix Controller-VM CVM on the target host. This step is for preventing CVM services from being impacted by possible connectivity issue. You may skip this step if the Nutanix CVM services are not running, or cluster has stopped.

STEP 1: Connect to Nutanix CVM via SSH

STEP 2: Find Nutanix AHV host ID of targeted running CVM needs to be enter into maintenance mode. To execute following command:

nutanix@cvm$ ncli host list

Command output would be similar like following

Note: You can find <host ID> in output of “ncli host ls” command:

Id : 00058977-c18c-af17-0000-004384300006f19::9812 <--- "9812" is host
ID Uuid : ddc9d93b-68e0-4220-85f9-63b7kjslf34349sdfj

STEP 3: Enter Nutanix Controller-VM CVM into maintenance mode via command:

nutanix@cvm$ ncli host edit id=<host ID> enable-maintenance-mode=true

Command Example :   nutanix@cvm$ ncli host edit id=00058977-c18c-af17-0000-004384300006f19  enable-maintenance-mode=true
OR
Command Example : nutanix@cvm$ ncli host edit id=9812 enable-maintenance-mode=true

STEP 4: Check Nutanix CVM maintenance mode status to execute following command

nutanix@cvm$ ncli host list

Command output:
Id                        : 00058977-c18c-af17-0000-004384300006f19::9812
    Uuid                      : ddc9d93b-68e0-4220-85f9-63b7kjslf34349sdfj
    Name                      : NTNX-SERIAL-X
    IPMI Address              : x.x.x.x
    Controller VM Address     :x.x.x.x
    Hypervisor Address        : x.x.x.x
    Host Status               : NORMAL
    Oplog Disk Size           : 428.02 GiB (459,587,190,986 bytes) (0.7%)
    Under Maintenance Mode    : True
    Metadata store status     : Metadata store enabled on the node

Exit Nutanix CVM from Maintenance Mode

If all tests are successfully completed exit / remove Nutanix Controller-VM CVM from maintenance mode:

STEP 1: Connect to Nutanix CVM via SSH

STEP 2: From one of the other CVMs run following command to exit affected Nutanix CVM from maintenance mode:

nutanix@cvm$ ncli host edit id=<host ID> enable-maintenance-mode=false

Read also: How to shutdown Nutanix Acropolis Cluster

Conclusion

To enter / enable the maintenance mode on Nutanix AHV and Controller-VM CVM is preferred in best practice to do before any software / hardware firmware activity it does automatically by the Nutanix automation system. but sometime needs to be done by manually. Nutanix offer maintenance mode feature through command line only but in future releases it would be available in Prism UI to avoid the command line interface.

Hopefully you enjoyed the Nutanix AHV and Controller-VM CVM maintenance mode procedure.

Thanks to being with HyperHCI Tech Blog.