Cannot run upgrade scripts on ESXi Host

Recently was given some tasks to upgrade few ESXi boxes (UCS-B200-M3) running with ESXi 5.0 update 3, to ESXi 5.5 Update 2 . As part of the process migrated some Monster Virtual Machines to neighbor ESXi boxes and put the host in the maintenance mode which made DRS to kick in and migrate remaining Virtual Machines to other host in the cluster.

Second thing on the list was to create a baseline and attached the baseline to the ESXi host, and proceed with remediation, initially it looked as if the host is being remediated properly until it was 92 percent completed and I saw the status of the host was still showing as ESXi5.0 Update 2, so thought of checking the tasks and events and was able to see something error. “Cannot run upgrade scripts on the Host”.

As part of initial troubleshooting I searched for KB article 2007163 which talks about the same issue so started following the steps in Kb article to locate the log files and any error in the log files, as suggested in the Kb article.

After connecting to my ESXi box using putty was not able to find any entries similar to below in the /var/log/vua.log file, :

OSError: [Errno 39] Directory not empty: /bootbank/state.XXXXXXX (where XXXXXXX is a number)

However was able to see some strange error messages “This Application is Not using Quick Exit ()” The Exit code will be set to 0.@ bora/vim/lib/vmacore/main/service.cpp

So started search for some more references to troubleshoot this issue and was able to find some references which talks about the FDM agent uninstallation and then remediating ESXi hosts.

Followed below steps to further troubleshoot this issue by uninstalling the FDM agent, as suggested in KB article 2012323 and then rebooting the ESXi host, Remediating and Exit the maintenance mode.

1 # cp /opt/vmware/uninstallers/VMware-fdm-uninstall.sh /tmp

2 # chmod +x /tmp/VMware-fdm-uninstall.sh

3 # /tmp/VMware-fdm-uninstall.sh

** Configuration shown above worked in my environment kindly raise a support request if you are also facing similar issue and unable to resolve with the help of KB articles