Please enable Javascript for better experience...
 
NSX-V upgrade fails with the error - Error occurred while communicating with EsxAgentManager
By Pranay Jha | May 14, 2021 | In Articles | Total Views [ 8447 ]
Taged In
(0 Like)
Rate

 

Issue Description

  • EAM Status is showing in “Starting” mode.
  • NSX Version is showing as “Not Running”.
  • It is showing error “Error occurred while communicating with EsxAgentManager.”

Troubleshooting Performed:

Vmware-eam service is already in running status, showing in VCSA putty console.

 

  • Check checked event logs of NSX, it is showing that Certificate ssoserverSign has expired.
    • Code: 360001
    • Event Source: Certificate Monitoring Service

When checked certificate status in NSX Manager console, it is showing that certificate is Valid till 18th Jan 2029.

STS Certificates are also not expired.

 

Solution

Checking into the logs I found the following output:

2021-05-11T19:01:50.121Z | INFO | vim-monitor | VcConnection.java | 635 | Connecting to https://vcenter.pjlab.local:8089/sdk/vimService via vCenter proxy http://localhost:80

2021-05-11T19:01:54.151Z | ERROR | vim-monitor | VcConnection.java | 210 | Failed to login to vCenter as extension.

vCenter has probably not loaded the EAM extension.xml yet.: Cannot complete login due to an incorrect user name or password.

2021-05-11T19:01:54.151Z | WARN | vim-monitor | VcListener.java | 110 | Trying to recover from error

2021-05-11T19:02:04.157Z | INFO | vim-monitor | VcConnection.java | 635 | Connecting to https://vcenter.pjlab.local:8089/sdk/vimService via vCenter proxy http://localhost:80

2021-05-11T19:02:08.196Z | ERROR | vim-monitor | VcConnection.java | 210 | Failed to login to vCenter as extension.

vCenter has probably not loaded the EAM extension.xml yet.: Cannot complete login due to an incorrect user name or password.

2021-05-11T19:02:08.197Z | WARN | vim-monitor | VcListener.java | 110 | Trying to recover from error

Run below commands to fix this issue:

Command> shell.set --enabled true

Command> shell

root@vcenter [ ~ ]# mkdir /certificate

root@vcenter [ ~ ]# /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.crt

root@vcenter [ ~ ]# /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.key

root@vcenter [ ~ ]# python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.vim.eam -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s vcenter.pjlab.local -u Administrator@vsphere.local

root@vcenter [ ~ ]# service-control --stop vmware-eam

root@vcenter [ ~ ]# service-control --start vmware-eam

 

Step by Step:

Login to vCenter Server Appliance through Putty console, and run above commands as showing below.

------------------------

login as: root

Command> shell.set --enabled true

Command> shell

Shell access is granted to root

root@vcenter [ ~ ]# mkdir /certificate

root@vcenter [ ~ ]# /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.crt

root@vcenter [ ~ ]# /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.key

root@vcenter [ ~ ]# python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.vim.eam -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s vcenter.pjlab.local -u Administrator@vsphere.local

Password to connect to VC server for user="Administrator@vsphere.local":

2021-05-14T05:55:15.464Z   Updating certificate for "com.vmware.vim.eam" extension

2021-05-14T05:55:15.542Z   Successfully updated certificate for "com.vmware.vim.eam" extension

Traceback (most recent call last):

  File "/usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py", line 173, in <module>

    update_extension_cert_in_VC()

  File "/usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py", line 161, in update_extension_cert_in_VC

    sessionMgr = si.content.sessionManager

  File "/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py", line 551, in __call__

    return self.f(*args, **kwargs)

  File "/usr/lib/vmware/site-packages/pyVmomi/VmomiSupport.py", line 371, in _InvokeAccessor

    return self._stub.InvokeAccessor(self, info)

  File "/usr/lib/vmware/site-packages/pyVmomi/StubAdapterAccessorImpl.py", line 23, in InvokeAccessor

    return self.InvokeMethod(mo, info, (prop,))

  File "/usr/lib/vmware/site-packages/pyVmomi/SoapAdapter.py", line 1409, in InvokeMethod

    conn.request('POST', self.path, req, headers)

  File "/usr/lib/python2.7/httplib.py", line 1082, in request

    self._send_request(method, url, body, headers)

  File "/usr/lib/python2.7/httplib.py", line 1122, in _send_request

    self.endheaders(body)

  File "/usr/lib/python2.7/httplib.py", line 1078, in endheaders

    self._send_output(message_body)

  File "/usr/lib/python2.7/httplib.py", line 894, in _send_output

    self.send(msg)

  File "/usr/lib/python2.7/httplib.py", line 856, in send

    self.connect()

  File "/usr/lib/python2.7/httplib.py", line 1303, in connect

    server_hostname=server_hostname)

  File "/usr/lib/python2.7/ssl.py", line 393, in wrap_socket

    _context=self)

  File "/usr/lib/python2.7/ssl.py", line 641, in __init__

    self.do_handshake()

  File "/usr/lib/python2.7/ssl.py", line 878, in do_handshake

    match_hostname(self.getpeercert(), self.server_hostname)

  File "/usr/lib/python2.7/ssl.py", line 316, in match_hostname

    % (hostname, dnsnames[0]))

ssl.CertificateError: hostname 'sdkTunnel' doesn't match u'vcenter.pjlab.local'

root@vcenter [ ~ ]# service-control --stop vmware-eam

Perform stop operation. vmon_profile=None, svc_names=['vmware-eam'], include_coreossvcs=False, include_leafossvcs=False

Successfully stopped service eam

root@vcenter [ ~ ]# service-control --start vmware-eam

Perform start operation. vmon_profile=None, svc_names=['vmware-eam'], include_coreossvcs=False, include_leafossvcs=False

2021-05-14T06:05:44.363Z   Service eam state STOPPED

Successfully started service eam

root@vcenter [ ~ ]#

 -------------------------------

After running these commands, EAM service should be showing in Running state in NSX Dashboard.

 

 


Thanks for visiting here. Share this article if you found it useful.
Like Facebook Page https://www.facebook.com/VMwareInsight/
Connect to twitter https://twitter.com/imPranayK
Subscribe my Channel https://www.youtube.com/vmwareinsight
Connect over Linkedin https://in.linkedin.com/in/impranayk
Share this on Social Media

About the Author

Pranay Jha
Pranay Jha
Founder, Contributer VMwareInsight.com

Public profile: user/profile/99900000


Follow me

facebook linkedin twitter G+ VMTN youtube

Thank you for visiting my profile. I am Pranay Jha, bring along a total of 11+ years of extensive experience with me in Information Technology sector for organizations from small business to large enterprises, wherein my current assignment I am associated with IBM as a Technical Solution Architect for Virtualization platform. I am vExpert x 3 (16/17/18), VCIX-DCV, VCAP5/6-DCD, VCAP5-DCA, VCP7-CMA, VCP5/6-DCV, VCA-DCV, VCA-Cloud, VSP, VCE-CIA, MCITP, MCSE, MCSA(Messaging). I am also an Independent blogger and founder of http://vmwareinsight.com and https://cloudpathshala.com. I can be reached via email at pranay1988jha@gmail.com or Direct Message via Contact Us form.

 
Please SignUp/Login to comment...

Or comment as anonymous...
* Name
* Email ID
Comment
 
Sponsors
 
 
 
 
 
Facebook Likes