We had an issue today where we were unable to logon
to a ESX host via the vSphere Client.
These were the sort of error messages I was getting.
I spent a lot of time on Google, which lead me to many
different knowledge base articles, which were
not related to the issue I was having.
Most were relating to file locks on the database on the
ESX host, but the errors didn’t tie up with the logs on the ESX host.
I ran Wireshark, and could see it had established a
TCP three way handshake, but started to drop packets
when trying to communicate over port 443.
I found online the log that contains information for the sign on process is
located here: /var/log/vmware/vpxd/vpxd.log
I checked in the log and found that port 443 was already in use.
2018-03-20T15:06:37.661Z [7F4321A15740 error ‘vpxdvpxdMoReverseProxy’] [VpxdReverseProxy] Failed to create https proxy: Resource is already in use: <acceptor p:0x00007f4308117200, h:33, >
2018-03-20T15:06:37.661Z [7F4321A15740 error ‘vpxdvpxdMain’] [Init] Init failed: ReverseProxyMo::Init()
I checked for open ports to work out what process was using port 443, and found it was using the process ID of 5911.
netstat -plnt | grep ‘:443’
tcp 129 0 0.0.0.0:443 0.0.0.0:* LISTEN 5911/vpxdtcp 4 0 :::443 :::* LISTEN 5911/vpxd
I then searched for the PID to find the name of this process.
ps -A | grep 5911
5911 ? 00:00:00 vpxd-worker
I stopped the parent process vmware-vpxd, which should stop the child process vpxd-worker but it didn’t.
service vmware-vpxd stop
I searched again for open ports, and even though the parent process vmware-vpxd had stopped, the port 443 was still in use.
The vpxd-worker process manages crash dumps. I killed off the process using the kill command.
kill -s KILL 5911
I then started the service, and it managed to successfully start.
service vmware-vpxd start