Reputation: 3233

Windbg serial debugging not working over Virtual Serial Port

I have the following configuration:

Host OS: Windows 7 SP1 Guest OS: Windows XP SP3 Pro

Guest OS is running inside VMWare Workstation version 7

I have configured a virtual serial port on the VM of type "output named pipe" and the following name:

\\.\pipe\whatever

I have also added the appropriate options to the boot.ini file on Win XP SP3 Pro image to allow for debugging (/debugport=com1 baudrate=115200)

in windbg, I configure the Kernel Debug option by going to:

File->Kernel Debug

and selecting the COM tab and configuring as follows:

Giving port name as:

\\.\pipe\whatever

and checking the Pipe box.

I boot up the Windows XP SP3 VM and select the second entry which has debug mode enabled. Right after I do this, I click on Ok inside the Windbg to start the kernel debugging session.

But even then it remains stuck at the following message:

Opened \.\pipe\whatever Waiting to reconnect...

It says "Debuggee not connected" and stays at that message even after Virtual Machine boots up completely. The status of the Virtual Serial port in the VmWare also shows as Connected.

I have checked the network connections.

The VM has the Network Adapter in the Bridged Mode and I am able to ping the Guest OS and Host OS from each other successfully.

Upvotes: 4

Answers (4)

Nick Westgate

Reputation: 3273

Your problem is likely to be that WinDbg isn't breaking on connect, so it's connected but it doesn't say so, and you think it's not working.

I had better luck running WinDbg from the command line with a -b, though I notice this option might have been removed in the later builds. I've tested with 6.12.0002.633 x86.

    windbg -b -k com:pipe,port=\\.\pipe\com_1

(From here.)

Upvotes: 2

Hanan

Reputation: 1179

I have had the same problem, and after changing the port at your windows guest (the one at the vmware) you should be connect. In my case the port at the boot.ini file was com1 and i have changed it to com2.

Upvotes: 1