Debugging win32 program with vs 2008

[h9uest]

Hi there:

With console application, we can use F10, F11 etc to check the memory status and interact with the app; I can check the variables for a win32 program in the same way, but find myself unable to interact with the WndProc callback function.

Problem 1, I insert break points into WndProc callback function, but when it returns, there pops out a dialogue window saying stuff like "no source code at this point". I then have to shift+f5 to quit debugging. What should I do in order to continue debugging?

Problem 2: In addition, Win32 program has different behaviors for different messages. How can I set the message values while debugging? It's not like console app where we switch to console and give it inputs. What should I do?

Any inputs are appreciated.
Also, if anyone could recommend any book/tutorial on win32 debugging with visual studio, that'll be great.

Thanks.

[Computergeek01]

Do you think at this point that debugging your application could be done easier WITHOUT the use of the debugger? What is the underlying issue that you are trying to fix here? Don't get me wrong, a debugger is exactly the right tool for certain situations. But I find those situations to be oddly specific most of the time.

[shacktar]

there pops out a dialogue window saying stuff like "no source code at this point". I then have to shift+f5 to quit debugging

Don't hit shift+f5 (unless you actually want to quit debugging). You should simply hit f5 to continue debugging. When it says "no source code at this point", that's when Windows code takes over. After this, your program should go back to listening for messages.

So, let's say you have a break point on mouse click. You are in the WndProc at your breakpoint. Hit F5 and your program will resume. Click the mouse again and your program would hit that breakpoint again.

Win32 program has different behaviors for different messages. How can I set the message values while debugging?

You could set up buttons that each send a specific message to your window(using the SendMessage function). Note that this may not duplicate any buggy behavior you may experience during the normal execution of your program. For instance, if you have a bug when you're moving the mouse on top of a control, having a button send a WM_MOUSEMOVE may not replicate the conditions necessary for the bug to occur.

[h9uest]

@shacktar:

Thanks for the reply.
Your suggestions work.

However, it would be great if I can interact with the application (such as click on a button) and see how variables (such as msg) changes in the memory. Is it possible in Visual Studio?



@Computergeek01:
Thanks for the reply.

I believe the debugger gives me a lot of help and I intend to learn using it effectively.

The underlying issue is, I'm new to win32 programming and I'm not comfortable debugging GUI win32 programs with visual studio. I have experience debugging console applications and the vs debugger exposes everything before me: memory status, which instruction is to be executed, etc;
I feel comfortable.

I'm trying to have the same comfortable feeling when I debug GUI programs. This is the motivation for this post.


As I said to shacktar, writing stuff like SendMessage functions is great, and I believe it will work. However, it's like writing cout for debugging when dealing with console apps. I personally don't think it is a very efficient way.

So if you have some suggestions or some books/tutorials to recommend, I'll appreciate it.

[shacktar]

However, it would be great if I can interact with the application (such as click on a button) and see how variables (such as msg) changes in the memory. Is it possible in Visual Studio?

Debugging a Win32 GUI app is the same as debugging a console app except for two differences. The first is with a console app, you essentially know the calling sequence of your code beforehand, whereas with a GUI app, your code only runs when a message has been sent (otherwise, its idling or waiting in a message loop). The second (major) difference is when you debug a GUI app, you may disrupt the message sequence. That is, when the debugger appears, your mouse is now hovering over the debugger, when in normal execution, you would have been initiating WM_MOUSEOVER events, for example. To get over this, you could use dual monitors, remote debugging, or simply trying to place your window and debugger window side by side.

That said, you can see how variables change in memory when debugging a Win32 app. Once your code hits a breakpoint, you're able to see the call stack just like in a console app (though it might start in an unfamiliar place) and you're able to watch variables, hover over variables and see their contents, open the memory window, etc. You can of course do all that until your app's execution is handed back over to the message loop.

You can't see how "[messages] change in memory" because they're just pre-defined values. However, I think what you might want is a spy tool, such as Spy++, which essentially lets you see window messages as they occur (amongst other things).
http://msdn.microsoft.com/en-us/library/dd460760.aspx

So if you have some suggestions or some books/tutorials to recommend, I'll appreciate it.

Sorry, I don't :s I've learned mostly through practice and sifting through MSDN. That said, if you're unsure of the workings of a particular Windows message or API call, there's always MSDN :)

[h9uest]

@shacktar:

Many thanks for the explanation and suggestions!
I'll play with it, and come back with problems, if any.