How to learn assembly code for debugging VC++?

[chingkui]

Sometimes when I try to debug my codes, VC++ brings me something I believe are assembly codes which I couldn't read at all. I want to know where I can learn these to better understand what is going on at low level. I am quite comfortable with high level programming, but I believe I really need to understand more about low level stuffs to improve my programming skills. Anyone has books/websites suggestion I can learn this skill? (a little background: I am a computational scientist/applied mathematician, learn high level programming myself, never taken any C.S. class in programming/architecture. So, I am almost completely clueless in low level operations)
Thanks.

 

[chiro]

Hey there chingkui.

I have a few suggestions for you.

The first suggestion is to get architecture manuals from CPU vendor websites like Intel and AMD. The documents from these sites will contain information on architecture/ instruction sets/ and standards that will help you understand the flow-control, optimizations, interaction with hardware and so on. The key document for you is the instruction set document. There should be documents for both 32 and 64 bit program environments.

If you are not dealing with things like device drivers, you will probably ignore a subset of the instruction set like hardware operations (IN, OUT) as well as some instructions like IRET. You said you're working with VC++ which implies Windows which implies no direct hardware access (hardware interrupts, ports, etc).

Using the above in conjunction with code samples and examples in conjunction with your own code that you write to gain experience is a good way to start, fiddle with, and eventually master assembly.

The reason I'm not recommending books per se, is because many books that I have read are very old. Its true that most of the commands and their use at the machine language level have mostly stayed the same, but there has been a big difference in memory addressing between segmented addressing and flat memory addressing. This is a primary reason why I am not advocating old assembler books.

If you have any questions, I'll try my best to answer them.

 

[Borek]

Sometimes when I try to debug my codes, VC++ brings me something I believe are assembly codes which I couldn't read at all.

That most likely means some earlier error resulted in instruction pointer being moved outside machine code generated by compilation of your program. While understanding assembly won't hurt, at this stage code you see is often either garbage, or completely unrelated to your program.

 

[pairofstrings]

Learning assembler is little easy when you learn it in conjunction with examples and by debugging those example programs using a separate program. Start by referring an easy to understand book. Good Luck!

 

[chiro]

Learning assembler is little easy when you learn it in conjunction with examples and by debugging those example programs using a separate program. Start by referring an easy to understand book. Good Luck!

That's part of the learning process. Its rare that you become a good programmer without having to debug a program, no matter what language, platform, environment its in.

 

[TylerH]

Here's a good reference for use in conjunction with the Intel/AMD manuals: http://flatassembler.net/docs.php?article=manual#2.1

Note: Only section 2.1 is related to x86 assembly, the rest is related to the Fasm assembler itself.

 

[chiro]

Here's a good reference for use in conjunction with the Intel/AMD manuals: http://flatassembler.net/docs.php?article=manual#2.1

Note: Only section 2.1 is related to x86 assembly, the rest is related to the Fasm assembler itself.

You brought up a great point that I want to emphasize to the OP: You will need to know specific issues about the assembler you are using. Different assemblers usually have different syntax for whatever reason so knowing this and how to use it is also important.

Personally I've only used the Microsoft Assembler (MASM), but there are others out there including NASM, TASM (Turbo Assembler), and probably others. I found MASM pretty good, but I don't have experience with others (the only exception is __asm code within VC).

Hopefully someone can chime in on their experiences with other Assemblers.

 

[TylerH]

Fasm is great. It has a focus on simplicity. But to be honest, I learned assembly with it for the singular reason that it has a GUI. I really didn't want to learn to use Nasm or Tasm with command line at the time. I discovered all of its other features well after the fact of deciding on which to use.

 

[rcgldr]

In the case of Microsoft tools, masm.exe has been replaced by ml.exe (masm + linker). ml.exe is still included with Microsoft's Visual Studio, but you have to manually edit a project to include the command line step to assemble source code using ml.exe. I've done a few projects that are assembly code only, using Visual Studio. You create an empty console project, then add the .asm source code to the project. Then for that .asm source file, you need to create a custom build step to invoke ml.exe:

ml /c /Fo$(outdir)\example.obj example.asm

or use this to include debugging info (just add the \Zi):

ml /Zi /c /Fo$(outdir)\ge32.obj ge32.asm

Once this is done, you can use use the Visual Studio standard GUI interface to build and debug the program. To trace through assembly code with the debugger, you'll want to enable the debugger's register view.

To determine the INCLUDELIB and EXTRN (library function calls) statements needed to use standard c library calls for a console project, I create a small C console project and output assembly code, then copy that to the test assembly source file.

One issue is that some of the free "express" versions of visual studio don't include the standard c library functions for console projects. There may be a work around but I haven't tired to figure this out since I use VS 2005, which I bought a long time ago.