Understanding X86 Assembly Parameters: Reading from the Stack vs. Elsewhere

[whitehorsey]

1. I need help understanding how to read the parameters from the stack and not from anywhere else. In addition, what is difference between reading from the stack and somewhere else?

For example, if the parameters had this:
logicUnit(unsigned long int value, unsigned long int flags, unsigned char *result )
How would I read the parameters from the stack?

3. I'm thinking that I have to store it in the registers but that seems like I'm calling it from somewhere else.
i.e. mov eax, value
mov ebx, flags
mov ecx, result

 

[Mark44]

1. I need help understanding how to read the parameters from the stack and not from anywhere else. In addition, what is difference between reading from the stack and somewhere else?

For example, if the parameters had this:
logicUnit(unsigned long int value, unsigned long int flags, unsigned char *result )
How would I read the parameters from the stack?

3. I'm thinking that I have to store it in the registers but that seems like I'm calling it from somewhere else.
i.e. mov eax, value
mov ebx, flags
mov ecx, result

There's not a simple answer here without knowing the calling convention (see http://en.wikipedia.org/wiki/X86_calling_conventions) in use by the compiler. I'm assuming that the declaration of the logicUnit function you show is from code written in C or C++. The parameters to a function can be pushed onto the stack in right-to-left order (cdecl or stdcall convention) or left-to-right order (pascal convention) or passed in registers. In addition, a particular convention specifies whether the caller or callee (the function being called) is responsible for cleaning up the stack before the function returns.

 

[rcgldr]

In addition, what is difference between reading from the stack and somewhere else?

In 32 bit mode, the stack segment register ss and the stack pointer esp are used for stack operations like push or pop. In 64 bit mode, most of the segment registers are not used, except fs and gs are still available, normally used for operating system type stuff. You didn't mention if you're working in 32 bit or 64 bit mode. In 64 bit mode, push and pop use rsp. Register esp / rsp can't be used for addressing, and register ebp / rbp are used instead. If ebp is used, the processor defaults to using the stack segement regiser ss.

For example, if the parameters had this:
logicUnit(unsigned long int value, unsigned long int flags, unsigned char *result )
How would I read the parameters from the stack?

As Mark44 mentioned, the calling conventions can vary. Microsoft compliers have 3 calling conventions in 32 bit mode for "C" type programs (there's a 4th convention for C++ class member functions, but these are the main 3):

__cdecl.htm

__stdcall.htm

__fastcall.htm

Microsoft only has one calling convention for 64 bit mode:

http://msdn.microsoft.com/en-US/library/zthk2dkh.aspx

Assuming 32 bit __cdecl type calling convention, then the calling code looks like this:

        push    result
        push    flags
        push    value
        call    logicUnit
        add     esp,12

The typical __cdecl function code will look like this:

        push    ebp             ;save ebp       
        mov     ebp,esp         ;set ebp=esp
        sub     esp,...         ;"allocate" space for locals
;
;                               ;[ebp+ 8] = last parameter = value
;                               ;[ebp+12] = next to last parameter = flags
;       ...
        mov     esp,ebp         ;restore esp
        pop     ebp             ;restore ebp
        ret                     ;return

 

[whitehorsey]

In 32 bit mode, the stack segment register ss and the stack pointer esp are used for stack operations like push or pop. In 64 bit mode, most of the segment registers are not used, except fs and gs are still available, normally used for operating system type stuff. You didn't mention if you're working in 32 bit or 64 bit mode. In 64 bit mode, push and pop use rsp. Register esp / rsp can't be used for addressing, and register ebp / rbp are used instead. If ebp is used, the processor defaults to using the stack segement regiser ss.

I'm working with 32 bits.

As Mark44 mentioned, the calling conventions can vary. Microsoft compliers have 3 calling conventions in 32 bit mode for "C" type programs (there's a 4th convention for C++ class member functions, but these are the main 3):

__cdecl.htm

__stdcall.htm

__fastcall.htm

Microsoft only has one calling convention for 64 bit mode:

http://msdn.microsoft.com/en-US/library/zthk2dkh.aspx

Assuming 32 bit __cdecl type calling convention, then the calling code looks like this:

        push    result
        push    flags
        push    value
        call    logicUnit
        add     esp,12

The typical __cdecl function code will look like this:

        push    ebp             ;save ebp       
        mov     ebp,esp         ;set ebp=esp
        sub     esp,...         ;"allocate" space for locals
;
;                               ;[ebp+ 8] = last parameter = value
;                               ;[ebp+12] = next to last parameter = flags
;       ...
        mov     esp,ebp         ;restore esp
        pop     ebp             ;restore ebp
        ret                     ;return

Thanks for the links!

Would it be something like this:
push flags
push value
push eax
push ebx
push ecx
Then store the flags/value into one of the registers or would that be considered not calling from the stack? If so, is the CALL function the method I should be using instead?

 

[rcgldr]

Would it be something like this:
...
push flags
push value
push eax
push ebx
push ecx
Then store the flags/value into one of the registers or would that be considered not calling from the stack? If so, is the CALL function the method I should be using instead?

There's no push eax ... push ecx. With __cdecl, all parameters are on the stack. If any registers need to be saved, this is done in the called function, not in the calling code (except for the add esp,12 to restore it).

 

[whitehorsey]

There's no push eax ... push ecx. With __cdecl, all parameters are on the stack. If any registers need to be saved, this is done in the called function, not in the calling code (except for the add esp,12 to restore it).

Ah I see. Thank You! Do you know what this means:

1. and ch, 0xc0

2. and ecx, 0x01010101

I'm having a hard time with figuring out what the 0x_______ means...

 

[rcgldr]

Ah I see. Thank You! Do you know what this means:
1. and ch, 0xc0
2. and ecx, 0x01010101
I'm having a hard time with figuring out what the 0x_______ means...

The 0x... means it's a hex number, but Microsoft assemblers use 0...h instead. I don't know which assemblers allow the 0x... format.

and ch, ... is an and for bits 15->8 of the ecx register. (cl would be bits 7->0).

 

[Mark44]

The 0x... means it's a hex number, but Microsoft assemblers use 0...h instead. I don't know which assemblers allow the 0x... format.

The 0x prefix is notation from C, I believe.

and ch, ... is an and for bits 15->8 of the ecx register. (cl would be bits 7->0).

The early 16-bit Intel CPUs (8088, 8086, and 80186) had registers named AX, BX, CX, DX and a few others. These registers were divided into 8-bit halves, with AL, BL, CL, and DL being the lower 8 bits (bits 0 ... 7), and AH, BH, CH, and DH being the upper 8 bits (bits 8 ... 15).

 

[whitehorsey]

The 0x... means it's a hex number, but Microsoft assemblers use 0...h instead. I don't know which assemblers allow the 0x... format.

and ch, ... is an and for bits 15->8 of the ecx register. (cl would be bits 7->0).

The 0x prefix is notation from C, I believe.
The early 16-bit Intel CPUs (8088, 8086, and 80186) had registers named AX, BX, CX, DX and a few others. These registers were divided into 8-bit halves, with AL, BL, CL, and DL being the lower 8 bits (bits 0 ... 7), and AH, BH, CH, and DH being the upper 8 bits (bits 8 ... 15).

Thank you! I understand it now!

I'm not sure if you guys could help me with this problem but my Visual Studio 2012 won't run my program and keeps crashing. Do you know what I could do?

 

[rcgldr]

I'm not sure if you guys could help me with this problem but my Visual Studio 2012 won't run my program and keeps crashing. Do you know what I could do?

You'll probably want to run your asm program using the C language environment. You'll may also need to create a custom build step (this was needed for VS 2010 and earlier versions). Link to example source assembly code that prints the value 64 to the console window:

http://rcgldr.net/misc/xasm.zip

To create a project, start with a directory that only includes the .asm source file. Make that the project name, and create an empty win32 console project. The option for creating an "empty" project shows up on the second screen when creating the project. Once the project is created, go to project, add existing item, and click on the .asm file name to add it.

To create the custom build step, right click on the name of the asm file in the solution explorer window, then click on custom build step.

For debug build it looks like this:

ml /Zi /c /Fo$(outdir)\example.obj example.asm

For release build it looks like this:

ml /c /Fo$(outdir)\example.obj example.asm

In both cases, "outputs" of the custom build step looks like this:

$(outdir)\example.obj

If you use my example source file, the name of the file is "x.asm", so use "x" instead of "example" in the custom build steps.

 

[whitehorsey]

You'll probably want to run your asm program using the C language environment. You'll may also need to create a custom build step (this was needed for VS 2010 and earlier versions). Link to example source assembly code that prints the value 64 to the console window:

http://rcgldr.net/misc/xasm.zip

To create a project, start with a directory that only includes the .asm source file. Make that the project name, and create an empty win32 console project. The option for creating an "empty" project shows up on the second screen when creating the project. Once the project is created, go to project, add existing item, and click on the .asm file name to add it.

To create the custom build step, right click on the name of the asm file in the solution explorer window, then click on custom build step.

For debug build it looks like this:

ml /Zi /c /Fo$(outdir)\example.obj example.asm

For release build it looks like this:

ml /c /Fo$(outdir)\example.obj example.asm

In both cases, "outputs" of the custom build step looks like this:

$(outdir)\example.obj

If you use my example source file, the name of the file is "x.asm", so use "x" instead of "example" in the custom build steps.

Thanks! I played with my code a bit more I found where it crashes:

__declspec(naked) void
function(unsigned long input, unsigned long *output)
{
__asm{
push eax
push ebx

mov ebx, [esp + 16]

push ebx
call MIRROR //another function that switches a byte from 100 to 001
pop ebx //right here is where it crashes

...
//some similar code as the last three lines above for other functions and each of
//them also crash at pop ebx

pop ebx
pop eax

ret
}
}
Why would it crash when I try to pop ebx?

 

[rcgldr]

mov ebx, [esp + 16]
push ebx
call MIRROR //another function that switches a byte from 100 to 001
pop ebx //right here is where it crashes

Why would it crash when I try to pop ebx?

What is at esp+16? Note that push eax and push ebx also subtract 8 from esp.

What does the MIRROR expect for it's parameter, a value or a pointer?

If it expexts a pointer, then you'd want:

lea ebx,[esp + 16]
push ebx
call MIRROR
add esp,4 (or pop ebx)

 

[whitehorsey]

What is at esp+16? Note that push eax and push ebx also subtract 8 from esp.

What does the MIRROR expect for it's parameter, a value or a pointer?

If it expexts a pointer, then you'd want:

lea ebx,[esp + 16]
push ebx
call MIRROR
add esp,4 (or pop ebx)

Thanks! I was able to figure it out. :]

I have another question:

What is the purpose of doing this?
mov dl, 0
rcl dl, 1

Isn't dl equal to 0000 0000? So by rotating dl to the left by one wouldn't the result still be 0000 0000?

And when you rotate does it affect any of the other bytes like the ones stored in dh and dx? I'm using 32 bits.

 

[rcgldr]

Thanks! I was able to figure it out. :]
What is the purpose of doing this?
mov dl, 0
rcl dl, 1

It clears the lower 8 bits of edx, then rotates the carry flag into the low order bit (bit 0) of edx, and also clears the carry flag. It would provide a way to test the carry flag (and also clear the carry flag) in C without having to use more assembler code (if edx was a register based variable in C).

I'm wondering what you're trying to accomplish with this mix of C and assembly code. Are you trying to figure out the code that the compiler generates? If you want to create functions that are written in assembler, then you could use the example I created, which is main() written in assembler.

 

[whitehorsey]

It clears the lower 8 bits of edx, then rotates the carry flag into the low order bit (bit 0) of edx, and also clears the carry flag. It would provide a way to test the carry flag (and also clear the carry flag) in C without having to use more assembler code (if edx was a register based variable in C).

I'm wondering what you're trying to accomplish with this mix of C and assembly code. Are you trying to figure out the code that the compiler generates? If you want to create functions that are written in assembler, then you could use the example I created, which is main() written in assembler.

Oh so wouldn't dl still be 0000 0000? Because your only rotating 0s. I'm kind of confused could you give a visual picture of what's happening?

Ah I'm actually working on a programming assignment where I have to move things around swapping bytes stuff like that. And I decided that using the rotate command seems much more efficient than shifting.

 

[rcgldr]

Oh so wouldn't dl still be 0000 0000? Because your only rotating 0s.

RCL is a 9 bit rotate, the 8 bits in DL and the 1 bit in the carry flag. ROL is an 8 bit rotate that doesn't involve the carry flag. Link to wiki list of X86 instructions:

http://en.wikipedia.org/wiki/X86_instruction_listings

 

[whitehorsey]

RCL is a 9 bit rotate, the 8 bits in DL and the 1 bit in the carry flag. ROL is an 8 bit rotate that doesn't involve the carry flag. Link to wiki list of X86 instructions:

http://en.wikipedia.org/wiki/X86_instruction_listings

Ohhhh I see. Do you know why I'm getting an unhandled exception by doing this line:

mov byte ptr [ecx], ah
pop eax

ecx is a register I use to store the value I get from ah. ah is used in another function and holds a value (something like 1000 0100). Eax then has to be popped but I want to save the value in ah so I stored it in ecx yet I'm getting an unhandled exception.

 

[rcgldr]

unhandled exception
mov byte ptr [ecx], ah
pop eax

This attempts to store the contents of ah in the memory location pointed to by ecx. What you probably want is:

mov cl,ah ;save ah in cl
pop eax ;restore eax

 

[whitehorsey]

This attempts to store the contents of ah in the memory location pointed to by ecx. What you probably want is:

mov cl,ah ;save ah in cl
pop eax ;restore eax

Ah.. Thank You!

 

[rcgldr]

You could also just save all of eax by using

mov ecx,eax

then using ch later when you needed it.