X86 assembly procedure prologue help.

Click For Summary
SUMMARY

The discussion centers on the function prologue in X86 assembly, specifically for a main function compiled with GCC. Key lines of code include "and esp, 0xblackf0" for aligning the stack pointer to a 16-byte boundary, while "mov eax, 0x0" and "sub esp, eax" are identified as redundant operations. Participants conclude that these lines may be related to memory optimization or debugging, although they do not contribute to functionality. The conversation highlights the importance of understanding assembly code generated by different compilers, particularly GCC.

PREREQUISITES
  • X86 assembly language fundamentals
  • Understanding of function prologues and stack management
  • Familiarity with GCC compiler options
  • Knowledge of CPU registers, specifically EAX
NEXT STEPS
  • Explore X86 assembly function prologues in detail
  • Learn about GCC compiler flags and optimization techniques
  • Investigate stack alignment and its impact on performance
  • Study the role of EAX in function return values and CPU flag management
USEFUL FOR

Assembly language programmers, systems programmers, and anyone interested in optimizing code generated by GCC or understanding low-level memory management.

perplexabot
Gold Member
Messages
328
Reaction score
5
Hey all, I was doing a bit of reading and ran into the following code (this is a function prologue for a main function, if you need more code or the actual C code let me know):

push ebp
mov ebp, esp
sub esp, 0x18
and esp, 0xblackf0
mov eax, 0x0
sub esp, eax

So I understand the first three lines. first save the base pointer to the stack, second assign the base pointer and lastly set the new stack pointer with enough space for the local variables.

What I don't understand are the last three lines. I know what the and, mov and sub opcodes do but I can't get the general picture. Also what exactly is eax used for? Is it a gpr? Those last three lines don't appear in regular functions, so I am guessing it has something to do with the main function.

Any help will be much appreciated, thank you.
 
Technology news on Phys.org
"and esp, 0xblackf0" - truncates esp to a 16 byte boundary

"mov eax, 0x0" "sub esp, eax" - accomplishes nothing.
 
  • Like
Likes 1 person
rcgldr said:
"and esp, 0xblackf0" - truncates esp to a 16 byte boundary

"mov eax, 0x0" "sub esp, eax" - accomplishes nothing.

Hmmm thank you. If you don't mind me asking though, why truncate it and why have the second to last line if it does nothing (I can see that it does nothing)?

Thanks.
 
One possibility is that some type of structure that is some multiple of 16 bytes long could be allocated on the stack. For example, the "mov eax, 0x0", could instead be "mov eax, 0x40", to allocate 64 bytes from the stack on a 16 byte boundary. I haven't seen this done with any of the microsoft compilers I use. Perhaps the space is allocated for some type of optional debug mode.
 
Last edited:
  • Like
Likes 1 person
rcgldr said:
One possibility is that some type of structure that is some multiple of 16 bytes long could be allocated on the stack. For example, the "mov eax, 0x0", could instead be "mov eax, 0x40", to allocate 64 bytes from the stack on a 16 byte boundary. I haven't seen this done with any of the microsoft compilers I use. Perhaps the space is allocated for some type of optional debug mode.

Thank you.
 
rcgldr said:
"and esp, 0xblackf0" - truncates esp to a 16 byte boundary

That may be to optimize the access speed to physical memory, and/or the CPU chip's memory cache(s).

"mov eax, 0x0" "sub esp, eax" - accomplishes nothing.

I'm not sure, but could that reset some of the CPU flags to a known state? (this sort of trick can be quicker than setting the flags directly).

It certainly sets eax to a known state, so it does accomplish "something".
 
  • Like
Likes 1 person
"mov eax, 0x0" "sub esp, eax" - accomplishes nothing.

AlephZero said:
I'm not sure, but could that reset some of the CPU flags to a known state? (this sort of trick can be quicker than setting the flags directly).
The flags are just set to indicate the unsigned result is greater than zero (esp - 0). I doubt it's used, as the cpu flags will get set again on any math like instruction that occurs later.

"mov eax, 0x0"

AlephZero said:
It certainly sets eax to a known state, so it does accomplish "something".
But it's likely that eax is going to get changed during the function. eax is the return values so a return(0) will translate into "mov eax,0", or more likely "xor eax,eax", but at the end of the function, just before "ret". As mentioned before, I haven't seen those last 3 lines of function header code with microsoft compilers.
 
Interesting, I didn't know that eax was for the return values. Good info. Thanks.
I also was wondering if those last three lines were compiler optimizations.
 
perplexabot said:
I also was wondering if those last three lines were compiler optimizations.
Just the opposite, optimization would not add lines of code that accomplish nothing. What compiler generated that code?
 
  • #10
rcgldr said:
Just the opposite, optimization would not add lines of code that accomplish nothing. What compiler generated that code?

Hmmm, that makes sense, thank you for correcting my logic. This was disassembled using gdb.
 
  • #11
perplexabot said:
Hmmm, that makes sense, thank you for correcting my logic. This was disassembled using gdb.
I was wondering what compiler created that code, not how you disassembled it.
 
  • #12
Oops, sorry, was compiled using gcc.
 
  • #13
perplexabot said:
Oops, sorry, was compiled using gcc.
See what the various options are for gcc, perhaps ones that disable any debug stuff and increase optimization to see if you get the same function entry code.
 

Similar threads

  • · Replies 19 ·
Replies
19
Views
6K
  • · Replies 4 ·
Replies
4
Views
9K
  • · Replies 5 ·
Replies
5
Views
4K
  • · Replies 11 ·
Replies
11
Views
5K
  • · Replies 11 ·
Replies
11
Views
2K
  • · Replies 6 ·
Replies
6
Views
6K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 32 ·
2
Replies
32
Views
4K
  • · Replies 5 ·
Replies
5
Views
4K