Debugging x86 Process Trace w/ Single Trap Interrupt

[Stephanus]

Dear PF Forum,
How does debugger in x86 process trace with single trap interrupt?
Supposed I have a simple program like this.

0100   B81000   mov AX,0010
0103   BB2000   mov BX,0020
0106   B93000   mov CX,0030
0109   BA4000   mov DX,0040
010C   CD20     int 20

And a debugger would like to trace line 0103: mov BX,0020.
So it has to single trap this instruction.
So, after hooking int 1.
Then does the debugger do something like this?

mov byte ptr [0102],9D ; the opcode for POPF
pushf
pop AX ; save the flag register int AX
or AX,100h ; set the TF bit
push ax
jmp 0102

And after this the int 1 handler restore the value that was in [0102] and process debugging procedure?
I have suspected that, but I need a clear confirmation. Perhaps anyone has any idea?
Thank you very much

 

[jedishrfu]

This article has some discussion on traps and interrupt as and talks about the need to preserve the CPU state while processing the trap:

http://flint.cs.yale.edu/cs422/doc/art-of-asm/pdf/CH17.PDF

I don't know if it answers your question but felt you might like to read about it while waiting for someone who knows something.

 

[rcgldr]

The debugger normally uses a hardware breakpoint (or uses the single byte opcode for int 3) on the first instruction. When that first breakpoint is reached, the exception handler turns on the single instruction trap flag and returns. This results in an exception after each instruction is executed in order to perform the trace. When the trace is complete, the single instruction trap flag is turned off.

 

[Stephanus]

The debugger normally uses a hardware breakpoint (or uses the single byte opcode for int 3) on the first instruction.

Yep, CC as oppossed to CD in x86 processor.

When that first breakpoint is reached, the exception handler turns on the single instruction trap flag and returns. This results in an exception after each instruction is executed in order to perform the trace.

So how does x86 turn it on? By POPF?

When the trace is complete, the single instruction trap flag is turned off.

Again, turns it off by POPF?
@jedishrfu Thanks for your reply. I'm sttill answering someone in Today I Learn. I"d like to respond to your answer later.
And thanks @rcgldr . Is it POPF?

 

[rcgldr]

So how does x86 turn it on? By POPF?

POPF or IRET.

 

[Stephanus]

POPF or IRET.

Thank you very much. I just need a confirmation for my theory. Whether x86 does use this instruction or another instruction that I'm not aware of.

This article has some discussion on traps and interrupt as and talks about the need to preserve the CPU state while processing the trap:

http://flint.cs.yale.edu/cs422/doc/art-of-asm/pdf/CH17.PDF

I don't know if it answers your question but felt you might like to read about it while waiting for someone who knows something.

It's said in page 6, below

The single step exception occurs after every instruction if the tracebit in the flags register is equal to one. Debuggers and other programs will often set this flag so they can
trace the execution of a program...

Actualy I have known that before, otherwise I woudln't ask about POPF or as @rcgldr says, IRET. Because they are the instructions that can (indirectly) set bits in Flags register. Aside from STI, CLI, CLD, STD, STC, CLC. And if you insist you can XOR AX,AX to set zero flag . But that is a very, very good article. I learn a lot of other things from it. Thank you very much @jedishrfu

 

[Stephanus]

POPF or IRET.

IRET is more sensible. While POPF requires you to change the instruction 1 byte before the intended instruction that needs to be single trapped.

 

[rcgldr]

IRET would be the normal way to do this. POPF might be able to be used with interrupts disabled (assuming this masks off single instruction trap), followed by ... CLI, RET, since CLI doesn't reenable interrupts until after the RET is executed.

 

[Stephanus]

IRET would be the normal way to do this. POPF might be able to be used with interrupts disabled (assuming this masks off single instruction trap), followed by ... CLI, RET, since CLI doesn't reenable interrupts until after the RET is executed.

Ahh, thanks for the answer @rcgldr , but while we are at it. Can I ask a question that's perhaps off topic?
CLI will ignore interrups from keyboard/timer, or in short an interrupt generated from IRQ pin.
What about INT 00? Divide by zero?
INT 01? But I think it's ignored considering your answer. But that's for INT 01 that's "called" by IRET.
INT 02?
INT 03? But it's not a real "interrupt" right. It's explicitly called. Either by CC or CD 03.
INT 04? Will the processor ignore INTO?

Thank you very much.
What about other interrupts? 10 - Video, 13 - Harddisk, DOS interrupts. Since they are explicitly called?

 

[rcgldr]

It's been a long time, I looked this up again, looks like it needs to be IRET. I don't think the trap flag is masked by the interrupt flag. Each time a single instruction trap exception occurs, the trap flag is reset, so in order to continue tracing, the stack has to be setup so IRET re-enables it.

CLI and STI are only supposed to affect hardware interrupts, not software interrupts.

 

[Stephanus]

It's been a long time, I looked this up again, looks like it needs to be IRET. I don't think the trap flag is masked by the interrupt flag. Each time a single instruction trap exception occurs, the trap flag is reset, so in order to continue tracing, the stack has to be setup so IRET re-enables it.

CLI and STI are only supposed to affect hardware interrupts, not software interrupts.

Trap flag is masked by interrupt flag? Yeah those kind of things can only be done by research. First you hook up interrupt 9 (keyboard) then see if IF is disabled (it should enabled before right, to allow the processor to process interrupt from IRQ) and then set up another unused interrupt, perhaps F0? Then force call it your self by int F0, see if IF is enabled or disabled.
first
CLI
INT F0
then
STI
INT F0
Calling single trap by IRET? Well that's the easiest way I think. POPF will force you to change the instruction 1 byte before the intended instruction that you need to debug. Then you JMP to the instruction (which is POPF) and in your debugging handler, you restore the 1 byte instruction that you change to POPF.
But, I just want to know if there is other way to trigger single trap interrupt beside POPF (now thanks to you now I know IRET ) or there's other assembly instruction.

[EDIT: Those kind of things can only be done by research not reset]

 

[Stephanus]

I have many questions answered this week from PF Forum, regarding the basic of x86 process. Protected mode, single trap. Which I should have studied looong time ago. I've never used assembly in my work. But still curiosities should be fulfilled.