Why segmentation fault when quitting the main

[jk22]

// Example of variable argument number function call and return adress deviation programm
// never program in a style like this

#include<stdio.h>
#include<string.h>
#include<stdlib.h>

void (*ret)();
void replaceexit(void);void prtarg(char *arg, ...)
{
	char **stack=&arg;

	printf("Return adr : %x\n",(int)(*(stack-1)));
	ret=(void (*)())*(stack-1);
	*(stack-1)=(char *)(replaceexit);

	
	while(*stack)
	{
		printf("%s",*stack);
		stack++;
	}
}

void replaceexit(void)
{
	printf("I come here\n");
	(*ret)();
}

int main(void)
{
	char *arg1;
	arg1=new char[10];
	strcpy(arg1,"Hello\n");

	prtarg(arg1,"Bonjour\n","Salut\n",0);

	printf("Return to main\n");
}

What it actually prints when runned is :

Hello
Bonjour
Salut
I come here // it is in the replace function for the exit
Return to main // it returns to main
Segmentation fault // when quitting the program error

I don't understand, maybe the stack is not treated in a good way ?

 

[jedishrfu]

It looks like you are trying to mess with the stack. Is this an exercise for a class?

The segmentation fault means you're referencing memory outside your program which means one of your pointers is messed up.

Try putting printf stmts to print the value of the pointer instead of the contents of where it points. IE if the pointer has the value of 1000 then print the 1000 don't print what's at memory location 1000. The segmentation is happening because you're trying to print the contents at memory location 1000.

In particular look at the ret pointer.

Better yet run your program in a debugger and step through each line.

I don't think the program has returned to main but the while loop has printed the message you see. A debugger would more clearly show this. ( Don't leave home without it. :-) )

 

[jim mcnamara]

This code is neither C nor C++. How the H%ll did you compile it?

char *arg1;
arg1=new char[10];  [color=red]// no way  in C [/color]

To get it to compile in C:

char *arg1=malloc(10);

After that on return to main the stack pointer is broken and you get an access violation - you attempt to execute non-executable memory.

FWIW - when C runs it starts with code you do not see: often named something __start, when main ends, there is some image rundown code (closing open files, etc.) that executes, kinda like the reverse of __start. It has lots of names depending on the platform. Lets's call it __fini. You forced __fini to execute read-only memory.
What happens is actually implementation dependent. I cannot know what your system actually does.

Most modern OS runtimes have a "noexec stack protective mechanism" to prevent malicious code from doing crashing the stack. Somebody is else is sure to say 'undefined behavior!' which is what I'm saying with a marginal explanation. Simply saying 'UB' is not an answer to this kind of thing, IMO. The fact that coders do this is proof , to me, they do not understand the full consequences of UB. You obviously did this intentionally. In the hope of learning something...

 

[voko]

Here is what happens. When the prtarg is called, the stack looks like this:

address to return to main
address of "Hello"
address of "Bonjour"
address of "Salut"
0

Then prtarg does a nasty thing and the stack becomes

address to "return" to replaceexit
address of "Hello"
address of "Bonjour"
address of "Salut"
0

I wrote "return" because it does not really return there in the ordinary sense of the word.

Then, when prtarg "returns" to replaceexit, the stack is:

address of "Hello"
address of "Bonjour"
address of "Salut"
0

Note the return address is gone, it was popped off the stack as part of the return sequence from prtarg. Then replaceexit does its thing and CALLS (!) into main using the return address previously stored in the ret variable. This makes the stack inside main() look like this:

address to return to into replaceexit
address of "Hello"
address of "Bonjour"
address of "Salut"
0

At this stage main executes the stack cleanup sequence, it has to remove the addresses of the strings it pushed there before calling prtarg, and the sequence expects the stack will be like this:

address of "Hello"
address of "Bonjour"
address of "Salut"
0

So it just pops four addresses off the stack. But the stack in reality is different, so after that sequence is complete, the stack is:

0
(data pushed on stack earlier)

This is where things go very wrong. main() probably expects that the (data pushed on stack earlier) starts with an address to return to, so it returns to the zero address, which produces the message you see.

 

[rcgldr]

As voko posed, the first problem occurs when printarg() returns, since the return address is popped off the stack, and the program junps to replaceexit() with no space allocated on the stack for a return address, so at the start of replaceexit(), what should be the return address is the first argument passed to printarg().

The next problem occurs because replaceexit() pushes even more stuff on the stack (usually ebp, maybe more stuff), then calls (*ret)() which pushes a return address onto the stack, and jumps back to just after where main called printarg(). At this point, the stack contains the return address from the call to (*ret)(), and whatever replaceexit() pushed onto the stack before the call to (*ret)().

 

[jk22]

This code is neither C nor C++. How the H%ll did you compile it?

I just used g++ but I know I make mix between c and c++.

Thanks for your help.

Still I don't see how to make the program todo this kind of replacement of the return adress that works (I suppose it's used by some virii program)

 

[rcgldr]

Still I don't see how to make the program todo this kind of replacement of the return adress that works (I suppose it's used by some virii program)

I'm not sure what would be gained by replacing the return address in C / C++, since it messes up the stack.

A call back (end action) pointer to function could be assigned to one of multiple functions, and this is useful for interrupt driven code used in device drivers or asynchronous I/O call for some operating systems.

 

[voko]

Still I don't see how to make the program todo this kind of replacement of the return adress that works (I suppose it's used by some virii program)

Malware will typically NOT return to the main function after it has gained control, because at that point the stack may be smashed so badly the application won't work anyway. It is possible, though, by manipulating the stack pointer directly, which would require use of the assembly language or some non-standard C language or library features. There may be some clever technique in "pure" C; I write pure as "pure" because that will again very significantly depend on the platform and the compiler.

 

[jedishrfu]

One C feature that does something like this is the longjmp feature that allows you to exit some subroutine skipping the return back up a few levels to an associated setjmp which resets the stack. Its kind of like the throw catch mechanism.

 

[rcgldr]

setjmp / longjmp example:

#include <setjmp.h>
#include <stdio.h>
#include <stdlib.h>

static jmp_buf cpustate;    /* this is an array of integers */

void fun0(void){
    printf("fun0\n");
    longjmp(cpustate, 1);
}

void fun1(void){
    printf("fun1\n");
    longjmp(cpustate, 2);
}

void fun2(void){
    printf("fun2\n");
    longjmp(cpustate, 3);
}

int main(int argc, char *argv[])
{
int i;
    i = setjmp(cpustate);   /* initial call returns 0 */
                            /* longjmp also returns here */
                            /* setting i to longjmp parameter */
    switch(i){
      case 0:
        fun0();
        break;
      case 1:
        fun1();
        break;
      case 2:
        fun2();
        break;
      case 3:
        printf("main\n");
        break;
      default:
        break;
    }
    return(0);
}