C Language: Runtime Library Standards & Operating Systems

[RedX]

Is it true that the the makers of the programming language C get to specify standards for runtime libraries, and operating manufacturers must implement those standards in the runtime libraries they provide for C?

Or do the operating system manufacturers tell the makers of C "here are my system calls to access the kernel, and you, the makers of C, have to write a runtime library using these system calls"?

Say I wanted to write my own high-level programming language for an x86 architecture. I create my preprocessor, compiler, and linker, and I assume that operating system manufacturers won't write the runtime libraries for me like they do for C. Therefore I am stuck writing runtime libraries using system calls that were optimized for use with C?

 

[schip666!]

There are specs for runtime libraries, e.g., POSIX, that make an attempt to match the "impedance" between application programs and operating system internals. Given that C-like languages are so prevalent it behooves OS builders to support the spec in one way or another. Of course there are always special cases and features that need their own interfaces.

Most "real" OS's have some kind of separation of church and state between the kernel and the user, and this is usually what gets translated by the system libraries. Since C code can be very "close to the machine" it doesn't really add much overhead. It's really more of a convenience than a matter of optimization for the language.

On top of the syslibs are a bunch of application libs that provide user oriented features like GUI's and buffered file I/O. If you have to write your program over-and-over for each OS you will soon come to appreciate these standard libraries too.

The Java Virtual Machine is written in C or C++ and provides, IMHO rather klunky, access to lowlevel C functions via JNI. It makes use of all the "C" system and application libraries to ease the job of porting between machines. It's still not all that easy to port because of things like different ideas of signal handling and system management, but it's do-able.

I would expect that your new-improved-language would probably have a C-like nugget at the bottom on which you can bootstrap your own syntax. Or else you can study up on syslib implementation for each OS in which you are interested...

 

[RedX]

I'm nowhere near knowledgeable enough to write my own language! I formulated a hypothetical to try to better understand the relationship between OSs and libraries more.

When you declare a variable in C, doesn't that require a system library, since the OS needs to make the space for that variable?

So if I have the definition:

int x=1;

is stdio.h the runtime library that declares int? And inside the actual stdio code, are there calls to system functions?

I understand things like a "for loop" can be translated directly into machine code without problem by a compiler. But variable declarations require system calls, and if you are working with C, that is fortunate because the system libraries are usually given in C by the OS manufacturer, but I'm unsure of what library in C allows you to declare variables.

 

[hamster143]

When you declare a variable in C, doesn't that require a system library, since the OS needs to make the space for that variable?

No. You use syscalls to allocate some initial amount of memory, and then do with that memory as you please. It would be extremely slow to have to travel into the kernel for every variable. Syscalls are needed to let your program communicate with the "outside world", for tasks like input/output or allocation/deallocation of additional memory. They are very general and not optimized for C.

 

[RedX]

No. You use syscalls to allocate some initial amount of memory, and then do with that memory as you please. It would be extremely slow to have to travel into the kernel for every variable. Syscalls are needed to let your program communicate with the "outside world", for tasks like input/output or allocation/deallocation of additional memory. They are very general and not optimized for C.

I was looking at unistd.h , which seems to be a runtime system call library for C, written by unix manufacturers and not the makers of C compilers. Even if you allocate all the memory at once (as you say you make just one system call to allocate the memory), don't you still need a C header like unistd.h for that one system call? I was thinking that maybe stdio.h included unistd.h, but I'm not sure.

 

[hamster143]

It's not a system call library, it's just a header that declares available system calls and provides your program with ways to execute them. The library itself is probably called libc.a.

 

[RedX]

It's not a system call library, it's just a header that declares available system calls and provides your program with ways to execute them. The library itself is probably called libc.a.

So somewhere in libc.a is a definition of the standard data types like int and double?

Because in the programming snippets for beginners that I've seen, it seems like they don't have to #include libc.a. , and the linker doesn't complain when they have:

int x=1;

All they include is stdio.h, so I thought maybe stdio.h includes declarations for int and double.

int x=1; looks like a system call function to create one byte of memory, and double y=100000; a system call to create two bytes of memory. One would think that maybe it'd look like this:

void int('argument')
{
create_memory_with_os(1 byte, int_store(argument));
}

void double('argument')
{
create_memory_with_os(2 bytes, double_store(argument));
}

But you say that a whole chunk of memory gets assigned, so it's not a one-by-one thing like this.

Are there any resources you could recommend to understand everything that's going on underneath the hood for a simple program, rather than how to write complex programs?

 

[schip666!]

Usually the basic libs, say libc.a, are included in your link automagically so you may not see an explicit reference in the commands to build an executable... or you might see something cryptic like "-lc"... To get to the functions in that library the C compiler may require you to include a header file which declares their signatures -- name, arguments, and return value.

Memory types like int and double are part of the language not the machine. A particular piece of hardware may have instructions to manipulate 1,2,4,and 8 byte quantities and interpret them as signed or un-signed collections of bits, as well as dealing with 4 or 8 byte lumps as floating-point values. But the language and it's compiler decide to call them ints and doubles and know how to allocate memory for them and use them appropriately. The fact that C's ints and doubles usually look identical to the hardware's registers and FP values derives from C being a very low level language. Java's Integer class is arithmetically identical to C's int, but way more complicated in it's implementation.

The point about memory allocation is that the program asks the system for a page of memory -- that's a syscall -- and then uses some local library functions to divide -- or allocate -- that memory into lumps that it can use. There are really three ways to do this in a C program, and all of them reflect common aspects of the hardware. You can do:

1. int x; outside of a function definition. This makes the compiler set aside 4 or so bytes in a "global" memory area that is allocated when the program is first loaded into memory.

2. int y; _inside_ a function definition. This makes the compiler set aside bytes on the program stack, which has been allocated at load time and changes size as needed.

3. your create_memory_with_os(2 bytes, double_store(argument)); which looks like a fancy way to say C's malloc() (actually malloc() is user level but I forget the syscall that gets new pages, so let's just pretend this is that...). This calls into the internals and returns a chunk of memory for the program to use that is neither in the pre-initialized area nor the stack.

Also hidden in the seemy underside of your program is some statup code that does all the stack and program area allocations, initializes everything, and then calls main(). That gets included in every link without much ado as well, although there is probably a way to override it.

I'm not sure where to send you to get more under-the-covers information. I tried googling "C language internals" and "C language startup" but didn't have the patience to dig very deep. This page has a bit of description of memory layout and how a program gets executed, but it's part of an embedded programming tutorial so it might be not the focus you want: http://www.bravegnu.org/gnu-eprog/c-startup.html But I'm sure that following some of those other links will cause you to stumble on the right thing.