To better understand memory safety in compilers, I analyzed and improved a single-pass compiler called “Smaller C.” The only memory safety consideration made in Smaller C natively is to store return address pointers at the beginning of the stack frame for functions. It is marginally easier to overwrite the end of a stack frame than the beginning with an overflow attack, so this feature protected against some basic attacks. Beyond this, Smaller C had no memory safety measures.

Some features would turn out to be impossible to implement, due to the single pass nature of the program, and the fact that the code generation module of the compiler was an unmodifiable pre-compiled add-on. T he primary improvement made to the compiler was a memory address sanitization procedure. Whenever memory is allocated, a “poisoned” memory zone is created on each side of the block. When the memory block is freed, it is also poisoned. Whenever an access is made to a poisoned memory block, an error is thrown. This prevents Use after Free and buffer overflow memory safety violations. This procedure has a massive memory overhead, which is only necessary due to Smaller C being a single-pass compiler. In multiple pass compilers, significantly more elegant methods can be used to poison memory addresses that have been freed, since these compilers can retain information about the allocations.

Smaller C was not designed to be a secure compiler, it was designed to be lightweight and fast. Adding security checks to code will often require a significant overhead. Even simple procedures, like Canaries to prevent stack smashing attacks add dozens of lines to every function call. A full analysis of the tradeoffs between security and speed can be found in the project report Here.