One of the nastiest kinds of computer security problems is the dreaded stack smashing attack. In a nutshell, this involves causing a program to write beyond the end of a buffer allocated on the stack in such a way that the return address for the current subroutine call is modified, diverting execution to a location of the attacker’s choosing. Most often, the chosen location is a snippet of malicious code which is also inserted on the stack immediately after the smashed return address as part of the same buffer overrun.

This kind of exploit relies on the fact that computer programs are very predictable. Usually, the code that reads (and overruns) the buffer will execute in an absolutely identical stack context every time. This allows the exploit code to predict the absolute address at which the malicious code will be inserted on the stack, which is important because return addresses are always absolute on pretty much every architecture…which leads to my real point in writing this. There are a lot of products out there which try to protect against stack-smashing attacks, and many of them are quite sophisticated, but my theory is that a very simple trick can reduce the severity of many stack-smashing attacks for almost zero cost.

My idea is this: replace the main() function in your program with an assembly-coded function that consumes a semi-random amount of stack space and then calls your real main() function. Ditto for your top-level function in any spawned threads. What this does is nullify the stack-smasher’s assumption that the stack state will be predictable when their code is called; their bogus return address will then point to somewhere other than where they thought. This will probably cause your program to crash, which seems bad except for two things:

  • A crash is usually preferable to having your machine compromised, taken over, and used to launch further attacks.
  • You’re more likely to notice a crash than the silent subversion of your machine, and more likely to do something about it.

This is absolutely not a comprehensive solution to stack-smashing attacks. Not even close. However, it should reduce the severity of a substantial subset of stack-smashing attacks, with a fairly simple implementation (achievable without special OS or library support) and zero runtime cost. If you disagree that it will work, or think it’s worthless, please feel free to say so on PlatSpot.