A. A buffer overflow is a common problem for today's users of computer systems.
It occurs if memory allocated by an application program does not provide enough storage to serve a current request.
If, for example, an application has an interface providing some simple input controls without field length validation it will be possible for the user to cause such an overflow, if the amount of data entered exceeds the range of memory allocated by the application. As a result memory would get corrupted. It would effectively destroy data stored in the location following the buffer that got overwritten.
This is just an example, because only few users would attempt to hack their own systems.
You might ask yourself, how can this be used to introduce malicious code into a system ?
If we look at the most common form of overflows, the stack based buffer overflow, the answer lies in the way the processor stores internal, control flow relevant data, together with application data on the stack. Local variables declared in a program's function are usually allocated on the stack. The processor itself uses the stack area during function calls to store the return address, in order to be capable of continuing the control flow, upon function completion, where the function has been called from. Because this return address is stored following a function's local variables, it can be potentially overwritten by a buffer overflow.
This allows an attacker to decide where the CPU should continue the control flow upon the function's completion. If the attacker correctly chooses the data he or she uses to overflow the buffer, it will be possible to let the CPU continue execution within some part of the data used to overflow the buffer, thus allowing the introduction and execution of malicious code.
A heap based overflow works differently. Applications use the heap area of memory to store data available during the entire lifetime of the process. There are two different kinds of heaps on Windows systems. The single instance of the application default heap is allocated by the system during process creation. Dynamic heap is allocated by an application.
To increase performance, multithreaded applications tend to allocate an additional heap for each running thread. The reason for this is that the single instance of the default heap requires synchronization mechanisms to prevent two threads from accessing the same address at one time.
Lets imagine an application using some function pointers stored on the heap, to allow some kind of dynamic data processing. It would be possible to overflow another data buffer located on the heap as well, to gain control of those function pointers. An attacker could potentially let those pointers point to some malicious code, contained in the data used to overflow the data buffer.
Another potential method is overwriting global variables or static variables defined within the scope of a function. These are stored within the data segment of an application. As a result static variables are not being removed from the stack upon function completion, opposed to stack based variables. A very common scenario for this are C++ class methods, because they are similar to globally available variables (struct) and they use function pointers to implement a classes methods causing them to end up in the data segment.
The described methods can be used by an attacker in many ways. Some potentially scenarios are
described below:
- Getting a network service to execute code upon receiving maliciously crafted requests. This recently happened to the Microsoft RPC Service in the form of the LoveSan virus. Other potential targets for this kind of attack are Web Servers, Mail Servers and others.
- Getting an eMail client to execute code upon receiving mail traffic by using a maliciously crafted e-mail message.
- Getting a Web Browser to execute code upon visiting a maliciously crafted web page.