Buffer overflow occurs when a program writes more data to a memory buffer than it was allocated to hold. This overflows adjacent memory addresses, potentially corrupting instructions.
1. Memory Layout & Stack Registers
- ESP (Stack Pointer): Points to the top of the stack memory.
- EBP (Base Pointer): Points to the base of the current stack frame.
- EIP (Instruction Pointer): Points to the memory address of the next command to execute.
2. Overwriting the EIP
If an application accepts input without boundary checks (e.g. using `strcpy` in C), an attacker can supply input long enough to overwrite the EIP. By replacing the EIP address with the location of custom shellcode, they can redirect execution to run commands.
3. Prevention: ASLR and DEP
Modern OS engines block this using Address Space Layout Randomization (ASLR) and Data Execution Prevention (DEP).