Understand NX Bit and Return-Oriented Programming

NX Bit and Return-Oriented Programming

The NX (No-eXecute) bit marks memory regions like the stack and heap as non-executable. This prevents attackers from injecting and executing shellcode directly on the stack, which was the classic exploitation technique before NX became standard.

How NX Changes Exploitation

Memory pages have permission bits: read (R), write (W), execute (X). Stack and heap are typically RW- (no execute). Attempting to jump to shellcode on the stack triggers SIGSEGV.

Return-Oriented Programming (ROP) bypasses NX by reusing existing executable code:

• Find short instruction sequences ending in ret (gadgets)
• Chain gadget addresses on the stack
• Each ret pops the next gadget address

Common x86-64 gadgets:

• pop rdi; ret: set first function argument
• pop rsi; ret: set second argument
• syscall; ret: invoke a system call

For This Exercise

You will document how NX prevents shellcode execution and explain how ROP reuses existing code fragments instead. ROP emerged as the dominant technique precisely because NX/DEP closed the shellcode-on-stack path.

From Gadgets to Execution

ROP does not execute new instructions; it reuses existing ones ending in ret. A chain calling mprotect can mark a stack page executable, reviving classic shellcode techniques. More commonly, chains call system or execve directly. Intel CET shadow stacks and ARM Pointer Authentication record return addresses to detect ROP, but the technique remains central to exploit development education.