Bismillah.
From this article we begin entering x86-64 Assembly to gain low-level understanding. The explanation will be simple and clear.
Prerequisite: a study of computer architecture and organization, plus basic knowledge of memory and registers (covered in the pointers and stack-vs-heap articles), is recommended before continuing this series.
Example environment: the gcc outputs here are from Linux (the SysV convention, where the first argument goes in RDI). On Windows x64 the argument registers differ (RCX/RDX/R8/R9) as explained in the virtual memory article.
Introduction to Assembly
Assembly is a low-level programming language (abbreviated asm) — called low-level because it is the closest to the CPU of all languages (C/C++, Java…), and it is the first human-readable layer above the code the processor executes.
C/C++ → Assembly → Machine Code (Hex) → Binary → CPU
Every instruction in asm is a mnemonic, translated into an opcode (a hex representation of the machine instruction), which then becomes binary the CPU executes.
Note: the semicolon
;in asm marks a comment.
The Essential Mnemonics (~90% of asm code)
1. Data movement:
mov destination, source ; move data from source to destination (like x = 5)
2. Arithmetic:
add dest, src ; dest = dest + src
sub dest, src ; dest = dest - src
inc operand ; operand++
dec operand ; operand--
3. Comparison and control:
cmp op1, op2 ; compare op1 and op2
jmp label ; unconditional jump to label
4. Stack management:
push value ; place a value on the stack
pop value ; retrieve a value from the stack
call func ; call a function (pushes the return address)
ret ; return from a function to the return address
Passing Arguments in C
Arguments reach functions in two main ways:
- Pass by Value
- Pass by Reference (via pointers)
1. Pass by Value
When an argument is passed by value, the function creates a copy of the original variable — any change inside the function does not affect the original.
#include <stdio.h>
void modify(int x) {
x = 20; // modifies only the local copy
printf("inside function: %d\n", x); // 20
}
int main() {
int num = 10;
modify(num); // pass by value
printf("outside function: %d\n", num); // 10 (unchanged)
return 0;
}
Memory Analysis
numinmain()sits at hypothetical address0x1000holding 10.- When
modify(num)is called a new copyxis created (say at0x2000) holding 10. - The assignment
x = 20changes only address0x2000. - The original
numstays at0x1000unchanged.
Turning C into ASM
Using gcc:
gcc -S -masm=intel -fno-asynchronous-unwind-tables -fno-pie -O0 file.c -o file.s
-masm=intel: Intel syntax (the easier one to read).-fno-asynchronous-unwind-tables: removes.cfidirective noise.-fno-pie: simplifies addressing.-O0: no optimizations — the clearest code.
The generated code:
.LC0:
.string "inside function: %d\n"
.text
.globl modify
.type modify, @function
modify:
push rbp
mov rbp, rsp
sub rsp, 16
mov DWORD PTR [rbp-4], edi ; the copy x = first argument (edi)
mov DWORD PTR [rbp-4], 20 ; x = 20 (the copy only)
mov eax, DWORD PTR [rbp-4]
mov esi, eax
mov edi, OFFSET FLAT:.LC0
mov eax, 0
call printf
nop
leave
ret
.size modify, .-modify
.section .rodata
.LC1:
.string "outside function: %d\n"
.text
.globl main
.type main, @function
main:
push rbp
mov rbp, rsp
sub rsp, 16
mov DWORD PTR [rbp-4], 10 ; num = 10
mov eax, DWORD PTR [rbp-4] ; eax = num
mov edi, eax ; first argument = 10 (the value!)
call modify
mov eax, DWORD PTR [rbp-4] ; num is still 10
mov esi, eax
mov edi, OFFSET FLAT:.LC1
mov eax, 0
call printf
mov eax, 0
leave
ret
Explaining the Assembly (Pass by Value)
In main:
main:
sub rsp, 16 ; reserve 16 bytes on the stack
mov DWORD PTR -4[rbp], 10 ; [rbp-4] = num = 10
mov eax, DWORD PTR -4[rbp] ; eax = num's value (10)
mov edi, eax ; copy 10 into edi
call modify ; call modify
In modify:
modify:
mov DWORD PTR -4[rbp], edi ; [rbp-4] = edi = 10 (a fresh copy, distinct from num)
mov DWORD PTR -4[rbp], 20 ; change the copy to 20 — num stays untouched
Back in main:
mov eax, DWORD PTR -4[rbp] ; eax = num (still 10)
The golden rule: with pass by value you see
movmoving the value into a register (mov edi, eax) — the value itself is copied.
What Is DWORD PTR?
A size directive — it specifies the size of the data an instruction operates on, because the processor does not automatically know how large the data it reads/writes from memory is.
| Term | Size |
|---|---|
| BYTE | 1 byte = 8 bits |
| WORD | 2 bytes = 16 bits |
| DWORD | 4 bytes = 32 bits |
| QWORD | 8 bytes = 64 bits |
PTR = Pointer — indicating we are dealing with a memory address.
2. Pass by Reference
C has no reference variables like C++ — so we use pointers. We pass the variable’s address, and modifications hit the original directly.
#include <stdio.h>
void modify(int *x) {
*x = 20; // modifies the original variable
printf("inside function:%d\n", *x); // 20
}
int main() {
int num = 10;
modify(&num); // pass the address
printf("outside function:%d\n", num); // changed to 20
return 0;
}
Memory Analysis
numholds 10 at address0x1000.- When
modify(&num)is called, address0x1000is passed into the pointerx. - The assignment
*x = 20changes the value at0x1000directly. - The original
numbecomes 20.
The Assembly (Pass by Reference)
.LC0:
.string "inside function:%d\n"
.text
.globl modify
.type modify, @function
modify:
push rbp
mov rbp, rsp
sub rsp, 16
mov QWORD PTR [rbp-8], rdi ; save the address (an 8-byte pointer)
mov rax, QWORD PTR [rbp-8] ; rax = the original address
mov DWORD PTR [rax], 20 ; *x = 20 (directly at the original address)
mov rax, QWORD PTR [rbp-8]
mov eax, DWORD PTR [rax]
mov esi, eax
mov edi, OFFSET FLAT:.LC0
mov eax, 0
call printf
nop
leave
ret
.size modify, .-modify
.section .rodata
.LC1:
.string "outside function:%d\n"
.text
.globl main
.type main, @function
main:
push rbp
mov rbp, rsp
sub rsp, 16
mov rax, QWORD PTR fs:40 ; stack canary setup begins
mov QWORD PTR [rbp-8], rax
xor eax, eax
mov DWORD PTR [rbp-12], 10 ; num = 10
lea rax, [rbp-12] ; rax = num's address (&num)
mov rdi, rax ; rdi = the address (passed as the argument)
call modify
mov eax, DWORD PTR [rbp-12] ; eax = num (now 20)
mov esi, eax
mov edi, OFFSET FLAT:.LC1
mov eax, 0
call printf
mov eax, 0
mov rdx, QWORD PTR [rbp-8] ; canary verification
sub rdx, QWORD PTR fs:40
je .L4
call __stack_chk_fail ; if the canary changed ← failure
.L4:
leave
ret
Explaining the Assembly (Pass by Reference)
In main:
mov DWORD PTR -12[rbp], 10 ; [rbp-12] = num = 10
lea rax, -12[rbp] ; rax = num's address (&num)
mov rdi, rax ; rdi = the address (passed as argument)
call modify
In modify:
mov QWORD PTR -8[rbp], rdi ; saves the original address in a local
mov rax, QWORD PTR -8[rbp] ; rax = the original address
mov DWORD PTR [rax], 20 ; *rax = 20 — directly at the original address
Back in main:
mov eax, DWORD PTR -12[rbp] ; eax = num (20) — the new value
The golden rule: with pass by reference you see
leacomputing an address, thenmovinto a 64-bit register (mov rdi, rax) — the address is passed, not the value.
The New Instruction: lea
Load Effective Address — computes the effective address of a memory operand and stores it in the destination without touching memory.
lea rax, [rbp-12] ; rax = &num — address computation only, no memory read
The fundamental difference:
mov eax, [rbp-4]reads the value from memory, whilelea rax, [rbp-12]computes the address only. That is why seeingleain a disassembly tells you the code was passing by reference.
A Special Case: Passing Arrays
Arrays are passed automatically by reference — the array name decays into a pointer to its first element.
#include <stdio.h>
void changeArray(int arr[]) {
arr[0] = 100; // modifies the original array directly
}
int main() {
int a[3] = {1, 2, 3};
changeArray(a); // pass the address without &
printf("%d\n", a[0]); // 100
return 0;
}
An Important Note: The Pointer Itself Is a Copy
When you pass a pointer, a copy of the address is passed — you can modify what it points to (*ptr = value), but if you change the pointer itself (ptr = &new) the original pointer outside the function is unaffected. To change the original pointer we use a pointer to pointer (int **ptr):
#include <stdio.h>
#include <stdlib.h>
void changePointer(int **ptrToPtr) {
int *new_ptr = malloc(sizeof(int));
*new_ptr = 100;
*ptrToPtr = new_ptr; // make the original pointer point at the new block
}
int main() {
int x = 5;
int *org_ptr = &x;
printf("pointer before change: %d\n", *org_ptr); // 5
changePointer(&org_ptr); // pass the pointer's address
printf("pointer after change: %d\n", *org_ptr); // 100
free(org_ptr);
return 0;
}
Important Questions (Answered)
1. How do we recognize pass by reference?
By the presence of lea — it computes addresses without touching memory. The pattern lea + mov rdi, rax = pass by reference.
2. Why did we use rax with by-reference and eax with by-value?
eax is the lower 32-bit half of rax. Addresses/pointers use full 64-bit registers because an address is 8 bytes — using 32-bit would truncate it (4 bytes only).
Extra detail: on x64, writing to a 32-bit register (like
mov eax, ...) automatically zeroes the upper half of the 64-bit register — important behavior when analyzing code.
3. Why [rbp-12] in by-reference but [rbp-4] in by-value?
Because the by-reference example includes a stack canary — a random value (8 bytes) placed between locals and the return address to protect it from buffer overflows:
- GCC adds the canary automatically when the address of a local variable is passed to another function (exactly what happens with by-reference).
- At function end the canary is verified; if it changed (a buffer was overrun) it calls
__stack_chk_failand stops the program.
Stack layout:
by value: by reference:
+-----------------+ +-----------------+
| num [rbp-4] | | num [rbp-12] |
+-----------------+ | canary [rbp-8] |
| return address | +-----------------+
+-----------------+ | return address |
+-----------------+
Why Bring Assembly into a C Series?
To reach the advanced levels of reverse engineering you must understand code line by line at the low level. Learning hands-on while watching C turn into assembly — and understanding how asm moves with C — is the most effective way to start.
The RE Connection
- The
lea= by-reference rule: when analyzing any binary, tracingleapatterns reveals arguments passed by reference — helping you interpret parameters in system functions and APIs. - The first argument in RDI/EDI (SysV x86-64): an iron rule — the first argument is always in RDI/EDI, making any disassembly far faster to read. On Windows x64 it is RCX instead — see the x64 convention in the virtual memory article.
- Stack canary: seeing
fs:40patterns in a disassembly means the code was compiled with buffer overflow protection — which determines in advance the kind of challenge an exploiter faces.
To read code at the low level, write small C programs and review their
gcc -Soutput — it is the fastest way to build intuition. And peace be upon you and the mercy of Allah and His blessings.