**Disclaimer**: there are probably easier ways to solve this, but let's check out the manual way to do it anyways.
## Challenge
Ever wonder how packed ELF works? Let's see this example from a CTF. Here you have a normal binary, statically linked, stripped of headers.
```
└─$ file chal
chal: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, no section header
```
Try to run it:
```
└─$ ./chal
Pls input the flag
flag{fake-flag}
ERROR
```
It is one of the typical reverse challenges that checks a flag. If you try to decompile this or other type of static analysis on Ghidra, you will see a great mess. Nothing makes sense there. But this does seem like a C compiled binary.
## Attempt
You run it with GDB, and break on some signals that you know will trigger, like "write" and "read" syscalls.
```
(gdb) catch syscall write
Catchpoint 1 (syscall 'write' [1])
(gdb) run
Starting program: /home/xxx/xxx/chal
Catchpoint 1 (call to syscall write), 0x0000000000451327 in ?? ()
(gdb) c
Continuing.
Pls input the flag
Catchpoint 1 (returned from syscall write), 0x0000000000451327 in ?? ()
(gdb) c
Continuing.
aaaaaaa
Catchpoint 1 (call to syscall write), 0x0000000000451327 in ?? ()
(gdb) c
Continuing.
ERROR
Catchpoint 1 (returned from syscall write), 0x0000000000451327 in ?? ()
(gdb) c
Continuing.
[Inferior 1 (process 267842) exited normally]
```
In the breakpoint, you notice the code that is in memory does not match up with the code in Ghidra. You wonder why is that? From the back of your head, you remember something called a packed binary.
When a binary is packed by a packer, it is hard to reverse or analyze it. Some packers obfuscate, encrypt, and move around bytecode, and might have dynamic anti-reverse techniques like GDB-detection, tamper-detection, root-detection, simulator-detection, etc. Their job is to make the reverse engineer miserable.
Luckily, there are pre-made tools to unpack binaries. A popular packer is UPX. If the binary is packed by UPX, you can unpack it with UPX. But first, you try to detect the packer by Detect It Easy (DIE).
```
└─$ upx -d ./chal
Ultimate Packer for eXecutables
Copyright (C) 1996 - 2024
UPX 4.2.4 Markus Oberhumer, Laszlo Molnar & John Reiser May 9th 2024
File size Ratio Format Name
-------------------- ------ ----------- -----------
upx: ./chal: NotPackedException: not packed by UPX
Unpacked 0 files.
```
Uh oh, that did not work. But don't worry! You are smart enough to figure this one out by hand!
## Manual reverse
Let's see, what is the problem here? What do you want to achieve?
First, you cannot understand the program, because you can't see the code. This makes it hard to approach this challenge. So it would be awesome if you can unpack this binary manually.
But you don't know how to do that unless you understand how the packer works. First you verify that the executable memory is different from the binary. Notice when you run your code, you go into the second range down below, and it is marked executable. The `vdso` memory region is also executable, but that is for kernel, which is not what you are interested in.
```
(gdb) info proc mappings
process 270385
Mapped address spaces:
Start Addr End Addr Size Offset Perms File
0x0000000000400000 0x0000000000401000 0x1000 0x0 r--p
0x0000000000401000 0x00000000004b4000 0xb3000 0x0 r-xp
0x00000000004b4000 0x00000000004dc000 0x28000 0x0 r--p
0x00000000004dc000 0x00000000004dd000 0x1000 0x0 ---p
0x00000000004dd000 0x00000000004e0000 0x3000 0x0 r--p
0x00000000004e0000 0x00000000004e4000 0x4000 0x0 rw-p
0x00000000004e4000 0x0000000000507000 0x23000 0x0 rw-p [heap]
0x00007ffff7fa2000 0x00007ffff7fa3000 0x1000 0x0 r--p /home/xxx/xxx/chal
0x00007ffff7ff9000 0x00007ffff7ffd000 0x4000 0x0 r--p [vvar]
0x00007ffff7ffd000 0x00007ffff7fff000 0x2000 0x0 r-xp [vdso]
0x00007ffffffde000 0x00007ffffffff000 0x21000 0x0 rw-p [stack]
```
How is it possible though? Because the memory region is not marked writable, then the program should not be able to change it on runtime.
This is a trick that packers use. When starting up, the packer mark the memory writable, fixes the bytecode, and mark it back to not writable. Here is a mental exercise: why would the packer unpack itself at runtime?
Let's see the output of `strace`:
```
└─$ strace ./chal
execve("./chal", ["./chal"], 0x7ffe19228c30 /* 62 vars */) = 0
open("/proc/self/exe", O_RDONLY) = 3
mmap(NULL, 350243, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f46eb879000
mmap(0x7f46eb879000, 349882, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0) = 0x7f46eb879000
mprotect(0x7f46eb8cd000, 6179, PROT_READ|PROT_EXEC) = 0
readlink("/proc/self/exe", "/home/xxx/xxx"..., 4095) = 56
mmap(0x400000, 933888, PROT_NONE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x400000
mmap(0x400000, 1328, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x400000
mprotect(0x400000, 1328, PROT_READ) = 0
mmap(0x401000, 729277, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0x1000) = 0x401000
mprotect(0x401000, 729277, PROT_READ|PROT_EXEC) = 0
mmap(0x4b4000, 162856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0xb4000) = 0x4b4000
mprotect(0x4b4000, 162856, PROT_READ) = 0
mmap(0x4dd000, 21040, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0xdc000) = 0x4dd000
mprotect(0x4dd000, 21040, PROT_READ|PROT_WRITE) = 0
mmap(0x4e3000, 2464, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x4e3000
mmap(NULL, 4096, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f46eb878000
close(3) = 0
munmap(0x7f46eb879000, 350243) = 0
arch_prctl(0x3001 /* ARCH_??? */, 0x7fffc75a7bd0) = -1 EINVAL (Invalid argument)
brk(NULL) = 0x2ade5000
brk(0x2ade61c0) = 0x2ade61c0
arch_prctl(ARCH_SET_FS, 0x2ade5880) = 0
uname({sysname="Linux", nodename="kali", ...}) = 0
readlink("/proc/self/exe", "/home/kali/Desktop/ctf/\350\203\275\346\272\220202"..., 4096) = 56
brk(0x2ae071c0) = 0x2ae071c0
brk(0x2ae08000) = 0x2ae08000
mprotect(0x4dd000, 12288, PROT_READ) = 0
fstat(1, {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0), ...}) = 0
write(1, "Pls input the flag\n", 19Pls input the flag
) = 19
fstat(0, {st_mode=S_IFCHR|0600, st_rdev=makedev(0x88, 0), ...}) = 0
read(0,
"\n", 1024) = 1
read(0, aaaaaaa
"aaaaaaa\n", 1024) = 8
write(1, "ERROR\n", 6ERROR
) = 6
lseek(0, -1, SEEK_CUR) = -1 ESPIPE (Illegal seek)
exit_group(0) = ?
+++ exited with 0 +++
```
Let's feed this into ChatGPT, and you see something like this:
### Quotes from ChatGPT
```bash
open("/proc/self/exe", O_RDONLY) = 3
```
This line shows that the binary is reading **its own file**. This is typical for unpackers or self-modifying code — they load their original contents to memory so they can decrypt or modify code at runtime.
```bash
mmap(NULL, 350243, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)
mmap(0x7f46eb879000, 349882, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0)
```
Here it: (1) Allocates a writable memory region. (2) Maps the contents of its own file descriptor (`3`, from `/proc/self/exe`) into that region. This strongly suggests the binary is **copying its own code into memory** to either: Decrypt it, Patch it, Or execute it later.
```bash
mmap(0x401000, 729277, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0x1000)
mprotect(0x401000, 729277, PROT_READ|PROT_EXEC)
```
This region has **RWX** permissions initially — meaning the program can **write executable code**, i.e., classic self-modifying code. Later, it's marked RX (just to execute it after patching). This memory region is likely where **decrypted/unpacked code** is placed and executed.
```bash
close(3)
munmap(0x7f46eb879000, 350243)
```
Once it’s done using its own binary contents, it cleans up the mapping — further indication it’s done **one-time decryption or transformation** of itself.
This isn't just an exec-stub that replaces itself via another `execve` — it's modifying itself **in memory**, not by replacing the process.
## Unpack
Now you know a few things.
1. It unpacks from the beginning, and only once
2. The unpacked the code is the executable memory between `0x401000` and `0x4b4000`
With that information, you have a plan. Maybe you can dump the memory after it unpacked, then put it in Ghidra for static analysis. When to dump? From `strace`, you know by the time it asks for the flag, the bytecode must be fixed. You can dump from there.
```
dump binary memory result.bin 0x0000000000401000 0x00000000004b4000
```
Now put the dump into Ghidra. Tt probably helped to use the unpacked memory dump, but it is still kind of a mess. It does not know where the functions are. It sees no headers, no symbols, no section information. If you can tell it, it will happily decompile for you.
How do you figure out code regions? A simple way is to have a trace of the program `rip`. The easiest way is to use GDB, and keep hitting `ni`. Then you get a list of actual executed addresses. Pick a few places to disassemble. I picked the addresses after `ret`, because I know it is the address returned to upper call stack.
There are only a few calls, and you can quickly see this suspicious block of code ( I renamed some functions):
```
bool UndefinedFunction_004021a2(void)
{
int iVar1;
long unaff_RBP;
long in_FS_OFFSET;
FUN_00410f40();
flagChanger(unaff_RBP + -0x30,0x20,unaff_RBP + -0x5a,*(unaff_RBP + -0x68));
iVar1 = strcmp(unaff_RBP + -0x30,unaff_RBP + -0x50,0x20);
if (iVar1 != 0) {
print(0x4b4022);
}
else {
print(0x4b401c);
}
if (*(unaff_RBP + -8) != *(in_FS_OFFSET + 0x28)) {
/* WARNING: Subroutine does not return */
stackCrash();
}
return iVar1 == 0;
}
```
Well, the Ghidra decompiled raw code does not look like this at the beginning. As you are somewhat experienced in CTF reversing, you guessed some of the functions.
Something changes the input first, and you can verify that your input is at `$rbp - 0x30`. Then your changed input is compared with another `char *` to check the flag.
Notice your GDB breakpoints might not work. If you set the breakpoints at the beginning, and break on the `flagChanger`, it might not work. I think maybe it is due to the reallocation of memory messes with GDB's breakpoints. However, if you disable the breakpoint, and enable it again after unpacking finishes, then it will work.
## Solve the CTF
The unpacking finished. Now you know what the program does. After analyzing the `flagChanger` function, you realized that every `char` you input is subtracted with a constant number. So a easy way to figure out the flag is to input 32 a's. Then use a breakpoint and see how the input has changed. Also read the "correct" `char *` by a breakpoint as well. Now you can do some simple math and figure out what the flag should be, to subtract each character with a constant number, and produce the desired character.
```
void flagChanger(char *input,ulong length,char *param_3,undefined8 const9)
{
long in_FS_OFFSET;
int a;
int b;
ulong i;
byte buff [264];
long canary;
byte temp;
canary = *(in_FS_OFFSET + 0x28);
prepareBuff(buff,param_3,const9);
a = 0;
b = 0;
for (i = 0; i < length; i = i + 1) {
a = (a + 1) % 0x100;
b = (buff[a] + b) % 0x100;
temp = buff[a];
buff[a] = buff[b];
buff[b] = temp;
input[i] = input[i] - buff[buff[b] + buff[a]];
}
if (canary != *(in_FS_OFFSET + 0x28)) {
/* WARNING: Subroutine does not return */
stackCrash();
}
return;
```
Sign in with Wallet
Connect another wallet