Skip to content

Instantly share code, notes, and snippets.

@topnotcher
Last active August 29, 2015 14:20
Show Gist options
  • Save topnotcher/26c9174040f22dee2630 to your computer and use it in GitHub Desktop.
Save topnotcher/26c9174040f22dee2630 to your computer and use it in GitHub Desktop.

Background on stdcall

  • eax, ecx, edx are clobbered
  • return through eax
  • Caller allocates stack space
  • arguments pushed right to left (i.e. the first argument has a lower address)
  • callee expected to deallocate stack space

stdcall example

Suppose we have a function:

extern int __attribute__((stdcall)) multiply(int x, int y);

If multiply(5,6) is called with an initial stack of ESP=100, we have:

# allocate space for two 32-bit ints
# post condition: ESP = 92
sub ESP, 8 

mov [esp+4], 6
mov [esp], 5

# stack now contains:
# 5 (4 bytes @92)
# 6 (4 bytes @96)
# --- bottom of stack at 100

call multiply
# pushes 4 byte return address (EIP) and 4 byte CS register so we have:
# EIP,CS (8 bytes@84)
# 5 (4 bytes @92)
# 6 (4 bytes @96)
# --- bottom of stack at 100

# at the end of square, there should be:
# ret 8, which will pop the return address (ESP += 8) and increment ESP by 8 (size of arguments)
# expected ESP = ESP_before_call (92) - 4 (call) + 4 (ret) + 8 (argument to ret)
# = ESP_before_call + 8 = 92 + 8 = 100 = ESP_before_call + number of bytes passed as arguments

ESP vs Arguments Passed

let:

post_esp = the value of esp immediately following the call
post_esp_exp = the value of esp the *caller* expects immediately following the call
pre_esp = the value of ESP immediately preceeding call (this is the same as post_esp_exp, so why did I use both???)
expected_args = The number of bytes the function expects as args
actual_args = The number of bytes the fnction is sent as args

The actual ESP after the call will always be:

post_esp = pre_esp + expected_args

The expected ESP after the call is:

post_esp_exp = pre_esp + actual_args

When (post_esp_exp - post_esp) is nonzero, the wrong number of bytes was passed. This leads to the following cases:

Case (a)

When (post_esp_exp - post_esp) > 0, we have:

((pre_esp + actual_args) - (pre_esp + expected_args)) > 0

And it follows that:

actual_args > expected_args

The number of extra bytes passed is given by:

extra_bytes = post_esp_esp - post_esp

Case (b)

When (post_esp_exp - post_esp) < 0, we have:

((pre_esp + actual_args) - (pre_esp + expected_args)) < 0

And it follows that:

actual_args < expected_args

The number of bytes missing is given by:

missing_bytes = post_esp - post_esp_exp

Example:

If multiply is passed 4 arguments, the stack before the call becomes:

arg1 (4 bytes)
arg2 (4 bytes)
              ---- post_esp
arg3 (4 bytes)
arg4 (4 bytes)
              ---- post_esp_exp

As indicated by case (a), post_esp_exp > post_esp

Simiarly, if multiply is passed only one argument instead of :

arg1 (4 bytes)
              ---- post_esp_exp
(missing 4 bytes)
              ---- post_esp

In this case, post_esp > post_esp_exp and it follows from (b) that actual_args < expected_args.

ffi_call_x86

Signature:

ffi_call_x86(
         void (* prepfunc)(char *, extended_cif *), /* 8 */
         extended_cif *ecif, /* 12 */
         unsigned bytes, /* 16 */
         unsigned flags, /* 20 */
         unsigned *rvalue, /* 24 */
         void (*fn)()); /* 28 */
# preserve ebp
push ebp

# copy the stack pointer to ebp
mov ebp, esp

# preserve esi
push esi

# copy the stack pointer to esi
# note that esp == ebp == esi right now
mov esi, esp

# [ebp+16] = bytes argument
mov ecx, [ebp+16]

# allocate as much stack space as indicated by the bytes argument
# This is the space for the arguments to the function that is going to be called
sub esp, ecx 

# stack poiner to eax.
mov eax, esp

# push ecif arg
push [ebp + 12]
# push current stack pointer
push eax

# call prepfunc
call [ebp+8]

# stack sent to prepfunc is:
# 0: EIP,CS: 8 bytes.
# 8: value stack pointer before call.
# 12: ecif 
# 16: Space allocated for function arguments. This address is stored in the first argument.
# ffi_prep_args puts the arguments onto the stack...


# hmmm... ffi_prep_args probably using CDECL?
# in which case this puts the stack back at the stack allocated to call the foreign function.
add esp, 8

# call the foriegn function
call [ebp+28] 

# in the std call case (I skipped a branch here)
# esi <- esi - esp
# esi = pre_esp
# esp = post_esp
# esi = post_esp_exp - post_esp 
sub esi, esp

# skip some more stuff and jump to sc_epilogue
# eax = post_esp_exp - post_esp ... the return value of ffi_call_x86()
mov eax, esi
pop esi
mov esp, ebp
pop ebp
ret
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment