Skip to content

Instantly share code, notes, and snippets.

@dumpmycode
Last active April 9, 2023 19:51
Show Gist options
  • Select an option

  • Save dumpmycode/f541e9c43c7ab25e360a01bdde1f8548 to your computer and use it in GitHub Desktop.

Select an option

Save dumpmycode/f541e9c43c7ab25e360a01bdde1f8548 to your computer and use it in GitHub Desktop.
==============================================================================================
FLAGS - Visual Studio
| Overflow | OV | 1 = Overflow | 0 = No Overflow
| Direction | UP | 1 = Down | 0 = Up
| Interrupt | EI | 1 = Enabled | 0 = Disabled
| Sign | PL | 1 = Negative | 0 = Positive
| Zero | ZR | 1 = Zero | 0 = Not Zero
| Auxiliary | AC |
| Parity | PE | 1 = Even | 0 = Odd
| Carry | CY | 1 = Carry | 0 = No Carry
==============================================================================================
Stack Frame.
The stack frame is created by the following sequential steps:
1. Passed arguments, if there are any, are pushed on the stack starting from right side to low address.
2. The subroutine is called, causing the subroutine return address to be pushed on the stack.
3. As the subroutine begins to execute, EBP is pushed on the stack.
4. EBP is set equal to ESP. From this point on, EBP acts as a base reference for all of the sub-routine parameters.
5. If there are local variables, ESP is decremented to reserve space for the variables on the stack.
from here, EBP+4 = subroutine return address, EBP+8 is the first 32bit argument passed on the INVOKE directive.
6. If any registers need to be saved, they are pushed on the stack
==============================================================================================
x64 / Win API calling convention.
Microsoft follows a consistent scheme for passing parameters and calling subroutines in 64-bit programs,
known as the Microsoft x64 calling convention. This convention is used by C and C++ compilers, as well as by the
Windows API library. The only time you need to use this calling convention is when you either call a Windows function,
or you call a function written in C or C++. Here are the characteristics and requirements of this calling convention:
1. The CALL instruction subtracts 8 from the RSP (stack pointer) register, since addresses are 64 bits long.
2. The first four parameters passed to a subroutine are placed in the RCX, RDX, R8, and R9,registers, in
that order. So if only one parameter is passed, it will be placed in RCX. If there is a second parameter,
it will be placed in RDX, and so on. Additional parameters are pushed on the stack, in left-to-right order.
3. Parameters less than 64 bits long are not zero extended, so the upper bits have indeterminatevalues.
4. If the return value is an integer whose size is less than or equal to 64 bits, it must bereturned in
the RAX register.
5. It is the caller’s responsibility to allocate at least 32 bytes of shadow space on the runtimestack,
so called subroutines can optionally save the register parameters in this area.
6. When calling a subroutine, the stack pointer (RSP) must be aligned on a 16-byte boundary.
The CALL instruction pushes an 8-byte return address on the stack, so the calling program must
subtrac 8 from the stack pointer, in addition to the 32 it subtracts for the registerparameters.
7. It is the calling program’s responsibility to remove all parameters and shadow space fromthe runtime
stack after the called subroutine has finished.
8. A return value larger than 64 bits is placed on the runtime stack, and RCX points to itslocation.
9. The RAX, RCX, RDX, R8, R9, R10, and R11 registers are often altered by subroutines, so if the calling
program wants them preserved, it will push them on the stack before the sub-routine call, and pop them off
the stack afterwards.
10. The values of the RBX, RBP, RDI, RSI, R12, R14, R14, and R15 registers must be preserved by subroutines
==============================================================================================
DATA RELATED OPERATORS & DIRECTIVES
OFFSET - get offset address from reg/mem + imm
MsgPrompt BYTE 011245887777h
data WORD 054154879999h
HelloPrompt DWORD 0448903971h
mov esi, OFFSET var1 ; esi = 00404000h (address of var1)
mov esi, OFFSET [var1+3] ; esi = 00404003h (address of var1 + 3 bytes forward and we reach var3)
PTR - move bigger data container to smaller one, vice versa.
can also be combined with OFFSET
var1 WORD 1234h
mov al, BYTE PTR var1 ; al = 34h
array1 WORD 5678h, 1234h
mov eax, DWORD PTR array1 ; eax = 12345678h
var2 DWORD 12345678h
mov ax, WORD PTR [var2+2] ; ax = 1234h, remember little endian.
; var2 start address is at 78h, var2+1 = 56h and var2+2 = 34h
; by moving 2 bytes (34h, 12h) to ax, we get 1234h
TYPE - operator that returns the number of bytes of a single element of a variable
can be used in conjunction with other operators, add xor, sub etc.
var1 BYTE 'a'
var2 WORD 0FFDDh
var3 DWORD 0DEADBEEFh
mov eax, TYPE var1 ; eax = 1
mov eax, TYPE var3 ; eax = 3
LENGTHOF - used to get number of elements in array.
arr1 BYTE 'Hello'
arr2 WORD 30 DUP(1),2,2
arr3 WORD 10 DUP(3 DUP(?))
mov eax, LENGTHOF arr2 ; eax = 0x20h
SIZEOF - used to calculate memory space in bytes? by (TYPE x LENGTHOF)
like in C, sizeof(var)
array1 WORD 20 DUP(0)
mov eax, SIZEOF array1 ; eax = 40 or 28h
LABEL - a directive to insert a 'label or name' and give it a SIZE attribute
such as BYTE, WORD etc without allocating any storage. Common use is to
provide alternative name and size attr for the variable declared next in .data segment
val16 LABEL WORD
val32 DWORD 0DEADBEEFh
mov ax, val16 ; ax = 0BEEFh
mov dx, [val16+2] ; dx = 0DEADh
ddVar LABEL DWORD
Str1 BYTE 'ab'
Str2 BYTE 'cd'
mov eax, ddVar ; eax = 'abcd'
; note that this only works in MASM, in other assembler you will need to enter
; mov eax, [ddVar] to get eax = 'abcd'. mov eax, ddVar is equal to mov eax, OFFSET ddVar
; in other assembler. Puts address of variable, not content.
==============================================================================================
Indirect operands. Protected Mode - An indirect operand can be any 32-bit general-purpose
register (EAX, EBX,ECX, EDX, ESI, EDI, EBP, and ESP) surrounded by brackets.
var1 BYTE 0FFh
mov esi, OFFSET var1 ; put address of var1 to esi
mov al, [esi] ; reference esi & put value in al. al = 0FFh
mov bl, 0AAh
mov [esi], bl ; var1 = 0AAh
inc [esi] ; ERROR: operand must have size
inc BYTE PTR [esi] ; var1 = 00h with Aux Carry = 1
*Another example is that we can cycle through an array
REMEMBER the data size & cycle according correct data size 1 byte, 2 word, 4 dword etc
In the example below we use BYTE, so we're cycling every 1 byte address.
arr1 BYTE 'abcd'
ptrArr1 DWORD arr1 ; put address of arr1 in ptrArr1
mov al, [ptrArr1] ; al = 'a'
inc ptrArr1 ; move 1byte, now to address where 'b' is
mov al, [ptrArr1] ; al = 'b'
...
*Indexed operands are better for array processing, any 32bit register can be used
Accepted notation in MASM: contant[reg] or [constant + reg]
mov esi, OFFSET myVar ; loads address of myVar to esi
[esi+2] would mean to add 2 bytes to the address in esi (myVar) or move address forward 2 bytes.
and [esi-2] would mean move backward 2 bytes and then dereference the address.
array1 BYTE 'ABCD'
mov esi, 0
mov al, BYTE PTR array1[esi] ; al = 'A'
mov al, BYTE PTR [array1 + esi] ; al = 'A' as well, diff notation.
*We can also index it with a scale factor.
array2 WORD 100h, 200h, 300h, 400h
mov esi, 3 ; subscript
mov ax, WORD PTR array2[esi * TYPE array2] ; ax = 400h, basically array2[3 * 2]
; the index cycle 3 times (from 0), with WORD(2bytes) size per cycle
==============================================================================================
Procedures:
CALL instruction pushes its return address on the stack and copies the called procedure’s address into EIP.
RET instruction pops the return address from the stack into EIP.
PUSH instruction subtract ESP by the operand size, ESP-2 for 16bit operand, ESP-4 for 32bit operand. Once decremented
then it copies source operand to the stack
POP instruction would copy value pointed by ESP to destination operand and increment ESP by destination operand. ESP+2
if 16bit, ESP+4 if 32bit operand.
PUSHAD pushes all of the 32-bit general-purpose registers on the stack in the following order:
EAX, ECX, EDX, EBX, ESP (value before executing PUSHAD), EBP, ESI, and EDI. POPAD retrieves from stack to register in reverse.
When your procedure returns value to EAX, don't use PUSHAD/POPAD as EAX will be overwritten when you POPAD.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment