Procedures

The Windows x64 calling convention, stack frames, arguments, results, and preserved registers.

A procedure is a contract between caller and callee. Windows x64 fixes where the first arguments arrive, which registers survive a call, how the stack is aligned, and where a return value goes.

Reserve stack space before calls

The caller reserves 32 bytes of shadow space for every call. The extra eight bytes in this prologue keep rsp 16-byte aligned at call sites. MASM's frame directives describe the allocation for Windows unwind support.

Reserve stack space before calls source/main.asm

main proc frame
    sub rsp, 40
    .allocstack 40
    .endprolog

Keep in mind: In this entry point, sub rsp, 40 means 32 bytes of shadow space plus 8 bytes for alignment.

Arguments go in registers first

The first four integer or pointer arguments use rcx, rdx, r8, and r9. A fifth argument goes on the stack after the shadow space. Return values normally come back in rax.

Arguments go in registers first source/main.asm

    mov rcx, rax    ; [0] hFile
    lea rdx, outputmessage    ; [1] lpBuffer
    mov r8, outputmessagelength    ; [2] nNumberOfBytesToWrite
    xor r9, r9    ; [3] lpNumberOfBytesWritten
    mov [rsp + 32], r9    ; [4] lpOverlapped
    call WriteFile    ; print

Keep in mind: Write the argument setup in call order; it makes the ABI visible during review.

Preserve the nonvolatile set

If a procedure uses rbx, rbp, rsi, rdi, or r12 through r15, it must restore the original values. This larger chapter saves each preserved register it needs.

Preserve the nonvolatile set source/chapters/addr.asm

addressing proc frame
    push rsi
    .pushreg rsi
    push rdi
    .pushreg rdi
    push r12
    .pushreg r12
    push r13
    .pushreg r13
    push r14
    .pushreg r14
    push r15
    .pushreg r15
    .endprolog

Keep in mind: Pushes in the prologue are popped in reverse order before ret.

A small leaf procedure

A leaf procedure calls nothing else, so it may need no stack allocation at all. This chapter routine simply places zero in the return register and returns.

A small leaf procedure source/chapters/procs.asm

procs proc frame
    .endprolog
    ; --- procedure ---
    ; procedure example
    ; -----------------------------------------------------------------------------
    xor rax, rax    ; return zero
    ret    ; return
procs endp    ; end proc

Keep in mind: Start with the smallest valid frame; add stack work only when the procedure needs it.

Keep these rules

rcx, rdx, r8, and r9 carry the first four integer arguments.
rax carries an integer or pointer return value.
The caller reserves shadow space; the callee restores preserved registers.