debugging-password-program.md

Debugging a Password Program with x64dbg

This guide outlines the steps to build and debug a simple C password-checking program using GCC in MSYS2 and x64dbg on Windows. The goal is to compile the program with specific flags, load it into x64dbg, set a breakpoint on strcmp, analyze function parameters, identify the correct password, and verify it. Mermaid diagrams are included to visualize the compilation and debugging processes.

Program Code

The C program (password.c) to debug is:

#include <stdio.h>
#include <string.h>

int main() {
    char input[50];
    printf("Enter password: ");
    scanf("%s", input);
    
    if (strcmp(input, "debug123") == 0) {
        printf("Access granted!\n");
    } else {
        printf("Access denied!\n");
    }
    return 0;
}

Step 1: Build the Program with GCC in MSYS2

The following flowchart illustrates the compilation process:

graph TD
    A[Save password.c] --> B[Open MSYS2 MinGW 64-bit Terminal]
    B --> C[Run: gcc -g -fno-stack-protect -no-pie -o password password.c]
    C --> D{Compilation Successful?}
    D -->|Yes| E[Verify: password.exe exists]
    D -->|No| F[Check GCC installation and code]
    E --> G[Proceed to Debugging]
    F --> C

Loading

1. Save the Code:

   • Save the C code in a file named password.c in your MSYS2 working directory (e.g., C:\msys64\home\user\password.c).

2. Open MSYS2 Terminal:

   • Launch the MSYS2 MinGW 64-bit terminal (mingw64.exe) to use the 64-bit toolchain.

3. Compile the Program:

   • Run the following command to compile with debugging symbols, no stack canary, and no PIE:

     gcc -g -fno-stack-protector -no-pie -o password password.c

     • -g: Includes debugging symbols for x64dbg.
     • -fno-stack-protector: Disables stack-smashing protection (stack canary).
     • -no-pie: Disables Position-Independent Executable for fixed addresses.
     • -o password: Names the output password.exe.

4. Verify Compilation:

   • Check for password.exe in the directory:

     ls

   • If compilation fails, ensure gcc is installed (pacman -S mingw-w64-x86_64-gcc) and the code is correct.

Step 2: Load the Program in x64dbg

1. Install x64dbg:

   • Download and extract x64dbg from its official site or GitHub releases to a directory (e.g., C:\x64dbg).

2. Launch x64dbg:

   • Open x64dbg.exe (use the 64-bit version for the 64-bit executable).

3. Load the Executable:

   • Go to File > Open and select password.exe.
   • The program loads, showing the entry point in the CPU window.

4. Verify Symbols:

   • Check the “Symbols” tab to confirm debugging symbols are loaded, making strcmp and variables like input visible.

Step 3: Set a Breakpoint on strcmp

1. Locate strcmp:

   • In the “Symbols” tab, find strcmp (likely in msvcrt.dll).
   • Right-click and select “Set Breakpoint” or press F2.

2. Alternative Method:

   • If strcmp isn’t in Symbols:
     • Navigate to the main function in the CPU window (double-click main in Symbols).
     • Step through the code to find the call to strcmp (e.g., call msvcrt.strcmp).
     • Set a breakpoint (F2) on this instruction.

Step 4: Run and Enter a Wrong Password

The debugging process is visualized in the following flowchart:

graph TD
    A[Load password.exe in x64dbg] --> B[Set breakpoint on strcmp]
    B --> C[Run program F9]
    C --> D[Enter wrong password e.g. wrongpass]
    D --> E[Breakpoint hit at strcmp]
    E --> F[Examine RCX: Input string]
    E --> G[Examine RDX: Hardcoded password debug123]
    F --> H[Compare strings: Mismatch]
    G --> H
    H --> I[Continue F9: Access denied]
    I --> J[Restart Ctrl+F2]
    J --> K[Run and enter debug123]
    K --> L[Breakpoint hit at strcmp]
    L --> M[Examine RCX and RDX: Both debug123]
    M --> N[Continue F9: Access granted]

Loading

1. Run the Program:

   • Press F9 to run until the breakpoint or F8 to step to the password prompt.
   • A console window appears with “Enter password:”.

2. Enter a Wrong Password:

   • Type a wrong password (e.g., wrongpass) and press Enter.
   • The program pauses at the strcmp breakpoint.

Step 5: Examine Function Parameters (RCX, RDX)

1. Inspect Registers:

   • At the strcmp breakpoint, view the Registers pane.
   • RCX: Points to your input string (e.g., wrongpass).
   • RDX: Points to the hardcoded password (debug123).

2. Follow in Dump:

   • Right-click RCX in Registers and select “Follow in Dump” to see your input string.
   • Repeat for RDX to see debug123.

3. Verify Strings:

   • In the Dump window, RCX shows wrongpass\0, and RDX shows debug123\0.
   • strcmp compares these, returning non-zero (mismatch), leading to “Access denied!”.

Step 6: Find the Correct Password

• From RDX in the Dump window, the correct password is debug123, hardcoded in the program.

Step 7: Verify with the Correct Password

1. Restart the Program:

   • Press Ctrl+F2 to restart in x64dbg.
   • Run (F9) to the password prompt.

2. Enter the Correct Password:

   • Type debug123 and press Enter.
   • The program hits the strcmp breakpoint.

3. Re-examine Parameters:

   • Check RCX and RDX:
     • RCX: Points to debug123\0 (your input).
     • RDX: Points to debug123\0 (hardcoded).
   • strcmp returns 0 (match).

4. Continue Execution:

   • Press F9 to continue.
   • The console outputs “Access granted!”.

5. Optional: Step Through Logic:

   • Step over (F8) the strcmp call to see RAX = 0 (indicating equality).
   • Observe the conditional jump to the “Access granted!” code.

Additional Notes

• Debugging Symbols: The -g flag ensures variable names are visible in x64dbg’s Symbols or Memory Map.
• No Stack Canary/PIE: These flags simplify debugging with static addresses and no stack checks.
• Potential Issues:
  • If strcmp isn’t found, verify msvcrt.dll is loaded in the Modules tab.
  • If the program crashes, confirm the executable is 64-bit (file password.exe).
  • scanf("%s", input) doesn’t handle spaces; consider fgets for robustness if modifying the code.

This process demonstrates compiling a program with specific flags, debugging it in x64dbg, analyzing strcmp parameters, and identifying/verifying the password.