Setting up your environment

The main piece of evidence for this puzzle is the core dump file core.dump. This file is a snapshot of the process's entire state frozen in time. Using a debugger you can explore this state.

Table of Contents

The first step to solving this puzzle is actually opening up the core dump.

Downloading the files

This guide will assume that the debug files are all downloaded to the working directory.

E.g.:

URL=https://www.robopenguins.com/fatal_core_dump/evidence
curl -O $URL/src/airlock_ctrl.c -O $URL/airlock_ctrl -O $URL/core.dump

Setting Up the Debugger

In the following sections I'll discuss getting a bunch of ways to get GDB to open the core dump. For each section I include instructions for:

If you're totally new, learning vanilla GDB is probably the most useful since understanding how it works will carry over to the other tools. For more experienced debuggers, pwndbg has some nice quality of life features, and VSCode has the best user interface.

Note: This binary was compiled as an AMD64 executable. While this means that x86_64 systems should be able to use the default gdb binary, systems with other architectures would have issues. To be more general, I'll be using gdb-multiarch in these guides. If you'd prefer the default gdb, just replace gdb-multiarch -> gdb.

Running in Browser

If you want to try debugging the core dump without access to a computer that can run GDB, I set up a virtual machine that runs in the browser:

https://axlan.github.io/webvm/

This is the easiest way to get started, but I do not recommend using this if you can run GDB natively, or can run it in Docker. Since this is a full OS being emulated in the processor it is extremely slow and resource intensive. The image comes preloaded with the binary and source files since setting up networking is a pain.

This is a fork of https://github.com/leaningtech/webvm with minor customizations.

Native Linux

To run natively on Linux, the simplest approach is to use the GDB CLI:

gdb-multiarch --core=core.dump airlock_ctrl

For some projects that build GDB that have some quality of life improvements, you can try:

https://www.gdbgui.com/

pip install gdbgui
gdbgui --gdb-cmd "/usr/bin/gdb-multiarch --core=core.dump airlock_ctrl"
# Note: you need to type a command like "backtrace" into the GDB prompt to get the "threads" to populate.
# You also need to fetch the airlock_ctrl.c source file

https://github.com/pwndbg/pwndbg

curl -qsL 'https://install.pwndbg.re' | sh -s -- -t pwndbg-gdb
pwndbg --core=core.dump airlock_ctrl

Visual Studio Code

Open the evidence directory in VS Code. Create the file:

.vscode/launch.json 
{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "GDB: Local Core Dump Analysis",
      "type": "cppdbg",
      "request": "launch",
      "program": "${workspaceFolder}/airlock_ctrl",
      "args": [],
      "stopAtEntry": false,
      "cwd": "${workspaceFolder}",
      "environment": [],
      "externalConsole": false,
      "MIMode": "gdb",
      "miDebuggerPath": "/usr/bin/gdb-multiarch",
      "coreDumpPath": "${workspaceFolder}/core.dump",
      "sourceFileMap": {
        "/fatal_core_dump/src": "${workspaceFolder}"
      },
      "setupCommands": [
        {
          "description": "Enable pretty-printing for gdb",
          "text": "-enable-pretty-printing",
          "ignoreFailures": true
        },
        {
          "description": "Set disassembly flavor to Intel",
          "text": "-gdb-set disassembly-flavor intel",
          "ignoreFailures": true
        }
      ]
    }
  ]
}

Then press F5 in Visual Studio Code

Running in Docker

With Docker installed, you can run GDB on non-linux systems, or avoid polluting your local environment with these tools. These example use ubuntu 22.04 as the base image, but really anything with a gdb-multiarch package should be fine.

With the basic example:

Dockerfile 
FROM ubuntu:22.04

RUN apt-get update && apt-get install -y --no-install-recommends \
    gdb-multiarch \
    && rm -rf /var/lib/apt/lists/*

WORKDIR /fatal_core_dump

COPY . .

CMD [ "gdb-multiarch","--core=core.dump", "airlock_ctrl" ]

Or https://github.com/pwndbg/pwndbg

Dockerfile 
FROM ubuntu:22.04

RUN apt-get update && apt-get install -y --no-install-recommends \
    curl \
    xz-utils \
    ca-certificates \
    && rm -rf /var/lib/apt/lists/*

RUN curl -qsL 'https://install.pwndbg.re' | sh -s -- -t pwndbg-gdb

WORKDIR /fatal_core_dump

COPY . .

CMD [ "pwndbg","--core=core.dump", "airlock_ctrl" ]

Create the Dockerfile and run:

# Create the Docker image
docker build -t core_dump_debug .
# Run the Docker image with a connected terminal
docker run -it --rm core_dump_debug

https://www.gdbgui.com/

Dockerfile 
FROM ubuntu:22.04

RUN apt-get update && apt-get install -y --no-install-recommends \
    gdb-multiarch \
    python3 \
    python3-pip \
    && rm -rf /var/lib/apt/lists/*

RUN python3 -m pip install --no-cache-dir gdbgui

WORKDIR /fatal_core_dump

COPY . .

CMD [ "gdbgui", "-r", "--gdb-cmd", "/usr/bin/gdb-multiarch --core=core.dump airlock_ctrl"]

Create the Dockerfile and run:

# Create the Docker image
docker build -t core_dump_debug .
docker run --rm -p 5000:5000 core_dump_debug

After running, it should be accessible at http://127.0.0.1:5000

Visual Studio Code

Build the core_dump_debug image with just the basic GDB package.

Open the evidence directory in VS Code. Create the file:

.vscode/launch.json 
{
  "version": "0.2.0",
  "configurations": [
    {
      "name": "Docker: GDB Core Dump Analysis",
      "type": "cppdbg",
      "request": "launch",
      "program": "/fatal_core_dump/airlock_ctrl",
      "args": [],
      "stopAtEntry": false,
      "cwd": "/fatal_core_dump",
      "environment": [],
      "externalConsole": false,
      "MIMode": "gdb",
      "coreDumpPath": "/fatal_core_dump/core.dump",
      "pipeTransport": {
        "pipeProgram": "docker",
        "pipeArgs": [
          "run",
          "--rm",
          "-i",
          "core_dump_debug",
          "/usr/bin/gdb-multiarch",
          "--interpreter=mi"
        ],
        "pipeCwd": "${workspaceFolder}"
      },
      "sourceFileMap": {
        "/fatal_core_dump": "${workspaceFolder}"
      },
      "setupCommands": [
        {
          "description": "Enable pretty-printing for gdb",
          "text": "-enable-pretty-printing",
          "ignoreFailures": true
        },
        {
          "description": "Set disassembly flavor to Intel",
          "text": "-gdb-set disassembly-flavor intel",
          "ignoreFailures": true
        }
      ]
    }
  ]
}

Then press F5 in Visual Studio Code

Copied to clipboard!