Emfuzzer fuzzing experiments orchestrator for embedded systems

When executing fuzzing experiment on embedded environment one often faces challenge of performing multiple tasks in repeatable and observable manner, for example: reset board, ensure embedded software booted, send fuzzing data using selected link, monitor peripheral state to detect changes in behaviour etc.

This is what Emfuzzer helps to orchestrate: it runs various tools and scripts in specific manner, then gathers their results for further inspections.

Note: although focused on fuzzing and embedded systems, Emfuzzer can help with any software-related experiments, that require repeatable order of tasks and results capture.

Installation

Emfuzzer is available on PyPI, it is recommended to install it in isolated environment, either by using Python venv or tools like pipx.

python -m venv .venv
source .venv/bin/activate
pip install emfuzzer

pipx install emfuzzer

Usage

To run experiments simply run:

emfuzzer --config=experiment.json test1.bin test2.bin

For each specified data file steps from experiment.json will be executed and gathered results stored in file named emfuzzer-CURRENTDATE.json. Application will output logs to the console and also store them in .log file next to the .json results file. The prefix for output files can be modified using --output-prefix command line switch.

See default-config.json in source directory for example of experiment definition (this file can be safely used - the "experiment" calls cat on each passed file).

To obtain complete command line switches documentation call emfuzzer --help.

Experiment

Each data file passed to the emfuzzer represents a single Test Case. For each test case following experiment steps will be performed:

1. Setup tasks will be executed and their results stored.
2. Monitoring tasks will be started.
3. Injector will use Test Case data to perform the main experiment task (e.g. fuzzing data injection).
4. Injector will observe system behaviour to capture result of the injection.
5. Monitoring tasks will finish, their results stored.
6. Check tasks will be executed and their results stored.
7. Go to 1 for next Test Case.

Configuration

Experiment configuration is stored in JSON format.

See chapters below for list of all types of injectors, tasks and monitors (with arguments).

Below is the default-config.json with comments:

{
  "delays": {
    "between_cases": 0.2, // delay between Test Cases
    "before_inject": 1    // delay after all setups
  },
  "injector": {
    "type": "subprocess", // type of injection
    "args": {             // arguments for given injector
      "cmd": [
        "cat"
      ],
      "shell": false,
      "timeout": 1
    }
  },
  "case": {                    // for each case
    "setups": [                // list of setups
      {
        "type": "subprocess",  // type of setup tasks
        "name": "setup",       // name used in results
        "args": {              // arguments for given setup
          "cmd": [
            "echo",
            "SETUP"
          ],
          "finish": {
            "timeout": 0.5,
            "signal": "NONE"
          },
          "shell": false
        }
      },
      {                        // second setup
        "type": "ping_alive",
        "name": "ping",
        "args": {
          "host": "127.0.0.1",
          "timeout": 10,
          "interval": 1
        }
      }
    ],
    "monitoring": [],          // monitoring tasks
    "checks": [                // list of checks tasks
      {                        // same as setups
        "type": "ping_stable",
        "name": "ping",
        "args": {
          "host": "127.0.0.1",
          "count": 2,
          "interval": 1
        }
      },
      {
        "type": "subprocess",
        "name": "teardown",
        "args": {
          "cmd": [
            "echo",
            "TEARDOWN"
          ],
          "finish": {
            "timeout": 0.5,
            "signal": "NONE"
          },
          "shell": false
        }
      }
    ]
  }
}

Injectors

Injectors are the tools that take the experiment data, use it to "inject" it into the system, and then observe the effects. Currently supported injectors' types:

• subprocess - execute script and captures its exit code. Arguments:
  • cmd - (list of strings) command to be executed, data will be passed as the last argument to the call
  • shell - (boolean) true when shell should be used todo interpret the command
  • timeout - (float) time to wait for command to finish
• coap - CoAP (Constrained Application Protocol) injector data will be sent over UDP to specified address, success when positive response is received from the system. Arguments:
  • target - dictionary containing host and port of the target
  • response_timeout - (float) timeout to wait for CoAP response after sending the data
  • observation_timeout - (float) additional time for detecting any unexpected messages after the response

SubTasks & Monitors

Sub Tasks are tasks that for each case can be executed as setups or checks.

Note: failure of "setup" does not interrupt the test case execution - it is logged and stored in results, next steps are still executed, to be analyzed later.

Monitors are tasks that their execution is started after setups, then they are active during the injection and finish before checks.

Available tasks:

• subprocess - execute script and capture its exit code. Arguments:
  • cmd - (list of strings) command to be executed
  • shell - (boolean) true when shell should be used todo interpret the command
  • finish - configuration of finishing the task:
    • signal - (string) signal name to be sent to the task (can be NONE)
    • timeout - (float) time to wait for command to finish (starts after signal is sent)
• ping_stable - pings a target number of times, expects all pings to reply. Arguments:
  • host - (string) host to be checked
  • count - (integer) number of pings to sent
  • interval - (integer) interval between pings
• ping_alive - pings a target and expects first response Arguments:
  • host - (string) host to be checked
  • timeout - (float) timeout to wait for response
  • interval - (integer) interval between pings
• remote - executes command over SSH and captures its exit code. Arguments:
  • connection - dictionary containing:
    • host
    • port
    • username
    • password
  • command - (string) command to be executed
  • start_key - (string) string expected in the output of the executed command for the command to be considered "started successfully"
  • start_timeout - (float) timeout for the start of the command
  • finish - configuration of finishing the task:
    • signal - (string) signal name to be sent to the task (can be NONE)
    • timeout - (float) time to wait for command to finish (starts after signal is sent)
• coap_monitor - listens for CoAP responses Arguments:
  • target - dictionary containing host and port of the target
  • response_timeout - (float) timeout to wait for CoAP response after sending any data (useful only when used as Injector)
  • observation_timeout - (float) additional time for detecting any unexpected messages when monitoring finishes