# Debugging

If you are using FRIDA mode and have hit some problems, then this guide may help
you to diagnose any problems you are encountering. This assumes you have
followed the [osx-lib](test/osx-lib) example to start fuzzing your target.

It should be noted that attempting to debug code using gdb which has been
instrumented in FRIDA is unlikely to be successful since the debugger will be
inserting breakpoints by patching the code in memory. FRIDA works by reading
this code and generating an instrumented copy to execute. In any case, unless
you are very familiar with the implementation of Stalker, the instrumented code
generated by FRIDA is likely to be very difficult to follow. For this reason,
the following debugging strategies are outlined below.

By convention, all files below should be provided with their path (they are
omitted for readability) and all items in `<braces>` are placeholders and should
be replaced accordingly.

## Select your version

Test with both the `dev` and `stable` branches of AFL++. The `dev` branch should
have the very latest version containing any fixes for identified issues. The
`stable` branch is updated less frequently, but equally might avoid a problem if
a regression has been introduced into the `dev` branch.

## Enable diagnostic information

Run your target specifying the `AFL_DEBUG_CHILD=1` environment variable. This
will print a lot more diagnostic information to the screen when the target
starts up. If you have a simple configuration issue, then you will likely see a
warning or error message in the output.

## Check your test harness

If any of the following steps fail, then there is a problem with your test
harness or your target library. Since this is running without FRIDA mode or
`afl-fuzz` that greatly reduces the search area for your defect. This is why it
is *VERY* important to carry out these basic steps first before taking on the
additional complexity of debugging with FRIDA mode or `afl-fuzz`.

- Run your harness outside of the fuzzer, passing it a representative seed as
  its input `./harness <input>`.
- Pass your harness multiple seeds to check that it is stable when running
  multiple tests as it will when running in fork server mode `./harness <input1>
  <intput2>`.
- Build your test harness with `CFLAGS=-fsanitize=address` and
  `LDFLAGS=-fsanitize=address`. Then run it again with multiple inputs to check
  for errors (note that when fuzzing, your harness should not be built with any
  sanitizer options).

## Check the samples

FRIDA mode contains a number of different sample targets in the `test` folder.
Have a look through these and find one which is similar to your real target.
Check whether you have any issues running the sample target and make sure you
compare the command line used to launch the sample with the one you are using to
launch your real target very carefully to check for any differences. If
possible, start with one of these samples and gradually make changes one at a
time re-testing as you go until you have migrated it to run your own target.

## FRIDA mode

### Basic

First, just try running your target with `LD_PRELOAD=afl-frida-trace.so
 ./harness <input>`. An error here means that your defect occurs when running
 with just FRIDA mode and isn't related to `afl-fuzz`.

Now you can try commenting out the implementation of `LLVMFuzzerTestOneInput` so
that the harness doesn't actually run your target library. This may also aid in
narrowing down the problem.

```c
int LLVMFuzzerTestOneInput(const unsigned char* data, size_t size){
    // fpn_crashme(data, size);
    return 0;
}
```

### Persistent mode

If your target is ok running in basic mode, you can try running it in persistent
mode (if that is the configuration you are having issues with) as follows (again
outside of `afl-fuzz`). This time, you will want to run it inside a debugger so
that you can use the debugger to send the `SIGCONT` signals (by continuing)
usually sent by `afl-fuzz` on each iteration.

```bash
gdb \
  --ex 'set environment __AFL_PERSISTENT=1' \
  --ex 'set environment AFL_FRIDA_PERSISTENT_CNT=3' \
  --ex 'set environment LD_PRELOAD=afl-frida-trace.so' \
  --ex 'set environment AFL_FRIDA_PERSISTENT_ADDR=<entry_address>' \
  --args ./harness <input>
```

Note:
- You have to manually set the `__AFL_PERSISTENT` environment variable which is
  usually passed by `afl-fuzz`.
- Setting breakpoints etc. is likely to interfere with FRIDA and cause spurious
  errors.

If this is successful, you can try additionally loading the hook library:

```bash
gdb \
  --ex 'set environment __AFL_PERSISTENT=1' \
  --ex 'set environment AFL_FRIDA_PERSISTENT_CNT=3' \
  --ex 'set environment LD_PRELOAD=afl-frida-trace.so' \
  --ex 'set environment AFL_FRIDA_PERSISTENT_ADDR=<entry_address>' \
  --ex 'set environment AFL_FRIDA_PERSISTENT_HOOK=frida_hook.so'
  --args ./harness <input>
```

Note that the format of the hook used for FRIDA mode is subtly different to that
used when running in QEMU mode as shown below. Thus the DSO used for the hook is
not interchangeable.

```c
void afl_persistent_hook(GumCpuContext *regs, uint8_t *input_buf,
                         uint32_t input_buf_len);

void afl_persistent_hook(struct x86_64_regs *regs, uint64_t guest_base,
                         uint8_t *input_buf, uint32_t input_buf_len);
```

### ASAN

It is also possible to enable ASAN (if that is the configuration you are having
issues with) without having to use `afl-fuzz`. This can be done as follows:

Note:
- The name of the asan DSO may need to be changed depending on your platform.
- The asan DSO must appear first in the `LD_PRELOAD` environment variable.

```bash
LD_PRELOAD=libclang_rt.asan-x86_64.so:afl-frida-trace.so \
ASAN_OPTIONS=detect_leaks=false,halt_on_error=0 \
AFL_USE_FASAN=1 \
  ./harness <input>
```

Note that care should be taken to ensure that if you set `AFL_INST_LIBS=1`, you
use `AFL_FRIDA_INST_RANGES` or `AFL_FRIDA_EXCLUDE_RANGES` to exclude the ASAN
DSO from coverage. Failure to do so will result in ASAN attempting to sanitize
itself and as a result detecting failures when it attempts to update the shadow
maps.

## Printf

If you have an idea of where things are going wrong for you, then don't be
scared to add `printf` statements to either AFL++ or FRIDA mode itself to show
more diagnostic information. Just be sure to set `AFL_DEBUG=1` and
`AFL_DEBUG_CHILD=1` when you are testing it.

## Core dumps

Lastly, if your defect only occurs when using `afl-fuzz` (e.g., when using
`CMPLOG` which cannot be tested outside of `afl-fuzz` due to its need for a
shared memory mapping being created for it to record its data), it is possible
to enable the creation of a core dump for post-mortem analysis.

Firstly, check if your `/proc/sys/kernel/core_pattern` configuration is set to a
filename (AFL++ encourages you to set it to the value `core` in any case since
it doesn't want any handler applications getting in the way).

Next, set `ulimit -c unlimited` to remove any size limitations for core files.

Lastly, when you `afl-fuzz`, set the environment variable `AFL_DEBUG=1` to
enable the creation of the `core` file. The file should be created in the
working directory of the target application. If there is an existing `core` file
already there, then it may not be overwritten.

## Reach out

Get in touch on Zulip and ask for help. The groups are pretty active so
someone may well be able to offer some advice. Better still, if you are able to
create a minimal reproducer for your problem, it will make it easier to diagnose
the issue.