Test-Driven Analysis¶

Test-driven analysis is the primary workflow in ADARE. Rather than running an experiment and then manually inspecting what happened, you define expected forensic outcomes before execution. ADARE runs the experiment, evaluates every assertion, and reports which artifacts matched expectations and which did not.

This approach turns forensic experiments into repeatable, auditable test suites that can be re-run across OS versions, software updates, or configuration changes to detect when artifact behavior diverges.

What Is Test-Driven Analysis?¶

In a test-driven workflow you:

Define expectations first – specify which files, registry keys, database entries, or log lines should exist (or not exist) after a set of user actions.
Automate the actions – write a playbook that performs those user actions inside a VM (browsing a website, opening a document, deleting a file, etc.).
Let ADARE evaluate – the framework executes the actions, runs every test assertion, and records pass/fail results with full context.

Each test maps to a testfunction – a small, reusable validation routine from one of the built-in testsets (standard, json, csv, xml, sqlite, linux, windows). See Test Functions for the complete catalog.

Designing a Test Plan¶

Before writing a playbook, decide what you want to verify.

Choosing testfunctions¶

Browse the testsets to find functions that match your artifact type:

standard – file existence, content matching, hash comparison, file size
json – key existence, value assertions in JSON files
csv – column checks, row matching in CSV/TSV files
xml – XPath queries, element assertions
sqlite – SQL queries against SQLite databases (browser history, logs)
linux – syslog entries, package state, service status
windows – registry keys/values, event log entries, Prefetch files

Defining pass/fail criteria¶

For each artifact, decide:

Must exist – e.g., a browser history database must be present after visiting a URL.
Must contain specific data – e.g., the visited URL must appear in the moz_places table.
Must not exist – e.g., a file must be absent after secure deletion.
Must match a value – e.g., a registry key must hold a specific DWORD.

Each criterion becomes one test entry in the playbook.

Writing the Playbook¶

A test-driven playbook has four main sections: settings, variables, tests, and actions.

settings:
  idle: 1.0
  timeout: 600
  screenshot:
    format: png
    quality: 95

variables:
  target_url: "https://example.com/research"
  history_db: "C:\\Users\\adare\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\*.default-release\\places.sqlite"

tests:
  - name: history_db_exists
    function: standard.file_exists
    parameter:
      dst: "{{ history_db }}"

  - name: url_recorded
    function: sqlite.query_result_contains
    parameter:
      dst: "{{ history_db }}"
      query: "SELECT url FROM moz_places WHERE url LIKE '%example.com%'"
      expected: "{{ target_url }}"

  - name: favicon_stored
    function: standard.file_exists
    parameter:
      dst: "C:\\Users\\adare\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\*.default-release\\favicons.sqlite"

actions:
  - action: gui.open_application
    parameter:
      name: firefox

  - action: gui.type_text
    parameter:
      text: "{{ target_url }}"

  - action: keyboard.press
    parameter:
      key: enter

  - action: general.wait
    parameter:
      seconds: 5

  - action: test
    parameter:
      name: history_db_exists

  - action: test
    parameter:
      name: url_recorded

  - action: test
    parameter:
      name: favicon_stored

  - action: gui.close_application
    parameter:
      name: firefox

How tests integrate with actions¶

Tests are declared in the tests section and executed as test actions within the actions sequence. This lets you control exactly when each assertion runs – after the relevant action has completed but before subsequent actions might alter state.

You can also set per-test timeouts for long-running validations:

tests:
  - name: large_db_check
    function: sqlite.query_result_contains
    timeout: 300
    parameter:
      dst: "/path/to/large.db"
      query: "SELECT count(*) FROM events"
      expected: "1000"

Running the Workflow¶

ADARE uses a two-phase workflow: test mode for iterative development and production mode for final, integrity-verified runs.

Test mode (default)¶

adare experiment run my-experiment -e win11

In test mode:

The playbook can be modified between runs.
Runs are marked as test runs in the database.
No integrity checks are enforced.
Use this phase to refine actions, fix timing issues, and tune test parameters until all assertions pass reliably.

Production mode¶

adare experiment run my-experiment -e win11 --production

In production mode:

The playbook is locked; integrity hashes are verified before execution.
Runs are recorded as official production results.
Any playbook modification after loading invalidates the hash and prevents execution.

The typical cycle is: iterate in test mode until satisfied, then execute one or more production runs to generate the official forensic record.

Interpreting Results¶

After a run completes, use the following tools to review outcomes.

Run info¶

adare run info

This displays a summary of every test execution: pass/fail status, execution time, and any error messages for failed assertions.

Screenshots¶

ADARE captures screenshots at key points during execution. They are stored in the run directory and are useful for diagnosing GUI automation failures or confirming that the correct application state was reached before a test ran.

Logs¶

Three log files provide progressively deeper detail:

adare.log – host-side orchestration log (experiment lifecycle, VM management, result collection).
adarevm.log – guest-side agent log (action execution, test evaluation, file operations inside the VM).
mcp_gui.log – GUI automation server log (screenshot analysis, element location, click/type operations).

End-to-End Example: Browser History Analysis¶

This example investigates which artifacts Firefox creates on Windows 11 when a user visits a URL, bookmarks it, and then clears recent history.

Step 1 – Design the test plan¶

We want to verify:

After visiting a URL, the history database records it.
After bookmarking, the bookmark entry exists.
After clearing recent history (last hour), the URL is removed from history but the bookmark persists.

Step 2 – Create the experiment¶

adare experiment create firefox-history-analysis

Step 3 – Write the playbook¶

experiments/firefox-history-analysis/playbook.yml¶

settings:
  idle: 1.5
  timeout: 600
  screenshot:
    format: png

variables:
  profile_glob: "C:\\Users\\adare\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\*.default-release"
  places_db: "{{ profile_glob }}\\places.sqlite"
  target_url: "https://www.iana.org/help/example-domains"

tests:
  - name: url_in_history
    function: sqlite.query_result_contains
    parameter:
      dst: "{{ places_db }}"
      query: "SELECT url FROM moz_places WHERE url LIKE '%iana.org%'"
      expected: "{{ target_url }}"

  - name: bookmark_exists
    function: sqlite.query_result_contains
    parameter:
      dst: "{{ places_db }}"
      query: >
        SELECT p.url FROM moz_bookmarks b
        JOIN moz_places p ON b.fk = p.id
        WHERE p.url LIKE '%iana.org%'
      expected: "{{ target_url }}"

  - name: history_cleared
    function: sqlite.query_result_not_contains
    parameter:
      dst: "{{ places_db }}"
      query: >
        SELECT url FROM moz_historyvisits v
        JOIN moz_places p ON v.place_id = p.id
        WHERE p.url LIKE '%iana.org%'
      expected: "{{ target_url }}"

  - name: bookmark_survives_clear
    function: sqlite.query_result_contains
    parameter:
      dst: "{{ places_db }}"
      query: >
        SELECT p.url FROM moz_bookmarks b
        JOIN moz_places p ON b.fk = p.id
        WHERE p.url LIKE '%iana.org%'
      expected: "{{ target_url }}"

actions:
  # --- Open Firefox and visit URL ---
  - action: gui.open_application
    parameter:
      name: firefox

  - action: gui.type_text
    parameter:
      text: "{{ target_url }}"

  - action: keyboard.press
    parameter:
      key: enter

  - action: general.wait
    parameter:
      seconds: 5

  # --- Verify history was recorded ---
  - action: test
    parameter:
      name: url_in_history

  # --- Bookmark the page ---
  - action: keyboard.hotkey
    parameter:
      keys: ctrl+d

  - action: general.wait
    parameter:
      seconds: 2

  - action: keyboard.press
    parameter:
      key: enter

  - action: general.wait
    parameter:
      seconds: 2

  # --- Verify bookmark exists ---
  - action: test
    parameter:
      name: bookmark_exists

  # --- Clear recent history (last hour) ---
  - action: keyboard.hotkey
    parameter:
      keys: ctrl+shift+delete

  - action: general.wait
    parameter:
      seconds: 2

  - action: gui.click_text
    parameter:
      text: "Last hour"

  - action: gui.click_text
    parameter:
      text: "Clear now"

  - action: general.wait
    parameter:
      seconds: 3

  # --- Verify history cleared but bookmark survived ---
  - action: test
    parameter:
      name: history_cleared

  - action: test
    parameter:
      name: bookmark_survives_clear

  - action: gui.close_application
    parameter:
      name: firefox

Step 4 – Iterate in test mode¶

adare experiment run firefox-history-analysis -e win11

Review results with adare run info. Adjust wait times, fix image references, or refine SQL queries as needed. Repeat until all four tests pass consistently.

Step 5 – Production run¶

adare experiment run firefox-history-analysis -e win11 --production

The production run locks the playbook and generates the official forensic record. To compare behavior across OS versions, run the same experiment against additional environments:

adare experiment run firefox-history-analysis -e win10 --production