The .tap file format¶

The .tap format is friTap's on-disk container for flow captures. It stores decrypted flows, TLS keylog material, file-level metadata, and analysis findings in a single binary file that can be replayed in the TUI, read programmatically with TapReader, or produced offline from a pcap with pcap_to_tap().

This page is the canonical reference for the format's magic values, struct layouts, record types, and version semantics. Other pages link here rather than restating the constants.

Overview & when to use¶

A .tap file is a binary envelope of JSON metadata plus raw binary blobs, one record per event. Unlike a pcap/pcapng, a .tap file stores friTap's reconstructed view of traffic: flows with their HTTP request/response parse results, protocol layer stacks, TLS enrichment, and any analysis findings — already decrypted.

Reach for .tap when you want to:

• Replay a capture offline in the TUI (fritap capture.tap) without re-running the target.
• Analyze traffic after the fact (fritap analyze capture.tap, see traffic analysis).
• Convert an existing pcap + keylog into friTap's flow model with --from-pcap.
• Read flows programmatically via the Python API.

If you only need wire-level packets for Wireshark, use friTap's --pcap/--pcapng output instead. .tap is friTap's higher-level, flow-centric format.

File layout¶

A .tap file is written in this order: a fixed 64-byte header, a stream of records (the META record first, then FLOW / KEYLOG / FINDING records as the capture runs), an optional FLOW_INDEX record written at close, and finally a fixed 16-byte footer pointing back at the index.

flowchart TD
    H["HEADER (64 bytes)<br/>magic · version · flags · capture_start<br/>flow_count · ext_header_len · capture_target"]
    M["REC_META (0x03)<br/>schema versions · fritap_version"]
    R["REC_FLOW / REC_KEYLOG / REC_FINDING<br/>streamed during capture"]
    IDX["REC_FLOW_INDEX (0x10)<br/>flow_id → file offset<br/>(written at close)"]
    F["FOOTER (16 bytes)<br/>FOOTER_MAGIC · index_offset"]

    H --> M --> R --> IDX --> F
    F -. index_offset points back at .-> IDX

Every record (including META, the FLOW_INDEX, and findings) is wrapped in the same 16-byte record envelope with a sync marker and a CRC32 over its payload.

Header¶

The header is a fixed 64-byte little-endian struct, optionally followed by a variable-length JSON extension block (ext_header_len bytes).

Struct format: <4sHHdII8s32s (_HEADER_STRUCT, asserted to be 64 bytes).

When ext_header_len > 0, the reader parses the following JSON object into TapHeader.ext_data. The total header size returned by decode_header() is 64 + ext_header_len.

Header flags¶

Record envelope & CRC32¶

Every record is prefixed by a fixed 16-byte little-endian envelope.

Struct format: <4sBBHII (_RECORD_ENVELOPE, asserted to be 16 bytes).

The CRC is computed over the payload only:

crc = zlib.crc32(payload) & 0xFFFFFFFF

On read, verify_payload_crc() recomputes the CRC and compares. A mismatch causes that record to be skipped (logged), but never aborts the whole read — other records remain usable.

Record types¶

The 1-byte type field selects the payload shape:

FLOW record body & de-duplication¶

A FLOW record payload is laid out as:

[json_meta_len (4 bytes LE)] [json_meta bytes] [blob bytes]

The JSON metadata holds flow scalars, the chunk list, request/response parse results, optional enrichment, and (schema v3) the protocol layer stack. Binary data — chunk bytes, request/response bodies, owned inner-layer bytes — lives in the trailing blob section and is never inlined. Each kind of payload is referenced from the JSON by its own offset/length field names, not a single generic blob_offset/blob_len pair:

Flow index & footer¶

When a capture closes cleanly, friTap writes a REC_FLOW_INDEX record mapping each flow_id to the byte offset of its FLOW record envelope, then a fixed 16-byte footer.

Footer struct format: <8sQ (_FOOTER_STRUCT, asserted to be 16 bytes).

The reader reads the last 16 bytes, validates FOOTER_MAGIC, seeks to index_offset, and loads the index — an O(1) open even for huge captures. If the footer magic is wrong, the index record is missing, or FLAG_HAS_INDEX is unset (a partial/crashed capture), the reader falls back to a full linear scan.

Versioning¶

There are two independent version numbers, both additive:

• FORMAT_VERSION governs the binary framing. If a file's header version is greater than the reader's FORMAT_VERSION, decode_header() raises ValueError ("Format version N is newer than supported … Please update friTap"). Equal or lower versions are accepted.

• FLOW_SCHEMA_VERSION is stamped into each FLOW record's JSON as _v and resolved per-flow by resolve_flow_schema_version(). Records with no _v are treated as schema version 1. The schema is additive across versions: v2 added optional enrichment keys (tls, tags, notes, local/remote addresses, process_name, hook_*) and the body_from_chunks de-dup flag; v3 added the ordered protocol layers stack. Readers use .get(...) with defaults, so v1, v2, and v3 FLOW records all decode correctly — older fields remain the source of truth and newer layers mirror them.

Reading with TapReader¶

TapReader (in friTap.flow.tap_reader, re-exported as friTap.TapReader) provides two-tier loading: fast metadata-only summaries for a flow list, plus on-demand full-flow loads with chunks and bodies.

from friTap import TapReader

reader = TapReader("capture_20260507_153933.tap")
meta = reader.open()                       # -> TapMeta
summaries = reader.read_flow_summaries()   # fast, no chunks/bodies
print(len(summaries))                      # number of flows

# Load one full flow on demand (chunks, bodies, findings)
if summaries:
    flow = reader.read_flow(summaries[0].flow_id)
    print(flow.dst_addr, flow.dst_port)

reader.close()

Run against the committed sample capture, this prints 276 — the number of flows in capture_20260507_153933.tap:

$ ./env/bin/python -c "from friTap import TapReader; \
    r=TapReader('capture_20260507_153933.tap'); m=r.open(); \
    s=r.read_flow_summaries(); print(len(s)); r.close()"
276

TapReader is also usable as a context manager:

with TapReader("capture_20260507_153933.tap") as reader:
    for summary in reader.read_flow_summaries():
        print(summary.flow_id, summary.protocol, summary.host)

Corruption recovery¶

The sync marker (SYNC_MARKER = b"\xF7\xA9\x00\x00") at the start of every record envelope lets the reader recover from a corrupt or partially written file. If the envelope at the current position fails to validate (bad sync marker), the reader scans forward in 4 KB chunks for the next SYNC_MARKER (find_sync_marker()) and resumes from there, skipping the damaged region.

This recovery, combined with the per-record CRC32, means a .tap file that was truncated mid-write (e.g. a crashed capture with no footer) is still partially readable:

• A missing/invalid footer or absent FLAG_HAS_INDEX triggers a linear scan that rebuilds the flow index by walking every record.
• Records with a CRC mismatch or undecodable payload are skipped (logged at debug level), not fatal.
• A truncated payload at EOF simply ends the scan.

Next steps¶

• Offline pcap → .tap conversion — produce a .tap from an existing pcap + keylog.
• Traffic analysis — run analyzers over a .tap and understand REC_FINDING records.
• Python API — the full TapReader / ReplayController / flow model surface.