Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

spvirit-scry and spvirit-scry_influx_collector

✅ Verified · no code on this page · docs-verify

The badge reports the whole docs-verify suite, not this chapter alone.

EPICS PVAccess packet capture, decoding, and real-time traffic monitoring. These two binaries capture live PVAccess traffic off a network interface and decode it as it goes past. They track connection state (channels, operations, PV names) and show it in a TUI dashboard, or export it to InfluxDB and Loki.

  • spvirit-scry: CLI/TUI packet capture with real-time traffic statistics.
  • spvirit-scry_influx_collector: packet capture with InfluxDB and Loki export.

They used to ship as the separate spvirit-scry crate and now live in spvirit-tools, decoding with the workspace's own codec. The binary names and flags are unchanged.

Installing them

Both ship in the Python wheel:

pip install spvirit-tools

The Linux wheels bundle libpcap and macOS has it built in, so nothing else is needed there. On Windows the Npcap runtime must be installed; the wheel does not bundle it (Npcap's licence forbids redistribution). If it is missing, the tools print a warning naming what is missing and where to get it, and exit with status 3, rather than failing to start. The other spvirit-tools work without Npcap. --help and --version work either way.

They look for wpcap.dll in Npcap's own folder (%SystemRoot%\System32\Npcap) first, then on the normal DLL search path (Npcap's "WinPcap API-compatible mode", or a legacy WinPcap install). Setting SPVIRIT_NPCAP_DIR to a directory makes that directory the only place checked; it exists for testing.

Building them

From source they need the optional scry feature, which is not on by default, because packet capture links against a system library:

  • Linux: libpcap-dev and pkg-config (or your distribution's equivalents). The InfluxDB collector's HTTP client uses rustls, so no OpenSSL is needed.
  • Windows: the Npcap SDK to build, and the Npcap runtime to capture. The build script finds the SDK automatically (see below), or set NPCAP_SDK to the extracted SDK root.
cargo build -p spvirit-tools --features scry --bins
# or
cargo install spvirit-tools --features scry

The build script searches for the Npcap SDK in this order:

  1. the NPCAP_SDK environment variable (explicit override)
  2. npcap-sdk/ relative to the package root
  3. C:\npcap\, C:\npcap-sdk\, C:\Program Files\Npcap\ (including versioned subdirs)
  4. %LOCALAPPDATA%\npcap-sdk\

Capturing needs privileges: sudo or CAP_NET_RAW on Linux, administrator rights on Windows.

Source builds need a Rust toolchain (edition 2024); the wheel needs none.

Packet capture without sudo (Linux)

After installing, grant the binary the CAP_NET_RAW capability so it can capture without sudo:

sudo setcap cap_net_raw=eip $(which spvirit-scry)
sudo setcap cap_net_raw=eip $(which spvirit-scry_influx_collector)

This only needs to be done once per install (re-run after cargo install or pip install --upgrade replaces the binary).

Usage

spvirit-scry

spvirit-scry --device eth0 --tui

Options:

  • --device / -i: network interface to capture from.
  • --integration-time SECS (default 1): stats integration period in seconds. There is no short form; -t is --tui.
  • --buffer-size / -b: capture buffer size (100–50000).
  • --resolve / -r: resolve IP addresses.
  • --tui / -t: enable TUI.
  • --tui-tick-ms: TUI refresh tick in milliseconds (default 100).
  • --quiet / -q: be quiet.
  • --show-payload / -p: show packet payloads.
  • --debug / -d: debug logging.
  • --version / -v: print version and exit.
  • --exclude-cmds / -x: comma-separated PVA commands to exclude (e.g. PUT,MONITOR).
  • --pva-ttl: PVA channel state TTL in seconds (default 86400).
  • --pva-max-memory-mb: ceiling on PVA tracker state in MiB, 0 to disable (default 1024; otherwise at least 16).
  • --pva-max-channels: max PVA channels to track per connection (default 40000).
  • --tracked-only: only show packets for tracked connections.
  • --pva-max-update-rate: max update timestamps kept per channel for rate calculation (default 10000).
  • --fingerprint: enable passive source fingerprinting (off by default). Results appear on the TUI's Fingerprints page (key 5).
  • --fp-window: fingerprint window in seconds (default 900).
  • --fp-min-samples: minimum samples before a fingerprint is emitted (default 30).
  • --fp-reservoir: per-channel reservoir size per window (default 200).
  • --fp-round-robin-slots: round-robin channel slots; N > 1 extends time-to-detection N-fold (default 1).
  • --fp-max-channels: maximum channels fingerprinted concurrently (default 5000).
  • --fp-d1: enable D1 source-change detection (off by default, see below).
  • --fp-suppress-secs: suppress D1 for this many seconds from startup (maintenance window).

spvirit-scry_influx_collector

spvirit-scry_influx_collector --device eth0 --config-file influxdb_config.toml

Options:

  • --device / -i: network interface to capture from.
  • --integration-time SECS (default 1): stats integration period in seconds. There is no short form; -t is --tui.
  • --buffer-size / -b: capture buffer size (100–50000).
  • --resolve / -r: resolve IP addresses.
  • --tui / -t: show the TUI.
  • --show-payload / -p: show packet payloads.
  • --debug / -d: debug logging.
  • --version / -v: print version and exit.
  • --config-file / -c: path to InfluxDB config file (default influxdb_config.toml).
  • --fingerprint: enable passive source fingerprinting (off by default).
  • --fp-window: fingerprint window in seconds (default 900).
  • --fp-min-samples: minimum samples before a fingerprint is emitted (default 30).
  • --fp-reservoir: per-channel reservoir size per window (default 200).
  • --fp-round-robin-slots: round-robin channel slots; N > 1 extends time-to-detection N-fold (default 1).
  • --fp-max-channels: maximum channels fingerprinted concurrently (default 5000).
  • --fp-d1: enable D1 source-change detection (off by default, see below).
  • --fp-suppress-secs: suppress D1 for this many seconds from startup (maintenance window).

Source fingerprinting

--fingerprint derives a per-channel fingerprint from four observables of the traffic itself, so it does not depend on what any other source claims:

  • Sub-second timestamp phase: where in the second a source stamps its updates, computed with circular statistics so a source phase-locked near the second boundary is not mistaken for an unlocked one. On real ISIS traffic two channels sat 434 ms apart with a within-channel spread under 0.2 ms.
  • Value quantisation lattice: the GCD of successive differences, which recovers the digitiser LSB (0.0175 °C, 0.01 pH on the reference channels).
  • Update cadence: modal interval plus p50/p95, a joint fingerprint of the scan rate and the deadband policy.
  • Arrival minus stamp: available only to a wire observer, and the best single signal for telling bridge-sourced from device-sourced data.

A fingerprint accumulates per (channel, server address) over a fixed window (default 15 minutes). When a window closes, the fingerprint is emitted if the channel has at least 30 samples; otherwise the channel is reported as insufficient (pending n/m in the TUI). Results appear on the TUI's Fingerprints page (key 5; v cycles list, clusters and the discontinuity log), as a badge column in the Connections detail table, and from the collector as InfluxDB measurements plus Loki events.

What is exported

pv_fingerprint: one point per channel per closed window. Tags: host, channel, server, struct_id (unknown when the channel carries no normative-type id), sampling (full, reservoir or round_robin). Fields: phase_mean_ms, phase_spread_us, phase_circular_var, value_lsb, lsb_confidence, cadence_modal_s, cadence_p50_s, cadence_p95_s, arrival_delta_mean_ms, arrival_delta_var_ms, sample_count, observed_count, has_nan_limits, struct_id_changed. value_lsb and arrival_delta_var_ms are omitted rather than zeroed when they could not be computed, so a missing value means "unknown", never zero.

pv_fingerprint_coverage: what the observer actually looked at: channels tracked, admitted, deferred at capacity, reservoir-sampled and rotated out, with monotonic totals and saturation flags.

pv_fingerprint_detect: everything the detector declined to emit: firings suppressed by the maintenance window, repeats, incomparable pairs, channels skipped by the D2 pair cap, populations too small for D4, and events dropped by the per-ingest cap.

pv_fingerprint_event (Loki): one structured event per rule firing, carrying both the prior and current fingerprint so the event is self-contained. Rules: D1 source change, D2 server/fingerprint disagreement, D3 synthesised metadata, D4 clock-offset outlier.

What it does not do

These limits come from passive observation itself, and more data does not remove them:

  • Identity: a fingerprint establishes sameness and difference. It can say two channels share a source, or that a channel's source changed; it cannot name an unknown source. Two unrelated IOCs started at the same moment with the same scan rate will look alike, so the fingerprint is strongest when it says "these differ" and weakest when it says "these match". There is no cryptographic assurance: traffic can be shaped to mimic a fingerprint.
  • Failover: planned failover is indistinguishable from silent re-sourcing. A legitimate gateway failover produces exactly the D1 signature, and nothing on the wire separates the two. This is why D1 is opt-in behind --fp-d1, with --fp-suppress-secs as a startup maintenance window. D2, D3 and D4 run whenever fingerprinting is on.
  • Coverage: it is partial, and is reported rather than assumed. A passive observer only sees the segments it is placed on; on a multi-node swarm with an internal gateway mesh one observation point sees a subset of channels. The pv_fingerprint_coverage measurement reports only the channels this observer saw.
  • Absence: a missing fingerprint is not evidence of anything. A hardware-clocked or event-driven source shows no narrow phase; a computed channel (calc record, unit conversion, aggregate) may show no lattice, or a lattice that reflects the arithmetic rather than the digitiser. The TUI keeps a measured phase, unlocked (measured but not a signature), — (computed, genuinely absent) and …n/m (still filling the window) as four distinct states.
  • D3: one-sided today. PVAccess carries no declared-provenance field, so D3 fires on the observable half of the contradiction: synthesised NaN or all-zero valueAlarm/display limits, with nothing to compare a declaration against.

Cost

Fingerprinting adds per-update work to the capture path, which is why it is off by default. Each window runs in one of three sampling regimes: full (every update used), reservoir (updates sampled within a channel) and round_robin (only a subset of channels admitted per window). Channels that have no fingerprint yet are always admitted. The regime is recorded in the sampling tag of every exported fingerprint.

Configuration

The collector reads a TOML configuration file. Required keys:

url = "http://localhost:8086"
token = "your-token"
org = "your-org"
bucket = "your-bucket"
loki_url = "http://localhost:3100"

Note: The collector reads token only from this file, in plain text. Keep the file out of version control and restrict its permissions.

Troubleshooting

  • No packets captured: verify the device name (--device) and ensure capture permissions (sudo or CAP_NET_RAW).
  • InfluxDB errors: verify url, token, org, and bucket in the config file and confirm network connectivity.
  • Buffer size errors: --buffer-size must be between 100 and 50000.
  • Diagnostics: use --version to confirm build version, --debug for verbose logging.