spvirit-py: Python Bindings
PyO3 bindings exposing the client, server, typed PV handles (tier 2), the
IOC engine (tier 3), NT payloads, dynamic sources, and the low-level
channel/codec to Python. Distributed on PyPI as spvirit; the complete
user guide is spvirit-py/README.md (the primary user doc). This chapter
covers the internals.
Packaging
Cargo.toml: packagespvirit-py,version.workspace = true, so it shares the single workspace version (0.4.2 at time of writing) with every Rust crate.edition = "2024".[lib] name = "spvirit",crate-type = ["cdylib", "rlib"]. Therlibis there only socargo test -p spvirit-py --no-default-features --features test-embedcan link a standalone test binary; the maturin build only ever produces/uses thecdylib. pyo3 0.24 (abi3-py39: one wheel for Python 3.9+),pyo3-async-runtimeshard-pinned to=0.24.0, tokio 1.47.1, plusserde_jsonandtracing. Depends onspvirit-ioc(tier 3).- Features:
default = ["extension-module"](pyo3 links no libpython; the hosting interpreter supplies the C API).test-embed = ["pyo3/auto-initialize"]is test-only, for the#[cfg(test)]unit tests that need a real embedded interpreter inside thecargo testbinary (e.g. the asyncio-bridge race test insource.rsand the ordered-notify tests in that file). pyproject.toml: maturin backend, distribution namespvirit, versiondynamic(from Cargo.toml, so it tracks the workspace version). Pins the runtime dependencyspvirit-tools == 0.4.2.- There is no Python source package. The whole API is the compiled
module plus three Rust-registered submodules
spvirit.codec,spvirit.lowlevelandspvirit.ioc(each injected intosys.modulesat import sofrom spvirit.ioc import aoworks). - No
.pyitype stubs. This is a known gap, since much of the API is about typed values. .venv/is not committed (venv tools write a self-ignoring.gitignoreinto it, but the repo.gitignoredoes not list it; the build/test commands assume it exists locally).dist/wheels/sdists are gitignored build output.
Dev loop:
cd C:\spvirit\spvirit-py
.\.venv\Scripts\maturin.exe develop # debug build, importable in the venv
.\.venv\Scripts\python.exe tests\test_pv_handles.py
Module map (spvirit-py/src/, ~7,600 LOC)
| File | LOC | Purpose |
|---|---|---|
lib.rs | 117 | #[pymodule]: the authoritative index of the public surface; registers classes, 16 factory functions, submodules; initializes the runtime bridge; pins threading._main_thread at import |
runtime.rs | 64 | Shared Tokio runtime (LazyLock); block_on_py (releases GIL, re-entrancy aware via block_in_place); future_into_py (asyncio bridge) |
errors.rs | 139 | Exception hierarchy under SpviritError; TimeoutError/IoError dual-inherit builtins |
convert.rs | 573 | Value conversion layer, including the typed (strict) coercion layer (see below) |
pv.rs | 995 | PyPv + PvKind, all factories, on_put/scan/calc bridging; the central tier-2 file |
server.rs | 1,144 | PyServerBuilder, PyServer, PyStore, PyEventDecorator (lifecycle/event hooks) |
client.rs | 583 | PyClientBuilder, PyClient, PyGetResult, PyDiscoveredServer, PySubscription, discovery |
nt.rs | 740 | NT wrapper classes, NtPayload↔Python bridging |
source.rs | 1,225 | Python-defined dynamic Sources: PyNotifier, PySourceAdapter, sync/async bridging, field-tier plumbing, and embedded ordered-notify/asyncio unit tests |
ioc.rs | 366 | Tier 3: spvirit.ioc.* record constructors, PyRecordSpec, and spvirit.Ioc (the IOC engine/store) |
channel.rs | 636 | spvirit.lowlevel.Channel: persistent single-PV TCP connection |
codec.rs | 500 | spvirit.codec: FieldDesc/StructureDesc wrappers, packet decode helpers |
discovery.rs | 300 | spvirit.lowlevel search/discover/pvlist functions |
packet.rs | 172 | spvirit.lowlevel.Packet: owned PVA frame |
monitor_update.rs | 88 | spvirit.lowlevel.MonitorUpdate: value + changed/overrun dotted paths handed to monitor callbacks; dispatch_monitor_update |
Threading model (read before touching)
Four design comments in the source explain the threading model. Read them first:
runtime.rs:29–42: one shared multi-thread Tokio runtime.block_on_pyreleases the GIL and detects being already on a runtime worker (usesblock_in_place), so a Python callback running inside the runtime can callpv.set()/client ops without deadlock. Covered bytest_on_put_can_set_other_pvsandtest_client_usable_inside_callbacks.pv.rs:435–443: scan closures run synchronously inside the async scan task and therefore cannot block-on other PV reads;None/raised exception falls back to the closure's cached last value.source.rs:324–331:async defsource methods are submitted to a dedicated long-lived asyncio event loop on its own Python thread (run_coroutine_threadsafe), then.result()blocks with the GIL released. This avoids the nested-run_until_completedeadlock.source.rs:250–284:Notifier.notifydelivers monitor updates synchronously and in order. The in-runtime branch used tohandle.spawn(...)the delivery fire-and-forget, which letnotify(v2)overtakenotify(v1)(computing v2's changed-bit delta against the wrong snapshot) and silently dropped delivery panics. Delivery now blocks (viablock_in_placewhen already on a runtime worker, plainblock_onotherwise), so each call returns only once its update has been applied. Covered by the ordered-notify tests at the foot ofsource.rs.
The "sync-only for phase 1" headers on server.rs:1/client.rs:1 are
historical: PyPv and Channel have async variants now; top-level
Server/Client ops remain blocking. Reconcile the comments when convenient.
The API surface, mapped to Rust
PvKind(pv.rs:29):F64/Bool/I32/Str/Arraywrapping the server crate'sPv<f64>/Pv<bool>/Pv<i32>/Pv<String>/PvArray, plusTyped(Pv<ScalarValue>, TypeCode), a dynamically typed scalar covering all twelve NTScalar wire types, backingspvirit.scalar()and anylong/unsigned handle minted byserver.pv().- Factories:
ai/ao/bi/bo/string_in/string_out/longin/longoutgenerated by thepv_ctor!macro (pv.rs:597), each with keyword-onlyunits/prec/desc/adel/mdel/drive_limits/alarm_limits;mbbi/mbbohand-written;waveform/aai/aaobuildPvKind::Arrayand accept a keyword-onlytype=for the element type;scalar(name, initial, *, type, writable=False, **opts)covers all twelve NTScalar wire types viaPvKind::Typed;calcbridgescallback(list[float]) -> float;pv()infers record type from the initial value (bool is checked before int, becauseisinstance(True, int)is True) or, withtype=, picks the wire type explicitly. - Callbacks:
on_put(pv, value)runs before apply; returningFalseor raising rejects the PUT on the wire; handle-drivenpv.set()bypasses it.scanworks as call or decorator; must be attached before serving (afterwards it is a silent no-op; the core only logs a tracing warning).calcexceptions/non-float returns post0.0(asymmetric with scan's cache fallback, documented in the README). Store: name-keyed runtime access (get_value/get_nt/set_value/ set_array_value/put_nt/pv_names), all throughblock_on_py.set_value/set_array_value/put_ntcoerce strictly to the record's existing wire type and never retype a scalar or scalar-array record (put_ntonNtTable/NtNdArrayreplaces the payload wholesale instead).MonitorUpdate(spvirit.lowlevel.MonitorUpdate, frozen): built byPyMonitorUpdate::from_updatefrom aspvirit_client::MonitorUpdate. Itschanged/overrungetters returnVec<String>copies of the resolved paths (the underlying decoded-value bitsets), which Python sees as plainlist[str].- NT classes:
NtScalar/NtScalarArrayconstructible with an optionaltype=picking the wire value type explicitly;NtTable(columns, *, labels=None, types=None, descriptor=None)andNtNdArray(value, dims, *, type=None)are constructible as well as returned byStore.get_nt.EnumandGenericpayloads cross the boundary as plain dicts (nt.rs:709, 716). - Dynamic sources: any duck-typed object with
claim/get/put/names(+optionalrpc/subscribe/on_start), sync or async.PvInfo.nt_scalar("double", writable)declares precise wire types via type strings, parsed byparse_type_codeinconvert.rs:223(source.rs:44 imports it) and shared with the rest of the typed API.
Server
The constructor is fully keyword-only:
Server(*, pvs=None, ioc=None, db_file=None, db_string=None, sources=None, port=None, udp_port=None, listen_ip=None, advertise_ip=None, compute_alarms=None, beacon_period=None) (server.rs:695). ioc= takes a
spvirit.Ioc (tier 3). db_file=/db_string= load .db text as ordinary
tier-2 records. They are deliberately not routed through the IOC engine, so
an existing user's records are not silently upgraded to a self-processing IOC.
run() blocks. start() (= start_background(), which also returns the
Store) first runs every on_start hook synchronously, so a raising hook
surfaces as RuntimeError. It then starts the event dispatcher and spawns a
std::thread running RUNTIME.block_on(server.serve_after_start_hooks()).
server.pv(name) sniffs the stored NT payload to mint a typed handle. It
attaches to any served record, including long/unsigned/float-distinct
ones, which get a dynamically typed PvKind::Typed handle (server.rs:840).
It raises KeyError for NTTable/NTNDArray/generic-structure records, which
have no handle representation (use Store for those).
Client
get/put/monitor/info/pvlist/subscribe. PySubscription runs on a spawned
task, has is_active/error/close, context-manager support, and a Drop that
aborts the task.
Monitor callbacks receive a MonitorUpdate, not the raw value: monitor,
subscribe and Channel.monitor all route the callback through
monitor_update::dispatch_monitor_update (client.rs:370, 449). The callback
gets .value (the decoded value), plus .changed/.overrun (dotted field
paths) and .has_overrun. Returning False unsubscribes; any other return,
including None, continues.
Tier 3 (ioc.rs)
The IOC layer. spvirit.ioc.ai/ao/bi/bo/longin/longout build a RecordSpec
(PyRecordSpec) using verbatim EPICS field names (EGU, PREC, DESC,
DRVL/DRVH, HIHI, …) passed as **fields, not tier 2's
units=/drive_limits= spellings. Passing a tier-2 spelling raises
ValueError (ioc.rs:79).
A spec is pending until handed to spvirit.Ioc(records=[...]) (or
db_file=/db_string=, exactly one). Before binding, its handle methods
raise RuntimeError ("Unbound"). Once bound, rec.get()/rec.set(v)/
rec.set_async(v) and rec["EGU"] (field read) work. rec["FIELD"] = … and
rec.on_put always raise: field writes are not implemented, and tier-3
records permanently exclude on_put (it would run Python inside process()
under a lock set). A bound spec belongs to exactly one engine; reusing it
raises RuntimeError.
Ioc exposes record_names() and run(*, port=None, udp_port=None) (a
Server(ioc=self).run() shortcut). add_record always raises, because
records are fixed at build.
The conversion layer
convert.rs has two parallel paths:
- Inference path (used when no
type=is given): full-fidelity Rust → Python (every width preserved;U8arrays becomebytes), but lossy Python → Rust.py_to_scalar(convert.rs:167):bool→Bool,int→I64,float→F64,str→Str;py_to_scalar_array(convert.rs:189):bytes→U8, empty list →F64, otherwise sniffs the first element. - Typed (strict) path (convert.rs:380 onward:
py_to_scalar_typed,py_to_scalar_array_typed,coerce_scalar_value,coerce_scalar_array_value): used whenever atype=/types=string is given, or when coercing against an existing record's wire type. Every Python value is checked strictly against the requestedTypeCode: out-of-range raisesOverflowError, wrong-kind raisesTypeError, an unrecognized type string raisesValueError. This is what makes all twelve NTScalar wire types (byte/short/int32/ubyte/ushort/uint32/uint64/ float32/...) reachable from Python. Seespvirit.scalar(), thetype=/types=kwargs across NT classes, factories, andServerBuilder, and chapter 08 for how this landed.
Tests and examples
-
Rust
#[cfg(test)]unit tests: the asyncio-bridge race test and the ordered/synchronous-notify tests at the foot ofsource.rs. They need a real embedded interpreter, so they run only under the test-only feature:cargo test -p spvirit-py --no-default-features --features test-embed. The rest of the handle-plumbing tests live inspvirit-server(cargo test -p spvirit-server pv::). -
Python tests (
tests/, ~1,500 LOC total). Five of the six files are plain-assert scripts with a__main__loop that calls everytest_*function, run directly with the venv's python aftermaturin develop. The exception istest_ioc_host_api.py, which usespytest.raisesand must be run withpython -m pytest tests/test_ioc_host_api.py:test_pv_handles.py(422 LOC) — typed handles.test_value_types.py(341 LOC) — the typed-conversion layer:NtScalar/NtScalarArray/NtTable/NtNdArraytype=/types=,spvirit.scalar(),server.pv()on long/unsigned records, strict coercion errors.test_ioc_host_api.py(200 LOC) — tier-3spvirit.Ioc/spvirit.ioc.*host API.test_lifecycle_events.py(283 LOC) —on_start/on_eventserver hooks.test_monitor_overrun.py(91 LOC) —MonitorUpdate.changed/overrun.test_source_fields.py(160 LOC) — dynamic-source field tiers.
Each server test uses a unique port pair from a reserved range (e.g.
test_pv_handles.py: 15075–15246;test_value_types.py: 16060–16083, within its reserved 16060–16099 range). -
Examples: ~38 scripts under
spvirit-py/examples/. Start withdemo_pv_handles.py(typed handles),demo_server.py, thedemo_source_*.pyfamily (dynamic sources),demo_channel*.py(low-level),demo_complete_ioc.py/demo_db_file.py(tier-3 IOC), anddemo_10k_farm.py/demo_stress.py(scale).