Real-time model#

A patch is edited on one thread and rendered on another, and messages reach it from several more. This page explains how that works without the audio thread ever waiting: how edits are published, when the memory they replace is freed, what happens at the start of each block, and which thread a message handler really runs on. It ends with the rules that follow for code on the message path, and for the host.

The pages on building, messages and the host routes already say what each call does. This page is about where it runs and why it is safe.

The threads#

Thread	What it does to a patch
Control thread	Any thread of yours that calls the patcher, `pHandle` or the C API: building and editing, `PassData`, `pHandle::Set*`, loading and saving, polling GUI values. There can be more than one.
Audio thread	Renders the patcher one block of 128 samples at a time. This is the engine’s audio callback, or a worker of the engine’s render pool that renders a channel on the callback’s behalf. Everything this page says about “the audio thread” applies to both.
Timer thread	One engine thread that runs every millisecond `.metro` and `.clocker` (see Time, scheduling and clocks).
Background pool	Frees retired graph memory (below), reads and writes files for `.coll`, `.textfile` and the other file objects (see File input and output), and opens MIDI ports.
MIDI input threads	One per open port. They only copy incoming bytes into lock-free queues. The patch reads them on the audio thread (see MIDI).
Update thread	The thread that calls `System::update()` / `yse_system_update()`. It delivers bus values published from other threads and writes queued log lines (see Talking to the host).

The audio thread never takes a lock the other threads hold, never allocates and never waits for them. Everything below is built around that.

Edits: build a new graph, swap one pointer#

The wiring of a patch lives in a graph snapshot, separate from the objects. A snapshot lists every object, the targets of every outlet, which inlets have a signal cord, the signal objects the render starts from, and the ~dac and ~adc objects. Once a snapshot is published, nothing changes it again.

Every structural edit (CreateObject, DeleteObject, Connect, Disconnect, Clear, a SetParams that rebuilds an object, a ParseJSON) does the same four things on the calling thread:

take the patcher’s lock, which serialises edits from several control threads. The audio thread never takes this lock;
change the objects and cords;
compile a complete new snapshot. This allocates, which is fine here;
publish it by swapping one atomic pointer.

At the start of each block the audio thread reads that pointer once and keeps the snapshot it found for the whole block. Every send, every “has this inlet got a signal?” question and the output sum of that block use the same snapshot. So:

A block is never half-edited. It renders the graph from before an edit or the graph after it. ParseJSON builds the whole file into one snapshot and publishes once, so a load is one step too (see Patch file format).
Objects keep their state. Objects are allocated once. A new snapshot points at the same objects with different cords, so an oscillator that the edit does not touch keeps its phase and a delay line keeps its contents.
Each call is published on its own. A block can render between two calls. Building and editing graphs explains how to order a group of edits so the in-between graph never sounds wrong.

Freeing what an edit replaced

The old snapshot, and an object that DeleteObject removed, may still be in use by a block that is rendering at that moment. So neither is freed at once, and neither is ever freed on the audio thread. The edit tags them with the current block count and hands them to a job on the background pool. The job frees them once the audio thread has started two more blocks. By then no block can still hold a pointer into them. The only thing the audio thread does for this is add one to its block counter.

That count only says something about the thread rendering the block, so a block’s snapshot is only ever read by that thread. A send made on a control thread (a host SetFloatData, a .preset recall, a .metro tick), whether a block is rendering or not, reads the latest published snapshot instead: the cords as the last finished edit left them.

A control thread never reads the cords themselves, because another control thread may be editing them: a DeleteObject or Disconnect rewrites them under the patcher’s lock, and a send cannot take that lock (it may need it again further down). So a send marks itself as in use before it loads the snapshot and drops the mark when its whole fan-out has returned. While any thread holds that mark the job frees nothing, neither old snapshots nor removed objects, so everything the send can reach stays allocated. An object deleted while a send is under way can therefore still receive that send, just as it can from a block that started before the delete. A .preset recall sets the same mark before it looks objects up under the lock, so an object it found cannot be freed before it is done with it. Setting the mark is one atomic add, so it never waits and never involves the audio thread. What was retired is freed by the first pass after the last mark is dropped.

When nothing renders (the engine is paused, or the patcher is not attached to anything), the block count stands still and nothing retired is freed. The memory is kept until rendering resumes and the next edit runs a new pass, or until the patcher is destroyed. A patch that is edited many times while nothing renders therefore grows until then.

Values from the host: queues, resolved at the block#

Two kinds of host call do not change the wiring, and they reach the audio thread through fixed-size lock-free queues instead of a new snapshot:

Call	Queue size	What rides the queue
`pHandle::SetParams` on an object whose arguments are all numbers	64	The parsed numbers. The audio thread writes them into the object.
`PassData` / `PassBang`	256	The value and the receiver name, inline, up to 64 and 256 characters.

Both queues are emptied at the start of every block. A queued item names its target by name or by address, and the audio thread looks it up again in the block’s snapshot. An object that was deleted after the call is simply not in that snapshot, so the item is dropped rather than delivered into freed memory. A full queue drops the new item and logs it, on the control thread. See Building and editing graphs and Talking to the host for the rest of these rules.

pHandle::SetBang, SetIntData, SetFloatData and SetListData do not use a queue. They run the object’s handler at once, on the calling thread (see below).

The start of a block#

Every block the patcher renders does these steps, in this order, on the audio thread:

Read the published snapshot and keep it for the block. Add one to the block counter.
Apply queued SetParams numbers.
Deliver queued PassData / PassBang values to their .r objects.
Give domain-clock bindings that have not found their clock yet another chance to find it (see Time, scheduling and clocks).
Deliver every deferred message that is due: .delay, .pipe, .line and the other block-clock and beat-clock objects.
Deliver finished file reads and writes (see File input and output).
Run the pollers (below).
Clear every signal buffer, then render the signal graph from its start points. Signal objects pass buffers along cords, and each one computes as soon as all of its signal inputs have arrived.
Add every ~dac into the patcher’s output.
Let go of the snapshot.

Steps 2 to 7 run before the render, so everything they cause is heard in the same block.

Pollers#

Most control objects only run when a message reaches them. A few have no inlet the outside world can push into, because their input arrives on another thread. The MIDI input objects are the main case: RtMidi’s thread puts events in their queues, and nothing would ever read them. Such an object asks to be a poller. The snapshot keeps a list of pollers, and step 7 calls each one once per block, whether or not the patch sends it anything. A poller does what any handler on the audio thread does: it drains its queue into its outlets without allocating, locking or waiting.

Which thread a handler runs on#

Messages between objects are synchronous: an outlet calls the next inlet’s handler directly (see Messages and dispatch). So a handler runs on whichever thread started the chain it is part of:

Where the chain starts	Thread
`pHandle::Set` / `yse_phandle_set_`, `.loadbang` after a load, teardown messages when an object is deleted	the control thread that made the call
`PassData` / `PassBang` (step 3)	the audio thread, at the next block
Deferred messages, file completions, pollers (steps 5 to 7)	the audio thread
A signal object sending a control message while it renders	the audio thread
A millisecond `.metro` or `.clocker`	the timer thread
A value arriving at a `.r` over the named bus	the update thread

Two things follow.

A message the host sends can be handled on the audio thread. PassData returns at once, and the .r, everything it is wired to, and any .s further down all run in the next block, on the audio thread. That is why a .s driven by PassData never reaches the send callback (see Talking to the host).

One object can be reached from two threads at once. A host pHandle::SetFloatData into an object can run while the audio thread delivers a queued value to the same object. Objects with state that must stay consistent protect it with a guard that never waits: the second thread’s message is dropped and counted (see Refusals). Named stores that several objects share are covered in Collections, dictionaries and arrays.

The thread tag is not the thread

Internally each delivery carries a tag. T_DSP means “render this now, we are inside the signal traversal”. T_GUI means “set the state, and let the block’s own render use it”. The drains in steps 3 to 7 deliver with T_GUI, because that is the right meaning for them, but they run on the audio thread. So an object must never read T_GUI as “I am on the control thread and may lock or allocate”. The engine asks the patcher which thread it is really on (patcherImplementation::CallingThread) whenever it has to choose between a lock-free route and a control-thread one. For example, a .s reached from a deferred delivery uses the lock-free, same-block route, not the queue that PassData from the host uses.

Changed in version 3.0: A .s or PassData reached from a deferred delivery used to take the patcher’s lock, and could build a log string, on the audio thread, because the T_GUI tag was read as “control thread” (issue #690).

Rules for code on the message path#

Since almost any handler can run on the audio thread, every message handler, poller and signal Calculate follows the audio thread’s rules. It must not:

allocate or free memory, including building a std::string that does not fit a buffer reserved in advance;
take a lock, or wait for another thread in any other way;
do file, socket or console I/O;
write a log line. Formatting one allocates and the log may write a file.

What the engine’s objects do instead:

Fixed capacity, reserved ahead. Lists, text and tables live in storage sized when the object is created or its arguments change, which happens on the control thread. Input that does not fit is refused, not cut short (see Messages and dispatch).
Hand slow work to another thread. File objects post a request that the background pool carries out, and pick up the result at step 6 (see File input and output). A MIDI output object opens its port on a background thread, and queues each message for a MIDI sender thread to send (see MIDI).
Guard, don’t wait. When two threads could reach the same state, the object uses a test-and-set flag. The thread that loses drops its message and counts it.
Log through the queue. .print does not write to the log. It copies the line into a lock-free queue that the update thread empties (see Talking to the host).

Writing a patcher object shows how a new object meets these rules.

Refusals#

Everything above has limits: queue sizes, list lengths, table slots. Whether the engine tells you about a refusal by a log line or by a counter depends on which thread refuses.

Refused on	Reported as	Examples
the control thread only	a log line, straight away	a refused `Connect` or `SetContainer`, a full `PassData` or `SetParams` queue, a receiver name or list too long for the queue, `PassData` to a name no `.r` has, a patcher name that is too long, too many creation arguments for a list object
any thread, the audio thread included	a counter the object keeps, no log line	a list over 256 atoms at an inlet (`.zl`), a full deferred-message table, a full file-request table, a note `.makenote` cannot track, a message that loses a test-and-set guard, a `.midiout` message that finds the MIDI sender’s queue full
`.print`	a count, logged at the next update	lines that do not fit the shared print queue

The counters are the Dropped() accessors on the engine’s object classes and schedulers. They only ever go up, and the test suite reads them. They are not reachable through pHandle or the C API. From the host, a refusal on the audio thread shows up as a message that had no effect.

What the host may do from where#

Call the patcher API only from control threads. That covers patcher, pHandle and the C API. Never from the audio callback, and never from a callback the engine runs on the audio thread. Edits take the patcher’s lock and allocate.
Keep the patcher alive while a sound plays it or an insert wraps it (see Patcher).
The send callback never runs on the audio thread. It runs on the control or timer thread that sent the message. Keep it short and thread safe. Talking to the host and Embedding through the C API have its install and destroy rules.
GUI reads are control-thread reads. Polling a GUI value takes no lock and never makes the audio thread wait, but some reads clear what they report. See GUI controls and presets.
A handle’s calls are safe during an edit. A SetParams that changes an object’s inlets or outlets builds a new object and swaps it in behind the same handle. Every pHandle call made from another thread meanwhile answers for the old object or the new one, never for freed memory: the lock-free calls (Type, GetName, GetID, GetInputs, GetOutputs and the GUI value reads) set the same in-use mark as a send, and GetParams and the GUI property calls take the patcher’s lock.
Queries see a moment, not a transaction. Objects, GetHandleFromList and the cord queries (GetConnections, GetConnectionTarget, GetConnectionTargetInlet) take the patcher’s lock and answer each call on its own: a cord counted by one call can be gone by the next, which then answers the “no such” value. Coordinate object lifetime yourself if one thread edits while another walks the graph (see Building and editing graphs).

Where to go next#

Building and editing graphs: the edit calls, and how to order edits while audio runs.
Messages and dispatch: synchronous dispatch, logical events and feedback limits.
Time, scheduling and clocks: the block clock, the timer thread and domain clocks.
Talking to the host: the queue, the bus and the send callback from the host’s side.
Embedding through the C API: the same model through the C API.

Real-time model

Contents