Review stop 3: the stream core, Source, Live Activity (Phases 1–3)

2026-11-11. Design c138 Phases 1, 2 and 3, run in sequence without stopping, on Mark's night ruling: "I'd like to move through the blockers if possible. We can do our best to examine our team's repos to take a good guess on the data shape. If there are phases that are blockers, let's undertake them in sequence." The ruling is recorded in design 9eec43f656. The calls are collected here. Built in ../qnative-c3 on branch c3/phases and fast-forwarded into main at each phase's end: Phase 1 2c01ebf, Phase 2 6a8999e, Phase 3 9d265fb (it was 8c979bb before the rebase onto c5's work), review stop 60461d8.

In short

• The stream core is built and held to qdash by test.
  • Subscribing to the sensor's stream, an SSE parser, the scenario model, and the envelope and Monitor types, each field marked read from code or proposed.
  • The stream scripts, written to qevents' payloads as read from code.
  • Five of qdash's lib/live modules ported to Rust. Each agrees with qdash's own TypeScript on cross-language vectors that regenerate byte-identical: ingest and previews, presence, the tally, entity-meta and the projections.
• Dev › Source (the plan's Sensor page) offers synthetic data (a world, six conditions, four streams) or a real sensor on this machine or at an address.
  • The ladder has six rungs, and the doctor is app --check-sensor.
  • Run for real, the doctor found this Mac's qcontrol lock stale. It also got the seed qdash serve refused with qdash's own 503 sentence.
• Dev › Live Activity is qdash's /live on the current envelope, plus the seven improvements.
• Guide page 3 is now the Live Activity document: how the app reaches a sensor, the data (read from code, or guessed), the pipeline, the states, and six asks of the sensor.
• Live Activity, as first built, cost 28–34% of a core at 100 and 1,000 events a second. Nearly all of that was redrawing the page once per arriving batch. Leaving the page returns the app to rest: 0.003% of a core, and its threads back to 6. The stream core alone costs 0.6–7%. The page now redraws at most 4 times a second, the monitor TUI's cadence. Readings for that build are still owed: the Mac was in use, then locked.

How to try it

cargo dev -- --data synthetic                                  # the full build, developer mode on
QPOINT_STREAM=file-storm@1000 cargo dev -- --data synthetic    # a stream at 1,000 a second
cargo run -p app -- --check-sensor                             # the doctor, this machine
cargo run -p app -- --check-sensor --sensor-url http://127.0.0.1:7878
scripts/live-vectors.sh                                        # regenerate the cross-language vectors

In the app:

• Dev › Source chooses the data.
• Dev › Live Activity shows the stream:
  • ↑ ↓ (or j and k) move the selection;
  • space pauses;
  • t switches the clock to ages;
  • Esc clears the selection.
• Open JSONL loads a run log. crates/stream/vectors/corpus.jsonl is a good one.

What was built

The record, phase by phase, is in CYCLE.md § Key Changes. In brief:

Parity, by test

The plan asked that "the native page and the web page agree by test". scripts/live-vectors.sh makes a corpus of 614 records from the four stream scripts plus edge lines. It runs qdash's own TypeScript over that corpus: crates/qdash/ui/app/lib/live/*.ts at 5c9175fa6, bundled read-only from the qcontrol checkout. It writes what qdash produced; the Rust tests compare. A second run gives byte-identical files.

• Quirks kept. Web quirks are kept for parity and marked web quirk, kept for parity: in the code; there are about 24. Three of them bear on what a person sees:
  • Presence counts every snapshot as activity. Every harness reads "working" for 10 s after each resync, as the survey inferred. The page shows this.
  • The tally keys on run.id. On today's stream that is one "Unattributed" row, which is why the page adds a tally by process, session and type.
  • exec.exit shows status=kind, not the exit code.
• Two places where Rust can't match JavaScript:
  • JavaScript can keep half of a UTF-16 surrogate pair, and Rust writes U+FFFD there.
  • JavaScript's localeCompare is matched exactly for ASCII only.
  
  No vector reaches either, and every id and type is ASCII.

The readings (Phase 3: Live Activity under a stream)

Method: the README recipe, gated: a reading starts after 60 s without input on an unlocked screen, and is taken again if anything touched the machine. These were taken on the full release build, with QPOINT_SPIKE_OPEN=/dev/live and a coding-session stream at the rate, the window on the Studio Display, 2026-11-11 17:00–17:40. Four of the six counted readings are below. The other two, the rest reading and a second-build reading at 10 a second, are under "What the readings say". The run is now a script: scripts/footprint/live-activity.sh, with scripts/footprint/measure.sh.

The page as first committed (8c979bb, now 9d265fb: one redraw per arriving batch, no row cache):

• * I took that reading while my own release build ran, at load 16. The CPU figure is the app's own, but the minute was not clean.
• The table has no rest reading from this build: six attempts were all touched.
• Recordings: rec_782642, rec_a281d8, rec_f7f4b0, rec_a770fa.

What the readings say:

• Leaving the page ends the feed. It falls to 0.003% of a core and 0.8 wakes a second, and to 6 threads, as at rest; while the page is open there are 8 (the producer and the page's feed). This is the lifecycle rule, checked from outside. Memory stays at 95 MB. The allocator and GPUI's atlases keep what the page used, as c2's page walk showed.
• The cost is the drawing, not the stream. Main-thread time is nearly all of the CPU at every rate. The page redrew once per arriving batch, and the synthetic stream delivers records about 10 ms apart, so that meant up to a redraw per record. A second build gathered arrivals for 50 ms and cached rows. Its reading at 10 a second (rec_0ca212, load 2.8) was 10.4% CPU with the main thread 9.6% busy: about ten redraws a second at about 10 ms each. So the cache did not move the cost of a redraw.
• The stream core itself is cheap. Summing every thread's CPU with ps over 20 s, with the window covered (GPUI draws nothing it cannot show), the record path costs:
  • 0.60% of a core at 10 a second;
  • 1.70% at 100;
  • 7.05% at 1,000.
  
  That covers ingest, presence, the tally and the entity index, for every record. The overview with the same stream configured costs 0: no page subscribes, so nothing runs. These are ps figures, not strobe.
• At rest, the second build reads 73.7 MB, 0.006% and 1.1 wakes a second (rec_ee3c1d).
• Changed because of the readings, and not yet measured: the page now gathers arrivals for 250 ms, so it redraws at most 4 times a second (the TUI's TICK = 250 ms), and rows on screen are cached. Three after-fix rounds were started and stopped. The first round measured the 50 ms build above. The second ran into a Mac in use. The third ran into a locked screen, which draws no windows. The readings at 4 redraws a second are owed (call 12).

The sensor's side: asks, not waits

Built on today's stream as it is. The six asks are in Guide page 3 § 7, each with what we would accept if the answer is "not now":

1. a cursor;
2. a gap marker with a drop count;
3. a filter on the server;
4. a snapshot for the new subscriber only;
5. the id spaces, stated;
6. a key that forbids developer mode.

Two findings to pass to Kamal:

• This Mac's qcontrol lock is stale (pid 54415, since 12 October). A session on his machine needs a running monitor.
• /api/stats serves no version, so the Answering rung can't say which sensor answered (c131 D17's /api/sensor would).

Calls for Mark

1. Pushing. Your instruction to this session was "commit locally, never push", and a peer relayed a new rule. My commits have stayed local. But c4's push of main carried two of them, Phase 1 (25cb477, 2c01ebf), to qpoint-io/qnative. Should this session push its own commits from now on (fast-forward only, as the peer described)?
2. Names.
   • The plan's Dev page "Sensor" is Source, because c5's V1 Sensor now sits under This machine.
   • Live is Live Activity, as you ruled.
   
   Keep both?
3. Retention. The default is 20,000 events and 32 MiB (the web's is 5,000 and 25 MiB), and the person can choose 5k, 20k or 100k. Keep the default?
4. Masking.
   • Masked by default: the documented secrets (installation.details env, headers, settings), and content-bearing fields (HTTP header values, MCP params and results, tool arguments and output, system instructions, argv).
   • Not masked: agent.message.prompt, which is user content after the adapter's redaction. Mask it too?
5. The harnesses row is shown by default, where the web hides it. Keep?
6. Presence's snapshot quirk. Keep parity, with "working" after every resync? Or improve, and count only the entity's own records as activity?
7. The doctor in the lean build. --check-sensor is in both builds, as a support tool on a customer's machine. Keep it there, or move it into developer mode?
8. Developer mode's cost on disk is now 4.54 MB: lean 24.93 MB against full 29.47 MB. At review stop 0 it was 1.07 MB.
   • Source added about 2.9 MB. My guess is the kit's text input pulling in its editor; I haven't measured that.
   • Live Activity added 0.65 MB.
   • The lean build is unchanged, and the lean check passes.
   
   Worth a cargo bloat pass before V1?
9. A saved scenario survives a restart, including a condition such as "sensor not running". The footer names it. Keep that, or reset conditions at each start?
10. tallyView.ts imports Vue, so its pure parts sit in the vectors harness as a marked copy that must be kept in step by hand. Ask qdash to move the pure parts into lib/live?
11. File mode replaces the live stream, as on the web. Keep, or open a run log in a second view?
12. The redraw rate and its readings.
    • The table redraws at most 4 times a second, about a quarter-second behind the stream. The TUI does the same. Keep it?
    • Its readings are owed: run scripts/footprint/live-activity.sh for about 10 minutes with the Mac idle and unlocked.
    • If a redraw at about 10 ms is still too dear, the next step is a cached view for the table (GPUI AnyView::cached), so only rows that changed redraw.
13. Next: Phase 4 (Events) needs your design first: a brief and a mock. Phase 5 is Monitor. Should a recording of a real session replace the invented stream scripts once a sensor runs?
14. The lean build after c6. Ruled 2026-11-11: (b), keep the row and change the check (design c138 ruling 10). It was built by c6 as qpoint-io/qnative#2, reviewed here, and merged by Mark as 2c435e4.
    • c6's info panel (8242467) draws a row in the lean build: "Developer mode", with "This build has no developer mode; it was built without --features dev." Since then the old check had found "Developer mode" in the lean binary.
    • scripts/check-lean.sh now looks for words only devmode carries: the not-allowed sentence, /dev/source, /dev/live, and a sentence each from Live Activity and Source. It allows the row: "Developer mode" at most once, counted per occurrence, and only beside the row's sentence. The dependency-graph check is unchanged.
    • Checked: CI run 43142674204 passed on Linux, Windows and macOS, every new marker present in each full build. On this Mac it fails with the full binary in the lean seat, and on crafted inputs (the name alone, and two names beside the sentence).

For the Rust expert (added to review-0's eight)

• Question 2 is answered with a crate. host holds the data handle (RepoHandle, Generation, reopen) and a Store of saved values; the app re-exports it and developer mode depends on it. The extension protocol gained an optional start hook, so an extension can apply its saved state at launch.
• Question 5 (teardown). Each page that reads a stream owns its subscription inside a GPUI Task. Dropping the page drops the task, the task drops the subscription, and the producer thread sees the flag within 250 ms and closes its socket. A detached task would have lived until the next keep-alive. The "closed" reading above checks this from outside: CPU and thread count after leaving the page.
• The SSE client speaks HTTP/1.1 over the socket itself, with chunked decoding and its own read timeout, so a closed page is noticed and the Host header is under our control. Is that acceptable, or should it be a crate (eventsource-client, reqwest with streaming)?
• Two copies of a JavaScript localeCompare (presence.rs, projections.rs), written by two port agents, should become one helper.

Not done

• The drill (c131 D15: inject one of three bad events).
• Recordings of a real session as stream scripts.
• The stretch items: a histogram to scrub the buffer, tailing a growing file.
• The Monitor's routes and page (Phase 5); Events (Phase 4); Config (Phase 6).
• CI on Linux and Windows:
  • At Phase 2's code, the macOS runner passes (run 43111428095): both variants build, every test passes, and the lean check passes (lean 24,931,728 bytes, full 28,820,880). That makes review stop 0's burden-test line 3 true on CI for macOS.
  • Linux and Windows failed on every push. The cause was the stories crate's manifest, which listed every dependency under the macOS target. c5's cloud session fixed it (3725719) and fixed two Windows failures in c3's code: the ladder's stale-lock check now uses tasklist, and the events fixture keeps to whole hundreds of nanoseconds. CI run 17, at 045eed1, is green on all three systems, the lean check included.
  • My stopgap, which left the stories crate out on those runners, was dropped when origin was merged in (78238ff).
  • c5 pushed main on Mark's word (4ccf511). Phase 3's first CI run (43138450683) passed on macOS and Linux and failed on Windows, in three of the stream crate's projections tests:
    • The cause: the corpus ends with a blank line. A Windows checkout read it as CRLF, so the test counted one line more, and every supplement line was one off.
    • The fix, local as 948a9b5: .gitattributes keeps crates/stream/vectors LF on every checkout, and the test's trim no longer assumes LF. Checked against a CRLF copy of the corpus.
    • It goes out with the next push.