This repository has been archived on 2026-09-17. You can view files and clone it, but you cannot make any changes to its state, such as pushing and creating new issues, pull requests or comments.
nixos-config/docs/research/intent-board-state-workflow-ui-2026-09-11.md

29 KiB
Raw Permalink Blame History

Intent Board: code, workflow, and live UI audit

Date: 2026-09-11. Investigator: the primary Codex conversation, without delegated research. Scope: assessment only. No production cards, configuration, jobs, deployments, or implementation files were changed.

Assessment

The board is a functioning orchestration application with significant implemented capability, but its end-to-end guarantees and user-facing state model lag behind that capability. It can dispatch work, split goals into dependency-aware children, review results, collect telemetry, and arrange deployments. It cannot yet consistently tell the operator what has actually succeeded, why work stopped, or which recovery action will work.

The most consequential findings are not cosmetic: review rejection can be parsed as approval; the deployment runner does not select the reviewed revision; and manual bookkeeping can become supposedly trusted deployment evidence. These behaviors are reproducible in isolated diagnostics. They do not establish that a wrong production deployment has occurred.

My recommendation is to stabilize outcome semantics, revision provenance, and recovery before adding further agents or workflow states. Preserve the existing intent/worker separation, isolated builder worktrees, deterministic operator validation, and report links.

Evidence and method

  • Read current board frontend, API, worker branches, completion watcher, orchestration, fanout, telemetry, tests, Nix wiring, and relevant builder/deployment code.
  • Used the existing Graphify graph for navigation: BFS query board intent phase review worker, 101 matching/traversed nodes, output capped at 37 nodes. Outcome: useful navigation, not current-source proof. Graph dated September 7; no graph metadata updated.
  • Inspected the live application in Chromium through Playwright at http://127.0.0.1:8060/, on blunix, around 18:45–18:49 CEST. Desktop 1440×1000 and mobile 390×844; light and dark appearances. Opened read-only intent and review dialogs, tested keyboard focus across polling, and simulated a failed API request inside that browser only. Browser routing blocked all methods other than GET/HEAD. No mutation request was attempted.
  • The HTTPS front door https://intent-board.0ff.space/ returned 401 without authentication. Browser inspection used the local backend; authenticated reverse-proxy/browser behavior was not tested.
  • Served HTML and installed trigger, worker, completion watcher, fanout, orchestration, and deployment runner matched the examined repository files by SHA-256.
  • Existing tests: 279 board/telemetry tests and 38 subtests passed; seven selected builder deployment/deferred-deployment tests passed. No system build or activation.
  • Additional temporary diagnostics reproduced action/transition mismatches, review parsing, unpinned deployment command construction, interleaved state overwrite, incomplete downstream queue inspection, and a research-transition exception. Subprocess calls in deployment/review probes were mocked; no model was invoked.
  • Historical audit docs/research/intent-board-architektur-ux-evaluation.md was context, not evidence of current behavior.

Temporary evidence: /tmp/intent-board-browser-audit.json, /tmp/intent-board-audit-live.json, /tmp/intent-board-family-telemetry.json, /tmp/intent-board-audit-probes.py, and screenshots /tmp/intent-board-{desktop,dark,mobile,intent}.png. Temporary files are not durable project artifacts.

Actual architecture

Intent and research-intent Markdown files provide titles and source documents. Runtime JSON overlays phases, results, relationships, profiles, and actions. Manual ideas and planning drafts originate in runtime state. The API merges these sources; it does not reconstruct a complete event history from Git.

The frontend projects many backend phases into seven lifecycle groups. It renders server-provided actions but independently derives lifecycle grouping, family progress, and status labels. These separate derivations are a major source of inconsistency.

The trigger creates files in the board inbox. The board worker executes research/review/operator/planning/inspection work or forwards implementation work into the separate agent-codex builder queues. Two builder lanes run through separate instances and isolated Git worktrees. Completion records return to the board watcher. Positive review creates a persistent deployment request; a separate runner performs the switch and reports back. Telemetry is stored separately in SQLite.

This separation has useful boundaries: model output is validated before operator decisions are applied, ordinary builder jobs request deferred deployment, and the deployment runner fixes its allowed host and repository. However, a request containing a commit is not equivalent to a deployment built from that commit.

Sources: board-agent/trigger.py:401, :1230; worker.py:935, :1089, :2001; completion_watcher.py:203; orchestration.py:585; agent-codex/deploy_runner.py:63. Paths below are relative to hosts/blunix/localServices/ unless otherwise stated.

Live snapshot and representative cases

The API snapshot had 50 cards: 28 archived, six planned, five deployed, two in implementation, two failed inspectors, one inspector question, two negative reviews, two completed research cards, one waiting parent, and one blocked parent. These are backend phase counts, not visible top-level card counts.

  1. full-system-audit-remediation: parent blocked, one of five children successful. mariadb-bind-local has a deployment completion; codeberg-mirror-disable and nix-gc-daily have negative reviews. The two remaining children are no_changes. Despite this, the parent displays “Kinder laufen” and an active wave, derived from children retaining implementation phases. The waiting/running presentation contradicts terminal outcomes.
  2. lldap-fail2ban-protection: the builder report says it could not read the required logs or use web search and therefore did not implement a guessed filter. This is a missing-capability/evidence outcome. The card is in implementation with no_changes, offering only Archive.
  3. gotosocial-orphaned-data-cleanup: the report challenges the task's source premise and contains a non-fast-forward push rejection. This should distinguish a task clarification from a publication failure. It is also no_changes with only Archive.
  4. board-session-usage-indicator: waiting parent, zero of three children successful. One child needs input and two are planned/dependency-blocked. The parent appears in Running with green “Status: abgeschlossen,” describing the completed operator job rather than the unfinished goal.
  5. board-intent-operator-orchestration: the inspector stored repo_status=partial and evidence, but the card exposes “inspected” and an old Builder report rather than a diagnosis report. Some stored evidence claims fields are absent that are present in current code. Its observation revision/freshness must be visible; this does not prove what was present when that older inspection ran.
  6. researcher-live-eval-tooling: phase deployed, negative review_decision, and no deployment status. It still offers Repair. This is an observed contradictory record, not proof that the current runner deployed a rejected revision.
  7. board-nested-child-intents and board-reviewer-visual-verification: failed inspection records remain visible with parse-related error details. Recent parser changes mean these historical messages are not evidence that the same output still fails today. Recovery still needs clear revalidation and a usable result view.

State-by-state workflow and information needs

Backend state/path What happens Current presentation / gap Information and next action needed
idea Manual runtime-only card Backlog; no visible conversion into a planning draft Refine into intent, edit idea, archive/delete
entwurf_laeuft Planner intake runs Running with conversation Queued vs executing, last activity, runtime, failure recovery
entwurf_frage Planner waits for answer Needs attention; conversation and reply controls Concrete question, retained answer, clear responsibility
entwurf_fehlgeschlagen Intake failed Needs attention; retry available Failure category and whether retry can resolve it
Draft → geplant Wrapper writes Markdown and replaces draft card Ready; original transcript no longer exposed as a draft Stable identity/link to the conversation that produced the intent
geplant Intent available; no automatic ordinary build Ready; Operator, two Builder choices, Diagnose Acceptance criteria, dependency readiness, recommended primary action and why
operator_angefordert / operator_laeuft Read-only operator returns direct/split/needs-human decision Running Queued/executing, decision and source revision
Operator direct Scheduler queues implementation Running Chosen lane/profile, rationale, immutable attempt identity
Operator split → wartet_auf_kinder Children dispatched as dependencies and capacity allow Running; nested children and verbose badges Distinguish runnable, queued, executing, blocked, and waiting-on-user counts
durch_kinder_blockiert At least one child failed/rejected Needs attention parent; can still say children running Actual blocking cause per child and recovery that preserves the family
operator_klaerung_noetig Operator needs help or failed Needs attention Required human decision, editable answer/replan path
in_umsetzung / in_umsetzung_leicht Work handed to separate builder lane Running or Needs attention depending on last status Queue position, executing job, capability requirements, last activity
Builder committed → review Completion queues review of recorded commit Running Review pending/running, exact source and result revision
Builder limit_reached Automatic escalation while ladder remains; otherwise terminal Escalation badges, then Needs attention Attempts used, next profile, total budget, exhausted reason
Builder no_changes Usually no phase change Needs attention, often Archive only Structured already-satisfied / needs-input / forbidden / publish-failed outcomes
aufteilen_angefordert → aufgeteilt Legacy splitter writes children Running → Done, even though children may remain unfinished “Decomposed; child work outstanding,” not goal completion
Review → planung_mit_feedback Negative result stored Needs attention; often green “abgeschlossen” Explicit rejected verdict, blocking findings, repair/replan choice
Review → bereit_fuer_deployment Positive result immediately queues deployment Ready to deploy Whether deployment is automatic, queued, running, or awaiting a person
Deployment → deployed Runner success accepted by watcher Done Actual activated revision/system generation and task-relevant health evidence
Deployment failure Watcher routes to fixer failure Needs attention Deployment-specific error, retained artifact, retry-deploy vs repair distinction
klaerung_reparatur / inspector_laeuft Typed inspection or repair request Running Clearly distinguish diagnosis from code mutation
Inspection completion Result stored; original phase restored unless question Often only “inspected” Readable diagnosis, checked revision/time, evidence and required action
inspector_rueckfrage / fixer_rueckfrage User answer persists; new session continues Needs attention with answer box Correct role label; avoid generic “answer the Fixer” for inspection
fixer_laeuft Isolated implementation repair Running Attempt limit, exact target, changes, separate review required
Fixer not_applicable Remains klaerung_reparatur Falls into Running although terminal Explain no applicable repair and offer routing/replanning
Fixer/inspector failure Error retained Needs attention; Archive can be offered but rejected Action list must agree with transition validation
Research requested/running/finished/failed Separate research lifecycle; no Builder trigger Ready/Running/Done/Needs attention Source intent, report, retry-research, use findings to create linked intent
archiviert Bookkeeping, not cancellation Archived; all archived cards expanded Reactivate, archive provenance, cancellation semantics, retained family visibility

Sources: trigger.py:360, :925, :1399, :1507; worker.py:1851, :2001; completion_watcher.py:350; board.html:816, :924, :1210, :1302.

Priority findings

P1 — Negative review text can release deployment

worker.py:1450 searches the entire report for an approval phrase before searching for rejection. Both isolated inputs returned bereit_fuer_deployment:

  • Not ready for deployment. Decision: back to planning.
  • Nicht bereit für Deployment. Entscheidung: zurück zur Planung.

The positive branch of process_job immediately queues deployment (worker.py:2177 vicinity). Negation and quoted prior decisions are enough to undermine the gate; no malicious input is required. Use one validated structured verdict bound to a full commit hash, reject conflicting/ambiguous results, and separate report prose from the decision. Add negative/quoted/multiple-verdict cases.

P1 — The deployment request is pinned; the build is not

agent-codex/deploy_runner.py:63 accepts a request with commit_hash but fetches/fast-forwards dev and runs switch --flake path:<shared-repo>#blunix. The hash does not select or verify the build source. _write_completion (:161) reports the requested hash afterward.

A mocked command-trace reproduction confirmed that the requested hash appeared in none of the executed commands. If another commit reaches dev before deployment, it can be included without the corresponding review having completed. The mutable checkout also weakens reproducibility. Fix by building an immutable reviewed revision or a separately reviewed integration revision and recording the actual activated closure/revision. Merely comparing request/completion metadata does not close this gap.

P1 — Mutable display state can masquerade as deployment evidence

An isolated planung_mit_feedback → deployed transition succeeded without a commit or deploy job. deployment_evidence() then returned status=deployed, source “trusted board wrapper state.” apply_routing_recommendation() also assigns phases directly instead of using the transition dispatcher: recommending in_umsetzung sets that phase while queuing zero jobs.

These do not directly execute unauthorized system changes in the reproduction; they create false progress/evidence and bypass action semantics. Separate operational facts from manual classification. Derive deployed status from a verified deployment record, and apply routing through the same validated command path used for ordinary actions. Sources: trigger.py:1453, :1491, :1760.

P1 — Shared state updates are not transactional across processes

trigger.py has a process-local threading lock. Worker state writers (worker.py:514, :544), fanout, failure handler, metrics, and operator scheduling maintain separate read/modify/write implementations. The operator lock is separate; it is not a lock shared by all writers. Multiple writers use the same .tmp filename.

An isolated legal interleaving—trigger reads state, worker records completion, trigger saves its older snapshot—lost the completion field. This establishes the vulnerability, not an observed production incident. Atomic rename alone does not make the whole transaction atomic. Use one cross-process transaction mechanism, ideally a shared SQLite state/event store or a single writer, with unique temporary files if JSON remains.

P1/P2 — Recovery semantics are missing, not just buttons

builder.py:815 classifies anything without a recognized success record or the string NICHT committed as no_changes. apply_commit():930 reports push failures in different prose. The live GoToSocial case shows the resulting misclassification.

available_actions() omits Diagnose for implementation phases; repair_eligible() excludes no_changes. Consequently these cards can be stranded with only Archive. Failed inspector cards conversely advertise Archive, but update_phase() rejects it until a reset to planned; the UI offers no such reset. Archived cards have no ordinary Reactivate action. route_next is an implemented task type without a normal action that requests it.

Return structured execution and publication outcomes from the harness, then derive both actions and state from those outcomes. Distinguish task clarification, missing tools/evidence, genuine no-op, failed checks, failed publication, and deployment failure. An infrastructure error should not automatically ask a code-writing Fixer to change the project.

P2 — Queue/replay guarantees are incomplete

  • builder_trigger_blocked() (trigger.py:1202) examines board queues only. After handoff, a downstream builder-processing record is not seen; an isolated same-card/lane probe returned no block. The operator scheduler does inspect downstream queues, so the two dispatch paths disagree. This is a guard gap; downstream serialization may limit simultaneous execution but does not establish duplicate-request prevention.
  • Completion records move to consumed before processing (completion_watcher.py:62, :529). A crash between those steps leaves a consumed-but-unapplied result; the shown main loop does not replay it. This crash window was inspected, not triggered in production.
  • A research reset from failed to requested writes state and then raises UnboundLocalError because job_id is initialized later in the non-research branch (trigger.py:1598 vicinity). Reproduced in temporary state.
  • Queue IDs based on slug and integer seconds and updates without current-attempt checks deserve explicit duplicate/stale-completion tests.

P2 — The UI confuses job completion, goal success, and deployment

statusLabel() maps done to completed regardless of a negative review. Family status treats an implementation phase as active even with terminal no_changes. Lifecycle grouping prioritizes some phase labels over contradictory failure data. Child cards appear only inside their parent, so a child needing attention can be hidden in the Running column.

Have the server emit a single typed display state and explanation: goal state, current job state, last outcome, blocking actor, next action. Preserve both family navigation and a flat “needs my attention” view. Count children separately from parent goals.

P2 — Diagnosis and provenance exist but are not adequately displayed

The API exposes repository evidence, deployment status, timestamps, profiles, and relationships, but much is never rendered. Inspector result summaries are not exposed as a readable report. The telemetry endpoint provides up to 20 runs; the dialog renders only current_pipeline. A Builder report is hidden once a review report exists. Review files are keyed by slug and overwritten, limiting accessible per-attempt provenance. Draft completion preserves its conversation in state, but ordinary intent serialization does not surface it.

Provide an intent detail view with Overview, Attempts, Evidence, Conversation, and Changes. Bind each observation to its source revision and time. Keep current recommendation distinct from old failures and old review verdicts.

Browser and interaction findings

The desktop board is legible at the individual-card level, and dark mode is implemented consistently through theme variables. Native dialogs, keyboard handlers on report links, escaped text rendering, orphan fallback, and preservation of clarification text are useful foundations. No uncaught page error appeared during successful loading and read-only dialog use.

The screen gives scarce space to current work:

  • Eight tracks including Agents; content spans 2014 pixels. At 1440 pixels, Done and Archived are outside the initial viewport.
  • Agents occupies the first 260 pixels; Backlog another 230 pixels despite being empty. On a 390-pixel phone, initial content is mostly settings and test controls; actual work starts offscreen at x=538.
  • The archived column was 5113 pixels tall; Needs attention 4382; Running 3128. Families expand all children and long operator explanations. There is no search, filtering, collapse, or urgency sorting.
  • Dense colored family cards contain repeated parent/child identifiers, raw phase keys, mixed German/English labels, and three or more competing action buttons. This makes a hierarchy that is technically present difficult to scan.
  • Prominent install/uninstall smoke-test controls occupy the main header. Everyday priority and freshness information is absent there.
  • Intent/review dialogs show raw Markdown. They are useful evidence viewers but do not emphasize decision, outstanding criteria, or next action. The tested intent title also duplicated the “Intent:” prefix.
  • A focused card Intent link became BODY after the five-second rerender. The edit guard protects input/select/textarea focus, not button or report-link navigation.
  • Blocking the next board GET in the browser left the prior board displayed without a stale/offline indicator or live alert. The catch logs only to console.

Suggested direction: move Agents configuration into a settings drawer; show attention/running/ready summaries first; use collapsible families and a searchable archive; provide a list view at narrow widths; preserve keyed DOM nodes and keyboard focus; show last successful refresh and reconnect state. These are product recommendations from observed behavior, not a formal accessibility conformance audit.

Metrics, operations, and maintainability

The live remediation family telemetry had ten jobs and reported 1,632,046 total tokens, of which 1,435,501 were cached input tokens, and 459.007 seconds of summed job duration. These are reported counters, not audited invoices or end-to-end elapsed time. Its reported cost aggregate was $0.9061986; coverage of missing costs is not indicated, so it must not be presented as a verified full pipeline cost.

Negative reviews and no_changes jobs had success=true in telemetry while the family remained blocked. That can sensibly mean a model call completed, but it must not be used as goal-success rate. The current dialog does not show the stored dollar cost and does not provide an attempt-history selector. Additional useful measures are time waiting for people, queue delay, time to accepted outcome, rework rate, verified deployment rate, missing-measurement coverage, and cost per accepted outcome.

All six live runtime configurations selected Claude at observation time. The user reports exhausted Claude session credits; the board does not expose provider availability in this view. Starting work should explain unavailable capability/quota and preserve the immutable chosen profile. The Codex UI currently lists only Default and gpt-5.6-luna; allowed model/configuration values are duplicated across frontend, orchestration, and Nix defaults.

The implementation spans roughly 14,000 lines across the board's Python/HTML files, including about 5,300 lines of tests. The concern is duplicated state rules, not line count itself. Documentation still describes the original single-transition pilot, dragging, and an automatic correction loop even though the current worker stores negative review results and exposes manual Repair. Tests preserve several old behaviors, including no-change staying in its phase, and many frontend assertions inspect source strings rather than browser behavior.

The inspected Restic configuration backs up LDAP, Headscale, and Forgejo dump paths, not /var/lib/board-agent. This is a backup coverage question, not proof that no other backup exists. Runtime state, queue/acknowledgment records, and telemetry need a documented consistency-aware restore procedure. Loaders that silently substitute an empty board for malformed state should expose degraded state rather than make old intents look newly ready.

Sources: board.html:1117, :1152, :1564, :1667; telemetry.py:315; completion_watcher.py:267; worker.py:101; restic.nix:7; trigger.py:229; README.md:1.

  1. Repair the release boundary: structured review verdict, immutable deployment source, actual deployed revision evidence, no deployment claims from bookkeeping.
  2. Introduce structured builder/publication outcomes and one shared transition/action contract. Cover every terminal outcome with a valid continuation; fix research reset and action mismatches.
  3. Make state transactions and completion acknowledgment/replay reliable; enforce attempt identity and shared queue occupancy across both dispatch paths.
  4. Make the board truthful: separate goal/job/deployment state; show explicit blockers and next actor; put diagnosis and per-attempt evidence in one detail view.
  5. Improve layout and interaction: collapse settings/families/archive, attention-first list, search, mobile presentation, focus preservation, stale-data indicator.
  6. Add invariant-focused tests and operational metrics: rejected verdict never releases; activated revision equals reviewed revision; advertised action is valid; every terminal failure is recoverable; concurrent updates survive; replay is idempotent; blocked children surface in attention.

There is no need for a framework rewrite to begin this work. A shared state/transition layer and small browser regression suite would address more risk than adding new roles.

Uncertainties and product decisions

No production phase transitions, real retries, cancellation, repairs, model calls, deployment, destructive actions, or restore exercises were performed. Historical rows may reflect older code or manual intervention. The board snapshot is a point-in-time sample, and the two security-sensitive code findings were not tied to an incident. Authentication was checked only through the unauthenticated HTTPS response and configuration. No broad dependency/security audit or formal accessibility certification was attempted.

The main product decision is whether positive review should automatically deploy or should wait for explicit release. Current code automatically queues deployment, while “Ready to deploy” suggests a waiting gate. Either policy can be coherent, but the interface, authorization model, and evidence must say the same thing. A second decision is whether the default view serves a person managing goals or a person debugging agent infrastructure; today it heavily favors the latter.

Implementation follow-up — September 12 (not yet activated)

The user selected automatic deployment after approval. The working-tree changes implement structured commit-bound verdicts, immutable deployment builds, exact activated-closure comparison, a deployment retry without another model run, and refusal of automatic rollback from the last verified deployment. Manual result-state assignment can no longer bypass worker evidence.

Board state now uses a shared cross-process transaction boundary, a durable queue outbox and claim markers. Completion processing records acknowledgments after transactional application and checks pipeline/attempt identity. The research-reset failure and archived-family scheduling were corrected. Failed publication preserves unpublished work; its location is shown as evidence.

The UI separates goal state from job outcome, surfaces blocked children and unverified legacy deployments, and adds search, archive/attention views, collapsible families, separate settings, detailed evidence/history/conversation views, focus preservation and stale-data warnings. These changes were exercised in Chromium with isolated fixtures and the earlier read-only live snapshot.

Regression verification: 419 Python tests and 38 subtests passed. Both Nix script bundles were built; flake evaluation was checked without building the VM checks or activating a system. The repeatable test_board_browser.cjs test checks desktop/mobile layout, filtering, dialogs, keyboard focus and API failure handling. It never sends live mutation requests or invokes a model.

Limits remain explicit: activation and live post-deployment smoke testing are pending; old cards are not bulk-certified or restarted. Historical missing attempt data cannot be recreated. Quota availability is not probed, and changing model choices does not validate provider access. Restic coverage added here is for board metadata and the deployment ledger, not a coordinated queue/SQLite restore. A linked research-to-implementation planner flow and exhaustive accessibility/production crash testing remain follow-up work. The original audit above describes the observed pre-change system, not a claim that every suggested product extension has been completed.

Final validation caveat: a later all-host nix flake check --no-build rerun, including a retry outside the sandbox, failed while evaluating tnix because a referenced cached flake source was no longer valid. This is not a passing final all-host check; the earlier successful evaluation and script-bundle builds do not replace resolution of that environment/input issue before a full rollout. The scoped final blunix evaluation subsequently passed with eval-cache=false, producing rbh29vyl8lwnj0dw3ssjx3scwna8klfz-nixos-system-blunix-26.11.20260910.8ce4ef6.drv. No system was activated. Unrelated flake.lock and journald edits must be excluded from any board-only activation snapshot.