Intro

Background: the bottleneck is human time

• A solo side project run alongside a day job at a game company: Unity 6, 2D turn-based tactics RPG
• Trial and error to experience an agentic project I can't pursue at work. This talk is the record
• Coding time itself is scarce, and the hours I can spare vary day to day
• Common failure patterns in AI-assisted development:
  • The same mistakes repeat every session (context evaporates)
  • It does only what you asked, or more than you asked (scope control fails)
  • It says "done," but it doesn't actually work (no verification)

The frame of this talk:

Don't just "make" the AI work. Build a system the AI can work in

Intro

The whole pipeline: today's map

• Each chapter that follows zooms into one section of this diagram
• Act 1: Following the pipeline (①~⑥)
• Act 2: Going unattended (⑦)
• Act 3: Why it keeps running, and how it stays that way

Act 1, Pipeline ②

The four thinking skills it routes to: origins and differences

In one line: first-principles breaks "the approach itself," why digs into "the reason to exist," brainstorming shapes "what's decided" into a design, and autoloop tightens "what works" by the numbers

Act 1, Pipeline ③-1

SPEC: if these 4 aren't filled in, we don't start

/plan-spec: collects and validates 4 required items interactively before implementation

• Output: vault features/{YYMMDD}-{feature-name}/1-SPEC-*.md, kept in Obsidian, not the repo
• Branch by size: medium/large → deeper DESIGN; small (only if all 4 conditions are met) → straight to PLAN
  • The default is going through DESIGN (a conservative branch: skipping is the exception)

Act 1, Pipeline ③-2

DESIGN: pin decisions down in a document

/plan-design: an 8-step design deep dive. Unneeded steps are skipped

Feasibility → Tech stack → User flow → Screens → API → Data → Architecture → Tech decisions

• What matters here: leaving the "decisions" behind
  • AI sessions get thrown away. Even when the next session or a different AI arrives, decisions in a document don't waver
  • Record even "why option B and not A" → when a review pushes back later, there's no re-arguing
• Output: 2-DESIGN-*.md in the same folder

A real example (vault features/ folder):

260620-balance-sim-harness/
  ├── 2-DESIGN-balance-sim-harness.md
  ├── 3-PLAN-balance-sim-harness-01-core-stats.md
  └── 3-PLAN-balance-sim-harness-02-...

Act 1, Pipeline ③-3

PLAN: leave the executor no room for judgment

/writing-plans: the implementation plan. It's executed by another AI (Codex), not a human

• Split responsibilities per file → bite-sized 2 to 5 minute steps → code, commands, expected results fully spelled out
• What matters here: ambiguity = where the executor gets creative = where accidents happen
  • If a phrase like "handle appropriately" is still there, the plan isn't finished
  • Every step also states how to verify it (compile, tests, grep)

Why the stages are split is the key:

Act 1, Pipeline ④-1

Division of roles: a three-party setup

• "Anything that touches code goes to Codex": the point is that the rule has no exceptions
  • The moment "it's a small change, I'll just do it" slips in, the whole role split collapses (this was actually called out once and logged as a rule)
• A side effect of the split: the implementer (Codex) and the manager (Claude) don't mix contexts → the foundation for independent review (continued in ⑥)

Act 1, Pipeline ④-2

Codex 4-tier: not every task gets the top model

Tier	Model/effort	Use
A	gpt-5.5 + xhigh	Architecture, large refactors, security audits, long-horizon debugging
B default	gpt-5.5 + high	General implementation, fixes, plan-based work
C	gpt-5.4 + xhigh	One-off deep analysis and algorithms (not long-horizon)
D	gpt-5.4 + high	Fixed for code review, exploration, lint, docs, research

• Everything on the top tier means about 2x the unit cost and high latency. The task type picks the tier
• Why review is D: many wide, shallow passes beat one deep pass on value
• Delegation briefs always carry pitfall clauses (e.g. "no new interfaces or public APIs," "check Unity assets via unity-cli"), each one born from a past accident (setup for Act 3)

Act 2, Unattended ⑦-1

issue-agent: PRs show up while you sleep

An unattended agent running on a Mac mini picks up an issue → branches → fixes → verifies → opens a PR automatically

The policy is declared as code in the repo's .issue-agent.yml:

auto_eligible_labels: [type:cleanup, type:bug, type:tech-debt]
exclude_labels: [type:feature]        # skip anything that needs a design call
max_diff_lines: 60                    # small fixes only
require_repro_test: true              # no repro test, no PR
verify:
  type: unity
  commands: [refresh, console]        # verify compile and console on local Unity

• Paying off Act 1's setup: because the type:* label convention is clean, the machine can tell which work is "OK to pick up"
• Why features are excluded: a new feature = a design decision = human territory

Act 2, Unattended ⑦-2

The real thing: what happened while I slept

Left: a real PR opened by issue-agent: Closes #N, with a change summary and even 0 compile errors, all self-reported
Right: a Discord notification at 3:21 a.m.: 23 issues triaged → 1 PR. The issue aborted for exceeding the 60-line cap was handed to a human (needs-human): the moment the guardrails on the next slide actually kicked in

Act 2, Unattended ⑦-3

Guardrails for going unattended: more autonomy ⇒ more guardrails

auto_merge:
  enabled: false                          # merging is always a human
  forbid_paths: [Tests/, .github/, CODEOWNERS]
notify:
  on_failure: true

• Auto-merge off: the agent only goes as far as the PR. Final approval is human
• No edits to Tests/ = defense against test gaming
  • The classic failure mode, an agent passing by "fixing the tests," is sealed off by blocking the path
• No .github/ or CODEOWNERS = the agent can't edit its own guardrails
• require_repro_test: a bug fix must come with a repro test (forces proof of the fix)

Lesson: every time autonomy goes up a step, the guardrails go up a step too.
Human-reviewed PRs and unattended PRs earn different levels of trust, and the policy absorbs that gap

Act 3, Why it runs ②

Codebase makeup: 37% of all code is tests

An unusual share for a game project: the games industry has traditionally been light on test code. Here, tests are how the AI proves its own changes, and the marginal cost of writing them is low (the AI writes them), so a culture of attaching a repro test to every bug fix holds → next slide: the rules that hold this pile of tests up

Act 3, Why it runs ③

AI-friendly conventions (tests): the AI's means of self-verification

Tests are the tool the AI uses to prove its own changes → the rules pin down "how far the AI can verify"

• Codify the EditMode/PlayMode boundary: pure logic = EditMode, scene and phase routing = PlayMode/runtime
  • Without that boundary, accidents happen where the AI reports "untestable" as if it were "tests pass"
• When behavior changes, grep every test that asserts the old behavior: AI tends to update only tests near the code it touched
• Before removing a public API, grep its consumers including Tests/: reflection-based tests compile fine and only break at runtime
• "No errors" ≠ "it works": the code path may never have run because there was no data. Confirm it actually triggers with real data

What these have in common: every one of them actually bit us at least once (→ next chapter)

Act 3, How it stays ①

The mistake-logging loop: the core of this talk

"AI repeats the same mistakes" → a 4-rung ladder that blocks repeats by design:

The point: don't climb the whole ladder. Most stop at ①; only repeat offenders move up to ② and ③

Act 3, How it stays ②

Real cases: every rule you've seen came from an accident

• ④-2's standing brief clauses, ⑦-3's guardrails: all products of this loop
• Tools can be copied, but this logging loop is what makes the workflow sustainable

Closing

Limits and the human's part

What humans still do, and what humans should keep doing:

• Design decisions: the AI goes as far as recommendations and reasons; the decision is human. Even two AIs agreeing isn't a mandate
• Fine-tuning UI visual design: layout, color, spacing. The AI handles structure and conventions only
• Final merge approval: no exceptions, whether the PR came from a human or an unattended agent
• Deciding which review findings to accept: review comments are information, not orders
• First detection of an accident: the loop automates the logging, not the detecting

Current limits:

• The unattended agent handles small fixes of 60 lines or fewer. Work that needs judgment can't be handed over yet
• Verifying runtime "feel" (hit impact, presentation quality) still comes down to human eyes
• Pipeline upkeep: rules, memory, and docs need constant gardening (left alone, they turn into a false-positive factory)

Closing

Wrap-up

Three things to take home:

1. A pipeline, not a tool: every stage needs an output and a gate, or the AI runs wild
2. Independence and self-verification create quality: separate implementation and review sessions, and let the AI drive its own changes
3. Log your failures into the system: the memory → rule → machine-enforcement ladder. Without it, everything else is an expensive toy that repeats the same accidents