How I Built AI OPS for subvocal

Why AI OPS?

I'm building subvocal — a Flutter app that reads subtitles aloud in sync with streaming video. It's for people who can't read, and for language learners who want to hear pronunciation while reading along.

The problem: I'm a solo developer working on a project that needs clean architecture, multiple API integrations, TTS synchronization, translation services, and production-quality code. I don't have a team. So I built one.

AI OPS is the system I set up to use AI agents as team members. Not as code generators that I point at a problem and hope for the best — but as structured roles with clear responsibilities, approval gates, and engineering principles baked into their context.

It was DevOps. Then DevSecOps. Now it's DevSecComplianceAIFinOps. The acronym is growing faster than the codebase.

The result: hundreds of commits, over a hundred tests, six agents, eight reference books, and a growing collection of automation scripts — all for a fraction of the cost of a single developer hour.

The Environment

Everything runs locally. There is no cloud counterpart, no remote dev environment, no hosted CI runner I pay for. The devcontainer runs on my machine — the same machine where I type, where the hardware token is plugged in, and where the API keys live. When I open this project in VS Code (or when an AI agent starts working), it gets a reproducible container environment with exactly the tools needed — and nothing more.

# devcontainer.json features
"ghcr.io/devcontainer-community/devcontainer-features/opencode.ai:1"
"ghcr.io/devcontainers/features/github-cli:1"
"ghcr.io/awf-project/devcontainer-features/flutter:1"
"ghcr.io/devcontainers/features/node:2"

The base image is Debian Bullseye. On top of it, four features install OpenCode (the AI agent framework), Flutter SDK, GitHub CLI, and Node.js. No manual setup, no "works on my machine" — every agent gets the same environment.

API Key Isolation

The clever part is API key forwarding. API keys are passed from my host machine into the container as environment variables via containerEnv:

"containerEnv": {
  "AI_FUN_TOKEN": "${localEnv:AI_FUN_TOKEN}",
  "OPENROUTER_API_KEY": "${localEnv:OPENROUTER_API_KEY}",
  "OPENCODE_ZEN_API_KEY": "${localEnv:OPENCODE_ZEN_API_KEY}",
  "GOOGLE_GENERATIVE_AI_API_KEY": "${localEnv:GOOGLE_API_KEY}",
  "OPENSUBTITLES_API_KEY": "${localEnv:OPENSUBTITLES_API_KEY}",
  "NVIDIA_API_KEY": "${localEnv:NVIDIA_API_KEY}",
  "omniroute_api_key": "${localEnv:omniroute_api_key}"
}

None are stored in the repository or the container image. The agents can use them (for OpenSubtitles, translation services, AI providers), but they never touch disk. If the container is destroyed, the keys vanish. This is AI isolation — the agents have access to the APIs they need, but the keys stay on my machine.

GPG Signing as a Hard Gate

Every git commit must be GPG-signed using a hardware token (YubiKey). The devcontainer forwards the GPG agent socket from the host machine, so signing works inside the container even though the hardware token is physically connected to the host.

This is a hard gate, not a suggestion. The commit-push.sh script enforces it: if GPG signing fails (hardware token timeout, PIN entry delayed), the commit is rejected. There's no fallback to unsigned commits. This prevents the "I'll just skip the signature this one time" anti-pattern that erodes security over time.

# commit-push.sh enforces GPG signing
🔐 Checking GPG card status...
✍️  Committing: feat: wire SubDL and Podnapisi providers
# If GPG fails:
❌ Commit failed. GPG signing required — cannot fall back to unsigned commits.

The practical effect: every commit in the repository is cryptographically signed. You can verify any commit with git log --show-signature. This matters for a project that uses AI agents — signed commits prove which changes were made by a human (via hardware token) versus which were automated.

LSP: Token Optimization for AI

One of the most impactful settings in opencode.json is "lsp": true. This enables Language Server Protocol support, which gives AI agents access to the Dart language server.

Why does this matter for token usage? Without LSP, an AI agent that needs to understand a function must read the entire file where it's defined, plus every file it imports, to trace types and references. With LSP, the agent can ask the language server: "what is the type of this variable?", "where is this method defined?", "what are all the callers of this function?" — and get precise answers without reading entire files.

The result: fewer tokens spent on context gathering. Instead of loading 500 lines of code to understand one function, the agent loads 20 lines of targeted, LSP-provided context. Across hundreds of agent interactions per day, this adds up to significant token savings.

Agent Permissions

Each agent has explicit permission boundaries defined in opencode.json. The architect — the primary agent I interact with — is read-only:

"architect": {
  "permission": {
    "*": "allow",
    "edit": "deny",
    "bash": "deny"
  }
}

It can read files, search the codebase, and use LSP — but it cannot edit files or run commands. This forces it to plan rather than implement. The developer and tester subagents have full access ("*": "allow"), but they work autonomously and return results for review. The security auditor and UX reviewer are read-only — they can inspect but not modify.

This permission model is the foundation of the HITL workflow. The architect can't accidentally edit code while planning. The developer can't skip the approval gate. Every agent operates within its defined boundaries.

VS Code Extensions

VS Code extensions install automatically — Dart, Flutter, SonarLint, GitLens, PlantUML, ESLint, Prettier, YAML, Markdown, EditorConfig, Spell Checker, Error Lens, and Git Graph. These aren't just developer conveniences — they provide the LSP infrastructure that makes token-efficient AI work possible. The Dart extension provides the language server. SonarLint provides static analysis. Error Lens surfaces issues inline. The environment is identical whether I'm working locally or an AI agent is working in a cloud session.

Building the AI Team

The core of AI OPS is the agent configuration in opencode.json. I defined six agents, each with a specific role, model configuration, and permission set:

Agent	Role	Permissions
architect	Plans, reviews, creates issues/PRs	Read-only (no edit, no bash)
developer	Implements code changes	Full access
tester	Writes and runs tests	Full access
security-auditor	Reviews for vulnerabilities	Read-only
ux-ui	Reviews UI/accessibility	Read-only

The key design decision: the architect is the primary agent — the one I talk to directly. It cannot edit files or run commands. This forces it to plan rather than implement. The developer and tester are subagents that the architect delegates to. They have full access but work autonomously and return results for review.

Temperature is configured per agent — lower for implementers who need precision, higher for planners who need creativity. The tester gets more steps than other agents because writing tests is the most token-intensive task. Models are also configured per agent, but if a specific model isn't available, OpenCode falls back to the default model.

How the Architect Plans

When I ask the architect to do something, it doesn't start coding. It explores the codebase using LSP and search tools, understands the existing patterns, and produces a structured plan in the form of a GitHub issue. Here's a real example — issue #143, fixing Danish character encoding during TTS playback:

## WHY
Danish locale characters (æ, ø, å) are corrupted when subtitles
are downloaded from OpenSubtitles and played via TTS. The
fetchContent() method uses response.body which internally decodes
bytes as UTF-8 (with allowMalformed). When the downloaded SRT
file is Latin-1/Windows-1252 encoded (common for European
subtitles), non-ASCII characters become Unicode replacement
characters (U+FFFD), appearing as ??? in both the UI and TTS.

## WHAT
1. Detect encoding from Content-Type header or byte pattern
2. Decode SRT content using correct encoding (Latin-1/CP1252)
3. Normalize to UTF-8 before returning to callers
4. Add encoding detection tests for European subtitle samples

## HOW
opensubtitles_api.dart: add _detectEncoding() helper.
fetchContent(): use detected encoding for body decoding.
Add unit tests with sample Latin-1 encoded SRT content.

Notice the structure: WHY explains the problem (Danish characters corrupted by wrong encoding). WHAT lists the changes needed. HOW specifies the exact files and methods. This isn't vague — it's a precise implementation plan that the developer can follow without asking clarifying questions.

The architect then calls create-issue.sh to create the issue on GitHub, which automatically creates the issue/{number}-{slug} branch. Then it calls watch-approval.sh and waits for me to approve before any code is written.

The Workflow That Won't Let Me Screw Up

The Human-in-the-Loop (HITL) workflow is the most important part of AI OPS. It's a pipeline with a hard gate that cannot be skipped:

This is the golden path. AI agents are non-deterministic — they sometimes stray from the workflow, skip steps, or make unexpected decisions. The hard gates (human approval, GPG signing, architect review) exist precisely because of this unpredictability. GPG signing, enforced by hardware token, is the one thing that never strays.

graph TD A[Architect Plans] --> B{Human Approves} B -->|Approved| C[Developer Implements] C --> D[Developer Returns to Architect] D --> E{Architect Reviews} E -->|Approved| F[Tester Writes Tests] F --> G[Tester Returns to Architect] G --> H{Architect Reviews Tests} H -->|Approved| I[Security Auditor Reviews] I --> J[Security Auditor Returns to Architect] J --> K{Final Architect Review} K -->|Approved| L[Architect Creates PR] L --> M{Human Reviews PR} M -->|Approved| N[Merge to Development] K -->|Changes Needed| C E -->|Changes Needed| C H -->|Changes Needed| F

The hard gate is step 2: human approval. I learned this the hard way. In early sessions, I let the architect proceed without explicit approval on "small" changes. Those changes introduced bugs, broke tests, and sometimes went in the wrong direction entirely.

Now the workflow is enforced in AGENTS.md with a prominent warning:

You MUST NOT create, edit, or modify any code files until the human has explicitly approved an architect's plan. Operational tasks (starting services, running commands, reading files) are exempt. Everything else requires: architect plans → human approves → developer implements.

Another lesson learned the hard way: never merge issue branches locally into development. I once merged a branch locally and pushed, which created orphaned branches with no PR trail and broke the review process. Now the workflow is always:

issue branch → implement → push branch → create PR → merge via GitHub

Case Study: Wiring Subtitle Providers

To show how this works in practice, here's the full journey of issue #167 — wiring SubDL and Podnapisi subtitle providers — from request to merge.

Step 1: The Request

I noticed that SubDL and Podnapisi APIs were fully implemented but never wired into the dependency graph. The app only used OpenSubtitles. I told the architect: "Wire SubDL and Podnapisi providers into the app."

Step 2: The Plan

The architect explored the codebase. It found that search_provider.dart only passed OpenSubtitlesApi to the repository, even though SubtitleProviderAggregator accepted optional params for SubDL and Podnapisi. It found that downloadFromProvider existed but was never called.

It produced the WHY/WHAT/HOW plan (shown earlier) and created issue #167 on GitHub with the issue/167-wire-subdl-podnapisi branch. Then it called watch-approval.sh and waited.

Step 3: Human Approval

I reviewed the plan. It was correct — the APIs were implemented but not wired. I commented "approved" on the issue. The watch-approval script detected the keyword and unblocked the architect.

Step 4: Developer Implements

The architect delegated to the developer subagent. The developer:

Added _subdlProvider and _podnapisiProvider in search_provider.dart
Wired them into subtitleRepositoryProvider
Added providerSource parameter to DownloadSubtitle use case
Updated search_screen.dart to pass result.providerSource
Added failure logging in SubtitleProviderAggregator

The developer committed each change with GPG-signed conventional commits, pushed the branch, and called workflow-notify.sh impl-done.

Step 5: Tests

The architect reviewed the implementation, then delegated to the tester. The tester follows a clear methodology:

Unit tests first — test individual functions and classes in isolation, mocking external dependencies
Widget tests for UI — verify Flutter widgets render correctly and respond to interactions
Integration tests for critical paths — run against the Android emulator for end-to-end flows

The tester validates that the implementation matches the architect's plan by reading the issue's WHY/WHAT/HOW and checking each item. If the developer deviated from the plan, the tester flags it. Tests are run with flutter test (unit/widget) or flutter test integration_test/ (integration). The tester also checks that existing tests still pass — no regressions.

When done, the tester calls workflow-notify.sh tests-done and returns results to the architect for review.

Step 6: PR and Merge

The architect created PR #168 with the WHY/WHAT/HOW body matching the issue. CI ran — analyze-and-test passed, integration-test was skipped (no integration paths changed). I reviewed the PR and merged it to development.

Total time: a few minutes of my time (reviewing plan + reviewing PR). The agents did the exploration, implementation, and testing autonomously.

Giving AI Engineering Principles

Raw AI models know how to write code, but they don't know your standards. I solved this by building a reference library of eight condensed books in library/, organized by agent role — not a flat list dumped into every agent's context.

library/
  release-it.mini.md                          # All agents
  architect/
    clean-architecture.mini.md                # @architect
    patterns-of-enterprise-application-architecture.mini.md
    domain-driven-design-distilled.mini.md
  developer/
    clean-code.mini.md                        # @developer, @tester
    refactoring.mini.md
  ux-ui/
    material-design-3.mini.md                 # @ux-ui
    ui-patterns.mini.md

The books are split between two sources. Six come from ciembor's agent-rules-books — condensed versions of Clean Code, Clean Architecture, Refactoring, Patterns of Enterprise Application Architecture, Domain-Driven Design Distilled, and Release It!. Each follows a consistent format: "When to use", "Primary bias to correct", "Decision rules", "Trigger rules", "Final checklist".

The other two — Material Design 3 and UI Patterns — are original references I wrote specifically for the UX/UI agent. There's no suitable condensed reference for Flutter UI design systems and interaction patterns, so I created them.

Role-Based Assignments

Not every agent needs every book. The architect needs architecture patterns but not Material Design. The UX/UI agent needs Material Design but not refactoring patterns. So the books are assigned per role:

Role	Books	Why
All agents	Release It!	Production readiness is everyone's responsibility
@architect	Clean Architecture, Patterns of EA, DDD Distilled	Dependency rules, layering, bounded contexts
@developer	Clean Code, Refactoring	Readability, behavior-preserving improvements
@tester	Clean Code, Refactoring	Readable tests, safe refactoring
@security-auditor	All books	Security review benefits from full design context
@ux-ui	Material Design 3, UI Patterns	Flutter design systems and interaction patterns

The implementation is in opencode.json: the global instructions array loads only the shared book (Release It!), while each agent's prompt field lists its role-specific books. This means the architect gets architecture context without wasting tokens on Material Design, and the UX/UI agent gets design context without loading refactoring patterns.

The result is that every agent operates with the engineering principles relevant to its role — and only those principles. No token waste, no irrelevant context.

Scripts That Save Tokens and Time

The scripts/ directory contains helper scripts. Some are general-purpose (test runners, git helpers), but the core ones are the AI workflow scripts that orchestrate the HITL process and track costs.

workflow-notify.sh — The Orchestrator

This is the central nervous system. When an agent completes a task, it calls:

./scripts/workflow-notify.sh impl-done "issue #167: SubDL/Podnapisi wired"

This script does three things automatically:

Looks up the latest open enhancement issue
Sends a push notification via ntfy.sh to my phone
Calls log-usage.sh to record token usage for FinOps tracking

Without this, I'd have to manually check which issue is being worked on, manually notify myself, and manually log costs. The script saves time per task boundary, and across many tasks per day, that adds up.

watch-approval.sh — The Gatekeeper

After the architect creates a plan, it needs my approval. Instead of me watching a screen, the architect calls:

./scripts/watch-approval.sh

This polls the GitHub issue comments every 10 seconds, looking for keywords like "approved", "lgtm", "looks good", or "go ahead". When it detects approval, it sends a notification and exits. The architect can then proceed to implementation.

This is a token-saving pattern: instead of the agent repeatedly asking "are you approved yet?" (which would consume tokens on each check), the script does the polling externally using gh CLI — zero tokens spent on the wait.

commit-push.sh — The Enforcer

Every commit must be GPG-signed (hardware token) and use conventional commit prefixes. The script handles this:

./scripts/commit-push.sh --all -m "feat: wire SubDL and Podnapisi providers"

It checks GPG card status, stages all changes, creates a signed commit, and pushes. If GPG signing fails (hardware token timeout), it reports the error instead of falling back to unsigned commits. This prevents the "I'll just skip the signature this one time" anti-pattern.

create-issue.sh and pr-create.sh — The Communicators

Issues and PRs use the same WHY/WHAT/HOW template. The scripts generate this format automatically:

./scripts/create-issue.sh \
  --title "feat: Wire SubDL and Podnapisi" \
  --why "SubDL and Podnapisi were implemented but never wired" \
  --what "Add providers, wire into repository, route downloads" \
  --how "search_provider.dart: add providers. repository: add providerSource" \
  --create-branch

This ensures every issue and PR has the same structured format, making them scannable for humans and parseable for agents. The --create-branch flag automatically creates the issue/{number}-{slug} branch.

log-usage.sh and usage-report.sh — The Accountants

Every token is tracked. The flow:

OpenCode SQLite DB → log-usage.sh → .aifinops/log.csv → usage-report.sh

log-usage.sh reads the OpenCode database, extracts token counts and costs, and appends a CSV row. It's called automatically by workflow-notify.sh at task boundaries — I never have to invoke it manually.

usage-report.sh reads the CSV and generates reports:

./scripts/usage-report.sh --summary
# Entries:     ...
# Input:       ... tokens
# Output:      ... tokens
# Cost:        ...

./scripts/usage-report.sh --by-agent
# Shows token usage per agent role

./scripts/usage-report.sh --by-model
# Shows token usage per AI model

Tracking Every Token

AI isn't free — but it can be cheap. The FinOps system tracks every token across every session, agent, and model. The data lives in .aifinops/log.csv (git-ignored, generated locally) and is surfaced via the AI Tokens badge in the README.

The badge shows current totals: input tokens, output tokens, and reasoning tokens. The cost breakdown is available via usage-report.sh, which can filter by agent, model, or issue.

This visibility matters. Without it, I'd have no idea whether AI OPS is cost-effective. With it, I can see exactly where tokens are being spent and optimize accordingly.

CI That Knows What Changed

The CI pipeline uses path-based filtering to skip jobs when their inputs haven't changed:

graph LR A[PR Created] --> B{paths-filter} B -->|lib/test changed| C[analyze-and-test] B -->|docs only| D[Skip CI] C -->|integration paths| E[integration-test] C --> F[ci-pass] E --> F F -->|required check| G[Ready to Merge]

If I only edit documentation (*.md, library/, site/), the entire CI workflow is skipped — no Flutter setup, no dependency install, no analysis. If I edit code but not integration tests, only analyze-and-test runs — skipping the Android emulator boot, KVM setup, Gradle caching, and the integration test suite entirely.

The ci-pass job is the unified required status check. It always runs and aggregates results from both jobs. This way, required checks never block PRs when jobs are legitimately skipped.

Structured Communication

Every issue and PR uses the same WHY/WHAT/HOW template. This isn't just convention — it's enforced by the create-issue.sh and pr-create.sh scripts, and by the issue templates in .github/.

The structure:

## WHY
Why is this change needed? What problem does it solve?

## WHAT
What was changed? Briefly list the major changes.

## HOW
How does the implementation work? Key design decisions.

Closes #

This format serves two purposes:

For humans: scannable in 10 seconds. WHY tells me if the work matters. WHAT tells me the scope. HOW tells me the approach.
For agents: parseable context. The architect reads this when picking up work. The developer reads it when implementing. The tester reads it when writing tests.

Having the same format for issues AND PRs means the PR body is essentially a copy of the issue with the implementation details filled in. This creates a clean audit trail: issue → plan → implementation → PR → merge.

Challenges and Limitations

AI OPS isn't perfect. Here's what's hard about it:

Setup Complexity

The devcontainer requires Docker, VS Code, and the Remote Containers extension. For a new contributor, getting the environment running takes time — installing Docker, cloning the repo, waiting for container build, configuring API keys. This is the trade-off for reproducibility: the environment is identical for everyone, but the initial barrier is higher than "just install Flutter."

Non-Determinism

AI agents are non-deterministic. The same prompt can produce different code, different file structures, different naming choices. The HITL workflow catches most drift, but not all. Sometimes the developer implements something slightly different from what the architect planned — a different variable name, a different error handling approach. The architect's review step exists for exactly this reason, but it adds a round-trip to every task.

Token Cost Reality

While the FinOps tracking shows low costs, tokens add up with larger codebases and more complex tasks. A single feature implementation can consume tens of thousands of tokens across the architect's planning, the developer's implementation, and the tester's validation. LSP optimization helps, but the fundamental cost is there.

Model Availability

The agents are configured with specific models, but those models aren't always available. When a model goes down or hits rate limits, OpenCode falls back to the default model — which may produce different quality results. The article doesn't specify models in the agent table for this reason: the configuration is aspirational, not guaranteed.

Local Only

Everything runs locally. There's no cloud counterpart, no remote dev environment, no hosted CI runner. This means the system is limited by my machine's resources — Docker memory, disk space, CPU for running the Android emulator in CI. It also means the system isn't easily shareable: someone else can't just log in and start using AI OPS without setting up the full local environment.

Lessons Learned

AI OPS didn't emerge perfectly. It evolved through mistakes — each one teaching a rule that's now baked into the system.

The Merge-Locally Disaster

Early on, I merged an issue branch locally into development and pushed. This created orphaned branches with no PR trail. There was no review record, no CI validation on the PR, and the branch list became a mess. The fix: never merge locally. Always push the branch, create a PR, and let GitHub handle the merge. This is now documented in AGENTS.md as a "LESSON LEARNED" with a prominent warning.

Skipping the HITL Gate

I once let the architect proceed without explicit approval on a "small" change — just renaming a variable. The rename broke three downstream references that the architect hadn't checked. The fix: the hard gate is non-negotiable. Even "obvious" changes need explicit approval. The memory aid in AGENTS.md now lists specific examples of what counts as a code change (adding tests, modifying source files, adding config files) and explicitly states: "But it's small!" is not an exemption.

The Token Waste Problem

Early sessions burned through tokens on context gathering — agents reading entire files to understand one function. Enabling LSP changed this dramatically. Instead of 500-line file reads, agents ask the language server for targeted answers. The result: same work, fraction of the tokens. This is now a default setting in opencode.json.

The Notification Gap

Before workflow-notify.sh, I had to manually check which issue was being worked on, manually notify myself, and manually log costs. This created gaps — I'd miss notifications, forget to log usage, or lose track of which agent was doing what. Automating the notification + logging pipeline closed these gaps. Every task boundary now triggers a notification and a cost log entry automatically.

Journal

This is a running log of findings and corrections, dated as they happen. The article above is a snapshot; the journal captures how the system evolves.

2026-07-31 — Subagent token usage was invisible, and the system wrote its own reference book

Two findings today, both discovered through the workflow itself.

Finding 1: the FinOps ledger was under-reporting subagent usage. The first version of log-usage.sh logged only top-level sessions, silently skipping subagents. Every @developer, @tester, and @security-auditor run was invisible in .aifinops/log.csv — I only caught it by comparing the report output against reality. The fix kept the data model simple: one CSV row per session (parent and subagents alike), a parent_session_id column added last to link a subagent row back to its parent without breaking existing rows, and dedup by session ID so re-runs never double-count. Each subagent row carries its own model and cost — architect, developer, tester, and security auditor may run on different models, and aggregating them into the parent would have hidden exactly the signal I was trying to see. The security audit that followed also surfaced injection hardening for the script: DB paths and session IDs are passed via environment variables instead of interpolated into Python source, CSV formula injection is escaped, and dedup uses fixed-string matching.

Finding 2: the system generated its own reference book. When we adopted a TDD/BDD development workflow (issue #219), the architect flagged that neither Clean Code nor Refactoring covered test-driven and behavior-driven development — and produced library/developer/tdd-bdd.mini.md in the same OBEY format as the other books, without me writing a line of it. The workflow didn't just follow my engineering principles; it extended them when it found a gap. The reference library grew from eight books to nine, and @developer and @tester now load it alongside their existing books.