Skip to content
Let's talk

Case study · Oct 2026 to present

Diligent

An audit skill for U.S. government websites that turned four failure modes of AI compliance review into a deterministic gate, a rule corpus, and a companion engine that ships zero promises and every limitation.

Role
Creator and product owner, coordinating AI executors
Company
Simple Cortex
Dates
Oct 2026 to present
The diligent.style marketing site hero: a D-tile mark before the wordmark, the headline reading 'Audit U.S. government sites with evidence, not assertions', and a subline stating the companion binary ships 61 automated and assisted checks across 48 rules
Diligent · Audit skillLive at diligent.style. The D-tile mark sits before the wordmark at cap height. The hero leads with what the tool does, not what it promises.

TL;DR

I created Diligent as a portable Agent Skill for auditing U.S. government websites and digital services. AI executors wrote the code from my specs while I ran audits, checked diffs, and held the design line. The skill started as a zero-dependency Node ESM package with a workflow, artifact formats, an authority registry, and a deterministic gate. The day after the first public release it had a Rust companion binary running 28 deterministic page checks. Version 0.2.1 ships storybook mode, component audit, source-pass template scanning across seven languages, SARIF output, a GitHub Action, and 45 rules reviewed against primary sources at draft lifecycle. The marketing site at diligent.style is an Astro static site with Arc UI, deployed to Cloudflare Pages.

The proof I trust most is what the skill refuses to do. It will not certify a site. It will not turn a missing check into a pass. It will not let the agent that wrote a repair declare the problem solved. The gate computes status from recorded results, and incomplete can never become a pass. The refusals are the product.

Four failure modes became a design

Ask a capable coding agent whether a government site is compliant and you get a confident answer. It is usually not an audit. Nobody decided which law applies, which version of the standard is binding, what was actually tested, or what was skipped. The agent produced a plausible checklist. Compliance is the reasoning that should have come first.

Four failure modes follow from that gap. Wrong rules, where a.gov hostname or a city name becomes a legal conclusion. Pass by silence, where a check that did not run is quietly counted as a pass. Evidence from assertion, where a policy page or a vendor report is accepted as proof of deployed behavior. Self-certified repair, where the agent that wrote the fix declares the problem solved.

Diligent makes the agent work the way a careful reviewer does instead. Resolve scope and applicability against a frozen set of sources. Collect evidence with provenance. Record results in explicit states, with unknown and untested kept visible. Compute the overall status mechanically. Hand repairs to a separate context for verification.

The design principle that shaped everything is one line: missing is not passing. Every result has one of six states. A skipped test is not_tested. An unknown applicability keeps the assessment incomplete. The overall status is one of fail, incomplete, pass_for_declared_scope, or not_applicable, and incomplete can never become a pass. An exception is recorded beside a failure. It never rewrites one.

What I did

  • Built the ten-phase audit workflow. Each phase writes a durable, schema-checked artifact before the next may start. Init records what is not yet known. Scope resolves which obligations apply with a cited predicate. Verify runs in a context that did not write the repair. The workflow is the product.
  • Created the authority registry and seven rule packs: 159 rules across accessibility, federal public web, privacy, infrastructure, and service UX. 45 rules were reviewed against primary sources and promoted to draft. The rest stay at research. The skill says which is which.
  • Wrote the deterministic gate. It recomputes status from recorded artifacts. Exit 0 is pass for declared scope. Exit 1 is a confirmed mandatory failure. Exit 2 is incomplete. Exit 3 is invalid configuration. The gate never reads prose. It never guesses.
  • Designed the verifier-is-not-the-author pattern. The verify phase runs from frozen evidence in a context that did not write the repair. A green build is not proof that a barrier is gone. The separation is structural, not contractual.
  • Shipped the Rust companion binary. Its page assessor runs 28 deterministic checks over captured pages without a browser: 17 accessibility, 11 federal web. Counting the source, network, and infrastructure channels, the site's hero line puts it at 61 automated and assisted checks across 48 rules. Built in CI for macOS and Linux with checksummed releases. The Node package stays as the shadow reference the suites compare against.
  • Added storybook mode. The audit captures stories from a built Storybook and assesses them with element-level checks. Whole-page obligations are recorded as not_tested. The component gate answers the component question. The exit code is useful. The record is honest. Both hold.
  • Built source-pass template scanning for seven languages: JSX, TSX, Vue, Svelte, Astro, Twig, Liquid. A dependency-free masking pre-pass. Every finding carries line, column, and byte offset. Expressions are opaque and become needs_review. That is the honest state.
  • Shipped SARIF output, a GitHub Action, and baseline-diff. The Action audits one URL, downloads the binary, verifies the checksum, uploads SARIF to code scanning, and fails only on the condition you choose. A team can accept a backlog and block only new problems.
  • Built the marketing site at diligent.style. Astro static, Arc UI, D-tile mark. Product facts are read from the skill repo at build time. The build fails if the repo is missing. Analytics are off by default. The footer says so.

The move that mattered most was keeping the Node package honest. The obvious path after the Rust binary shipped was to say the Node package was retired. I did not. The binary is the default executor when present. Node is what the suites compare against. The marketing site carries the same distinction. A tool that audits government sites should not be ambiguous about its own claims.

Audit workflow · Ten phases

01 INITRUN.json02 DISCOVERPROJECT-CONTEXT.json03 SCOPEAPPLICABILITY.json04 INVENTORYINVENTORY.json05 PLANAUDIT-PLAN.json06 COLLECTEVIDENCE.jsonl07 ASSESSRESULTS.jsonl08 PROPOSEREPAIRS.json09 VERIFYVERIFICATION.json10 REPORTREPORT.mdthe verifier did not write the repair
Every phase writes a durable artifact before the next may start. The verifier runs in a context that did not write the repair. The boundary between propose and verify is the design, not a convention.

The decision that defined the product was what the skill would not do. Diligent will not certify, will not grant an authorization to operate, will not approve a privacy assessment, and will not decide eligibility. For generic accessibility linting it tells you to use a general accessibility tool. The refusals are why the claims it does make hold.

Deterministic gate · Exit codes

GATEfrozen records only0EXIT 0 PASSfor declared scope1EXIT 1 FAILmandatory failure2EXIT 2 INCOMPLETEevidence missing3EXIT 3 INVALIDinvalid configurationincomplete can neverbecome a pass
The gate computes status from recorded artifacts. It never reads prose and never guesses. Incomplete can never become a pass. An exception is recorded beside a failure and never rewrites one.

Reflection

The through-line of this build is that an audit tool for government should be inspectable, honest about its own limits, and structurally unable to turn silence into a pass. Every design decision traces back to those three commitments.

The shadow suite is the trust mechanism. The Rust binary and the Node gate run the same fixtures and must agree byte for byte. When they disagree, the Node gate is authoritative and the binary is wrong. That relationship is not a migration plan. It is a permanent verification contract. The binary is faster and runs the automated checks. The Node gate is what the suites compare against. Neither can drift without the shadow suite catching it.

A tool that audits claims should not make claims it cannot support. The rule corpus has two lifecycle states: research and draft. A rule at research is cataloged for review, not reviewed. A rule at draft was checked against its primary source with an exact clause locator. The skill will not promote a rule whose locator is still unatomized, even when the rule is clearly correct. Two rules stay at research because a single clause could not be isolated. The marketing site carries the same distinction. That restraint is the product.

Component mode taught me that a gate can answer a smaller question honestly. The first instinct when auditing a Storybook was to say the audit is incomplete because whole-page obligations were not tested. That is true and useless. The component gate answers the component question and exits on it. The assessment still records the site-level incompleteness. The exit code is useful. The record is honest. Both hold because they are separate artifacts.

I would reach for the companion binary sooner next time. The Node-first approach was the right call for the workflow design. The three-day sprint to a Rust binary was reactive, not planned. If I were starting again, I would design the artifact contract in Node and build the engine in Rust from day one, with the shadow suite as a first-class artifact rather than a proof added later.