Auditor

Setup

1. Open Team → Auditor in the sidebar and click Install on GitHub. Pick the account and the repositories the app can see. GitHub asks you to authorize the app so Inference Bureau can confirm the installation is yours. The app reads code, pull requests, issues and Actions logs and writes checks and comments; its write permission on contents and pull requests is used only by @auditor fix and by automations set to propose or push, and an installation made before that permission existed approves it in the app's settings on GitHub when you first need it.
2. Back on the Auditor page, turn on the repositories to review. Auditor is set up once per team: every repository shares the team's settings, a repository can override some of them in its settings (when reviews run, what re-reviews post, the deep review, its depth and models, PR descriptions, ticket checks and Theme Store readiness), and every review is billed to the team.
3. Add rules to the repository's default branch under .rules/ (the format), then open a pull request.

Turning on the first repository creates the team's Auditor API key, in an Auditor project of its own that is not listed with your projects. The key has no secret anyone can copy; it exists so Auditor's requests are admitted, limited and billed like any other key, under your team Directives. What that costs, and how to cap it, is on Costs and spend caps.

When reviews run

Draft pull requests are skipped unless Review draft pull requests is on. With Incremental review on, later reviews only check what changed since the last reviewed commit; after a rebase or force-push Auditor reviews the whole pull request again. Each repository can override the mode, whether the deep review runs, its depth and its models. Branches Auditor opens itself (auditor/fix-…, auditor/pull-…) are never reviewed automatically. To review changes before they are a pull request, use the Auditor CLI.

Skipping pull requests

Under Skip pull requests on the Auditor page's Reviews tab, list the pull requests no review should start on by itself: dependency bumps, release branches, work in progress. A skipped pull request gets no review, no check and no charge, and someone with write access can still comment @auditor review. The rules are the team's, for every repository.

To stop reviews of one pull request, comment @auditor pause: pushes to it are not reviewed until someone comments @auditor resume. Commands, @auditor review included, still work while it is paused.

Settings in the repository

A repository can keep its review settings in an .auditor.yaml file (or .auditor.yml) at its root, so a change to them is reviewed and merged like any other change. Keys are the settings' own names, nested as they are in the API:

trigger: push            # once | push | manual
reReviews: verify
failOn: medium           # never | high | medium
deep:
  depth: standard        # standard | thorough
  staticAnalysis: false
skip:
  branches:
    - dependabot/**
  titles:
    - WIP
pathIgnores:
  - "**/*.snap"
walkthrough:
  diagram: false
fix:
  target: branch

What the file sets wins over the team's settings and the repository's own on the Auditor page; anything it leaves out keeps those. Auditor reads it from the default branch whenever a push there changes it, so a pull request can't loosen its own review: a change to the file applies from the first pull request after it merges. The repository's settings on the Auditor page show what the file sets, and any line Auditor couldn't use.

Models, spend caps, sharing findings to improve reviews, ticket trackers and keys, documentation crawls, and everything Shopify (previews, theme documentation, Theme Store readiness, performance) can't be set in the file: they cost money or reach outside the repository, so they stay on the Auditor page. A line that sets one of them, names no setting, or holds a value Auditor can't use is skipped and the rest of the file applies. A pull request that changes the file gets a finding on each such line, so it can be fixed before it merges.

Merge checks

Merge checks are rules about the pull request itself, written in plain English: what its description must say, what has to come with a change to certain files, how its title reads. Add them under Merge checks on the Auditor page's Reviews tab, or in a repository's settings file:

mergeChecks:
  - name: Rollback note
    instructions: A change to a migration says in the description how to roll it back.
    paths:
      - migrations/**
    mode: error          # error | warning | off
  - name: Clear title
    instructions: The title says what changed, not only a ticket number.

Auditor judges them on every push and every edit of the pull request's title or description, whatever when reviews run says, and reports them as a check of their own, Auditor merge checks, apart from the review's. It fails when a required check fails; a warning, or a check Auditor couldn't decide from the pull request, leaves it neutral. Each check gets a one-line reason. Require Auditor merge checks in branch protection to block merging on them. Pull requests the skip rules leave out, and paused ones, get a skipped check, which branch protection accepts.

A pull request from a fork is held to more, since the person opening it picks its branch name and title: only the author skip rule skips its merge checks, and a required check Auditor couldn't judge (an unclear verdict, a spend cap reached, no allowed model) fails them instead of leaving them neutral.

Auditor reads the title, the description, the labels, the files changed and their diffs; it doesn't explore the rest of the repository, so write checks about what the pull request shows. Each run is one model call, billed to the team like a review and counted toward the spend caps.

Commands

Comment on a pull request, at the start of a line. The @ is optional: auditor review and /auditor review work too, and under a white label so does your own command word. Quoted lines are never commands. Commands work in every mode, on drafts too, from anyone with write access to the repository; anyone else gets a 👎 and nothing is billed. When a review started by a comment is done, Auditor replies in the conversation with what it found (or that it found nothing), what is still open, and a link to the check run.

Commands work in pull request comments, not in review threads. A reply under a finding is answered as a conversation instead: see Replying to a finding. Any other comment that mentions @auditor is a question: see Asking Auditor.

Asking Auditor

Mention @auditor anywhere in a pull request comment that isn't a command, and Auditor answers it: @auditor what else calls chargeCustomer?, is this migration safe to run on a live table, @auditor?. In the conversation, the answer is a new comment that quotes the question. On a line of the diff, or in a review thread someone else started, it is a reply in that thread, and "this" means the code there.

Answers come from the code. Auditor reads the pull request, its diff, the conversation so far and its open findings, then reads and searches the repository at the pull request's head before answering, and cites the files it relies on. It never changes code or settings from a question; to have findings fixed, use @auditor fix or reply "fix this" under one. Quoted lines don't count, so quoting a question doesn't ask it again.

Only people with write access to the repository can ask; anyone else gets a 👎 and nothing is billed. Each answer is billed to the team like a review, shows in Logs as session = athk_<repo>_<pr>, and counts toward the spend caps: once a cap is reached, Auditor says so instead of answering. Turn questions off with Answer questions under When to review on the Auditor page's Reviews tab, or chat: false in a repository's settings file.

Working on issues

On an issue, mention @auditor and say what you want, as you would to a teammate. Auditor reads the issue, its comments and the code, and does what you asked: @auditor how would you approach this? gets a plan (what changes, the files and functions, what it leaves out, how it is tested, and its questions); @auditor what calls chargeCustomer? gets an answer; @auditor fix this, keep the old endpoint gets the change, opened as a pull request that ends with Fixes # the issue. When it can't tell whether you want the change made, it plans and asks.

Each issue is one conversation: every mention continues it, so you can push back on a plan and then say go ahead. @auditor plan asks for a plan only (it changes nothing), and @auditor implement asks for the change. Adding the label auditor:plan to an issue plans it too. A change is checked in the same sandbox as fixes before its pull request opens, and Auditor reviews that pull request like any other.

It runs as your team's Issue assistant, an automation made the first time someone uses it. Its instruction, model and budget are yours to change on the Automations tab, where each issue's run shows with its whole conversation; turn it off and mentions on issues aren't answered. Only people with write access to the repository can ask, and every turn is billed like an automation run.

Analytics

The Auditor page's Analytics tab shows what Auditor does for the repositories you can see, over 7, 30 or 90 days: how often your team acts on findings, findings posted, cost per pull request and time to merge, each against the period before; findings by how they ended; What lands, each kind of finding by how often it is acted on, with the ones tuning muted; and every repository, or every author, with findings posted, resolved and open, reviews started automatically or asked for, commands, previews and cost.

A repository's page adds every way Auditor was started there (each event, each command, questions, replies, coding agents and the CLI), with who started it, what came of it and what it cost; the people who worked with it and what each opened, asked for, fixed, dismissed and taught it; a timeline of feedback, rules and tuning; every setting in effect with where it comes from (the team, the repository, or .auditor.yaml); the models used; and merge check results. Commands, previews, pull request authors and who decided a finding are counted from November 4, 2026.

The rest of these pages

• Reviews: the deep review, review depth, static analysis, coding agent instructions and the walkthrough comment.
• Findings and fixes: replying to a finding, Fix in your agent, @auditor fix, findings for coding agents over MCP, and stale comments.
• Costs and spend caps: what reviews and fixes cost, where to see it, and the monthly and per pull request caps.
• Knowledge: the .rules/ format, exceptions, team rulesets, learnings, tuning, documentation sources and built-in Shopify knowledge.
• Descriptions: generated pull request descriptions and tickets.
• Automations: an agent run on a schedule, a pull request or a push across the repositories you choose, with an instruction, a permission, outputs and a budget, and runs you can watch, send notes to and continue.
• Add-ons: white label.
• Auditor CLI: review local changes before you open a pull request, and set up repositories, theme previews and theme documentation from a terminal.
• MCP server: findings for coding agents on every repository Auditor reviews, and the Shopify store behind a theme repository.
• For repositories that hold a Shopify theme: theme previews, theme documentation, and Shopify checks (merchant content on the store, translation keys and Theme Store readiness).