Knowledge

The .rules/ format

Rules are plain Markdown in .rules/*.md, so people and coding agents read the same files Auditor checks. Each rule is a ### block with an ID in capitals, optional severity and applies lines, a description, and optional ok and bad examples.

### NO-TODO — Track work in issues, not comments
severity: warning
applies: **/*.{ts,tsx}

Don't leave TODO comments in new code; open an issue and link it instead.

```bad
// TODO: fix later
```
```ok
// Tracked in #123
```

Rules are read from the pull request's base branch, so a pull request can't loosen the rules it is checked against. A rule with its examples must fit in 3,200 characters. Rules that can't be read are listed in the Auditor check run. To give Claude Code the same rules while it writes, add @.rules/ to your CLAUDE.md.

Exceptions

List exceptions in .rules/ignore.md, one per list item: a - or * list marker, a rule ID (or * for every rule), a path glob in backticks, and a reason. Lines without a list marker are ignored.

# Exceptions
- NO-TODO `scripts/**` — one-off scripts (@dev, #412)
- * `src/generated/**` — codegen output

Paths that should never be reviewed at all go in Never review these paths on the Auditor page. Lockfiles, build output, minified and generated files, images, fonts, source maps, snapshots, vendor/ and coverage/ are always skipped.

Team rulesets

Team rulesets hold rules for many repositories at once. Write them on the Auditor page under Team rulesets, in the same format as a .rules/ block without its heading line: an optional # Title, the severity and applies lines, a description, and examples. Each rule's ID gets a team/ prefix, so it never clashes with a repository's own rules. When a review starts, the team rulesets that apply are checked together with the repository's .rules/*.md in the same Jev requests.

• Applies to: repository patterns such as inference-bureau/api or inference-bureau/web*, one per line. Empty means every repository Auditor reviews.
• Mode: Dry run checks the rules and records findings on the ruleset's page without posting them, so a new rule can be tried across repositories first. New rulesets start in dry run. Enforced posts findings like any other rule.
• Required: repositories in scope can't detach the ruleset, and neither .rules/ignore.md nor dismissing a finding turns its rules off. A team admin changes the ruleset instead. Optional rulesets can be detached per repository in its overrides, and excepted in ignore.md like a repository rule (for example - team/NO-SECRETS `test/**` — fixtures).

A team rule's finding says which ruleset it came from. The walkthrough comment ends with the rulesets the review checked, and the check run lists them with any dry-run findings. Each rule adds its text to every rules-pass request, so the ruleset page shows about what it adds per review.

Theme Store readiness template

Theme vendors can start a ruleset from Theme Store template under Team rulesets. It holds twelve rules for the Theme Store requirements that need judgement: alt text, form labels, visible focus, keyboard access, rel="nofollow" on links to Shopify, Shopify's payment icons, default header and footer menus, fonts through font_picker with font_modify variants, foreground colors for backgrounds, product page parts as blocks, no credits, affiliate links or app dependencies, and American English settings text. Set Applies to to your theme repositories; like every new ruleset it starts in dry run, and you can edit or delete any rule before saving. The requirements a file can show are checked without a model by Theme Store readiness.

Learnings

Auditor learns from what your team dismisses. Resolve an Auditor thread yourself while the flagged code is still there, react 👎 to its comment, or reply in the thread with pushback Auditor accepts or a pattern you say is intentional, and the finding is recorded as a learning for that repository and directory when the pull request is next reviewed. Resolving a thread after fixing the code is a fix, not a dismissal: Auditor marks the finding resolved and learns nothing from it. A rule dismissed twice in a directory stops being checked there, and the deep review is told not to report findings like the ones you dismissed. Learnings are listed on the Auditor page; Forget one to restore the old behavior.

Tuning

Learnings act on one rule in one directory. Tuning looks at a repository's whole record. Every finding that gets decided is either acted on (the flagged code changes, its suggestion is applied, or a coding agent marks it fixed with resolve_finding) or dismissed. By default, when at least 8 findings of one kind were decided in the last 60 days and fewer than 20% of them were acted on, Auditor stops posting that kind on that repository. A kind is one rule, or deep findings of Low severity, or of Medium severity.

Muted findings are still checked and recorded, and the check run says how many were held back. They are listed under Tuning on the Auditor page with the record that decided it; Restore one and it is posted again and never muted automatically. High-severity deep findings, static analysis and rules of a required team ruleset are never muted. Turn tuning off for the team with the switch in that section.

Help improve reviews

The findings your team decides are the best test of the reviewer: a fix means it was right, a thread resolved with the code left alone, a 👎 or a reply it conceded to means it was wrong. Help improve reviews, under Tuning on the Auditor page, lets such deep findings become cases in the tests Auditor's review quality is tuned against. It is off by default, and cannot be turned on while your Directives require zero data retention.

A case holds the hunk the finding sits in with some lines of context on both sides of the change, the file's name and the folder it is in, the finding's text with links, commit ids, mentions and your repository's and account's names removed, how it ended, and for a fixed finding the lines of the fixing commit that touch it. It never holds your repository's name, the pull request number, authors, the rest of the file or other files, and an excerpt that looks like it holds a credential is skipped. Nothing is copied at review time: the lines are read through the GitHub App when we build the test set, and the cases stay with us.

Documentation

Point Auditor at the documentation of the frameworks and platforms your repositories use, and the deep review knows how they behave. On the Auditor page, under Documentation, add a site's address. Auditor reads the pages at and below that address (it uses llms.txt or the sitemap when the site has one, and obeys robots.txt) and distils them into short facts: defaults, ordering, what fails silently, deprecated APIs. Each fact carries a few keywords, and a review gets the fact when the change mentions one of them.

A site can apply to every repository or to the ones matching a pattern such as inference-bureau/web*, and only to changes that touch certain paths (*.liquid, src/components/**). Open a site's facts to read them: Switch off any that are wrong, or Edit the wording and keywords. A fact you switched off stays off, and one you edited is kept, when the site is read again. When a review used a site, its check run says Documentation used with the site's name.

Sites are read again every 7 days by default; a page that did not change is skipped. Reading is billed to the Auditor key like a review, appears in Logs under session = the source's id, and stops at a budget per read ($2 by default). Only public HTTPS sites can be read. Change the schedule, the budget, or turn documentation facts off for the team under Deep review.

Built-in Shopify knowledge

Most of what goes wrong in Shopify code is only visible if you know the platform: a visible_if that hides a setting but keeps its saved value, a theme editor event that fires for every section, a Function reading an input field its query never asks for. So when a change touches Shopify code, the deep review's passes and its verifier get short lists of platform facts, chosen by the files the change touches and what they import, with no setup and nothing to turn on:

• Themes: Liquid, section and block schema, the theme editor, escaping, translation keys, metafields and images, and render cost (all_products lookups, snippets and filters inside loops).
• Theme blocks, as Horizon uses them: the blocks/ folder, @theme, private, static and nested blocks, and what renaming a block file does to placed content.
• Theme app extensions: app blocks and app embeds, @app, block handles and deep links, available_if and app metafields.
• Shopify Functions, checkout and customer account UI extensions, and apps (the Admin API, webhooks and authentication).

They are facts, not rules: nothing is posted because a fact exists, and each adds only a little to the requests of a review it applies to. The documentation sites you add work alongside them. Shopify themes also get checks that need no model.