What CI/CD for Shopify should look like, and what we've built so far

For most web apps, CI/CD is a solved shape. Each pull request runs its tests, gets a preview deployment and has its database migration checked, and a merge ships it. The repository holds nearly everything the app is, so checking the repository checks the change.

Shopify doesn't work that way. A theme's code is in git, but much of what a merchant sees isn't. Merchants place sections and set options in the theme editor, assign products to alternate templates in the admin, and let Shopify's AI write blocks. All of that is stored on the store, keyed by ids the theme's code defines, and most repositories keep it out of git with .shopifyignore. A Shopify app is similar: its code is in the repository, but the records it has already written are in production.

So a pull request can pass every test, look tidy in review and still wipe out a merchant's homepage. This is the gap we've been working on, and here's what we think closing it takes.

A preview on a real store, with real content

Reviewers should click through a change, not read a Liquid diff. That means every pull request needs its own theme on a real store, updated on every push, with the link in the pull request.

A preview built from the branch alone isn't enough, though. Without the merchant's templates and settings it shows default content and looks broken, so nobody trusts it.

What we built: Auditor deploys each reviewed pull request to its own unpublished theme and copies the content files the repository ignores from the published theme, so the preview shows the merchant's real homepage with the pull request's code. The pull request links the preview and the theme editor, and a render check confirms the main pages load with no Liquid errors. Previews are never published and are deleted when the pull request closes. More on theme previews.

Checks against what the store uses

Removing a section, a block type, a setting id or an alternate template can drop content merchants placed on the live store. The diff can't show this, because the content isn't in the repository. The only way to know is to ask the store.

What we built: when a pull request removes or renames something a store's content can depend on, Auditor reads the published theme for exactly that thing and comments on the line with what would be lost, such as "12 products use product.bundle; removing it falls them back to product." It never writes to the store and calls no model, so it costs nothing. More on the merchant content check.

A branch that keeps up with the store

Merchants and Shopify's AI keep editing the live theme, so the default branch falls behind the store. The next deploy from that branch can quietly undo their work.

What we built: after a preview deploy, Auditor compares the branch with the published theme and lists the templates, section groups and settings that are missing or different. Comment @auditor pull and it opens a pull request into your branch with the store's versions, so the store's changes go through review like anyone else's.

Review that knows Shopify

A general code reviewer reads Liquid as text. A Shopify reviewer should know what the platform does with it.

What we built: Auditor flags translation keys that aren't in the theme's default locale before the storefront shows "translation missing", and checks theme blocks and theme app extensions the way Shopify runs them. Theme vendors can turn on Theme Store readiness to check a theme against the Theme Store requirements on every review. Auditor follows your own rules in AGENTS.md, CLAUDE.md and Cursor rules, and its MCP server gives your coding agent the same store reads, so it can check what a change would remove before it opens a pull request.

Deploys that show what they'll do to production data

For a Shopify app the risky part of a deploy is often the data: a removed field that still holds values, or a new unique rule that existing records break. A migration that fails halfway through a deploy is a bad time to learn that.

What we built: in Apps, a pull request into a branch an environment follows gets a check saying what deploying it would do to production: the files it changes, each model change and how many of production's records it reaches. A change that existing records would break fails the check, before the merge. Web apps can also get a preview environment for each pull request, with records that start empty or as an anonymized copy of production's. Teams that keep their own CI can push and deploy with a deploy token from a GitHub Actions workflow.

What we haven't built yet

This is a start, not a finished list. Some limits are Shopify's: a store holds a limited number of themes, so a repository keeps five preview themes at once by default. Others are ours. Pull requests from forks are skipped, and the checks cover the themes and apps we've seen most, so a theme built very differently may find gaps.

If your Shopify workflow has a step you still do by hand, we'd like to hear about it. Vote on or suggest it on the roadmap, or tell us from the Beta badge in Apps.