Key takeaways
- With Preview environments on, each pull request into a followed branch gets pr-<number>: a development environment that follows the pull request's branch, with its address on the pull request.
- Its records start empty, or as a copy of production's taken when the pull request opens, with email, password, encrypted and name-like fields replaced or dropped. A copy takes up to 50,000 records and 2,000 files.
- Merging or closing the pull request deletes the preview. Up to five can be open at once, pull requests from forks get none, and previews are for web apps only.
- Previews run with development's variables, bill only their dev server hours, and are reachable by anyone with the address, so the app's own sign-in is what protects them.
The weeks before Black Friday are when a team most wants to ship a fix and least wants to break production. The usual safeguard is someone clicking through the change before it merges. But on a shared development environment, with a handful of hand-made records, that click-through doesn't tell you much.
Web apps on Inference Bureau can now give each pull request an environment of its own, with a copy of production's records in it and the personal fields anonymized. This article covers what that copy contains, what it changes, and where its limits are. Following a branch covers the setup it builds on.
Turning previews on
Previews build on an environment that follows a branch. Once one does, open Settings → Git, go to Pull requests and turn on Preview environments. They're off by default. There are two settings:
- Records: start empty, or as a copy of production's records. Empty is the default.
- At once: how many previews can be open together, from 1 to 5. The default is 3.
Each preview is one of the app's development environments, and an app can have ten, development included. If your app already has several, the form shows how much room is left.
Open, push, close
When someone opens a pull request into a followed branch, Inference Bureau creates pr-<number>. It starts as a copy of the files in the environment that follows the base branch, then follows the pull request's own branch. The first sync brings in the branch's files, and every later push syncs it again, like any followed environment.
Its address follows the usual pattern for environments:
https://<app-host>--pr-42.inferencebureau.app
The address shows on the pull request as a GitHub Deployment. The deploy-plan check's summary carries it too, with what the copy brought, for example "with 3,214 of production's records (4 fields anonymized)". The Previews table on Settings → Git lists the open ones, with each one's last sync and check.
Merging or closing the pull request deletes the preview. Its files go, its dev server stops and its address stops answering. Its GitHub Deployments are marked inactive. Delete in the Previews table removes one early, and the pull request's next push makes it again.
Some pull requests get no preview. The check's summary says why:
- The limit is reached. Closing another pull request frees a place, and the next push to this one creates its preview.
- The branch is in a fork. Its code would run with the app's variables, so it never gets a preview. The deploy-plan check still runs.
- The app has no room for another development environment.
What the copy brings, and what it changes
With Records set to production, the preview's records are copied once, when the preview is created. Later pushes sync code, not data. Production isn't changed by the copy.
Each copied record gets a new id, and links follow the copies, so a review still points at its product. Vectors come along with their records. Files that records name are copied into the preview's own storage, so deleting the preview never touches production's files.
The fields anonymized are picked from production's schema:
- every
email,passwordandencryptedStringfield - text, rich text and url fields whose names look like personal details: email, phone, mobile,
name, first, last, full, author or customer name, address, street, city, zip, postal and birth
Here's what each kind of field becomes:
- Passwords and encrypted strings are dropped.
- Email fields become a stand-in address at example.com.
- URLs become a stand-in under
https://example.com/. - Other text fields become the field's name followed by a short hash, such as
customerName 3f9a1c0b7e42. - Vectors embedded from an anonymized field aren't copied.
Within one copy, the same value always gets the same stand-in. Two orders from one customer still share an email, so grouping and lookups behave as they do in production. Each copy uses a new random salt, so stand-ins don't match between previews.
What the copy doesn't change
Anonymizing by field name is a heuristic, and it can miss things:
- A text field called
notes,messageorbiois copied as it is, whatever people typed into it. - A
jsonfield is never anonymized, whatever it's called. - Uploaded files are copied as they are.
- A field named exactly
nameis always anonymized, even on a model where it holds a product's name.
Before turning the copy on, read your models with this in mind. If a model holds free text you wouldn't want on a preview, keep Records empty. Or use Copy from production on a development environment's Settings → Backups, where you choose the fields to anonymize yourself.
There are limits, too. A copy takes at most 50,000 records and 2,000 files. Past that, the preview is still created, but with no records, and the check's summary says why.
Who can open a preview
A preview's address is reachable by anyone who has it, like any development environment's. Inference Bureau doesn't put a sign-in in front of it. The app's own sign-in, and its permissions.ib.ts, decide what a visitor sees. If your app has pages that work signed out, they work signed out on the preview as well.
A preview's variables are development's, never production's. Payment, email and other third-party keys are whatever development uses, so it's worth confirming those are test keys before you turn previews on.
Shopify apps get the check, not a preview
Previews are for web apps only. A Shopify app's preview would need its own Shopify app and its own install on a development store for every pull request, and we haven't built that. Turning previews on for a Shopify app is refused with that reason.
Shopify apps do get the deploy-plan check on every pull request. It runs before the merge and fails when a change would break records production already holds. For hands-on testing, an admin can create a named development environment, which gets its own Shopify app, and point it at a branch.
What it costs
Previews are development environments, and development's runs, requests and storage aren't billed. Their dev servers' awake hours count toward the team's dev server hours: 100 a month on Free, 150 on Pro and 1,000 on Scale, then $0.25 an hour on Pro and Scale. A preview's dev server sleeps when nobody uses it, as other development environments do, and wakes on the next request.
Frequently asked questions
Does a push to the pull request copy production's records again?
No. The records are copied once, when the preview is created. A push syncs the branch's files. Deleting the preview from the Previews table and pushing again makes a new one with a fresh copy.
Which fields are anonymized?
Every email, password and encrypted field, and text, rich text and url fields whose names look like personal details, such as phone, address, city or customerName. Other fields, json fields and uploaded files are copied as they are.
Can Shopify apps have preview environments?
Not yet. Each one would need its own Shopify app and store install. Shopify apps get the deploy-plan check on every pull request instead.
Is a preview's address private?
No. Anyone with the address can reach it, as with any development environment. The app's own sign-in and permissions decide what they see.
Do pull requests from forks get a preview?
No. A fork's code would run with the app's variables. The deploy-plan check still runs on them.