Key takeaways
- Inference Bureau is building hosted apps and cloud functions: run the code behind an app, a webhook or a scheduled job next to your models.
- Model calls made from that code are ordinary requests, so they follow the same Directives, budgets and Logs as the rest of your traffic.
- Development and production stay separate, each with its own data, settings and secrets, and changes move between them through a deploy you can review and roll back.
- It's on the roadmap as Soon; follow the roadmap item to hear when it opens.
Most AI features aren't just a model call. Something has to receive the webhook, run the job at 3am, keep a little data, call the model and do something with the answer. Today that usually means a second platform: a host for the code, a queue, a cron service, a secrets store, and a separate bill for each, all outside the rules you've set for your model traffic.
We're building hosted apps and cloud functions to close that gap. In the roadmap's words: run the code behind an app, a webhook or a scheduled job on Inference Bureau, next to your models and on the same bill.
Why next to your models
When the code runs where your models already run, a model call made from it is an ordinary request. It follows your team's Directives, counts against the same budgets and shows up in Logs with a receipt, like every other request. There's no second set of keys to rotate and no second place to look when spending jumps.
It also means one bill. The compute behind your app sits on the same invoice as the inference it uses, instead of a per-seat infrastructure plan you pay for whether the app is busy or not.
The problems we want to remove
- Hosting webhooks and scheduled jobs. Receiving an event or running something every hour shouldn't require standing up and watching your own servers or cron.
- Keeping development and production apart. Each should have its own data, settings and secrets, and work should move one way, through a deploy you can review and roll back.
- Re-solving the same plumbing. Hosting, queues and isolation are the same for every app. They should be building blocks you use, not projects you redo.
What we're not sharing yet
This post is deliberately light on details. We'll write up how it works, what it costs and how to get started when it's ready, rather than describe something that's still changing week to week.
Follow along
Update: this shipped as Apps, in beta. Getting started walks through a first app.