A request can stop being visible before it stops being billable.
An inference gateway has to connect the API that clients already target with routing, model availability, payment, and accounting. Streaming makes that relationship structurally interesting: the request can stop being visible to the client well before the system has finished accounting for it, and the moment of disconnection is precisely the moment the usual accounting path stops working.
Phantom is an OpenAI-compatible gateway with prepaid keys. It routes inference through Vercel AI Gateway and uses PostgreSQL for the application's data layer.
The request lifecycle
Start with a familiar contract
The OpenAI-compatible interface means existing clients require no modification to route through the gateway. Behind that contract, the application verifies the prepaid key and confirms available budget before any upstream call is made, so an exhausted key fails at the gateway rather than after incurring cost.
Keep metering attached to the stream
A completed response reports upstream usage that can be reconciled directly. An interrupted response never produces that report, and it still consumed inference. Phantom captures usage while tokens are arriving rather than after the response completes, and defines explicit settlement behavior for disconnects, falling back to estimation when final upstream usage is unavailable and flagging the estimate as such.
The demonstration above illustrates both paths. Its example request and receipt are synthetic; it is not making a paid inference call.
Keep money representable
Billing uses integer micro-USD throughout, which removes floating-point accumulation error from the ledger entirely rather than managing it. The product also includes payment handling, webhook validation, key recovery, and a model catalog synchronized against the upstream gateway. These boundaries carry the same failure cost as the visible chat interface and receive the same attention.
What I owned
I built the gateway, the streaming metering path, the payment integration, the model directory, and the product interface. The work includes disconnect handling and the operational checks that keep the application's assumptions aligned with both its database and the upstream contract it depends on.
Architecture changes over time
Earlier versions explored confidential-compute backends. The current version routes through Vercel AI Gateway. I state that distinction explicitly because a historical architectural property should never be allowed to become an implied promise about the product as it exists today, and the drift from one to the other happens without anyone deciding it should.
The current gateway consolidates routing and model discovery behind a single upstream contract. Changes to that contract are part of what the application has to track.
Verification has levels
The project's feature inventory distinguishes behavior exercised live in production, behavior covered by tests, and paths that still require end-to-end verification. A route existing in the codebase is a materially different claim from a funded request completing successfully through production, and collapsing the two is the most common way a project's documentation stops being true.
That distinction now shapes how I think about product ownership generally: the interface, the implementation, and the evidence that the two agree have to stay aligned, and the third one degrades first.