An “offline AI” switch hides several different questions: is a model present, are its companion files present, and is the runtime actually using it? Treating those as one boolean produces confusing settings and difficult support reports.
The local-model catalog in GPT Realtime Assistant makes those distinctions explicit. This article focuses on the inspected download service rather than making a latency claim about every device or language.
Model identity is more than a display name
RealtimeDownloadableModel entries identify a repository, a remote filename, a destination filename, and an installation kind. The catalog includes Whisper Large v3 Turbo variants and Qwen 3 8B variants with different quantizations.
That metadata is operational configuration. “Qwen” is not enough information to select a compatible file or explain its storage cost. The precise artifact matters, and companion artifacts can matter too.
Installed and selected answer different questions
The status builder resolves the runtime’s chosen Whisper, Qwen, and Qwen-ASR paths separately. For each catalog entry it inspects the local file size and checks that required companion files exist. It reports isInstalled separately from isSelected.
| State | Evidence used by the status builder | Limitation |
|---|---|---|
| File present | Filesystem metadata | Presence does not prove valid model contents |
| Companions present | Each required companion has metadata | Still not an inference test |
| Selected | Runtime path equals model path | Does not prove the runtime has loaded successfully |
| Ready for a conversation | Requires runtime/session checks beyond this catalog | Cannot be concluded from a download progress bar |
This avoids a common UI trap: showing a model as active simply because a download finished.
Stage large downloads separately
The downloader is an actor. It creates the model destination directory and uses a .part path while transferring a file. It reports downloaded bytes, expected bytes when known, speed, and an estimated remaining duration.
Progress remains below completion while transfer work is still in progress. When expected size is unavailable, the UI must tolerate uncertainty rather than inventing an accurate-looking completion time.
The downloader processes companion files after the primary model. That ordering reinforces why “main file finished” cannot be the only readiness signal.
Do not confuse a local pipeline with the cloud voice path
A local speech-recognition model and a local language model are distinct dependencies. Cloud live voice and local inference also have different startup and failure behavior. The useful interface makes the selected path and missing assets visible rather than silently switching providers.
For regression testing, start with a missing companion, an interrupted .part download, a selected file that has been moved, and two installed quantizations. Assert the displayed state, then separately run a synthetic inference check. File length checks are not cryptographic integrity checks; the code inspected here should not be described as providing such verification.
Product demos: train timetable photo Q&A and live voice context
These short demos show the app's photo-question and live voice workflows in action. They are not tests or benchmarks of the local Mac model pipeline described above.
Asking about an illustrative train timetable (connections, departure times, and platforms)
When a station departure board or timetable has multiple connections and platform changes, attaching a photo and asking a direct route question is often faster than scanning rows manually:
Live translation, lecture tasks, and interview preparation
The scenes and presenter voices are AI-generated. The timetable is illustrative; the route answer (connections, departure times, and platforms) comes from a real app test with an attached sample timetable image and is edited for time. Some on-screen text in the first demo was adapted.
For shorter cuts, watch the 13-second train timetable edit, the family translation Short, or the technical interview Short, and see our guide on asking AI about a photo or screenshot.
The model lifecycle is a first-class part of a native AI product. It determines whether “offline” is an understandable capability or an ambiguous toggle. GPT Realtime Assistant’s comparison pages describe the product-level choices around that architecture.
Available on GPT Realtime Assistant on the App Store.
Top comments (0)