Docs  →  Zero-Cost On-Device Architecture

Zero-Cost Infrastructure: Running AI Completely in Client Browsers

How Cloudflare Pages static hosting and an in-browser WebGPU LLM (WebLLM) combine into a fully serverless generative-AI product with zero egress and absolute privacy.

Category: Architecture & Privacy Reading time: ~12 min Level: Intermediate

1. Overview

Modern consumer AI products are almost universally built on a familiar but expensive pattern: an API gateway in front of an inference service, a queuing layer, GPU autoscaling, observability, and a billing loop. That model delivers extraordinary power, but it also delivers a per-request marginal cost that grows with success, and it concentrates every user's data in a server-side datastore. MooduTools was designed from the first line of code to invert that paradigm.

This document walks through the architecture that lets MooduTools operate as a generative product with zero persistent compute, zero model-inference cost, and zero network egress of user mood data. The entire intelligence payload — the model that turns words into moving art — is executed not on a rented GPU in a faraway region, but inside the visitor's own browser GPU via the WebLLM runtime, which loads a lightweight instruction-tuned model (Qwen2.5-0.5B-Instruct) over a CDN (esm.run) with no server, no API key, and no Chrome flag.

Key number: The operational cost difference between hosting one user and hosting one million users is effectively zero. The only moving part with any real budget is static file delivery, which is absorbed by a free static-hosting edge.

2. Why Traditional AI Cloud Infrastructure Feels Heavy

To appreciate the design, it helps to name precisely what a conventional stack adds. A typical "send a prompt to an LLM" product needs: an origin server to authenticate and rate-limit, a load balancer, autoscaling worker pods, an inference runtime with GPU capacity, persistent storage for conversation logs, a caching layer, and an observability/alerting pipeline. Each stage multiplies the cost curve and, more importantly for an emotional-expressiveness product, each stage is a place where user data could be inspected, logged, or breached.

MooduTools is deliberately small: a single page, a prompt, a stylesheet. There is no login session to protect, no conversation history to encrypt at rest, and no inference cluster to schedule. Removing the server does not mean removing engineering rigor — it means relocating trust to the browser's own security model, where data can be processed and immediately discarded without ever crossing a network boundary.

3. The Static Hosting Layer (Cloudflare Pages)

The only piece of the product that is actually hosted is the code itself. Cloudflare Pages serves the HTML, CSS, and JavaScript from its global edge network with a free tier and generous traffic allowance. Because every file is immutable and content-addressed on deployment, caching behavior is deterministic: the edge and the browser both cache aggressively, so repeat visits often cost nothing more than a round trip to the nearest edge node.

From a cost-accounting perspective, Pages is ideal because it bills on delivery, not compute. There is no always-on virtual machine and no concurrency ceiling tied to inference. The deployment pipeline is a simple git push that triggers a build, which in practice means a one-developer project can ship production features with the same discipline as a large organization's CD pipeline, but without a single cloud bill line item for model usage.

It is important to separate two concerns that many products conflate: caching the static shell and serving live compute. MooduTools uses the edge only for the former. The latter — token generation — never touches the edge at all. That division is the entire financial and privacy thesis of the system.

4. The Runtime AI Layer (In-Browser WebGPU LLM — WebLLM)

The second half of the architecture is a runtime that runs entirely inside the visitor's browser. The @mlc-ai/web-llm engine is pulled straight from a CDN (esm.run) with no build step, and it downloads a lightweight instruction-tuned model (we ship Qwen2.5-0.5B-Instruct) whose weights are stored once in the browser's Cache API and reused on every later visit. The model's forward pass runs on the device GPU through the WebGPU standard — no backend, no API key, no Chrome flag, and no browser-vendor lock-in:

// Zero-setup, cross-browser WebGPU engine — no build, no key, no flag.
const webllm = await import("https://esm.run/@mlc-ai/web-llm");

// Persist model weights in the browser Cache API (download once).
webllm.prebuiltAppConfig.cacheBackend = "cache";

const engine = await webllm.CreateMLCEngine(
  "Qwen2.5-0.5B-Instruct-q4f16_1-MLC",
  { initProgressCallback: (report) => setProgress(report.progress, report.text) }
);

for await (const chunk of engine.chat.completions.create({
  stream: true,
  messages: [
    { role: "system", content: SYSTEM_PROMPT },
    { role: "user",   content: MOOD_TEXT },
  ],
})) {
  stream(chunk.choices[0].delta.content);
}

Location is meaningful. Because the model runs on the visitor's own GPU within the browser, a prompt executes with inference happening with zero network round-trip, at zero token cost, and under the browser's origin security boundary. There is no API key, no egress firewall exception, and no server it can reach even if a prompt were maliciously crafted. This is fundamentally different from a thin-client chat product: here the weight, the reasoning, and the output all live where the user already is. And because the runtime is a portable WebGPU engine rather than a browser-bundled model, it works across any WebGPU-capable browser — zero setup, cross-browser by design.

5. The Zero-Egress Privacy Guarantee

Privacy claims in software are only meaningful when they can be reduced to facts about packets. MooduTools' guarantee rest on a verifiable property: a user's mood text and the generated artwork never appear in a network frame. Let us trace the packet inventory for a complete generation cycle.

  • Document load. The browser fetches HTML, CSS, JS, and fonts over HTTPS. This is identical to visiting any documentation page.
  • Prompt construction. The typed mood is concatenated into a system prompt string in memory. No AJAX, no WebSocket, no fetch. The strings exist only in the page's JavaScript heap.
  • Inference. Each token is resolved entirely by the local WebGPU model. No network request is initiated by the call.
  • Rendering. The returned HTML fragment is sanitized and injected into the DOM. Everything renders within the same origin.
Zero egress: precisely because no fetch(), XMLHttpRequest, or WebSocket is ever invoked with user content, there is no packet to intercept, no server log to subpoena, and no third-party SDK to leak through. The guarantee is structural, not aspirational.

This is where the architecture earns its privacy language in a way that a server-side product cannot. A remote LLM provider can promise encryption in transit and at rest, but those promises still route the plaintext through a party that is not the user. On-device inference removes the third party from the threat model by removing the transmission itself.

6. Lifecycle, Progress & Fallbacks

Because products must degrade gracefully, the integration distinguishes between "no WebGPU", "model weights not yet downloaded", and "model ready." The engine exposes an initialization callback so the UI can show a friendly, animated download bar instead of a frozen button:

const engine = await webllm.CreateMLCEngine(modelId, {
  initProgressCallback: (report) => {
    // report.progress ∈ [0,1]; report.text is a human-readable stage label.
    progressBar.style.width = `${Math.round(report.progress * 100)}%`;
  },
});

When WebGPU is unavailable on the visitor's machine, or when the model cannot be started, the product falls back to a deterministic offline mood engine — small hand-authored templates and text mixers that run the same tools with no model download. The user never sees a dead button; they see a graceful, explained transition between on-device-intelligence and offline modes. This fallback is not an afterthought, but a core part of the architecture, because it keeps the surface trustworthy and 100% functional even on browsers without WebGPU.

7. The Cost Model: From One User to a Million

Consider a conventional hosted AI product: at a modest token price and reasonable prompt length, 100,000 monthly active users might generate tens of thousands of dollars in inference spend. Add GPU autoscaling overhead and cold-start latency, and the unit economics quietly decide what features a small team can offer. MooduTools has a structurally different ledger.

  • Inference cost: $0.00. The user's own hardware performs every forward pass.
  • Compute staging: $0.00. There is no runtime, no container, no worker pool.
  • Hosting + egress: Static edge delivery on a free tier. Scaling is linear with bytes served, not with intelligence served.
  • Privacy risk surface: Effectively zero, because no PII or mood data is collected, stored, or transmitted server-side.

The result is that marginal cost per user approaches zero, which changes product strategy. Features can be shipped for generosity rather than gatekeeping, and the free tier is not a loss-leader — it is the whole product. For an indie project this is not merely efficient; it is liberating.

8. Security & Threat Model

A serverless client architecture does not remove security work — it moves it. The surface that matters now is the browser sandbox and the integrity of the served code. Three controls deserve emphasis.

  • Subresource integrity and a tight Content-Security-Policy. The page pins its own static assets so a compromised CDN cannot silently substitute a malicious script, and restricts sources so no unexpected domain can load code into the page.
  • Output sanitization. The model is treated as untrusted. Generated output is scrubbed of <script>, event-handler attributes, and URL schemes like javascript: before it is ever allowed near the DOM. (The rendering side is covered in detail in the companion document on the generative CSS art engine.)
  • No third-party SDKs. There are no analytics beacons, ads, or telemetry embeds in the runtime path. Fewer remote parties means a smaller supply chain and no silent data collector acting on the user's behalf.
A remote model provider could still, in principle, be compelled to log prompts. An on-device model removes that class of risk at the root: there is no remote provider holding plaintext, because there is no transmission.

9. Conclusion

MooduTools is evidence that a generative product does not have to rent intelligence. By serving static code from a free edge and letting a lightweight WebGPU LLM (WebLLM) do the reasoning inside the visitor's own browser, the architecture achieves what most AI startups measure but cannot honestly claim: zero marginal inference cost and zero egress of user emotions.

The trade-off is a dependency on browser capability distribution. Not every visitor's machine exposes a WebGPU adapter yet, so the product ships a deterministic offline fallback and keeps the interface identical in every environment. But for the growing population of WebGPU-capable browsers, the experience is indistinguishable from a hosted product with the privateness of a locally installed application. That inversion — where the most private option is also the most cost-effective option — is the quiet revolution this architecture represents.

Continue to the companion deep-dive to see how the raw text returned by the local model is turned into living, interactive HTML/CSS animation: From Text to Canvas: The Generative CSS Animation Pipeline.