Exposing a Local Ollama to a Vercel App: A 90-Line Token Gateway and a Cloudflare Quick Tunnel

This is for anyone who wants a hosted web app to call a model running on their own hardware. RPOMS AI is a Next.js app on Vercel. Its vision model, Qwen2.5-VL 7B under Ollama, cannot run there: Vercel functions cannot hold a multi-gigabyte model, and the project's rule is zero mandatory AI cost. So the model runs on a machine we control, and the site calls it over HTTPS.

The obvious move is to tunnel Ollama's port, 11434, to the internet. That would be a mistake. Ollama has no authentication, so an exposed port gives free GPU time, and every model on the machine, to whoever finds the address. The design puts a small gateway in front and points the tunnel only at the gateway.

The gateway is one Node file of under ninety lines with no dependencies beyond Node itself. It binds to 127.0.0.1, so the only way in from outside is the tunnel. For each request it does the following, in order:

1. Checks Authorization: Bearer <token> with a constant-time comparison, and answers 401 otherwise. It refuses to start at all if the token is shorter than 32 characters.
2. Forwards exactly two routes: GET /api/tags and POST /api/chat. Everything else gets 404. Pulling, deleting or creating models is simply not reachable.
3. Allows only the models named in ALLOWED_MODELS. A chat request for any other model gets 403, and even the tags list is filtered, so the caller cannot learn what else is installed.
4. Caps the request body at 8 MB and forces stream: false.
5. Runs chat requests one at a time.

const expected = Buffer.from("Bearer " + TOKEN);
const authorised = (h) => {
  const got = Buffer.from(h || "");
  return got.length === expected.length && timingSafeEqual(got, expected);
};

let busy = Promise.resolve();
const serial = (fn) => {
  const run = busy.then(fn, fn);
  busy = run.catch(() => {});
  return run;
};

The serial queue is deliberate. The host is a laptop with an integrated GPU that is already saturated by one analysis. Two in parallel would both slow down and both miss the timeout. Queuing at least lets the first one finish. The cost is written down: two people submitting together means the second waits about 40 seconds for the first and will usually time out.

The tunnel is a Cloudflare quick tunnel from cloudflared, which is free and needs no account: cloudflared tunnel --url http://127.0.0.1:8787. It prints a random HTTPS address that changes on every restart. A named tunnel would give a stable address but needs a domain on Cloudflare, so it is the planned upgrade, not the current setup.

One command, npm run inference, starts whatever is not already running: Ollama, then the gateway, then the tunnel. It then prints the two values to paste into Vercel, the endpoint URL and the token. The token is generated once with 32 random bytes and saved to a gitignored file, written with mode 0600. So after a restart only the address changes, and the token in Vercel stays valid.

The application enforces the same rule from its side. A vision endpoint that is not localhost is ignored unless it is HTTPS and a token is configured:

const local = /^http:\/\/(localhost|127\.0\.0\.1)(:\d+)?/.test(url);
if (!local && (!token || !url.startsWith("https://"))) return null;

Three hops can fail, and each needs a different fix. "AI offline" as a single red dot sends you to the wrong one. The provider's diagnosis reads the answer to one GET /api/tags through the gateway:

The gateway already distinguishes these cases on the wire, returning 502 when it cannot reach Ollama. The admin page simply reads them off it, and shows the detail line without ever including the token.

On 20 September 2026, probes against the live tunnel gave these results. No token: 401. A wrong token: 401. The correct token: 200, listing the allowed model only. A path the gateway does not forward: 401, because the token check comes first. Port 11434 was unreachable from the internet. A real inspection then went end to end through the browser, the Vercel function, the tunnel, the gateway and Ollama.

The limits are the honest part. Inference exists only while the laptop is awake and the command is running. The address changes on restart. One analysis runs at a time. RPOMS AI has no model activated in production and has not been used on real line photos, so this path has carried test traffic only. For the speed of the host behind the gateway, see Running Qwen2.5-VL 7B on an Intel Iris Xe. For why a full-size photo needed capping to fit the time budget, see the 896 px article.