Messages consist of "parts" (parts): text, model reasoning, tool calls, files, sources. Therefore, the same template shows both the regular answer and the reasoning from Claude or Qwen3, and the tool call.

The API key remains on the server. The server part is a regular Web-standard request → response handler that works in Nuxt/Nitro, Node, Bun, Deno, and edge runtimes:

// server/api/chat.post.ts (Nuxt) import { createLLMHandler } from 'vue-llm/server' import { createAnthropic } from 'vue-llm/providers' const handler = createLLMHandler({ provider: createAnthropic({ apiKey: process.env.ANTHROPIC_API_KEY! }), model: 'claude-sonnet-4-5', rateLimit: { maxRequests: 30, interval: 60_000 }, }) export default defineEventHandler(event => handler(toWebRequest(event)))

Built-in providers: OpenAI (and any OpenAI-compatible API), Anthropic, Google Gemini, Ollama. If you already have your own backend with streaming, createSSEProvider connects useChat to it: you describe how your SSE events are converted into answer fragments, and the rest works as usual.

Generative UI: the model responds with components

The most interesting part for me. When the model calls a tool, you can show your Vue component instead of text. Arguments are streamed into the component as they are generated, meaning the card starts filling before the model finishes writing the call.

import { defineTool } from 'vue-llm' export const getWeather = defineTool<{ city: string }, Weather>({ name: 'get_weather', description: 'Weather in the city', parameters: { type: 'object', properties: { city: { type: 'string' } }, required: ['city'] }, execute: ({ city }) => fetch(`/api/weather?city=${city}`).then(r => r.json()), })

The tool with execute runs on its own. The tool without execute pauses the chat and waits for a response from the interface — this is how human-in-the-loop works: the model suggests booking a table, the user clicks "Confirm" in the card, addToolResult passes the response, and the conversation continues. Live demo (works offline): https://vue-llm.js.org/ru/guide/generative-ui

Under the hood, we had to write a parser for "incomplete" JSON: {"city":"Мос превращается в { city: 'Мос' }, and incomplete keys are discarded until they are completed.

Models directly in the browser

createWebLLM launches open models (Llama 3.2, Qwen, SmolLM, Phi) on WebGPU via WebLLM. No server or key required, data never leaves the device.

import { CreateMLCEngine } from '@mlc-ai/web-llm' import { createWebLLM } from 'vue-llm/providers' const ai = createWebLLM({ createEngine: CreateMLCEngine, onProgress: ({ progress }) => console.log(`${Math.round(progress * 100)}%`), }) useChat({ provider: ai, model: 'Llama-3.2-1B-Instruct-q4f16_1-MLC' })

This is the same useChat as with OpenAI — the only difference is the provider. For example, you can provide users without WebGPU with a server model and others with a local one. The first model load takes up to 400 MB, after that it is fetched from the browser cache in a couple of seconds.

Ready-to-use chat interface in the shadcn style

Writing a chat window from scratch is tiring: auto-scroll that doesn't pull the user down when they scroll up; Enter for sending that doesn't trigger when typing in Chinese or Japanese; inserting images from the clipboard; drag-and-drop files. Therefore, the library is hybrid:

  • the npm package contains headless logic: usePromptInput and useAutoScroll;

  • ready components are copied into your project as in shadcn — with one command, after which your code:

npx shadcn-vue@latest add https://vue-llm.js.org/r/chat.json

Components use shadcn theme tokens, so they immediately look like the rest of your interface, including dark mode.

Debugging in Vue DevTools

vue-llm/devtools adds an LLM tab to Vue DevTools: a list of all useChat, tokens, time to first token, generation speed, tool calls and timeline for each request. Without the plugin, the library does not trace anything and does not spend resources on it.

Nuxt

The module is added with one command:

npx nuxi module add vue-llm

It does auto-import of composable functions and providers, registers components and automatically adds the tab to DevTools in dev mode. Tested on Nuxt 3 and 4.

Why not use a ready AI SDK

A good question, and I started with it myself. vue-llm is Vue-first: Nuxt module, Vue components for Generative UI, integration with Vue DevTools, provider for models in the browser, adapter for existing SSE backend and support for Vue 2.7 for composables (many legacy admin panels still use Vue 2). No runtime dependencies except Vue itself.

What's next

Library under MIT, version 1.3, covered with tests for Vue 3 and Vue 2.7. Planned: MCP client (connect MCP servers as tools), quoted responses for RAG and voice input.

I'd be happy to receive feedback: what is inconvenient, what is missing, where the documentation is unclear. Issues and PRs are welcome, and if the project is useful, please star it on GitHub, it really helps promote it.

  • Playground: https://vue-llm.js.org/ru/playground

  • Documentation: https://vue-llm.js.org/ru/

  • GitHub: https://github.com/dadashi44/vue-llm