AI Plugin

The aiChat Elysia plugin is the server-side entry point for AI streaming. It creates WebSocket and REST endpoints, manages conversations, and orchestrates provider streaming with tool execution.

#Basic Setup

At minimum, aiChat needs a provider factory. Everything else has sensible defaults.

import { aiChat } from '@absolutejs/ai';
import { anthropic } from '@absolutejs/ai/anthropic';

new Elysia().use(
  aiChat({
    // WebSocket path (default: '/chat')
    path: '/chat',

    // Provider factory : receives provider name, returns config
    provider: (providerName) =>
      anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }),

    // System prompt sent with every request
    systemPrompt: 'You are a helpful assistant.',

    // Maximum tool-use turns before stopping (default: 10)
    maxTurns: 5,
  })
);

#Full Configuration

A production setup with multiple providers, dynamic tool and thinking configuration, custom message parsing, and completion callbacks:

import { aiChat } from '@absolutejs/ai';
import { anthropic } from '@absolutejs/ai/anthropic';
import { openaiResponses } from '@absolutejs/ai/openai-responses';

const getProvider = (name: string) => {
  switch (name) {
    case 'anthropic':
      return anthropic({ apiKey: process.env.ANTHROPIC_API_KEY });
    case 'openai':
      return openaiResponses({ apiKey: process.env.OPENAI_API_KEY });
    default:
      throw new Error(`Unknown provider: ${name}`);
  }
};

new Elysia().use(
  aiChat({
    path: '/chat',
    provider: getProvider,

    // Model can be static or dynamic per provider
    model: (providerName) => {
      if (providerName === 'anthropic') return 'claude-sonnet-4-6';
      return 'gpt-4o';
    },

    // Tools can vary by provider/model
    tools: (providerName, model) => {
      return TOOL_CAPABLE_MODELS.has(model) ? myTools : undefined;
    },

    // Extended thinking : static or per provider/model
    thinking: (providerName, model) => {
      return THINKING_MODELS.has(model)
        ? { budgetTokens: 8000 }
        : undefined;
    },

    // Custom message parser for provider:model:content format
    parseProvider: (raw) => {
      const first = raw.indexOf(':');
      const providerName = raw.slice(0, first);
      const rest = raw.slice(first + 1);
      const second = rest.indexOf(':');
      return {
        content: rest.slice(second + 1),
        model: rest.slice(0, second),
        providerName,
      };
    },

    // Called when a response completes
    onComplete: (conversationId, fullResponse, usage) => {
      console.log(`[${conversationId}] ${usage?.inputTokens}in / ${usage?.outputTokens}out`);
    },
  })
);

#Config Options

providerrequired(name) => AIProviderConfig
Factory function that receives a provider name and returns a provider config. This is the only required option.

modelstring | (name) => string
Static model name or function to select model per provider. If omitted, the client must specify the model.

toolsAIToolMap | (name, model) => AIToolMap
Static tool map or function returning tools per provider/model. Return undefined to disable tools for a model.

thinkingboolean | { budgetTokens } | fn
Enable extended thinking globally, with a token budget, or dynamically per provider/model.

parseProvider(content) => { content, model, providerName }
Custom parser to extract provider, model, and content from the raw message string sent by the client.

onComplete(conversationId, response, usage) => void
Callback fired when a response finishes streaming. Useful for logging token usage or saving to a database.

pathstring
WebSocket and REST endpoint path. Defaults to '/chat'.

maxTurnsnumber
Maximum tool-use rounds before the plugin stops the loop. Defaults to 10.

#Configuration Type

type AIChatPluginConfig = {
  path?: string;
  provider: (providerName: string) => AIProviderConfig;
  model?: string | ((providerName: string) => string);
  tools?:
    | AIToolMap
    | ((providerName: string, model: string) => AIToolMap | undefined);
  thinking?:
    | boolean
    | { budgetTokens: number }
    | ((
        providerName: string,
        model: string
      ) => boolean | { budgetTokens: number } | undefined);
  systemPrompt?: string;
  maxTurns?: number;
  parseProvider?: (content: string) => {
    content: string;
    model?: string;
    providerName: string;
  };
  onComplete?: (
    conversationId: string,
    fullResponse: string,
    usage?: AIUsage
  ) => void;
};

#Endpoints

The plugin automatically creates these endpoints under the configured path (default /chat):

Method	Path	Description
WS	`/chat`	Main streaming endpoint. Handles messages, cancellation, and branching.
GET	`/chat/conversations`	List all conversations sorted by most recent.
GET	`/chat/conversations/:id`	Get a single conversation with its full message history.
DELETE	`/chat/conversations/:id`	Delete a conversation and all its messages.

By default, conversations are stored in memory. This works for single-instance deployments but breaks when you run multiple server instances behind a load balancer. Pass a custom store to persist conversations to Redis, Postgres, or any backend.

type AIConversationStore = {
  get: (id: string) => Promise<AIConversation | undefined>;
  getOrCreate: (id: string) => Promise<AIConversation>;
  set: (id: string, conversation: AIConversation) => Promise<void>;
  list: () => Promise<AIConversationSummary[]>;
  remove: (id: string) => Promise<void>;
};

The default memory store:

import { aiChat, createMemoryStore } from '@absolutejs/ai';

// Default : same as not passing store at all
new Elysia().use(
  aiChat({
    provider: getProvider,
    store: createMemoryStore(),
  })
);

A Redis implementation for multi-instance deployments:

import { aiChat } from '@absolutejs/ai';
import type { AIConversationStore } from '@absolutejs/absolute';
import { Redis } from 'ioredis';

const redis = new Redis(process.env.REDIS_URL);

const redisStore: AIConversationStore = {
  get: async (id) => {
    const data = await redis.get(`conv:${id}`);
    return data ? JSON.parse(data) : undefined;
  },
  getOrCreate: async (id) => {
    const existing = await redis.get(`conv:${id}`);
    if (existing) return JSON.parse(existing);
    const conv = { id, messages: [], createdAt: Date.now() };
    await redis.set(`conv:${id}`, JSON.stringify(conv));
    return conv;
  },
  set: async (id, conversation) => {
    await redis.set(`conv:${id}`, JSON.stringify(conversation));
  },
  list: async () => {
    const keys = await redis.keys('conv:*');
    const convos = await Promise.all(
      keys.map(async (key) => {
        const data = JSON.parse(await redis.get(key) ?? '{}');
        return {
          id: data.id, title: data.title ?? 'Untitled',
          messageCount: data.messages?.length ?? 0,
          createdAt: data.createdAt, lastMessageAt: data.lastMessageAt,
        };
      })
    );
    return convos.sort((a, b) =>
      (b.lastMessageAt ?? b.createdAt): (a.lastMessageAt ?? a.createdAt)
    );
  },
  remove: async (id) => { await redis.del(`conv:${id}`); },
};

new Elysia().use(
  aiChat({ provider: getProvider, store: redisStore })
);

#HTMX Support

Enable htmx: true to add SSE-based endpoints alongside the WebSocket ones. HTMX clients can stream AI responses with zero JavaScript. Just use hx-post and sse-connect.

import { aiChat } from '@absolutejs/ai';
import { anthropic } from '@absolutejs/ai/anthropic';

new Elysia().use(
  aiChat({
    provider: () => anthropic({ apiKey: process.env.ANTHROPIC_API_KEY }),
    systemPrompt: 'You are a helpful assistant.',
    htmx: true,
  })
);

Customize the HTML fragments with render functions:

new Elysia().use(
  aiChat({
    provider: getProvider,
    htmx: {
      render: {
        chunk: (text, fullContent) =>
          `<div class="prose">${markdownToHtml(fullContent)}</div>`,
        thinking: (text) =>
          `<details class="think"><summary>Reasoning</summary>${text}</details>`,
        toolRunning: (name) =>
          `<div class="tool"><span class="spinner"></span> ${name}</div>`,
        toolComplete: (name, result) =>
          `<details class="tool done"><summary>${name}</summary><pre>${result}</pre></details>`,
        complete: (usage, durationMs, model) =>
          `<footer>${model} · ${usage?.outputTokens} tokens · ${(durationMs! / 1000).toFixed(1)}s</footer>`,
      },
    },
  })
);

PreviousProviders NextTools