Introducing Axion: multi-model ensemble

Every model makes mistakes. That's not a flaw — it's a statistical reality. Given the same prompt, two models will disagree on roughly 15–25% of factual questions, and the one that's right isn't always the biggest or most expensive one. Axion exists because we wanted Cyanix Intelligence to be more reliably right, without simply scaling up to a single massive model.

The core idea is old: ensembles beat individuals. Random forests, boosted classifiers, mixture-of-experts — the pattern shows up everywhere in ML. We just applied it at the inference layer for a live chat product, using Groq's low-latency API to make the fan-out fast enough that users don't notice the extra work happening under the hood.

Why an ensemble?

Before Axion, Cyanix ran a single-model pipeline. The model was good, but we saw a specific failure pattern: confident incorrectness. The model would produce a polished, fluent response that was quietly wrong about a specific detail — a date, a function signature, a nuanced technical trade-off. Users rarely caught it because the surrounding text was convincing.

We explored a few solutions:

• Bigger model — reduces error rate, but increases latency and cost substantially. Not practical for a free-tier chat product.
• Self-critique pass — ask the same model to review its own answer. Works sometimes, but the model is often blind to its own blind spots.
• Ensemble + synthesis — fan out to multiple models, then synthesise. Errors made by one model are usually absent in the others, so a synthesis pass can catch and correct them.

The third option won on all three dimensions that matter to us: accuracy, cost, and latency. Running all calls in parallel means even 3× the API calls can resolve in under 3 seconds total, because they overlap instead of stacking up.

Architecture

Axion runs entirely inside a Supabase Edge Function — a Deno V8 runtime that sits close to the database and handles the orchestration without a separate backend server. The function receives the user's message from Cyanix's frontend, manages the fan-out and synthesis, then streams the final response back.

Phase 1 — Fan-out

// Phase 1 — fan out to all models in parallel
const modelCalls = AXION_MODELS.map((m) =>
  fetch(`https://api.groq.com/openai/v1/chat/completions`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${GROQ_API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      model: m.id,
      messages: [
        { role: 'system', content: m.systemPrompt },
        ...conversationHistory,
        { role: 'user', content: userMessage },
      ],
      max_tokens: 1024,
      temperature: m.temperature,
    }),
  })
);

const results = await Promise.allSettled(modelCalls);
const drafts = results
  .filter((r) => r.status === 'fulfilled')
  .map((r) => extractContent(r.value))
  .filter(Boolean);

Phase 2 — Synthesis

Once all drafts are collected, Axion runs a dedicated synthesis call with Claude Haiku — it needs to read and reconcile potentially conflicting information, which is a harder reasoning task than producing the initial draft. The synthesis model receives the original user message and all drafts, and is instructed to:

• Identify where the models agree — these points are almost certainly correct
• Identify where they disagree — flag the conflict and pick the most defensible answer
• Produce a response in the style of a single, direct, confident assistant (no "Model A said… Model B said…" hedging)

const synthesisPrompt = `You are a synthesis engine.
You will receive a user question and ${drafts.length} draft responses from different AI models.
Your task: produce ONE definitive, accurate, well-reasoned answer.

Rules:
- Where models agree, that's your signal to be confident.
- Where models conflict, reason through which answer is most defensible and pick it.
- Do NOT mention the drafts, models, or that you are synthesising.
- Write as a single authoritative assistant. Be direct, not verbose.
- If none of the drafts are satisfactory, say so honestly.`;

const synthesisMessages = [
  { role: 'system', content: synthesisPrompt },
  {
    role: 'user',
    content: [
      `Original question: ${userMessage}`,
      ...drafts.map((d, i) => `Draft ${i + 1}:\n${d}`),
    ].join('\n\n---\n\n'),
  },
];

Model roster

We run three fan-out models plus one synthesis model. The roster changed several times during development — here's the current lineup and the reasoning behind each pick:

Streaming the result

One UX challenge with ensembles is that you can't stream during the fan-out phase — you're waiting for all models to finish before you can synthesise. We experimented with streaming model outputs as they arrived ("Draft 1 is ready…") but users found it noisy and confusing.

The current approach is a two-phase UX:

1. Show a "Axion is thinking…" indicator with an animated ring while the fan-out runs in the background (usually 1.5–2s)
2. Stream the synthesis response token-by-token once it begins, which feels instant and responsive

This means the perceived latency is roughly: fan-out time + synthesis time to first token. In practice, first token appears about 2.2 seconds after the user sends — which is acceptable and comparable to a single large model on a slower inference backend.

What we learned

Building Axion surfaced a few non-obvious lessons:

Diverse prompts matter more than diverse models. We initially put three different models on the same system prompt and got near-identical drafts. Differentiated per-model system prompts — asking each model to emphasise different things — produced much more useful disagreement for the synthesis pass to work with.

The synthesis prompt is the product. Axion's quality ceiling is set by how well the synthesis model is prompted. A weak synthesis prompt produces hedged, verbose output. A tight one produces something that noticeably exceeds what any single draft achieved.

allSettled is non-negotiable. Using Promise.all early in development meant a single model rate-limit hit would crash the entire ensemble. allSettled with graceful degradation to two or even one draft made the system robust against transient provider errors.

Token budget allocation is delicate. Fan-out models get 1024 tokens. Synthesis gets 2048. If you're too generous with fan-out tokens, you feed the synthesis model massive context and it loses focus. Too stingy, and drafts are truncated. We calibrated these limits over a week of dogfooding.

What's next for Axion

The initial Axion shipped as a "pick-winner" system — the synthesis pass could drop drafts entirely and just return the best one. The current version always synthesises. We're exploring a routing layer that detects query type and decides whether to run full ensemble (complex factual or reasoning tasks) or single-model fast path (casual chat, short creative tasks) — because not every message benefits from the overhead.

We're also looking at letting users toggle Axion on or off per conversation, with a visible quality indicator showing how much the synthesis diverged from the best individual draft. If the synthesis barely changed anything, that's a signal the task was simple. If it changed a lot, that's where Axion added the most value.

Axion is live in Cyanix Intelligence today. Try asking it something technical and see if you can catch it being wrong.