> ## Documentation Index
> Fetch the complete documentation index at: https://dev-doc.dualmindlab.tech/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming Protocol

> How DualMind Lab uses Server-Sent Events for real-time AI responses — SSE protocol, event types, and frontend integration.

## What is Streaming?

Streaming delivers AI responses incrementally as they're generated, rather than waiting for the complete response.

**User Experience:**

* Non-streaming: Wait 3 seconds → Full response appears
* Streaming: Text appears word-by-word as generated

**Why this matters:** Perceived latency is drastically reduced. Users see progress immediately.

## Why Server-Sent Events?

DualMind uses **Server-Sent Events (SSE)**, not WebSocket.

### SSE vs WebSocket

| Feature      | SSE                       | WebSocket                      |
| ------------ | ------------------------- | ------------------------------ |
| Direction    | Server → Client only      | Bidirectional                  |
| Protocol     | HTTP                      | Custom (upgrade from HTTP)     |
| Reconnection | Automatic                 | Manual implementation required |
| Debugging    | Standard browser DevTools | Specialized tools              |
| Complexity   | Low                       | Higher                         |
| Use Case     | Server pushes data        | Client-server chat             |

<Info>
  DualMind only needs **server-to-client streaming** during response generation. There's no client data to stream back during inference, making SSE the perfect protocol.
</Info>

### Why Not WebSocket?

WebSocket adds unnecessary complexity:

* Requires connection upgrade handshake
* No automatic reconnection on disconnect
* Harder to debug (non-HTTP protocol)
* Bidirectional capability unused

**DualMind principle**: Use simplest protocol that meets requirements.

## SSE Event Format

Events follow a strict format:

```
data: {JSON_PAYLOAD}\n\n
```

**Rules**:

* Each event prefixed with `data: `
* JSON must be single-line (no newlines in payload)
* Each event terminated with double newline (`\n\n`)
* UTF-8 encoding required

### Event Types

All events use `ai.stream.*` namespace:

<Accordion title="ai.stream.start">
  **Purpose**: Stream initialization, provides model metadata

  **Payload**:

  ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  {
    "type": "ai.stream.start",
    "model": {
      "name": "llama-3.3-70b-versatile",
      "provider": "groq"
    }
  }
  ```

  **When sent**: First event immediately after connection established
</Accordion>

<Accordion title="ai.stream.delta">
  **Purpose**: Incremental text chunk

  **Payload**:

  ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  {
    "type": "ai.stream.delta",
    "delta": {
      "text": "Quantum computing"
    }
  }
  ```

  **When sent**: Repeatedly as model generates text (0+ times)

  **Client handling**: Append `delta.text` to displayed response
</Accordion>

<Accordion title="ai.stream.done">
  **Purpose**: Stream completion with final metrics

  **Payload**:

  ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  {
    "type": "ai.stream.done",
    "usage": {
      "promptTokens": 15,
      "completionTokens": 120,
      "totalTokens": 135
    },
    "timing": {
      "responseTimeMs": 2145
    }
  }
  ```

  **When sent**: After final delta event, stream terminates
</Accordion>

<Accordion title="ai.error">
  **Purpose**: Stream failure notification

  **Payload**:

  ```json theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
  {
    "type": "ai.error",
    "code": "PROVIDER_ERROR",
    "message": "Groq API timeout after 45 seconds"
  }
  ```

  **When sent**: On provider failure, timeout, or other errors

  **Stream behavior**: Terminates after error event
</Accordion>

## Event Lifecycle

A successful stream follows this sequence:

<Steps>
  <Step title="Connection Established">
    Client opens HTTP connection to streaming endpoint with JWT authentication.
  </Step>

  <Step title="Start Event">
    Server sends `ai.stream.start` with model information.
  </Step>

  <Step title="Delta Events">
    Server sends 0+ `ai.stream.delta` events as text generates.
  </Step>

  <Step title="Done Event">
    Server sends `ai.stream.done` with usage and timing statistics.
  </Step>

  <Step title="Connection Closes">
    HTTP connection terminates cleanly.
  </Step>
</Steps>

**Failure path**: `start` → `error` (connection closes)

<Warning>
  Stream MUST NOT send `delta` events after `done` or `error`. Event ordering is strictly enforced.
</Warning>

## Client Implementation

### Browser (Fetch API with ReadableStream)

Modern browsers can consume SSE via the Fetch API:

```javascript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
// Note: EventSource does NOT support custom headers or POST requests.
// Use the Fetch API with ReadableStream instead:
const response = await fetch('http://localhost:5079/api/arena/chat/stream', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${jwt}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({ prompt: 'Hello' })
});

const reader = response.body.getReader();
const decoder = new TextDecoder();
let fullResponse = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const chunk = decoder.decode(value);
  const lines = chunk.split('\n\n');

  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const data = JSON.parse(line.substring(6));

      switch (data.type) {
        case 'ai.stream.start':
          console.log('Model:', data.model.name);
          break;

        case 'ai.stream.delta':
          fullResponse += data.delta.text;
          displayText(fullResponse); // Update UI
          break;

        case 'ai.stream.done':
          console.log('Usage:', data.usage);
          break;

        case 'ai.error':
          console.error('Stream error:', data.message);
          break;
      }
    }
  }
}
```

### Custom HTTP Client

For environments without EventSource:

```javascript theme={"theme":{"light":"github-light","dark":"github-dark-dimmed"}}
const response = await fetch('/api/arena/chat/stream', {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${jwt}`,
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({ prompt: 'Hello' })
});

const reader = response.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  
  const chunk = decoder.decode(value);
  const lines = chunk.split('\n\n');
  
  for (const line of lines) {
    if (line.startsWith('data: ')) {
      const json = line.substring(6); // Remove "data: " prefix
      const event = JSON.parse(json);
      handleEvent(event);
    }
  }
}
```

## Server Behavior

### Connection Lifecycle

<Steps>
  <Step title="Request Received">
    Backend validates JWT, extracts user ID, performs user sync.
  </Step>

  <Step title="Headers Set">
    Response headers configured: `Content-Type: text/event-stream`
  </Step>

  <Step title="Provider Streaming">
    AI provider streams response via callback function.
  </Step>

  <Step title="Event Transformation">
    Provider chunks transformed into SSE event format.
  </Step>

  <Step title="Client Detection">
    Monitors `HttpContext.RequestAborted` token for disconnects.
  </Step>
</Steps>

### Disconnect Handling

If client disconnects during streaming:

1. `RequestAborted` cancellation token fires
2. Provider receives cancellation signal
3. Stream processing terminates immediately
4. **No database writes occur** for partial streams

<Info>
  Partial streams are not persisted to thread messages. Only complete non-streaming requests write to database.
</Info>

### Timeout Behavior

Streaming requests have same 45-second timeout as non-streaming:

1. Provider timeout (45s)
2. Fallback to alternative model (45s)
3. Fallback to Bytez provider (45s)
4. Return error if all fail

**Difference from non-streaming**: Error delivered as `ai.error` event instead of JSON response.

## Streaming vs Non-Streaming

| Aspect               | Non-Streaming              | Streaming               |
| -------------------- | -------------------------- | ----------------------- |
| Response time        | Wait for full response     | Immediate first token   |
| Perceived latency    | High                       | Low                     |
| Response format      | Single JSON object         | SSE event stream        |
| Database persistence | Yes (if threadId provided) | No                      |
| Client complexity    | Simple (await response)    | Higher (event handling) |
| Bandwidth            | Single burst               | Gradual delivery        |

## When to Use Streaming

**Use streaming when:**

* User experience priority (perceived speed)
* Long responses (>500 tokens)
* Interactive chat interfaces
* Real-time feedback important

**Avoid streaming when:**

* Database persistence required immediately
* Processing full response programmatically
* Client doesn't support SSE
* Response needs to be votable (dual-chat)

<Warning>
  Streaming endpoint does NOT write to thread\_messages table. For persistent threads, client must call non-streaming endpoint separately after stream completes.
</Warning>

## Dual-Chat Streaming

**Current status**: Not supported

**Why not?**

* Two parallel SSE streams complicate client-side handling
* Arena comparisons require complete responses for fair voting
* Use case unclear (voters need full text anyway)

**Workaround**: If streaming needed for dual-chat, call single-chat streaming endpoint twice sequentially.

## Error Scenarios

### Provider Timeout

If provider exceeds 45 seconds:

```
data: {"type":"ai.stream.start","model":{...}}\n\n
data: {"type":"ai.error","code":"PROVIDER_TIMEOUT","message":"Groq API timeout"}\n\n
```

Stream terminates. Client should display error.

### Network Interruption

If network drops during stream:

* Browser's EventSource auto-reconnects (with `Last-Event-ID` header)
* DualMind doesn't support resumption (no event IDs)
* Client must start new request

### Malformed Events

Provider sends invalid JSON:

* Backend catches parse errors
* Sends `ai.error` event
* Stream terminates

## Performance Considerations

### Bandwidth

Streaming uses more total bandwidth than non-streaming:

* Event overhead: `data: ` prefix + `\n\n` suffix per chunk
* JSON overhead: Repeated `{"type":"ai.stream.delta","delta":{"text":"..."}}`

**Example**: 100-word response might generate 50-100 delta events vs 1 non-streaming response.

**Tradeoff**: Higher bandwidth for better perceived latency.

### Server Resources

Streaming holds HTTP connection open longer:

* Non-streaming: 2-3 seconds
* Streaming: 2-3 seconds (same inference time, different delivery)

**Impact**: Minimal. Inference time dominates connection time.

## Next Steps

<CardGroup cols="2">
  <Card title="Chat Modes" icon="comments" href="/concepts/chat-modes">
    Single vs dual chat explained
  </Card>

  <Card title="Thread Management" icon="folder" href="/concepts/threads">
    Persisting conversations
  </Card>

  <Card title="System Overview" icon="cube" href="/concepts/overview">
    Architecture decisions
  </Card>
</CardGroup>