Architecture Overview

System Architecture

text

┌─────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                   LinkSoul Platform (灵心平台)                                     │
│                                                                                                 │
│  ┌─────────────────────────┐       Bridge (桥接器)        ┌─────────────────────────┐           │
│  │ LinkskyGateway          │ ◄───────────────────────────►│  Interaction Gateway    │           │
│  │ (二开网关 / Secondary    │   Boss Channel (control)     │  (交互网关)              │           │
│  │  Dev Gateway)           │   Worker Channel (data)       │                         │           │
│  │ Accepts SDK connections │   Bidirectional signaling     │ Connects to robots      │           │
│  │ Auth & message routing  │   handshake                   │ Audio/video/cmd relay    │           │
│  └────────────┬────────────┘                              └────────────┬────────────┘           │
│               │                                                        │                        │
└───────────────│────────────────────────────────────────────────────────│────────────────────────┘
                │                                                        │
                │ WebSocket (wss)                                        │ WebSocket / TCP
                │ wss://open.agibot.com/api/V1/open-portal/app/wss/agent-sdk                  │
                │                                                        │
┌───────────────▼────────────────┐                          ┌────────────▼─────────────┐
│  Your Application (AgentSdk)   │                          │   Robot / Agent Endpoint  │
│                                │                          │                          │
│  ┌──────────────────────────┐  │                          │  ┌────────────────────┐  │
│  │  LinkskyClient           │  │                          │  │  Microphone (MIC)  │  │
│  │  (Netty WebSocket)       │  │                          │  │  Camera            │  │
│  └───────────┬──────────────┘  │                          │  │  Speaker (TTS)     │  │
│              │                 │                          │  │  Motor (Actions)   │  │
│  ┌───────────▼──────────────┐  │                          │  └────────────────────┘  │
│  │  MsgHandler              │  │                          └──────────────────────────┘
│  │  (passive dispatch)      │  │
│  ├──────────────────────────┤  │
│  │  ActiveMsgHandler        │  │
│  │  (active dispatch)       │  │
│  └───────────┬──────────────┘  │
│              │                 │
│  ┌───────────▼──────────────┐  │
│  │  PassiveCallbacks        │  │     Passive: Robot initiates → SDK processes → returns result
│  │  (8 callback types)      │  │     Audio → ASR → LLM → TTS
│  ├──────────────────────────┤  │
│  │  ActiveRequests          │  │     Active: SDK initiates → gateway relays → robot responds
│  │  (TaskFlow)              │  │     registerTaskFlow / startRequest / taskRequest / endRequest
│  └──────────────────────────┘  │
└────────────────────────────────┘

Data Flow

Passive Interaction (Robot → SDK)

text

Robot Mic → Interaction GW → Bridge Worker → LinkskyGateway → AgentSdk → Callback.onRequest()
                                                                              │
Callback returns → AgentSdk → LinkskyGateway → Bridge Worker → Interaction GW → Robot Speaker

Active Interaction (SDK → Robot)

text

AgentSdk.registerTaskFlow() → LinkskyGateway → Bridge Boss → Interaction GW → Robot
                                                                          │
Robot responds → Interaction GW → Bridge Boss → LinkskyGateway → AgentSdk → Callback.onResponse()

Bridge (Signaling Relay)

The Bridge is the communication hub between LinkskyGateway (the secondary-development gateway) and the Interaction Gateway:

Channel	Layer	Responsibility
Boss Channel	Control plane	Agent registration/deregistration, callId broadcast, signaling sync
Worker Channel	Data plane	Audio/video/ASR/TTS data forwarding, state sync

Each gateway pair establishes Boss + Worker WebSocket connections
ZooKeeper service discovery for automatic instance detection
CountDownGate(2) ensures both channels are ready before message forwarding begins
Supports cluster deployment with multiple LinkskyGateway instances

Module Structure

text

com.agibot.aiem.sdk/
├── AgentSdk.java              # Entry point (singleton factory, registration, lifecycle)
├── AgentParam.java            # Type-safe parameter wrapper
├── AgentMeta.java             # Agent metadata
├── AgentAuthCallback.java     # Auth callback interface
├── IdGenerator.java           # ID generator
├── client/                    # WebSocket client
├── enums/                     # Enum definitions
├── passive/                   # Passive interaction (8 Callback + Response pairs, plus shared base methods like GreetResponse)
├── active/                    # Active interaction (TaskFlowRequest)
├── mgr/                       # Managers (callback type, response cache, thread pool)
└── delayed/                   # Delayed tasks (heartbeat timeout)

Communication Protocol

Authentication

HTTP Headers on WebSocket connect:

AGIBOT_APP_ID: Application ID
AGIBOT_AUTHORIZATION: Bearer <JWT Token>

Message Format (JSON)

json

{
    "type": "agentsdk.audio_request.start",
    "agentId": "AGENT_0000001",
    "eventId": "event_xxxxx",
    "robotCid": "cid_xxx"
}

Threading Model

Netty EventLoop (1 thread): WebSocket I/O
AgentThreadPool: Per-agent worker threads ensuring ordered message processing (core=3, max=50)
DelayedPool (2 threads): Heartbeat timeout detection
Cleanup timer (10s): Garbage-collects expired responses/callbacks

Reconnection

Auto-reconnect on disconnect with 3-second delay
Infinite retries until AgentSdk.release(appId) is called
Heartbeat: Ping every 10 seconds

Below is the end-to-end sequence of one conversation using Audio2Tts (audio in → ASR + LLM + TTS). The other passive callbacks differ only in what data arrives at onRequest; the overall rhythm is the same.

mermaid

sequenceDiagram
    participant Robot
    participant IGW as Interaction GW
    participant Bridge as Bridge Relay
    participant Linksky as LinkskyGateway<br/>(Secondary Dev GW)
    participant SDK as AgentSdk<br/>(your app)

    Note over SDK,Linksky: Startup
    SDK->>Linksky: WebSocket connect + JWT auth
    Linksky-->>SDK: onAuthState(code=0)
    Robot->>IGW: robot online
    IGW->>Bridge: online event
    Bridge->>Linksky: online event
    Linksky-->>SDK: onRobotOnline(agentId, agentMeta)

    Note over Robot,SDK: One conversation
    Robot->>IGW: user starts speaking (VAD start)
    IGW->>Bridge: audio_request.start
    Bridge->>Linksky: forward
    Linksky-->>SDK: onRequest(flag=0)
    loop audio frames
        Robot->>IGW: PCM audio frame
        IGW->>Linksky: audio_request.append
        Linksky-->>SDK: onRequest(flag=1, buf=audio)
    end
    Robot->>IGW: VAD end
    IGW->>Linksky: audio_request.commit
    Linksky-->>SDK: onRequest(flag=2)

    Note over SDK: business ASR → LLM → TTS
    SDK->>Linksky: response.onAsrMiddle / onAsrFinal
    Linksky->>IGW: ASR text
    IGW->>Robot: render on screen

    SDK->>Linksky: response.onLlmItemDelta × N
    SDK->>Linksky: response.onLlmItemDone / onLlmDone
    Linksky->>IGW: LLM streaming text
    IGW->>Robot: render on screen

    SDK->>Linksky: response.onSkill(skill command)
    Linksky->>IGW: skill command
    IGW->>Robot: execute skill

    SDK->>Linksky: response.onTtsItemDelta × N
    SDK->>Linksky: response.onTtsItemDone / onTtsDone
    Linksky->>IGW: TTS audio
    IGW->>Robot: speaker playback

Key points:

A whole conversation is tied together by a single eventId that is shared across every onRequest / response.on* call.
ASR/LLM/VLM/TTS are all streaming — emit *Delta zero-or-more times and close with *Done.
response.onSkill(...) can fire in parallel with text/audio output and does not block them; the full skill catalogue is in Semantic Skills.
Report business failures with response.onError(eventId, code, msg).