Skip to content

COMPLETE_ARCHITECTURE.md

Latest commit

 

History

History
911 lines (705 loc) · 21.5 KB

COMPLETE_ARCHITECTURE.md

File metadata and controls

911 lines (705 loc) · 21.5 KB

🏗️ Complete SDK Architecture & Flow

Comprehensive end-to-end architecture aligned with source code v1.0.0

📋 Table of Contents

🎯 System Overview

graph TB
    subgraph "User Layer"
        User[👤 User/Developer]
    end
    
    subgraph "SDK Core"
        Agent[🤖 Agent Class]
        Runner[⚡ AgenticRunner]
        Tools[🔧 Tool System]
        Guards[🛡️ Guardrails]
    end
    
    subgraph "State Management"
        RunState[📊 RunState]
        Session[💾 Session]
        Usage[📈 Usage Tracking]
    end
    
    subgraph "Execution Layer"
        Exec[🔄 Execution Engine]
        Transfer[🔀 Transfer System]
        HITL[👤 Human-in-Loop]
    end
    
    subgraph "Observability"
        Trace[📊 Tracing Context]
        Langfuse[🔍 Langfuse]
    end
    
    subgraph "External"
        LLM[🧠 LLM Provider]
        Storage[💾 Storage]
    end
    
    User -->|run/runStream| Agent
    Agent --> Runner
    Runner --> Guards
    Guards --> Exec
    Exec --> Tools
    Exec --> Transfer
    Exec --> HITL
    
    Runner <--> RunState
    RunState <--> Session
    RunState --> Usage
    
    Runner --> Trace
    Trace --> Langfuse
    
    Exec <--> LLM
    Session <--> Storage
    
    style Agent fill:#4a90e2
    style Runner fill:#e74c3c
    style Guards fill:#9b59b6
    style Trace fill:#f39c12
Loading

📁 Directory Structure

Source Code Organization

src/
├── core/                       # Core execution engine
│   ├── agent/                 # Modular agent (types, class, run, tools)
│   ├── runner.ts              # AgenticRunner (main executor)
│   ├── execution.ts           # Single-step execution
│   ├── transfers.ts           # Multi-agent transfer system
│   ├── runstate.ts            # State management
│   ├── usage.ts               # Token tracking
│   ├── result.ts              # Result types
│   ├── approvals.ts           # HITL approvals
│   ├── coordination.ts        # Agent coordination
│   ├── race-agents.ts         # Race patterns
│   └── hitl.ts                # Human-in-the-loop
│
├── guardrails/                # Safety & validation
│   └── index.ts               # All guardrails
│
├── tracing/                   # Observability
│   ├── context.ts             # Tracing context
│   ├── tracing.ts             # Tracing logic
│   └── tracing-utils.ts       # Utilities
│
├── sessions/                  # State persistence
│   ├── session.ts             # Session interface
│   └── index.ts               # Exports
│
├── lifecycle/                 # Event system
│   ├── events.ts              # Hook definitions
│   └── langfuse/              # Langfuse integration
│       └── index.ts
│
├── tools/                     # Built-in tools
│   ├── audio/                 # Speech & transcription
│   ├── embeddings/            # Vector embeddings
│   ├── image/                 # Image generation
│   ├── rag/                   # RAG with Pinecone
│   └── rerank/                # Result reranking
│
├── helpers/                   # Utilities
│   ├── message.ts             # Message helpers
│   ├── safe-execute.ts        # Safe tool execution
│   └── toon.ts                # Token optimization
│
├── mcp/                       # Model Context Protocol
│   ├── index.ts               # MCP client
│   ├── enhanced.ts            # Enhanced features
│   └── utils.ts               # MCP utilities
│
├── approvals/                 # HITL system
│   └── index.ts               # Approval manager
│
├── types/                     # TypeScript types
│   ├── types.ts               # Core types
│   └── helpers.ts             # Type utilities
│
└── index.ts                   # Main exports

🏛️ Core Architecture

Component Relationships

classDiagram
    class Agent {
        +name: string
        +instructions: string
        +model: LanguageModel
        +tools: ToolSet
        +subagents: Agent[]
        +guardrails: Guardrail[]
        +run(input) RunResult
    }
    
    class AgenticRunner {
        -agent: Agent
        -state: RunState
        +execute() Promise~Result~
        +executeAgentLoop() Promise
        +executeStep() Promise
    }
    
    class RunState {
        +messages: ModelMessage[]
        +context: any
        +steps: StepResult[]
        +currentAgent: Agent
        +usage: Usage
        +trace: Trace
    }
    
    class Execution {
        +executeStep() Promise
        +executeTools() Promise
        +detectTransfer() boolean
    }
    
    class Transfers {
        +createTransferTools()
        +detectTransfer()
        +createTransferContext()
    }
    
    class Guardrails {
        +runInputGuardrails()
        +runOutputGuardrails()
    }
    
    class Tracing {
        +createTrace()
        +createSpan()
        +createGeneration()
    }
    
    Agent --> AgenticRunner : creates
    AgenticRunner --> RunState : manages
    AgenticRunner --> Execution : uses
    AgenticRunner --> Guardrails : validates
    AgenticRunner --> Tracing : observes
    Execution --> Transfers : handles
    Agent --> Agent : subagents
Loading

⚡ Execution Flow

Main Execution Pipeline

sequenceDiagram
    participant U as User
    participant A as Agent
    participant R as AgenticRunner
    participant G as Guardrails
    participant E as Execution
    participant L as LLM
    participant T as Tracing
    
    U->>A: run(input, options)
    A->>R: execute()
    
    Note over R: Create RunState
    R->>T: Create trace
    T-->>R: trace object
    
    loop For each turn (max turns)
        Note over R: Turn start
        
        R->>G: Run input guardrails
        alt Guardrail passes
            G-->>R: ✅ Pass
        else Guardrail fails
            G-->>R: ❌ Fail + feedback
            R->>E: Regenerate with feedback
        end
        
        R->>E: executeStep(messages)
        E->>L: generateText()
        L-->>E: response
        
        E->>R: step result
        R->>R: Update state
        
        alt Has tool calls
            R->>E: Execute tools
            E-->>R: tool results
            Note over R: Continue to next turn
        else Transfer detected
            R->>R: Switch agent
            Note over R: Continue to next turn
        else No tools
            R->>G: Run output guardrails
            alt Guardrail passes
                G-->>R: ✅ Pass
                Note over R: Exit loop
            else Guardrail fails
                G-->>R: ❌ Fail + feedback
                Note over R: Continue to next turn
            end
        end
    end
    
    R->>T: End trace
    R-->>A: Result
    A-->>U: RunResult
Loading

🔀 Multi-Agent Flow

Agent Transfer & Coordination

sequenceDiagram
    participant C as Coordinator
    participant R as Runner
    participant A1 as Agent A
    participant A2 as Agent B
    participant T as Tracing
    
    Note over C: User request
    
    C->>R: run(Coordinator, input)
    R->>T: Create trace "Agent Run"
    
    R->>T: Create span "Agent: Coordinator"
    R->>A1: Execute turn
    A1-->>R: transfer_to_B(reason, query)
    
    Note over R: Detect transfer
    R->>T: End span "Agent: Coordinator"
    
    R->>R: Switch to Agent B
    R->>R: Create isolated context
    
    R->>T: Create span "Agent: B" (sibling)
    R->>A2: Execute turn
    A2-->>R: Final response
    
    R->>T: End span "Agent: B"
    R->>T: End trace
    
    R-->>C: Result with path: A → B
Loading

Context Isolation

graph LR
    subgraph "Agent A Context"
        A1[Messages A]
        A2[Tools A]
        A3[Context A]
    end
    
    subgraph "Transfer"
        T1[Transfer Tool]
        T2[Transfer Context]
        T3[Isolated State]
    end
    
    subgraph "Agent B Context"
        B1[Messages B]
        B2[Tools B]
        B3[Context B]
    end
    
    A1 --> T1
    A3 --> T2
    T2 --> B1
    T3 --> B3
    
    style T1 fill:#f39c12
    style T2 fill:#f39c12
    style T3 fill:#f39c12
Loading

🛡️ Guardrails Flow

Input & Output Validation

flowchart TD
    Start([User Input]) --> IG{Input<br/>Guardrails?}
    
    IG -->|Yes| RunIG[Run Input Guardrails]
    IG -->|No| LLM
    
    RunIG --> IGPass{Pass?}
    IGPass -->|✅ Yes| LLM[Generate with LLM]
    IGPass -->|❌ No| IGFeed[Generate Feedback]
    IGFeed --> LLM
    
    LLM --> OG{Output<br/>Guardrails?}
    
    OG -->|Yes| RunOG[Run Output Guardrails]
    OG -->|No| Success
    
    RunOG --> OGPass{Pass?}
    OGPass -->|✅ Yes| Success([Return Result])
    OGPass -->|❌ No| OGFeed[Generate Feedback]
    
    OGFeed --> Check{Max<br/>Turns?}
    Check -->|No| LLM
    Check -->|Yes| Error([Max Turns Exceeded])
    
    style IG fill:#9b59b6
    style RunIG fill:#9b59b6
    style OG fill:#9b59b6
    style RunOG fill:#9b59b6
    style Success fill:#27ae60
    style Error fill:#e74c3c
Loading

Available Guardrails

graph TB
    subgraph "Core Guardrails"
        LG[Length Guardrail]
        PII[PII Detection]
        Custom[Custom Guardrail]
    end
    
    subgraph "Advanced Guardrails"
        CS[Content Safety]
        TR[Topic Relevance]
        Sent[Sentiment]
        Tox[Toxicity]
        Lang[Language]
        Rate[Rate Limit]
    end
    
    Input[📝 Input] --> LG
    Input --> PII
    Input --> Custom
    
    Output[📤 Output] --> CS
    Output --> TR
    Output --> Sent
    Output --> Tox
    Output --> Lang
    Output --> Rate
    
    style LG fill:#9b59b6
    style PII fill:#9b59b6
    style CS fill:#e74c3c
    style TR fill:#e74c3c
Loading

📊 Tracing & Observability

Langfuse Integration

sequenceDiagram
    participant R as Runner
    participant TC as Tracing Context
    participant LF as Langfuse
    
    Note over R: Start execution
    
    R->>TC: runWithTraceContext()
    TC->>LF: Create trace "Agent Run"
    LF-->>TC: trace object
    
    loop For each agent
        R->>TC: createContextualSpan()
        TC->>LF: Create span "Agent: X"
        
        loop For each LLM call
            R->>TC: createContextualGeneration()
            TC->>LF: Create generation
            Note over LF: Log tokens, latency
        end
        
        loop For each tool
            R->>TC: createContextualSpan()
            TC->>LF: Create span "Tool: X"
        end
        
        loop For each guardrail
            R->>TC: createContextualSpan()
            TC->>LF: Create span "Guardrail: X"
        end
        
        R->>TC: End agent span
    end
    
    R->>TC: End trace with metadata
    TC->>LF: flushAsync()
Loading

Trace Hierarchy

graph TB
    Trace[🔍 Trace: Agent Run]
    
    Trace --> IG[📥 SPAN: Input Guardrail]
    Trace --> A1[🤖 SPAN: Agent A]
    Trace --> A2[🤖 SPAN: Agent B]
    Trace --> OG[📤 SPAN: Output Guardrail]
    
    A1 --> G1[🧠 GENERATION: LLM Call 1]
    A1 --> T1[🔧 SPAN: Tool 1]
    A1 --> T2[🔧 SPAN: Tool 2]
    
    A2 --> G2[🧠 GENERATION: LLM Call 2]
    A2 --> T3[🔧 SPAN: Tool 3]
    
    style Trace fill:#f39c12
    style IG fill:#9b59b6
    style OG fill:#9b59b6
    style A1 fill:#4a90e2
    style A2 fill:#4a90e2
    style G1 fill:#e74c3c
    style G2 fill:#e74c3c
Loading

💾 Session Management

Session Flow

sequenceDiagram
    participant U as User
    participant A as Agent
    participant R as Runner
    participant S as Session
    participant DB as Storage
    
    U->>A: run(input, {session})
    A->>R: execute()
    
    R->>S: getMessages()
    S->>DB: Load history
    DB-->>S: messages[]
    S-->>R: messages[]
    
    Note over R: Execute agent
    
    R->>R: Generate response
    
    R->>S: addMessages(new)
    S->>DB: Save messages
    
    R-->>A: result
    A-->>U: result
    
    Note over U: Next turn
    
    U->>A: run(input2, {session})
    Note over R: History available
Loading

Session Types

graph LR
    subgraph "Memory Session"
        M1[In-Memory Storage]
        M2[Fast Access]
        M3[No Persistence]
    end
    
    subgraph "Redis Session"
        R1[Redis Storage]
        R2[Distributed]
        R3[Persistent]
    end
    
    App[Application] --> M1
    App --> R1
    
    M1 --> M2
    M2 --> M3
    
    R1 --> R2
    R2 --> R3
    
    style M1 fill:#4a90e2
    style R1 fill:#e74c3c
Loading

🔧 Tool Execution

Tool Call Flow

flowchart TD
    Start([LLM Response]) --> HasTools{Has Tool<br/>Calls?}
    
    HasTools -->|No| End([Continue])
    HasTools -->|Yes| Parse[Parse Tool Calls]
    
    Parse --> Parallel{Parallel<br/>Enabled?}
    
    Parallel -->|Yes| ExecPar[Execute in Parallel]
    Parallel -->|No| ExecSeq[Execute Sequentially]
    
    ExecPar --> Collect[Collect Results]
    ExecSeq --> Collect
    
    Collect --> Trace[Trace Each Tool]
    Trace --> AddMsg[Add Tool Results to Messages]
    AddMsg --> Next([Next Turn])
    
    style Start fill:#50c878
    style ExecPar fill:#4a90e2
    style ExecSeq fill:#f39c12
    style Next fill:#50c878
Loading

Built-in Tools

graph TB
    subgraph "Audio Tools"
        TTS[Text-to-Speech]
        STT[Speech-to-Text]
    end
    
    subgraph "Vision Tools"
        IMG[Image Generation]
    end
    
    subgraph "Knowledge Tools"
        EMB[Embeddings]
        RAG[RAG Search]
        RANK[Rerank]
    end
    
    subgraph "Integration"
        MCP[MCP Tools]
    end
    
    Agent[🤖 Agent] --> TTS
    Agent --> STT
    Agent --> IMG
    Agent --> EMB
    Agent --> RAG
    Agent --> RANK
    Agent --> MCP
    
    style Agent fill:#4a90e2
    style TTS fill:#27ae60
    style IMG fill:#27ae60
    style RAG fill:#27ae60
Loading

🎯 Complete End-to-End Flow

Full System Flow with All Features

flowchart TD
    Start([👤 User Request]) --> Init[Initialize Agent]
    
    Init --> CreateRunner[Create AgenticRunner]
    CreateRunner --> CreateState[Create RunState]
    CreateState --> CreateTrace[🔍 Create Langfuse Trace]
    
    CreateTrace --> LoadSession{Session<br/>Exists?}
    LoadSession -->|Yes| LoadHist[💾 Load History]
    LoadSession -->|No| NewHist[Create New History]
    
    LoadHist --> PrepMsg[Prepare Messages]
    NewHist --> PrepMsg
    
    PrepMsg --> InputGuard{Input<br/>Guardrails?}
    
    InputGuard -->|Yes| RunInputG[🛡️ Run Input Guardrails]
    InputGuard -->|No| StartLoop
    
    RunInputG --> IGPass{Pass?}
    IGPass -->|❌ No| GenFeedback1[Generate Feedback]
    IGPass -->|✅ Yes| StartLoop
    
    GenFeedback1 --> StartLoop
    
    StartLoop[📍 Start Turn Loop] --> CheckTurns{Max Turns<br/>Reached?}
    CheckTurns -->|Yes| MaxTurnsError([❌ Max Turns Error])
    CheckTurns -->|No| CreateSpan[📊 Create Agent Span]
    
    CreateSpan --> LLMCall[🧠 LLM Generation]
    LLMCall --> TraceGen[📊 Trace Generation]
    
    TraceGen --> CheckResp{Response<br/>Type?}
    
    CheckResp -->|Text Only| OutputGuard
    CheckResp -->|Tool Calls| ExecTools[🔧 Execute Tools]
    CheckResp -->|Transfer| HandleTransfer
    
    ExecTools --> TraceTools[📊 Trace Each Tool]
    TraceTools --> ToolApproval{HITL<br/>Approval?}
    
    ToolApproval -->|Yes| AskApproval[👤 Request Approval]
    ToolApproval -->|No| RunTools[Run Tools]
    
    AskApproval --> ApprovalResp{Approved?}
    ApprovalResp -->|Yes| RunTools
    ApprovalResp -->|No| SkipTool[Skip Tool]
    
    RunTools --> AddResults[Add Tool Results]
    SkipTool --> AddResults
    
    AddResults --> EndSpan1[End Agent Span]
    EndSpan1 --> StartLoop
    
    HandleTransfer[🔀 Handle Transfer] --> EndCurrentSpan[End Current Agent Span]
    EndCurrentSpan --> SwitchAgent[Switch to New Agent]
    SwitchAgent --> IsolateCtx[Isolate Context]
    IsolateCtx --> StartLoop
    
    OutputGuard{Output<br/>Guardrails?} -->|Yes| RunOutputG[🛡️ Run Output Guardrails]
    OutputGuard -->|No| SaveSession
    
    RunOutputG --> OGPass{Pass?}
    OGPass -->|❌ No| GenFeedback2[Generate Feedback]
    OGPass -->|✅ Yes| SaveSession
    
    GenFeedback2 --> CheckTurns2{Max Turns?}
    CheckTurns2 -->|Yes| MaxTurnsError
    CheckTurns2 -->|No| StartLoop
    
    SaveSession{Session?} -->|Yes| SaveMsg[💾 Save Messages]
    SaveSession -->|No| EndTrace
    
    SaveMsg --> EndTrace[End Trace]
    EndTrace --> FlushTrace[🔍 Flush to Langfuse]
    FlushTrace --> ReturnResult([✅ Return Result])
    
    style Start fill:#50c878
    style LLMCall fill:#e74c3c
    style ExecTools fill:#27ae60
    style HandleTransfer fill:#f39c12
    style SaveMsg fill:#f39c12
    style FlushTrace fill:#f39c12
    style ReturnResult fill:#ffd700
    style MaxTurnsError fill:#e74c3c
    style RunInputG fill:#9b59b6
    style RunOutputG fill:#9b59b6
Loading

🔍 Component Details

1. Agent Class (core/agent/agent-class.ts)

Purpose: Main interface for creating and running agents

Key Methods:

  • run(input, options) - Execute agent
  • runStream(input, options) - Execute with streaming
  • tool(config) - Create tool definition

Key Properties:

  • name - Agent identifier
  • instructions - System prompt
  • model - LLM model
  • tools - Tool definitions
  • subagents - Child agents for transfers
  • guardrails - Validation rules

2. AgenticRunner (core/runner.ts)

Purpose: Core execution engine

Key Methods:

  • execute() - Main execution loop
  • executeAgentLoop() - Agent turn loop
  • executeStep() - Single LLM call
  • runInputGuardrails() - Input validation
  • runOutputGuardrails() - Output validation

Features:

  • Turn-based execution
  • Guardrail integration
  • Transfer handling
  • Tracing integration
  • Token tracking

3. Execution Engine (core/execution.ts)

Purpose: Single-step execution logic

Key Functions:

  • executeStep() - Execute one LLM call
  • executeToolCalls() - Execute tools
  • detectTransfer() - Check for transfers

4. Transfer System (core/transfers.ts)

Purpose: Multi-agent coordination

Key Functions:

  • createTransferTools() - Generate transfer tools
  • detectTransfer() - Detect transfer in response
  • createTransferContext() - Isolate context

Features:

  • Context isolation
  • Transfer detection
  • Query/reason passing

5. RunState (core/runstate.ts)

Purpose: State management during execution

Properties:

  • messages - Conversation history
  • steps - Execution steps
  • usage - Token usage
  • currentAgent - Active agent
  • transferChain - Agent path
  • agentMetrics - Per-agent stats

6. Guardrails (guardrails/index.ts)

Available Guardrails:

  • lengthGuardrail - Text length validation
  • piiDetectionGuardrail - PII detection
  • contentSafetyGuardrail - Content safety (LLM-based)
  • topicRelevanceGuardrail - Topic validation
  • sentimentGuardrail - Sentiment analysis
  • toxicityGuardrail - Toxicity check
  • languageGuardrail - Language validation
  • rateLimitGuardrail - Rate limiting

7. Tracing Context (tracing/context.ts)

Purpose: Langfuse integration

Key Functions:

  • withTrace() - Create trace scope
  • createContextualSpan() - Create span
  • createContextualGeneration() - Create generation
  • runWithTraceContext() - Execute with tracing

Features:

  • Async context tracking
  • Hierarchical spans
  • Token tracking
  • Error tracking

8. Session Manager (sessions/)

Types:

  • MemorySession - In-memory storage
  • RedisSession - Redis storage (external)

Methods:

  • getMessages() - Load history
  • addMessage() - Save message
  • clear() - Clear history

9. Tool System (tools/)

Built-in Tools:

  • Audio: Text-to-speech, transcription
  • Embeddings: Vector generation
  • Image: Image generation
  • RAG: Pinecone search
  • Rerank: Result reranking

10. Lifecycle Hooks (lifecycle/)

Hook Events:

  • Agent hooks: onStart, onEnd, onError
  • Run hooks: onStepStart, onStepFinish, onToolCall

📊 Data Flow Summary

graph LR
    Input[User Input] --> Agent
    Agent --> Runner
    Runner --> State
    State --> Execution
    Execution --> LLM
    LLM --> Response
    Response --> Tools
    Tools --> State
    State --> Guardrails
    Guardrails --> Output
    
    State -.-> Session
    State -.-> Usage
    Runner -.-> Tracing
    
    style Input fill:#50c878
    style Output fill:#ffd700
    style LLM fill:#e74c3c
Loading

✅ Key Takeaways

  1. Agent - User-facing interface
  2. AgenticRunner - Core execution engine
  3. RunState - Centralized state management
  4. Execution - LLM interaction logic
  5. Transfers - Multi-agent coordination
  6. Guardrails - Safety & validation
  7. Tracing - Observability via Langfuse
  8. Sessions - Conversation persistence
  9. Tools - Extensible functionality
  10. Lifecycle - Event-driven hooks

This architecture ensures:

  • ✅ Clear separation of concerns
  • ✅ True agentic patterns
  • ✅ Comprehensive observability
  • ✅ Production-ready reliability
  • ✅ Extensible design

Made with ❤️ by Tawk.to