orchestrator

command module
v0.0.0-...-391b619 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 5, 2026 License: MIT Imports: 42 Imported by: 0

README ΒΆ

Shannon Orchestrator (Go)

The orchestrator is Shannon's central coordination service, managing AI agent workflows through Temporal with pattern-based cognitive architectures, enforcing policies, and handling session state.

🎯 Core Responsibilities

  • Pattern-Based Orchestration - Routes tasks to cognitive patterns (CoT, ToT, ReAct, Debate, Reflection)
  • Workflow Management - Coordinates multi-agent execution via Temporal workflows
  • Budget Management - Tracks and enforces token usage limits across agents
  • Policy Enforcement - Integrates OPA for security and compliance rules
  • Session Management - Maintains conversation state across interactions
  • Service Coordination - Routes between Rust agent core and Python LLM services

πŸ—οΈ Architecture

User Request β†’ gRPC Server (:50052)
    ↓
OrchestratorRouter (Pattern Selection)
    ↓
Pattern Analysis β†’ Cognitive Pattern Selection
    β”œβ”€β”€ Chain of Thought (CoT) - Sequential reasoning
    β”œβ”€β”€ Tree of Thoughts (ToT) - Exploration with backtracking
    β”œβ”€β”€ ReAct - Reasoning + Acting loops
    β”œβ”€β”€ Debate - Multi-agent argumentation
    └── Reflection - Self-improvement iterations
    ↓
Pattern Execution β†’ Agent Coordination
    ↓
Results β†’ Synthesis β†’ Session Update β†’ Response

πŸ“ Project Structure

go/orchestrator/
β”œβ”€β”€ main.go                      # Service entry point
β”œβ”€β”€ Dockerfile                   # Container build
β”œβ”€β”€ internal/
β”‚   β”œβ”€β”€ activities/              # Temporal activity implementations
β”‚   β”‚   β”œβ”€β”€ agent.go            # Agent execution activities
β”‚   β”‚   β”œβ”€β”€ budget.go           # Budget tracking activities
β”‚   β”‚   β”œβ”€β”€ decompose.go        # Task decomposition
β”‚   β”‚   β”œβ”€β”€ synthesis.go        # Result synthesis
β”‚   β”‚   └── metrics.go          # Pattern metrics tracking
β”‚   β”œβ”€β”€ workflows/               # Temporal workflow definitions
β”‚   β”‚   β”œβ”€β”€ orchestrator_router.go  # Main pattern router
β”‚   β”‚   β”œβ”€β”€ supervisor_workflow.go  # Retry & supervision
β”‚   β”‚   β”œβ”€β”€ simple_workflow.go      # Simple task execution
β”‚   β”‚   β”œβ”€β”€ patterns/               # Cognitive patterns
β”‚   β”‚   β”‚   β”œβ”€β”€ chain_of_thought.go
β”‚   β”‚   β”‚   β”œβ”€β”€ tree_of_thoughts.go
β”‚   β”‚   β”‚   β”œβ”€β”€ react.go
β”‚   β”‚   β”‚   β”œβ”€β”€ debate.go
β”‚   β”‚   β”‚   └── reflection.go
β”‚   β”‚   β”œβ”€β”€ strategies/            # Legacy workflow strategies
β”‚   β”‚   β”‚   β”œβ”€β”€ dag.go
β”‚   β”‚   β”‚   β”œβ”€β”€ exploratory.go
β”‚   β”‚   β”‚   β”œβ”€β”€ research.go
β”‚   β”‚   β”‚   └── scientific.go
β”‚   β”‚   └── execution/             # Execution patterns
β”‚   β”‚       β”œβ”€β”€ parallel.go
β”‚   β”‚       β”œβ”€β”€ sequential.go
β”‚   β”‚       └── hybrid.go
β”‚   β”œβ”€β”€ server/                  # gRPC service implementation
β”‚   β”œβ”€β”€ policy/                  # OPA policy engine integration
β”‚   β”œβ”€β”€ budget/                  # Token budget management
β”‚   β”œβ”€β”€ auth/                    # Authentication & authorization
β”‚   β”œβ”€β”€ db/                      # PostgreSQL operations
β”‚   β”œβ”€β”€ health/                  # Health checks & degradation
β”‚   β”œβ”€β”€ config/                  # Configuration management
β”‚   β”œβ”€β”€ streaming/               # SSE/WebSocket streaming
β”‚   └── circuitbreaker/          # Failure protection
β”œβ”€β”€ histories/                   # Workflow replay test files
β”œβ”€β”€ tests/                       # Integration tests
β”‚   └── replay/                  # Determinism testing
└── tools/replay/                # Temporal replay tooling

πŸš€ Quick Start

Prerequisites
  • Go 1.21+
  • Docker & Docker Compose
  • PostgreSQL, Redis, Temporal running
Development
# Install dependencies
go mod download

# Run tests
go test -race ./...

# Build binary
go build -o orchestrator .

# Run locally (requires services)
./orchestrator
Docker Deployment
# Build image
docker build -t shannon-orchestrator .

# Run with compose (recommended)
make dev  # From repository root

βš™οΈ Configuration

Configuration is loaded from /app/config/shannon.yaml (mounted in Docker):

# Key configuration sections
service:
  port: 50052           # gRPC port
  health_port: 8081     # Health check HTTP port

policy:
  enabled: true         # OPA policy enforcement
  mode: "dry-run"      # off | dry-run | enforce
  path: "/app/config/opa/policies"

temporal:
  host_port: "temporal:7233"
  namespace: "default"
  task_queue: "shannon-task-queue"

patterns:
  chain_of_thought:
    max_iterations: 10
    timeout: "5m"
  tree_of_thoughts:
    max_depth: 5
    branching_factor: 3
  react:
    max_steps: 15
    timeout: "10m"

budget:
  max_tokens_per_request: 10000
  max_cost_per_request: 1.0
Environment Variables
  • PRIORITY_QUEUES (default: empty)

    • When set to on/true/1, the orchestrator starts one Temporal worker per priority queue:
      • shannon-tasks-critical, shannon-tasks-high, shannon-tasks (normal), shannon-tasks-low
    • Concurrency per queue is tuned in main.go
  • ENABLE_TOOL_SELECTION (default: 1)

    • When enabled, the orchestrator calls the LLM service /tools/select to auto-populate context.tool_calls
    • This enables parallel tool execution in Agent Core when TOOL_PARALLELISM > 1
  • Priority worker concurrency (optional overrides):

    • WORKER_ACT_CRITICAL / WORKER_WF_CRITICAL (default: 12 / 12)
    • WORKER_ACT_HIGH / WORKER_WF_HIGH (default: 10 / 10)
    • WORKER_ACT_NORMAL / WORKER_WF_NORMAL (default: 8 / 8)
    • WORKER_ACT_LOW / WORKER_WF_LOW (default: 4 / 4)
  • Single-queue mode concurrency (when PRIORITY_QUEUES is off):

    • WORKER_ACT / WORKER_WF (default: 10 / 10)
Submit with Priority

Set the priority via metadata.labels["priority"] in SubmitTaskRequest.

Valid values: critical, high, normal, low (case-insensitive). Invalid values fall back to the default queue.

Example:

req := &pb.SubmitTaskRequest{
    Metadata: &common.TaskMetadata{
        UserId: "user-123",
        Labels: map[string]string{"priority": "critical"},
    },
    Query: "Plan and execute task",
}
resp, err := client.SubmitTask(ctx, req)

πŸ”§ Key Features

Pattern-Based Workflows

Cognitive Patterns:

  • ChainOfThought - Step-by-step logical reasoning
  • TreeOfThoughts - Explores multiple solution paths with backtracking
  • ReAct - Combines reasoning with action for interactive tasks
  • Debate - Multi-agent argumentation for complex decisions
  • Reflection - Iterative self-improvement

Core Workflows:

  • OrchestratorRouter - Main entry point that selects patterns
  • SupervisorWorkflow - Handles retries and supervision
  • SimpleWorkflow - Direct execution for simple tasks

Key Activities:

  • DecomposeTask - Analyzes complexity and creates subtasks
  • ExecuteAgent - Runs individual agent tasks
  • SynthesizeResults - Combines agent outputs
  • UpdateSessionResult - Persists session state
  • RecordPatternMetrics - Tracks pattern performance
Budget Management

Token usage is tracked at multiple levels:

  • Per-request budgets with backpressure
  • Per-user quotas with circuit breakers
  • Cost estimation before execution
  • Real-time usage monitoring
Policy Enforcement

OPA policies control:

  • Task execution permissions
  • Agent access controls
  • Resource usage limits
  • Data access boundaries
Health & Degradation

Automatic degradation under load:

  • Complex β†’ Standard mode fallback
  • Circuit breakers for external services
  • Graceful timeout handling
  • Health endpoint at :8081/health

πŸ“Š Observability

Metrics (Prometheus format)
  • Endpoint: :2112/metrics
  • Workflow execution times
  • Pattern selection distribution
  • Token usage per pattern
  • Error rates by workflow type
Logging
  • Structured JSON logging with zap
  • Correlation IDs for request tracing
  • Debug mode available via LOG_LEVEL=debug
Streaming
  • SSE: GET /stream/sse?workflow_id=<id>
  • WebSocket: GET /stream/ws?workflow_id=<id>
  • Notes:
    • All child workflow and agent events are unified under the parent workflow_id for a single stream.
    • SSE/WS events are buffered in Redis Streams (~24h TTL) and persisted to Postgres when configured.
Synthesis Events
  • LLM_OUTPUT (AgentID: synthesis): Final synthesized content (truncated to 10k chars). Emitted on LLM success and all fallback/simple paths.
  • DATA_PROCESSING summary: Lightweight token usage message (for example, "Used 1.5k tokens"), based on model/tokens reported by synthesis.
  • Ordering: LLM_OUTPUT β†’ summary (DATA_PROCESSING "Used … tokens") β†’ DATA_PROCESSING "Answer ready" β†’ WORKFLOW_COMPLETED.
  • Bypass behavior: when synthesis is bypassed (single suitable result), no extra synthesis events are emitted; the agent’s own LLM_OUTPUT serves as the final result.

πŸ§ͺ Testing

Unit Tests
go test ./internal/...
Integration Tests
# Requires running services
go test ./tests/integration/...
Replay Testing
# Export workflow history
make replay-export WORKFLOW_ID=task-xxx OUT=histories/test.json

# Test determinism
make replay HISTORY=histories/test.json

# Run all replay tests
go test ./tests/replay

🚨 Common Issues

Workflow Non-Determinism
  • Ensure no time.Sleep() in activities
  • Use workflow.Sleep() in workflows
  • Register all activities with consistent names
Budget Exceeded
  • Check token limits in config
  • Monitor usage via metrics
  • Adjust max_tokens_per_request
Pattern Selection
  • Review decomposition results
  • Check pattern confidence scores
  • Monitor pattern metrics

πŸ“š Further Documentation

Documentation ΒΆ

The Go Gopher

There is no documentation for this package.

Directories ΒΆ

Path Synopsis
cmd
gateway command
gateway/internal/openai
Package openai provides OpenAI-compatible API endpoints for Shannon.
Package openai provides OpenAI-compatible API endpoints for Shannon.
internal
db
skills
Package skills implements a markdown-based skill system compatible with Anthropic's Agent Skills specification.
Package skills implements a markdown-based skill system compatible with Anthropic's Agent Skills specification.
validation
Package validation provides utilities for validating workflow configurations.
Package validation provides utilities for validating workflow configurations.
workflows/strategies
DomainAnalysisWorkflow performs company/entity research as a child workflow.
DomainAnalysisWorkflow performs company/entity research as a child workflow.
tools
replay command

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 捒网址