Shannon Orchestrator (Go)
The orchestrator is Shannon's central coordination service, managing AI agent workflows through Temporal with pattern-based cognitive architectures, enforcing policies, and handling session state.
π― Core Responsibilities
- Pattern-Based Orchestration - Routes tasks to cognitive patterns (CoT, ToT, ReAct, Debate, Reflection)
- Workflow Management - Coordinates multi-agent execution via Temporal workflows
- Budget Management - Tracks and enforces token usage limits across agents
- Policy Enforcement - Integrates OPA for security and compliance rules
- Session Management - Maintains conversation state across interactions
- Service Coordination - Routes between Rust agent core and Python LLM services
ποΈ Architecture
User Request β gRPC Server (:50052)
β
OrchestratorRouter (Pattern Selection)
β
Pattern Analysis β Cognitive Pattern Selection
βββ Chain of Thought (CoT) - Sequential reasoning
βββ Tree of Thoughts (ToT) - Exploration with backtracking
βββ ReAct - Reasoning + Acting loops
βββ Debate - Multi-agent argumentation
βββ Reflection - Self-improvement iterations
β
Pattern Execution β Agent Coordination
β
Results β Synthesis β Session Update β Response
π Project Structure
go/orchestrator/
βββ main.go # Service entry point
βββ Dockerfile # Container build
βββ internal/
β βββ activities/ # Temporal activity implementations
β β βββ agent.go # Agent execution activities
β β βββ budget.go # Budget tracking activities
β β βββ decompose.go # Task decomposition
β β βββ synthesis.go # Result synthesis
β β βββ metrics.go # Pattern metrics tracking
β βββ workflows/ # Temporal workflow definitions
β β βββ orchestrator_router.go # Main pattern router
β β βββ supervisor_workflow.go # Retry & supervision
β β βββ simple_workflow.go # Simple task execution
β β βββ patterns/ # Cognitive patterns
β β β βββ chain_of_thought.go
β β β βββ tree_of_thoughts.go
β β β βββ react.go
β β β βββ debate.go
β β β βββ reflection.go
β β βββ strategies/ # Legacy workflow strategies
β β β βββ dag.go
β β β βββ exploratory.go
β β β βββ research.go
β β β βββ scientific.go
β β βββ execution/ # Execution patterns
β β βββ parallel.go
β β βββ sequential.go
β β βββ hybrid.go
β βββ server/ # gRPC service implementation
β βββ policy/ # OPA policy engine integration
β βββ budget/ # Token budget management
β βββ auth/ # Authentication & authorization
β βββ db/ # PostgreSQL operations
β βββ health/ # Health checks & degradation
β βββ config/ # Configuration management
β βββ streaming/ # SSE/WebSocket streaming
β βββ circuitbreaker/ # Failure protection
βββ histories/ # Workflow replay test files
βββ tests/ # Integration tests
β βββ replay/ # Determinism testing
βββ tools/replay/ # Temporal replay tooling
π Quick Start
Prerequisites
- Go 1.21+
- Docker & Docker Compose
- PostgreSQL, Redis, Temporal running
Development
# Install dependencies
go mod download
# Run tests
go test -race ./...
# Build binary
go build -o orchestrator .
# Run locally (requires services)
./orchestrator
Docker Deployment
# Build image
docker build -t shannon-orchestrator .
# Run with compose (recommended)
make dev # From repository root
βοΈ Configuration
Configuration is loaded from /app/config/shannon.yaml (mounted in Docker):
# Key configuration sections
service:
port: 50052 # gRPC port
health_port: 8081 # Health check HTTP port
policy:
enabled: true # OPA policy enforcement
mode: "dry-run" # off | dry-run | enforce
path: "/app/config/opa/policies"
temporal:
host_port: "temporal:7233"
namespace: "default"
task_queue: "shannon-task-queue"
patterns:
chain_of_thought:
max_iterations: 10
timeout: "5m"
tree_of_thoughts:
max_depth: 5
branching_factor: 3
react:
max_steps: 15
timeout: "10m"
budget:
max_tokens_per_request: 10000
max_cost_per_request: 1.0
Environment Variables
-
PRIORITY_QUEUES (default: empty)
- When set to
on/true/1, the orchestrator starts one Temporal worker per priority queue:
shannon-tasks-critical, shannon-tasks-high, shannon-tasks (normal), shannon-tasks-low
- Concurrency per queue is tuned in
main.go
-
ENABLE_TOOL_SELECTION (default: 1)
- When enabled, the orchestrator calls the LLM service
/tools/select to auto-populate context.tool_calls
- This enables parallel tool execution in Agent Core when
TOOL_PARALLELISM > 1
-
Priority worker concurrency (optional overrides):
WORKER_ACT_CRITICAL / WORKER_WF_CRITICAL (default: 12 / 12)
WORKER_ACT_HIGH / WORKER_WF_HIGH (default: 10 / 10)
WORKER_ACT_NORMAL / WORKER_WF_NORMAL (default: 8 / 8)
WORKER_ACT_LOW / WORKER_WF_LOW (default: 4 / 4)
-
Single-queue mode concurrency (when PRIORITY_QUEUES is off):
WORKER_ACT / WORKER_WF (default: 10 / 10)
Submit with Priority
Set the priority via metadata.labels["priority"] in SubmitTaskRequest.
Valid values: critical, high, normal, low (case-insensitive). Invalid values fall back to the default queue.
Example:
req := &pb.SubmitTaskRequest{
Metadata: &common.TaskMetadata{
UserId: "user-123",
Labels: map[string]string{"priority": "critical"},
},
Query: "Plan and execute task",
}
resp, err := client.SubmitTask(ctx, req)
π§ Key Features
Pattern-Based Workflows
Cognitive Patterns:
ChainOfThought - Step-by-step logical reasoning
TreeOfThoughts - Explores multiple solution paths with backtracking
ReAct - Combines reasoning with action for interactive tasks
Debate - Multi-agent argumentation for complex decisions
Reflection - Iterative self-improvement
Core Workflows:
OrchestratorRouter - Main entry point that selects patterns
SupervisorWorkflow - Handles retries and supervision
SimpleWorkflow - Direct execution for simple tasks
Key Activities:
DecomposeTask - Analyzes complexity and creates subtasks
ExecuteAgent - Runs individual agent tasks
SynthesizeResults - Combines agent outputs
UpdateSessionResult - Persists session state
RecordPatternMetrics - Tracks pattern performance
Budget Management
Token usage is tracked at multiple levels:
- Per-request budgets with backpressure
- Per-user quotas with circuit breakers
- Cost estimation before execution
- Real-time usage monitoring
Policy Enforcement
OPA policies control:
- Task execution permissions
- Agent access controls
- Resource usage limits
- Data access boundaries
Health & Degradation
Automatic degradation under load:
- Complex β Standard mode fallback
- Circuit breakers for external services
- Graceful timeout handling
- Health endpoint at
:8081/health
π Observability
- Endpoint:
:2112/metrics
- Workflow execution times
- Pattern selection distribution
- Token usage per pattern
- Error rates by workflow type
Logging
- Structured JSON logging with zap
- Correlation IDs for request tracing
- Debug mode available via
LOG_LEVEL=debug
Streaming
- SSE:
GET /stream/sse?workflow_id=<id>
- WebSocket:
GET /stream/ws?workflow_id=<id>
- Notes:
- All child workflow and agent events are unified under the parent
workflow_id for a single stream.
- SSE/WS events are buffered in Redis Streams (~24h TTL) and persisted to Postgres when configured.
Synthesis Events
LLM_OUTPUT (AgentID: synthesis): Final synthesized content (truncated to 10k chars). Emitted on LLM success and all fallback/simple paths.
DATA_PROCESSING summary: Lightweight token usage message (for example, "Used 1.5k tokens"), based on model/tokens reported by synthesis.
- Ordering:
LLM_OUTPUT β summary (DATA_PROCESSING "Used β¦ tokens") β DATA_PROCESSING "Answer ready" β WORKFLOW_COMPLETED.
- Bypass behavior: when synthesis is bypassed (single suitable result), no extra synthesis events are emitted; the agentβs own
LLM_OUTPUT serves as the final result.
π§ͺ Testing
Unit Tests
go test ./internal/...
Integration Tests
# Requires running services
go test ./tests/integration/...
Replay Testing
# Export workflow history
make replay-export WORKFLOW_ID=task-xxx OUT=histories/test.json
# Test determinism
make replay HISTORY=histories/test.json
# Run all replay tests
go test ./tests/replay
π¨ Common Issues
Workflow Non-Determinism
- Ensure no
time.Sleep() in activities
- Use
workflow.Sleep() in workflows
- Register all activities with consistent names
Budget Exceeded
- Check token limits in config
- Monitor usage via metrics
- Adjust
max_tokens_per_request
Pattern Selection
- Review decomposition results
- Check pattern confidence scores
- Monitor pattern metrics
π Further Documentation