Imagine you're on call. Your application suddenly starts returning HTTP 500 errors.
You have Prometheus for metrics, Loki for logs, Kubernetes for workload information, and GitHub for deployments.
You ask an LLM:
"Why did our application start returning 5xx errors around 10:30 AM?"
To answer this accurately, the LLM needs to investigate several things:
- Query Prometheus to identify when the error rate increased.
- Check whether request volume or latency changed.
- Search Loki for corresponding application errors around that exact time window.
- Inspect Kubernetes pods and recent restarts.
- Correlate the findings and identify a probable root cause.
This is where three concepts become critical for Site Reliability Engineers: Model Context Protocol (MCP), Universal Tool Calling Protocol (UTCP), and Code Mode.
They are often conflated in discussions, but they operate at two completely different layers of the stack:
- The Tool Interface Layer (MCP vs. UTCP): How an LLM discovers and invokes external operational tools (via intermediate wrapper servers with MCP, or direct native calls via declarative manuals with UTCP).
- The Orchestration Layer (Standard Tool Calling vs. Code Mode): Whether the LLM invokes tools one by one in a conversational loop, or writes and executes code in a sandbox to orchestrate those tools and filter the data.
Importantly, Code Mode is an orchestration pattern, not a competing protocol—it can run on top of either MCP tools or UTCP manuals underneath.
Let's explore how these work in practice through a real-world SRE incident investigation, complete with Python implementations for each pattern.
1. Understanding MCP (The Standard Proxy Approach)
The Model Context Protocol (MCP) is an open standard that allows LLM applications to connect to external tools and data sources through a consistent client-server interface.
Without a standard protocol, every integration requires custom glue code. Prometheus, Loki, Kubernetes, and GitHub each expose different APIs and authentication mechanisms. MCP provides a unified way to wrap and expose these capabilities to an LLM.
For example, an SRE monitoring MCP server could expose tools such as:
query_prometheus
query_loki
get_pod_status
get_recent_deployments
The LLM discovers these tools, reads their descriptions and input schemas, and invokes them when needed.
How MCP Works
flowchart TD
User([On-Call SRE]) --> LLM[LLM Client]
LLM --> MCP_Client[MCP Client]
MCP_Client -->|JSON-RPC| MCP_Server[MCP Server Wrapper]
MCP_Server --> Prom[(Prometheus API)]
MCP_Server --> Loki[(Grafana Loki API)]
MCP_Server --> K8s[(Kubernetes API)]
The MCP server acts as a middleman that handles the integration with the underlying systems. The LLM does not need to know every API endpoint or authentication detail.
A Simple Python MCP Server
The following example uses the Python MCP SDK (FastMCP) to expose a Prometheus query tool.
Install the SDK and HTTP client:
pip install "mcp[cli]" httpx
Implementation:
import os
import httpx
from mcp.server.fastmcp import FastMCP
mcp = FastMCP("sre-monitoring")
PROMETHEUS_URL = os.environ["PROMETHEUS_URL"]
PROMETHEUS_TOKEN = os.environ["PROMETHEUS_TOKEN"]
@mcp.tool()
async def query_prometheus(query: str) -> dict:
"""Execute an instant Prometheus query."""
headers = {
"Authorization": f"Bearer {PROMETHEUS_TOKEN}"
}
async with httpx.AsyncClient(timeout=15) as client:
response = await client.get(
f"{PROMETHEUS_URL}/api/v1/query",
params={"query": query},
headers=headers,
)
response.raise_for_status()
payload = response.json()
return payload["data"]
if __name__ == "__main__":
mcp.run(transport="streamable-http")
You could implement similar tools for Loki, Kubernetes, and GitHub. In production, you must add authentication, authorization, query restrictions, timeouts, rate limits, and response-size limits to avoid exposing unrestricted access to infrastructure APIs.
The SRE Trade-off with MCP: While MCP standardizes tool access, it introduces two operational challenges at scale:
- Infrastructure Toil: You have to deploy, monitor, and maintain intermediate MCP server processes for APIs that already exist.
- Sequential Orchestration & Schema Bloat: MCP preloads tool schemas into the LLM context and, by default, the LLM calls tools one at a time—pulling huge raw responses back into its context window.
2. Enter UTCP (The Direct "Manual" Alternative to MCP)
Before looking at orchestration (Code Mode), let's look at how Universal Tool Calling Protocol (UTCP) addresses the infrastructure and discovery side of MCP.
Where MCP acts as a middleman server in the data plane, UTCP acts as a declarative manual in the control plane. Its core premise is simple: if your internal systems already have HTTP APIs, OpenAPI specs, or CLIs (kubectl, promtool), you shouldn't need to deploy and maintain a wrapper server just so an AI agent can call them.
Instead of running a Python server process like FastMCP, UTCP uses a static JSON manual (sre_manual.json) that tells the UTCP Client how to call Prometheus, Loki, or kubectl directly using native protocols and local environment variables.
How UTCP Works
flowchart TD
User([On-Call SRE]) --> LLM[LLM / Agent]
LLM --> UTCP_Client[UTCP Client]
UTCP_Client -.->|Reads Static JSON Spec| Manual[UTCP Manual / OpenAPI]
UTCP_Client -->|Direct HTTP| Prom[(Prometheus API)]
UTCP_Client -->|Direct HTTP| Loki[(Grafana Loki API)]
UTCP_Client -->|Direct CLI / HTTP| K8s[(Kubernetes API / kubectl)]
Example UTCP Manual (sre_manual.json)
Instead of writing and hosting the FastMCP server above, the exact same Prometheus, Loki, and Kubernetes tools in UTCP are defined in a single declarative JSON file:
{
"manual_version": "1.0.0",
"utcp_version": "1.0.0",
"tools": [
{
"name": "query_prometheus",
"description": "Execute a Prometheus range query for metrics.",
"tags": ["prometheus", "metrics", "error_rate"],
"inputs": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "PromQL expression" },
"start": { "type": "number", "description": "Start unix timestamp" },
"end": { "type": "number", "description": "End unix timestamp" }
},
"required": ["query", "start", "end"]
},
"tool_call_template": {
"call_template_type": "http",
"http_method": "GET",
"url": "${PROMETHEUS_URL}/api/v1/query_range",
"query_params": {
"query": "{query}",
"start": "{start}",
"end": "{end}",
"step": "60s"
},
"auth": {
"auth_type": "api_key",
"api_key": "Bearer ${PROMETHEUS_TOKEN}",
"var_name": "Authorization",
"location": "header"
}
}
},
{
"name": "query_loki",
"description": "Query Grafana Loki for application logs in a time window.",
"tags": ["loki", "logs", "errors"],
"inputs": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "LogQL query" },
"start": { "type": "number", "description": "Start unix timestamp" },
"end": { "type": "number", "description": "End unix timestamp" }
},
"required": ["query", "start", "end"]
},
"tool_call_template": {
"call_template_type": "http",
"http_method": "GET",
"url": "${LOKI_URL}/loki/api/v1/query_range",
"query_params": {
"query": "{query}",
"start": "{start}",
"end": "{end}",
"limit": "2000"
}
}
},
{
"name": "get_pod_status",
"description": "Get Kubernetes pod status and restart counts via kubectl CLI.",
"tags": ["kubernetes", "k8s", "pods", "restarts"],
"inputs": {
"type": "object",
"properties": {
"namespace": { "type": "string" },
"deployment": { "type": "string" }
},
"required": ["namespace", "deployment"]
},
"tool_call_template": {
"call_template_type": "cli",
"commands": [
{
"command": "kubectl get pods -n {namespace} -l app={deployment} -o json"
}
]
}
}
]
}
With UTCP, there is zero extra server infrastructure to deploy or get paged for at 3 AM. Furthermore, the UTCP client includes native progressive discovery (client.search_tools()), so you don't have to preload all 100 tool schemas into the LLM prompt on Turn 1.
3. What is Code Mode?
With a conventional tool-calling approach, the LLM invokes tools individually in a conversational "ping-pong" loop:
- Call Prometheus → Read raw result in LLM context.
- Decide to call Loki → Read thousands of log lines in LLM context.
- Decide to inspect Kubernetes → Read pod statuses in LLM context.
- Correlate everything and produce an answer.
This works for simple checks, but during an incident, the LLM makes multiple sequential round-trips and ingests megabytes of intermediate telemetry data.
Code Mode takes a programmatic approach.
Instead of asking the LLM to orchestrate every individual tool call through conversation turns, Code Mode allows the LLM to write a small program that performs the orchestration inside an isolated execution runtime.
For example, the LLM generates Python code that:
- Queries Prometheus for the error rate.
- Programmatically identifies the exact timestamp when the error rate crossed the threshold.
- Searches Loki only for that narrow time window.
- Groups matching errors by type in memory.
- Retrieves relevant Kubernetes pod statuses.
- Returns a compact JSON summary back to the LLM.
How Code Mode Works
flowchart TD
User([On-Call SRE]) --> LLM[LLM]
LLM -->|1. Search Needed Tools Only| Discovery[Progressive Tool Discovery]
Discovery -->|Returns 3 Compact Stubs| LLM
LLM -->|2. Generates Orchestration Script| Runtime[Isolated Python Execution Runtime]
subgraph Sandbox [In-Memory Data Processing & Filtering]
Runtime -->|3a. Direct HTTP| Prom[(Prometheus)]
Runtime -->|3b. Direct HTTP| Loki[(Loki)]
Runtime -->|3c. Direct CLI| K8s[(Kubernetes)]
Prom & Loki & K8s --> Correlate[Process, Filter & Aggregate in Python]
end
Correlate -->|4. Return Compact Summary Only| LLM
LLM --> Final([Final Root Cause Analysis])
Important architectural distinction: Code Mode is an execution pattern, not a replacement for tool definitions. Code Mode can call MCP tools through a bridge, or it can pair with UTCP—where the agent dynamically searches the UTCP manual for only the tools it needs and executes the chained calls directly inside the sandbox.
4. Comparing the Approaches: MCP vs. UTCP vs. Code Mode
To keep the mental model clean, here is how traditional tool calling compares to Code Mode, and where MCP and UTCP fit:
| Area | Traditional Tool Calling (MCP) | Code Mode + MCP | Code Mode + UTCP |
|---|---|---|---|
| Main Purpose | Expose & call individual tools via proxy | Orchestrate MCP tools using code | Discover & orchestrate native APIs/CLIs using code |
| Infra Footprint | Requires running MCP wrapper servers | MCP servers + Code execution sandbox | 0 wrapper servers (JSON manual) + Sandbox |
| Upfront Schema Cost | High (All tool schemas loaded on Turn 1) | Medium/High (Unless behind a search bridge) |
Ultra-Low (search_tools loads only matching stubs) |
| Execution Flow | N sequential LLM round-trips | 1 generated script executes multiple calls | 1 generated script executes direct native calls |
| Intermediate Data | Enters and compounds in LLM context | Stays inside Python runtime memory | Stays inside Python runtime memory |
| Protocols | JSON-RPC over stdio / HTTP / SSE | JSON-RPC to MCP server | Native HTTP, gRPC, CLI, WebSockets, OpenAPI |
A useful rule of thumb for SREs:
- MCP standardizes tool access via proxy servers.
-
UTCP standardizes tool access via direct manuals (zero proxy servers + native
search_toolsdiscovery). - Code Mode lets the LLM compose those tools programmatically so raw telemetry never floods your context window.
5. Real-World SRE Example: Investigating a 5xx Incident (3 Ways)
Let's walk through our 10:30 AM HTTP 500 outage.
The on-call engineer asks:
"Investigate why the 5xx error rate increased around 10:30 AM. Check Prometheus, Loki, and Kubernetes. Identify the most likely cause."
Let's compare how this exact investigation looks across Approach A (Individual MCP Calls), Approach B (Code Mode over MCP), and Approach C (UTCP + Code Mode in Python).
Approach A: Individual Tool Calls (The Conversational Loop)
The LLM executes the investigation across four separate conversational turns:
-
Turn 1 (Prometheus): LLM calls
query_prometheus. The raw time-series array (500 data points) is dumped into the LLM context window. The LLM reads the points and sees the5xxerror rate jumped from0.5%to18%at10:30 AM. -
Turn 2 (Loki): LLM calls
query_loki. Loki returns 2,000 matching log lines (~80,000 tokens) directly into the LLM context window. -
Turn 3 (Kubernetes): LLM calls
get_pod_statusand receives full pod manifests (~15,000 tokens) in context, along with the previous 95,000 tokens of conversation history. - Turn 4 (Correlation): The LLM reasons across the massive accumulated context history and outputs its conclusion.
Approach B: Code Mode with Python (Conceptual Runtime Bridge)
Now imagine the LLM generates Python code and executes it in an isolated runtime. Instead of making 4 conversational turns and pulling 110,000 tokens of raw telemetry into the prompt, the LLM writes a single orchestration function that processes the data in Python memory:
async def investigate_5xx_incident(start: int, end: int):
# 1. Query the error rate from Prometheus
metrics = await tools.query_prometheus(
query='sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))',
start=start,
end=end,
)
# 2. Identify the first abnormal point in Python memory (not in LLM context!)
first_bad_point = find_threshold_crossing(metrics, threshold=0.05)
if not first_bad_point:
return {"status": "No significant increase detected"}
# 3. Search Loki ONLY around the abnormal time window (±5 mins)
logs = await tools.query_loki(
query='{namespace="production", app="checkout"} |= "ERROR"',
start=first_bad_point["timestamp"] - 300,
end=end,
)
# 4. Group and count 2,000 log lines in Python memory
error_summary = summarize_errors(logs)
# 5. Check Kubernetes pod states
pods = await tools.get_pod_status(
namespace="production",
deployment="checkout",
)
# 6. Return ONLY the distilled findings (~100 tokens) to the LLM
return {
"error_rate": first_bad_point,
"log_summary": error_summary,
"pod_status": summarize_pods(pods),
}
Here, tools is a controlled Python bridge inside the sandbox. Even if tool schemas stay in the prompt, none of the 110,000 tokens of intermediate metrics, logs, or pod manifests ever enter the LLM context window.
Approach C: Full UTCP + Code Mode Implementation in Python
What if we want to eliminate both the MCP wrapper servers and the upfront tool schema bloat in the LLM context window?
By combining the Python UTCP SDK (pip install utcp utcp-http utcp-cli) with a Code Mode execution loop, the workflow happens in two clean phases:
-
Progressive Schema Discovery (
search_tools): Instead of preloading 100 tool schemas into the prompt, the agent searches the UTCP manual for"prometheus loki kubernetes"and loads only the 3 compact Python call signatures (~400 tokens). -
Sandboxed UTCP Execution (
execute_code): The LLM writes a Python script usingutcp_client.call_tool(...)that runs inside the sandbox, talks directly to Prometheus (HTTP), Loki (HTTP), andkubectl(CLI) viasre_manual.json, filters the data in memory, and returns the compact JSON summary.
Here is the complete, runnable Python implementation showing how UTCP and Code Mode work together:
import asyncio
import json
from collections import Counter
from utcp.utcp_client import UtcpClient
# =====================================================================
# STEP 1: Progressive Tool Discovery (Evicts Idle Schemas from Context)
# =====================================================================
async def discover_sre_tools(client: UtcpClient, incident_query: str) -> str:
"""
Instead of loading 100+ MCP tool schemas into the LLM prompt on Turn 1,
we search the UTCP manual and return ONLY the relevant tool stubs (~300 tokens).
"""
matched_tools = await client.search_tools(query=incident_query, limit=5)
compact_stubs = []
for tool in matched_tools:
props = list(tool.inputs.get("properties", {}).keys())
compact_stubs.append(
f"- {tool.name}({', '.join(props)}): {tool.description}"
)
return "\n".join(compact_stubs)
# =====================================================================
# STEP 2: Code Mode Sandbox Execution (Generated by LLM, Run in Sandbox)
# =====================================================================
async def llm_generated_code_mode_script(client: UtcpClient, start_ts: int, end_ts: int) -> dict:
"""
In Code Mode, the LLM generates this Python orchestration block.
It calls Prometheus, Loki, and kubectl DIRECTLY via the UTCP manual,
processes 100K+ tokens of raw data in Python RAM, and returns ~100 tokens.
"""
# 1. Direct HTTP call to Prometheus via UTCP manual
raw_prom = await client.call_tool(
"sre_manual.query_prometheus",
{
"query": 'sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))',
"start": start_ts,
"end": end_ts,
},
)
prom_data = json.loads(raw_prom)
# 2. Find the exact timestamp where 5xx error rate crossed 5% (0.05)
spike_ts = None
peak_rate = 0.0
for series in prom_data.get("data", {}).get("result", []):
for ts, val in series.get("values", []):
rate_val = float(val)
if rate_val > 0.05 and spike_ts is None:
spike_ts = int(ts)
peak_rate = max(peak_rate, rate_val)
if not spike_ts:
return {"status": "healthy", "message": "No 5xx spike > 5% detected."}
# 3. Direct HTTP call to Loki ONLY for the 15-min window around spike_ts
raw_loki = await client.call_tool(
"sre_manual.query_loki",
{
"query": '{namespace="production", app="checkout"} |= "ERROR"',
"start": spike_ts - 300,
"end": spike_ts + 600,
},
)
loki_data = json.loads(raw_loki)
# 4. Aggregate 2,000 raw log lines in Python memory (Zero LLM context used!)
error_counts = Counter()
total_logs = 0
for stream in loki_data.get("data", {}).get("result", []):
for _, log_line in stream.get("values", []):
total_logs += 1
if "Database connection timeout" in log_line:
error_counts["Database connection timeout"] += 1
elif "Connection pool exhausted" in log_line:
error_counts["Connection pool exhausted"] += 1
else:
error_counts["Other application error"] += 1
# 5. Direct CLI call to Kubernetes (`kubectl get pods`) via UTCP manual
raw_k8s = await client.call_tool(
"sre_manual.get_pod_status",
{"namespace": "production", "deployment": "checkout"},
)
k8s_data = json.loads(raw_k8s)
restarting_pods = 0
for pod in k8s_data.get("items", []):
Statuses = pod.get("status", {}).get("containerStatuses", [])
if any(c.get("restartCount", 0) > 0 for c in Statuses):
restarting_pods += 1
# 6. Return ONLY the compact distilled summary back to the LLM context!
return {
"incident": "checkout-5xx-increase",
"spike_timestamp": spike_ts,
"peak_error_rate": f"{peak_rate * 100:.1f}%",
"total_error_logs_analyzed": total_logs,
"top_errors": error_counts.most_common(3),
"pods_with_restarts": restarting_pods,
}
# =====================================================================
# STEP 3: Main Agent Runner
# =====================================================================
async def main():
# Initialize UTCP Client pointing to our declarative JSON manual (No MCP server!)
client = await UtcpClient.create(
config={
"manual_call_templates": [
{
"name": "sre_manual",
"call_template_type": "text",
"file_path": "./sre_manual.json",
}
]
}
)
try:
# Phase 1: Discover only the tools needed for this incident
stubs = await discover_sre_tools(client, "prometheus metrics loki logs kubernetes pods")
print("=== 1. Compact Tool Stubs Sent to LLM (~120 tokens) ===")
print(stubs)
# Phase 2: Execute the LLM-generated Code Mode script inside the runtime
summary = await llm_generated_code_mode_script(
client, start_ts=1730523000, end_ts=1730526600
)
print("\n=== 2. Compact Investigation Output Returned to LLM (~90 tokens) ===")
print(json.dumps(summary, indent=2))
finally:
await client.close()
if __name__ == "__main__":
asyncio.run(main())
Example Investigation Output Returned to the LLM
Instead of ingesting 50+ preloaded tool schemas, 500 metric samples, and 2,000 raw log lines, the LLM receives this compact JSON payload (~90 tokens):
{
"incident": "checkout-5xx-increase",
"spike_timestamp": 1730524200,
"peak_error_rate": "18.0%",
"total_error_logs_analyzed": 2000,
"top_errors": [
["Database connection timeout", 1450],
["Connection pool exhausted", 420],
["Other application error", 130]
],
"pods_with_restarts": 4
}
The LLM can now explain the findings, separate observed facts from hypotheses, and recommend next steps—without drowning in raw telemetry or maintaining intermediate MCP servers.
6. Why Context Windows Matter: The Per-Turn Token Math
A common question engineers ask is: "Does Code Mode only shrink the context window if tool schemas are removed from the prompt?"
The answer is no—because in SRE observability workflows, Static Schema Tax and Dynamic Telemetry Payload Tax both compete for the context window, and traditional tool calling compounds payloads across every turn.
Let's count what the LLM actually receives per turn in our 10:30 AM 5xx investigation. Assume:
-
50 preloaded tool schemas:
8,000tokens -
Base system prompt + user query:
2,000tokens -
Raw telemetry responses:
15,000tokens (500 Prometheus samples),80,000tokens (2,000 Loki log lines), and15,000tokens (Kubernetes pod manifests)
Per-Turn Breakdown Across All 3 Architectures
Because LLM APIs are stateless, every subsequent turn in traditional tool calling re-sends the schemas plus all prior raw tool responses in the conversation history:
-
Architecture 1: Traditional MCP Tool Calling (Schemas Preloaded, Sequential Calls)
-
Turn 1 (Call Prometheus):
8K (schemas) + 2K (prompt)= 10,000 tokens -
Turn 2 (Read Prom, Call Loki):
8K + 2K + 15K (Prom raw)= 25,000 tokens -
Turn 3 (Read Loki, Call K8s):
8K + 2K + 15K + 80K (Loki raw)= 105,000 tokens -
Turn 4 (Read K8s, Output RCA):
8K + 2K + 15K + 80K + 15K (K8s raw)= 120,000 tokens
-
Turn 1 (Call Prometheus):
-
Architecture 2: Code Mode Without Schema Eviction (Schemas Kept, Payloads Filtered in Sandbox)
-
Turn 1 (Write Python Script):
8K (schemas) + 2K (prompt)= 10,000 tokens - (Sandbox runs Prom → Loki → K8s in memory and distills 110,000 tokens down to 150 tokens)
-
Turn 2 (Output RCA from Summary):
8K + 2K + 0.4K (script) + 0.15K (summary)= 10,550 tokens
-
Turn 1 (Write Python Script):
-
Architecture 3: UTCP + Code Mode With Schema Eviction (
search_tools+ Sandbox)-
Turn 1 (Search Tools):
0.4K (meta-tools) + 2K (prompt)= 2,400 tokens -
Turn 2 (Write Script with 3 Matched Stubs):
0.4K + 2K + 0.4K (3 stubs)= 2,800 tokens -
Turn 3 (Output RCA from Summary):
0.4K + 2K + 0.4K + 0.4K (script) + 0.15K (summary)= 3,350 tokens
-
Turn 1 (Search Tools):
| Metric | 1. Standard MCP | 2. Code Mode (Schemas Kept) | 3. UTCP + Code Mode (Schemas Evicted) |
|---|---|---|---|
| Upfront Schema Tokens / Turn | 8,000 | 8,000 | ~400 – 800 |
| Raw Telemetry in LLM Context | 110,000 | 150 | 150 |
| LLM Inference Turns | 4 turns | 2 turns | 2–3 turns |
| Peak Single-Turn Context | 120,000 | 10,550 | 3,350 |
| Cumulative Tokens Billed | 260,000 | 20,550 | 8,550 |
Key Takeaway:
-
Evicting schemas (via UTCP's
search_tools) solves theO(N_tools)static tax when your platform scales to dozens or hundreds of tools. -
Code Mode execution solves the
O(N_steps × Payload_Size)dynamic tax by keeping megabytes of time-series points and log lines inside Python memory rather than compounding them across conversational turns.
7. Where Code Mode + UTCP Makes Sense for SRE Teams
Not every operational query needs Code Mode. If you ask "What is the current CPU usage of payment-service?", a single direct tool call is fast and sufficient.
Code Mode and UTCP shine when the workflow requires multi-step correlation, conditional branching, or heavy data reduction:
| SRE Use Case | Why UTCP + Code Mode Helps |
|---|---|
| Incident Investigation | Correlates metrics, logs, and K8s state in one execution pass |
| Root Cause Analysis (RCA) | Runs fan-out queries across multiple microservices and compares results |
| Capacity Planning | Aggregates 30-day historical metrics and computes statistical trends locally |
| Deployment Canary Analysis | Compares error/latency distributions before and after a GitHub rollout |
| Alert Storm Triage | Groups 50+ firing alerts and extracts common upstream dependencies |
| High-Volume Log Analysis | Filters and deduplicates thousands of Loki/Elasticsearch lines in memory |
| SLO & Error Budget Burn | Calculates multi-window burn rates across services deterministically |
8. Production Safeguards for Python Execution Runtimes
If you introduce a Python execution runtime for Code Mode, safety and isolation are non-negotiable. An LLM-generated script must never have unrestricted access to your production environment.
flowchart TD
LLM[LLM Code Generator] --> Validator[AST / Policy Validator]
Validator --> Sandbox[Isolated Python Sandbox: gVisor / MicroVM]
Sandbox --> Bridge[Allowlisted Read-Only Tool Bridge]
Bridge --> Client[UTCP / MCP Client with Scoped Auth]
Client --> Infra[(Prometheus / Loki / K8s)]
Here is the production checklist every SRE team should enforce:
-
Strict Sandbox Execution: Never run generated code inside your host process using raw
exec(). Use isolated sandboxes (e.g., gVisor, Firecracker microVMs, ephemeral containers, or restricted WebAssembly/Deno sandboxes). -
Tool Allowlisting: Do not give the sandbox arbitrary network access. Expose only an explicit bridge of approved, read-only operational tools (
query_prometheus,query_loki,get_pod_status). -
Scoped Authentication: Use short-lived, read-only service account tokens. With UTCP, credentials (
${PROMETHEUS_TOKEN}) are injected natively at the client execution boundary so secrets are never exposed to the LLM prompt or generated code. - Hard Resource Limits: Enforce strict CPU, memory, execution timeout (e.g., 30s max), and payload-size limits on the sandbox.
- Query Guardrails: Validate PromQL and LogQL queries to block high-cardinality regex scans or unbounded time ranges that could OOM your monitoring stack.
- Full Audit Logging: Record the prompt, the exact generated code, every underlying tool call, and the user/session identity for post-incident compliance.
- Human-in-the-Loop for Mutations: Keep investigation 100% read-only. Any state-changing action (pod restarts, rollbacks, scaling) must require explicit human approval.
9. A Sensible Adoption Path for SRE Teams
You don't need to over-engineer your stack on Day 1. Follow a phased rollout:
- Phase 1: Expose Read-Only Tools (UTCP or MCP): Start by exposing Prometheus, Loki, and Kubernetes read-only queries. If you already have clean REST APIs, OpenAPI specs, and CLIs, UTCP manuals let you skip deploying wrapper servers; if you need a centralized proxy boundary, use MCP.
- Phase 2: Validate Basic Workflows: Let engineers run simple single-step queries to verify authentication, query guardrails, and accuracy.
- Phase 3: Identify Context-Heavy Bottlenecks: Spot the workflows where investigations hit token limits, suffer from slow multi-turn latency, or drown in raw logs.
-
Phase 4: Introduce Progressive Discovery & Sandboxed Code Mode: Use
search_toolsto keep idle tool schemas out of the prompt, and let the LLM generate orchestration scripts that filter telemetry outside the context window. - Phase 5: Measure SRE Impact: Track per-turn token reduction, investigation latency (MTTR), number of LLM turns, and root-cause accuracy.
Conclusion
MCP, UTCP, and Code Mode solve distinct pieces of the SRE automation puzzle:
- MCP standardizes how LLMs access tools through a consistent server interface.
- UTCP streamlines that access by letting agents discover and call your existing APIs and CLIs directly via declarative manuals, eliminating middleware server toil and upfront schema bloat.
- Code Mode gives the LLM a safe way to programmatically orchestrate those tools, filter noisy telemetry in memory, and return focused insights.
An incident investigation is rarely a single query—it is a sequence of hypotheses across metrics, logs, deployments, and cluster state. The future of AI in SRE isn't just giving models access to more tools; it's giving them a lightweight, safe architecture to combine those tools and reason over signal instead of noise.
Top comments (3)
This is a masterclass in breaking down architectural patterns for production operations! Comparing traditional individual tool calls to Code Mode using an SRE 5xx incident scenario makes the benefits of data preprocessing crystal clear.
Sending 2,000 raw Loki log lines directly into an LLM context window is a recipe for hallucinations and massive token waste. Letting an isolated Python runtime aggregate, filter, and summarize errors before handing compact findings back to the model is the exact way AI should be integrated into on-call workflows. Brilliant write-up! 👏
Code Mode helps only when tool schemas leave the context. Count what the model receives per turn before and after the switch. Does your context shrink after it?
Great question—let's run the exact per-turn token math for both Tool Schemas and Telemetry Payloads, because in SRE observability workflows, the context dynamics look very different from standard CRUD APIs.
You are spot on that if an agent preloads 100+ tool schemas into the system prompt and switches to Code Mode without stripping those schemas out (via progressive discovery like
search_tools), the Turn 1 static overhead doesn't shrink.However, saying Code Mode helps only when tool schemas leave the context misses the biggest bottleneck in SRE incident response: intermediate payload accumulation across sequential turns.
Let's count what the model actually receives per turn in our 10:30 AM
5xxinvestigation across three setups. Assume our prompt/history base is2,000tokens, and our raw telemetry payloads are15,000tokens (Prometheus),80,000tokens (2,000 Loki logs), and15,000tokens (K8s pod status).Scenario 1: Traditional MCP Tool Calling (50 tools preloaded = ~8,000 schema tokens)
Because LLM APIs are stateless, every subsequent turn re-sends the schemas plus all prior tool responses in the conversation history:
8K (schemas) + 2K (prompt)= 10,000 tokens8K (schemas) + 2K + 15K (Prom raw)= 25,000 tokens8K (schemas) + 2K + 15K + 80K (Loki raw)= 105,000 tokens8K (schemas) + 2K + 15K + 80K + 15K (K8s raw)= 120,000 tokensScenario 2: Code Mode WITHOUT Schema Eviction (Same 8,000 schema tokens stay in context)
Suppose we keep the exact same 8,000 tokens of tool definitions/stubs in context so the LLM knows what
tools.*functions it can write in Python, and execute the investigation in the sandbox:8K (schemas) + 2K (prompt)= 10,000 tokens8K (schemas) + 2K (prompt) + 0.4K (script) + 0.15K (summary)= 10,550 tokensEven if zero tool schemas leave the context, peak context shrinks from 120,000 tokens to 10,550 tokens (~91% reduction) and eliminates 2 LLM inference round-trips.
Scenario 3: Code Mode WITH Progressive Schema Discovery (UTCP
search_tools/ Meta-Tools)Now we also evict the upfront tool schemas—exposing only
search_toolsandexecute_code(~400 tokens) on Turn 1, and fetching only the 3 needed tool signatures (~600 tokens):0.4K (meta-tools) + 2K (prompt)= 2,400 tokens0.4K + 2K + 0.6K (3 stubs)= 3,000 tokens0.4K + 2K + 0.6K + 0.4K (script) + 0.15K (summary)= 3,550 tokensSummary Table: What the Model Receives
So in short:
O(N_tools)static tax when your platform scales to dozens/hundreds of tools.O(N_steps × Payload_Size)dynamic tax by keeping megabytes of time-series points and log lines inside the Python runtime rather than compounding them across conversational turns.