DEV Community

StareBrain
StareBrain

Posted on

Building StareBrain, Part 21: Natural Language Planning — Turning a Raw Instruction into an AgentTask

#android #kotlin #llm #mobile

Building StareBrain, Part 21: Natural Language Planning — Turning a Raw Instruction into an AgentTask

Parts 1–20 built the execution engine. We've covered how StareBrain runs tasks, handles failures, manages context, and resumes across sessions. Today we go one layer up: how a raw user instruction becomes an AgentTask with concrete AgentSteps before any execution begins.

This is the planning layer.

The Problem

A user says: "Order the same thing I got last time."

That's not a task. It's an intent. Before StareBrain can execute anything, it needs to decompose this into steps — open the app, find the last order, reorder it, confirm payment. Each step needs a target package, a concrete action, and enough context to know what success looks like.

The planning layer is what bridges the gap between what the user said and what the executor needs.

TaskPlanner

TaskPlanner takes the user's instruction and the current AgentContext, sends a prompt to the on-device model, and parses the response into an AgentTask:

class TaskPlanner(
    private val modelClient: OnDeviceModelClient
) {

    suspend fun plan(
        instruction: String,
        context: AgentContext
    ): PlanResult {
        val prompt = buildPlanningPrompt(instruction, context)
        val response = modelClient.generate(prompt)
        return parsePlanResponse(response, instruction)
    }
}
Enter fullscreen mode Exit fullscreen mode

The Planning Prompt

The prompt gives the model the instruction, the current app context, and a strict output format:

private fun buildPlanningPrompt(
    instruction: String,
    context: AgentContext
): String {
    return buildString {
        appendLine("You are a task planner for an Android agent.")
        appendLine("Break the following instruction into a list of concrete steps.")
        appendLine()
        appendLine("Instruction: $instruction")
        appendLine()
        if (context.activePackage != null) {
            appendLine("Current app: ${context.activePackage}")
        }
        if (context.lastUserIntent != null) {
            appendLine("Previous intent: ${context.lastUserIntent}")
        }
        appendLine()
        appendLine("Output format (JSON array):")
        appendLine("""
            [
              {
                "instruction": "Open Uber Eats",
                "targetPackage": "com.ubercooks.orderapp",
                "requiresConfirmation": false
              },
              ...
            ]
        """.trimIndent())
        appendLine()
        appendLine("Rules:")
        appendLine("- Each step must be a single, atomic action")
        appendLine("- Set requiresConfirmation true for any step that spends money, sends a message, or deletes data")
        appendLine("- If the instruction is ambiguous, include a clarification step first")
        appendLine("- Maximum 8 steps")
    }
}
Enter fullscreen mode Exit fullscreen mode

Parsing the Response

The model returns a JSON array. We parse it into AgentSteps:

private fun parsePlanResponse(
    response: String,
    originalInstruction: String
): PlanResult {
    return try {
        val json = extractJsonArray(response)
        val steps = Gson().fromJson(json, Array<StepJson>::class.java)
            .map { it.toAgentStep() }
        PlanResult.Success(
            AgentTask(
                description = originalInstruction,
                steps = steps
            )
        )
    } catch (e: Exception) {
        PlanResult.Failure("Could not parse plan: ${e.message}")
    }
}

private data class StepJson(
    val instruction: String,
    val targetPackage: String?,
    val requiresConfirmation: Boolean = false
)

private fun StepJson.toAgentStep() = AgentStep(
    instruction = instruction,
    targetPackage = targetPackage,
    requiresConfirmation = requiresConfirmation
)

sealed class PlanResult {
    data class Success(val task: AgentTask) : PlanResult()
    data class Failure(val reason: String) : PlanResult()
}
Enter fullscreen mode Exit fullscreen mode

Ambiguity Handling

If the instruction is too vague to plan ("do the thing"), the model is instructed to include a clarification step first:

// Model output for ambiguous instruction:
[
  {
    "instruction": "Ask user: which app do you want to use for this?",
    "targetPackage": null,
    "requiresConfirmation": false
  }
]
Enter fullscreen mode Exit fullscreen mode

A step with targetPackage: null and an instruction starting with "Ask user:" is treated as a ClarificationStep by TaskExecutor — it surfaces a prompt to the user via the overlay rather than attempting an action.

Connecting to the Pipeline

In StareBrainService, planning happens before execution:

private fun launchPipeline(instruction: String, context: AgentContext) {
    lifecycleScope.launch {
        overlayManager.setState(OverlayState.Planning)
        when (val result = taskPlanner.plan(instruction, context)) {
            is PlanResult.Success -> {
                taskPersistence.save(result.task)
                taskExecutor.execute(result.task)
            }
            is PlanResult.Failure -> {
                overlayManager.showError(result.reason)
            }
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

OverlayState.Planning is a new state — a brief indicator that StareBrain is thinking before it starts acting. It prevents the overlay from jumping straight to Running before any steps exist.

What's Next

Part 22 will cover plan validation — how StareBrain checks a generated plan for safety before execution begins, and what happens when a plan contains a step that ConfirmationGuard would catch at runtime.

Top comments (0)

⌂ 换网址