autoui

package
v0.9.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 17, 2026 License: MIT Imports: 21 Imported by: 0

Documentation

Overview

Package autoui provides EbitenUI automation helpers for autoebiten. It enables CLI-based widget tree inspection, search, method invocation, and visual debugging for LLM-assisted E2E testing.

Usage:

ui := ebitenui.UI{Container: root}
autoui.Register(&ui)

Commands registered:

  • autoui.tree: Export full widget tree as XML
  • autoui.at: Get widget at (x,y) coordinates
  • autoui.find: Simple key=value attribute search
  • autoui.xpath: XPath 1.0 queries
  • autoui.call: Reflection-based method invocation
  • autoui.highlight: Visual debugging highlights

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func AddHighlight

func AddHighlight(rect image.Rectangle)

AddHighlight adds a highlight for the given rectangle using the global manager.

func ClearHighlights

func ClearHighlights()

ClearHighlights removes all highlights immediately using the global manager.

func DrawHighlights

func DrawHighlights(screen *ebiten.Image)

DrawHighlights draws all active highlights onto the screen. This is the public API to be called in the game's Draw method after ui.Draw(screen).

func GetRadioGroup added in v0.9.0

func GetRadioGroup(name string) *widget.RadioGroup

GetRadioGroup returns a registered RadioGroup by name, or nil if not found.

func GetRegisteredRadioGroups added in v0.9.0

func GetRegisteredRadioGroups() []string

GetRegisteredRadioGroups returns all registered RadioGroup names. Used by tree traversal to inject RadioGroups into output.

func GetUI

func GetUI() *ebitenui.UI

GetUI returns the registered UI instance. Returns nil if no UI has been registered.

func InvokeMethod

func InvokeMethod(w widget.PreferredSizeLocateableWidget, methodName string, args []any) (any, error)

InvokeMethod invokes a method on a widget using reflection with whitelist safety. Also handles proxy methods that bypass reflection (e.g., SelectEntryByIndex).

Only whitelisted method signatures are allowed for security: - func() - func(any/interface{}) - func(bool), func(int), func(float64), func(string) - func(enum types) - types with underlying basic kind

Return values are converted to Go's "largest" representation for JSON compatibility: - Integers (int, int32, int64, enums) → int64 - Floats (float32, float64) → float64 - bool, string, slices → unchanged

func InvokeRadioGroupMethod added in v0.9.0

func InvokeRadioGroupMethod(rg *widget.RadioGroup, method string, args []any) (any, error)

InvokeRadioGroupMethod invokes a named method on a RadioGroup. Returns the result of the method call or an error if the method is not found or the method execution fails.

func MarshalWidgetTreeXML

func MarshalWidgetTreeXML(widgets []WidgetInfo) ([]byte, error)

MarshalWidgetTreeXML converts a flat list of widgets to a tree XML structure. It reconstructs the parent-child relationships and wraps the result in a <UI> root element. RadioGroups appear as children of the UI node (siblings of the root widget).

func MarshalWidgetXML

func MarshalWidgetXML(info WidgetInfo) ([]byte, error)

MarshalWidgetXML converts a single WidgetInfo to XML bytes. The widget type becomes the XML element name.

func MarshalWidgetsXML

func MarshalWidgetsXML(widgets []WidgetInfo) ([]byte, error)

MarshalWidgetsXML converts a flat list of widgets to XML elements. Unlike MarshalWidgetTreeXML, this returns widgets directly without hierarchy reconstruction or <UI> wrapper. Follows XPath convention.

func Register

func Register(ui *ebitenui.UI)

Register registers all autoui commands with the default "autoui." prefix. The UI must be non-nil; otherwise Register panics.

func RegisterRadioGroup added in v0.9.0

func RegisterRadioGroup(name string, rg *widget.RadioGroup)

RegisterRadioGroup registers a RadioGroup with a unique name. If a RadioGroup with the same name exists, it will be replaced. Use this to make RadioGroups discoverable via autoui.call and autoui.tree.

func RegisterWithOptions

func RegisterWithOptions(ui *ebitenui.UI, prefix string, opts *RegisterOptions)

RegisterWithOptions registers all autoui commands with a custom prefix and options. The UI must be non-nil; otherwise RegisterWithOptions panics. Use RegisterOptions.RWLock if you modify the UI tree from goroutines.

Example:

var uiLock sync.RWMutex
ui := &ebitenui.UI{Container: root}
autoui.RegisterWithOptions(ui, "autoui", &autoui.RegisterOptions{
    RWLock: &uiLock,
})

// In a goroutine that modifies the UI:
uiLock.Lock()
container.AddChild(newWidget)
uiLock.Unlock()

func RegisterWithPrefix

func RegisterWithPrefix(ui *ebitenui.UI, prefix string)

RegisterWithPrefix registers all autoui commands with a custom prefix. The UI must be non-nil; otherwise RegisterWithPrefix panics.

func SetConfig

func SetConfig(c Config)

SetConfig updates autoui configuration.

func SetHighlightDuration

func SetHighlightDuration(d time.Duration)

SetHighlightDuration configures the duration for new highlights.

func UnregisterRadioGroup added in v0.9.0

func UnregisterRadioGroup(name string)

UnregisterRadioGroup removes a registered RadioGroup by name. Call this when the RadioGroup is no longer needed.

Types

type CallRequest

type CallRequest struct {
	// Target is the query string to find the target widget.
	// Supports "key=value" or JSON format.
	Target string `json:"target"`

	// Method is the name of the method to invoke.
	Method string `json:"method"`

	// Args are the arguments to pass to the method.
	Args []any `json:"args"`
}

CallRequest represents a method invocation request.

type CallResponse

type CallResponse struct {
	// Success indicates if the invocation was successful.
	Success bool `json:"success"`

	// Error contains the error message if invocation failed.
	Error string `json:"error,omitempty"`

	// Result contains the return value from the method (if applicable).
	// Used for getter methods like ActiveIndex, TabIndex, etc.
	Result any `json:"result,omitempty"`
}

CallResponse represents the response from a method invocation.

type Config

type Config struct {
	HighlightDuration time.Duration
	HighlightColor    color.Color
}

Config holds autoui configuration.

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns default configuration.

type CoordinateRequest

type CoordinateRequest struct {
	X int `json:"x"`
	Y int `json:"y"`
}

CoordinateRequest represents a coordinate request in JSON format.

type ExistsResponse

type ExistsResponse struct {
	// Found indicates if any widgets matched the query.
	Found bool `json:"found"`

	// Count is the number of matching widgets.
	Count int `json:"count"`
}

ExistsResponse represents the response from the exists command. Returns JSON format for use with wait-for command.

type Highlight

type Highlight struct {
	// Rect is the screen rectangle to highlight.
	Rect image.Rectangle

	// Color is the highlight color.
	Color color.Color

	// ExpiresAt is the time when this highlight should be removed.
	ExpiresAt time.Time
}

Highlight represents a visual highlight rectangle for debugging.

type HighlightRequest

type HighlightRequest struct {
	Query string `json:"query"` // Query to find widgets to highlight
	X     int    `json:"x"`     // Optional: X coordinate for direct highlight
	Y     int    `json:"y"`     // Optional: Y coordinate for direct highlight
}

HighlightRequest represents a highlight request in JSON format.

type ProxyHandler added in v0.9.0

type ProxyHandler func(w widget.PreferredSizeLocateableWidget, args []any) (any, error)

ProxyHandler handles a named operation on a widget, bypassing the whitelist.

func GetProxyHandler added in v0.9.0

func GetProxyHandler(name string) ProxyHandler

GetProxyHandler returns the handler for a proxy method name, or nil if not found.

type RadioGroupElementInfo added in v0.9.0

type RadioGroupElementInfo struct {
	Type   string `json:"type"`
	Active bool   `json:"active"`
	Label  string `json:"label,omitempty"`
}

RadioGroupElementInfo represents information about a RadioGroup element.

type RadioGroupHandler added in v0.9.0

type RadioGroupHandler func(rg *widget.RadioGroup, args []any) (any, error)

RadioGroupHandler handles a named operation on a RadioGroup.

type RegisterOptions

type RegisterOptions struct {
	// RWLock protects concurrent UI tree access.
	// Provide this if you modify the UI tree from goroutines.
	// Acquire WriteLock when modifying, autoui acquires RLock during traversal.
	RWLock *sync.RWMutex
}

RegisterOptions holds optional configuration for autoui registration.

type TabInfo added in v0.9.0

type TabInfo struct {
	Index    int    `json:"index"`
	Label    string `json:"label"`
	Disabled bool   `json:"disabled"`
}

TabInfo represents information about a tab in a TabBook.

type WidgetInfo

type WidgetInfo struct {
	// Widget is the underlying widget instance.
	Widget widget.PreferredSizeLocateableWidget

	// Type is the widget type name (e.g., "Button", "Container").
	Type string

	// Rect is the widget's screen rectangle.
	Rect image.Rectangle

	// Visible indicates if the widget is currently visible.
	Visible bool

	// Disabled indicates if the widget is disabled.
	Disabled bool

	// State contains widget-specific state attributes.
	// Keys depend on widget type (e.g., "text", "checked", "value").
	State map[string]string

	// CustomData contains extracted custom data attributes.
	// Flattened from the widget's CustomData field.
	CustomData map[string]string

	// Addr is the widget pointer address for unique identification.
	// Format: "0x14000abc0" (hex string).
	Addr string
}

WidgetInfo holds extracted information about a widget. This is the internal representation used before XML conversion.

func ExtractWidgetInfo

func ExtractWidgetInfo(w widget.PreferredSizeLocateableWidget) WidgetInfo

ExtractWidgetInfo extracts widget information from a widget instance. It captures the widget's type, geometry, visibility, state, and custom data.

func FindAt

func FindAt(widgets []WidgetInfo, x, y int) *WidgetInfo

FindAt finds the top-most widget at the given coordinates. It searches in reverse order (last widget first) to find widgets added later. Returns nil if no widget contains the point.

func FindByQuery

func FindByQuery(widgets []WidgetInfo, query string) []WidgetInfo

FindByQuery finds widgets matching a simple key=value query. The query format is "key=value" for exact match. Empty query returns all widgets.

func FindByQueryJSON

func FindByQueryJSON(widgets []WidgetInfo, query string) []WidgetInfo

FindByQueryJSON finds widgets matching JSON criteria with AND logic. The query format is `{"key":"value",...}` - all criteria must match. Invalid JSON returns empty results.

func QueryXPath

func QueryXPath(widgets []WidgetInfo, xpathExpr string) ([]WidgetInfo, error)

QueryXPath executes an XPath 1.0 expression against the widget tree. It converts widgets to XML, executes the XPath query, and maps results back to WidgetInfo. Returns an error if the XPath expression is invalid or XML parsing fails.

func SnapshotTree

func SnapshotTree(ui *ebitenui.UI) []WidgetInfo

SnapshotTree returns a snapshot of the widget tree from the UI. If RWLock was provided via RegisterOptions, it acquires RLock before traversal.

func WalkTree

WalkTree traverses the widget hierarchy depth-first and returns a flat list of all widgets. It starts from the root widget and recursively visits all children in Container widgets. Registered RadioGroups are appended at the end of traversal.

type WidgetNode

type WidgetNode struct {
	XMLName  xml.Name
	Attrs    map[string]string
	Children []*WidgetNode
}

WidgetNode represents a widget as an XML node with dynamic element names. It implements xml.Marshaler for custom XML element name generation.

func (*WidgetNode) MarshalXML

func (n *WidgetNode) MarshalXML(e *xml.Encoder, start xml.StartElement) error

MarshalXML implements xml.Marshaler for custom element name generation. It writes the element with its type as the name and all attributes sorted alphabetically.

func (*WidgetNode) UnmarshalXML

func (n *WidgetNode) UnmarshalXML(d *xml.Decoder, start xml.StartElement) error

UnmarshalXML implements xml.Unmarshaler for WidgetNode. It reads the element and populates the Attrs map.

Directories

Path Synopsis
Package internal provides helper functions for autoui package.
Package internal provides helper functions for autoui package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
⌂ 换网址