<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Mohammad Bahauddin</title>
    <description>The latest articles on DEV Community by Mohammad Bahauddin (@mohd_bahauddin_afb498f213).</description>
    <link>https://dev.to/mohd_bahauddin_afb498f213</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4219487%2Fc2f13a4b-ac78-4694-a75b-d734bf13d9e8.png</url>
      <title>DEV Community: Mohammad Bahauddin</title>
      <link>https://dev.to/mohd_bahauddin_afb498f213</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/mohd_bahauddin_afb498f213"/>
    <language>en</language>
    <item>
      <title>Understanding Playwright's Accessibility Tree: Providing Better Context for AI-Driven Locator Generation</title>
      <dc:creator>Mohammad Bahauddin</dc:creator>
      <pubDate>Tue, 03 Nov 2026 05:26:29 +0000</pubDate>
      <link>https://dev.to/mohd_bahauddin_afb498f213/understanding-playwrights-accessibility-tree-providing-better-context-for-ai-driven-locator-e4c</link>
      <guid>https://dev.to/mohd_bahauddin_afb498f213/understanding-playwrights-accessibility-tree-providing-better-context-for-ai-driven-locator-e4c</guid>
      <description>&lt;p&gt;&lt;a href="https://dev.to/t/testing"&gt;#testing&lt;/a&gt; &lt;a href="https://dev.to/t/automation"&gt;#automation&lt;/a&gt; &lt;a href="https://dev.to/t/playwright"&gt;#playwright&lt;/a&gt; #ai #agenticai&lt;/p&gt;

&lt;p&gt;In the &lt;a href="https://dev.to/"&gt;first article&lt;/a&gt; of this series, we explored why generic locator generation can become challenging when automating complex enterprise applications such as Salesforce Lightning.&lt;/p&gt;

&lt;p&gt;We discussed how multiple elements can share identical accessible names and why identifying an element without understanding its surrounding context can lead to unreliable automation.&lt;/p&gt;

&lt;p&gt;This raises an interesting question:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What if we could provide AI with a structured representation of the page instead of asking it to identify elements from an entire page?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;One approach worth exploring is Playwright's Accessibility Tree and its accessibility snapshot capabilities.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Understanding the Accessibility Tree
&lt;/h2&gt;

&lt;p&gt;A browser does not simply render HTML elements visually. It also exposes information about the interface to assistive technologies such as screen readers.&lt;/p&gt;

&lt;p&gt;This information is represented through the Accessibility Tree.&lt;/p&gt;

&lt;p&gt;Unlike the DOM, which describes the document structure, the Accessibility Tree focuses on how elements are presented and understood by assistive technologies.&lt;/p&gt;

&lt;p&gt;For example, consider the following HTML:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight html"&gt;&lt;code&gt;&lt;span class="nt"&gt;&amp;lt;button&lt;/span&gt; &lt;span class="na"&gt;aria-label=&lt;/span&gt;&lt;span class="s"&gt;"Edit customer record"&lt;/span&gt;&lt;span class="nt"&gt;&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;svg&amp;gt;&lt;/span&gt;
    &lt;span class="c"&gt;&amp;lt;!-- Edit icon --&amp;gt;&lt;/span&gt;
  &lt;span class="nt"&gt;&amp;lt;/svg&amp;gt;&lt;/span&gt;
&lt;span class="nt"&gt;&amp;lt;/button&amp;gt;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The DOM contains a button and an SVG element.&lt;/p&gt;

&lt;p&gt;However, its accessibility representation can expose information such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Role: button&lt;/li&gt;
&lt;li&gt;Accessible name: Edit customer record&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This semantic information is particularly useful when working with Playwright.&lt;/p&gt;

&lt;p&gt;Instead of depending entirely on CSS selectors or DOM attributes, we can use accessible roles and names to identify elements.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;button&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Edit customer record&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;click&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is one of the reasons Playwright recommends user-facing locators such as roles and labels.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Exploring Accessibility Snapshots in Playwright
&lt;/h2&gt;

&lt;p&gt;Playwright provides the &lt;code&gt;locator.ariaSnapshot()&lt;/code&gt; API to retrieve an accessibility-oriented representation of a locator's contents.&lt;/p&gt;

&lt;p&gt;Let's look at a simple example.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;snapshot&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;locator&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;body&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;ariaSnapshot&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

&lt;span class="nx"&gt;console&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nx"&gt;snapshot&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The output is a YAML-like representation of accessible elements.&lt;/p&gt;

&lt;p&gt;For example, a simplified representation might look like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;main&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;heading "Customer Cases" [level=1]&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;region "Parent Case"&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;heading "Case&lt;/span&gt; &lt;span class="c1"&gt;#1001"&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;button "Edit"&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;region "Child Case"&lt;/span&gt;&lt;span class="err"&gt;:&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;heading "Case&lt;/span&gt; &lt;span class="c1"&gt;#1002"&lt;/span&gt;
    &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;button "Edit"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The example above is illustrative. Actual output depends on the application's accessible markup and browser state.&lt;/p&gt;

&lt;p&gt;Notice something interesting.&lt;/p&gt;

&lt;p&gt;Instead of receiving only a collection of buttons named Edit, we can potentially understand the sections in which those buttons appear.&lt;/p&gt;

&lt;p&gt;This additional information can help distinguish between otherwise similar elements.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Why structured accessibility information matters
&lt;/h2&gt;

&lt;p&gt;Consider a Salesforce workflow involving parent and child cases.&lt;/p&gt;

&lt;p&gt;Both records contain an Edit button.&lt;/p&gt;

&lt;p&gt;A generic locator might look like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;button&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Edit&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If multiple buttons match, Playwright's strictness checks prevent the automation from arbitrarily selecting one.&lt;/p&gt;

&lt;p&gt;We could use positional selection:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;button&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Edit&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="p"&gt;}).&lt;/span&gt;&lt;span class="nf"&gt;nth&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;However, this introduces a dependency on element ordering.&lt;/p&gt;

&lt;p&gt;A more context-aware approach would be to identify the relevant section first.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;parentCase&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;
  &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;region&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Parent Case&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
  &lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;parentCase&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;button&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Edit&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;exact&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;click&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This approach limits the search to the intended region.&lt;/p&gt;

&lt;p&gt;The important distinction is that we are no longer identifying the button independently. We are identifying it within the context of the parent case.&lt;/p&gt;

&lt;p&gt;Of course, this assumes the application exposes a meaningful accessible region with the appropriate name. Real Salesforce markup may require a different scoping strategy.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Where AI-driven automation fits into this
&lt;/h2&gt;

&lt;p&gt;Now let's introduce AI into the workflow.&lt;/p&gt;

&lt;p&gt;Imagine an AI agent receiving the following instruction:&lt;/p&gt;

&lt;p&gt;"Open the parent case and change its status to In Progress."&lt;/p&gt;

&lt;p&gt;The agent needs to understand:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Which record represents the parent case?&lt;/li&gt;
&lt;li&gt;Where is the Edit button associated with that record?&lt;/li&gt;
&lt;li&gt;Which Status field belongs to the selected record?&lt;/li&gt;
&lt;li&gt;Which option should be selected?&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;If we provide only a large collection of page elements, the model needs to infer these relationships from potentially ambiguous information.&lt;/p&gt;

&lt;p&gt;However, if we provide a structured accessibility snapshot, the model gets additional semantic information.&lt;/p&gt;

&lt;p&gt;A simplified instruction could look like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Task:
Update the Parent Case status to In Progress.

Relevant page structure:

Region: Parent Case
  Heading: Case #1001
  Button: Edit

Region: Child Case
  Heading: Case #1002
  Button: Edit

Instruction:
Identify the Edit button within the Parent Case region.
Do not interact with the Child Case.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This is not a complete AI automation implementation. It illustrates how structured context and explicit instructions can help reduce ambiguity during locator identification.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Accessibility snapshots are useful, but not a complete solution
&lt;/h2&gt;

&lt;p&gt;There is an important limitation worth understanding.&lt;/p&gt;

&lt;p&gt;The Accessibility Tree does not necessarily expose every element present in the DOM.&lt;/p&gt;

&lt;p&gt;Some elements may be excluded because they are not accessible, are hidden, or do not contribute meaningful accessibility information.&lt;/p&gt;

&lt;p&gt;Additionally, Salesforce Lightning contains dynamically rendered components, custom controls and complex interactions.&lt;/p&gt;

&lt;p&gt;An accessibility snapshot may not always provide enough information to identify every required element.&lt;/p&gt;

&lt;p&gt;For example:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Custom dropdowns may expose different accessibility structures.&lt;/li&gt;
&lt;li&gt;Dynamically rendered components may change the snapshot.&lt;/li&gt;
&lt;li&gt;Multiple controls can still have identical accessible names.&lt;/li&gt;
&lt;li&gt;Some elements may require additional DOM-level inspection.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Therefore, accessibility snapshots should not be treated as a replacement for the DOM or as a guaranteed solution to locator ambiguity.&lt;/p&gt;

&lt;p&gt;Instead, they can serve as another source of structured information for an automation agent.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. What can we take away from this?
&lt;/h2&gt;

&lt;p&gt;Playwright already provides powerful locator strategies.&lt;/p&gt;

&lt;p&gt;The challenge in AI-driven automation is not simply generating a valid locator. It is generating a locator that identifies the correct element within the intended business context.&lt;/p&gt;

&lt;p&gt;Accessibility information provides a useful foundation because it exposes semantic information such as roles, accessible names and relationships between interface elements.&lt;/p&gt;

&lt;p&gt;When combined with contextual instructions and scoped locator strategies, this information can help an AI agent make more informed locator decisions.&lt;/p&gt;

&lt;p&gt;However, there is still an important question:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Can we make AI identify the relevant section first, and then generate locators only within that section?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;That is what I want to explore next.&lt;/p&gt;

&lt;h2&gt;
  
  
  What's next?
&lt;/h2&gt;

&lt;p&gt;In Part 3, I'll explore a context-aware locator generation approach using Playwright, TypeScript and AI.&lt;/p&gt;

&lt;p&gt;We'll look at how an automation agent can:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Understand the requested business context.&lt;/li&gt;
&lt;li&gt;Identify the relevant section of the application.&lt;/li&gt;
&lt;li&gt;Extract useful accessibility information.&lt;/li&gt;
&lt;li&gt;Generate scoped Playwright locators.&lt;/li&gt;
&lt;li&gt;Validate that the identified element matches the intended target.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The goal is to move beyond generic locator generation towards context-aware AI-driven browser automation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Part 3: Building a Context-Aware AI Locator Generation Approach with Playwright.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;#playwright #typescript #javascript #testautomation #AI #AIBasedAutomation #softwaretesting&lt;/p&gt;

</description>
      <category>playwright</category>
      <category>automationtesting</category>
      <category>genai</category>
      <category>agentaichallenge</category>
    </item>
    <item>
      <title># Automating Salesforce Lightning with Playwright: Why Generic Locators Become a Challenge</title>
      <dc:creator>Mohammad Bahauddin</dc:creator>
      <pubDate>Sun, 01 Nov 2026 10:25:02 +0000</pubDate>
      <link>https://dev.to/mohd_bahauddin_afb498f213/automating-salesforce-ui-with-playwright-why-generic-locators-become-a-challenge-9el</link>
      <guid>https://dev.to/mohd_bahauddin_afb498f213/automating-salesforce-ui-with-playwright-why-generic-locators-become-a-challenge-9el</guid>
      <description>&lt;p&gt;Salesforce Lightning applications can be challenging to automate, particularly when dealing with complex page structures, dynamically rendered components and repeated UI interactions.&lt;/p&gt;

&lt;p&gt;One challenge that becomes increasingly noticeable when working with browser automation is &lt;strong&gt;locator identification&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;In a simple application, identifying an element by its role, accessible name or text may be sufficient. However, enterprise applications such as Salesforce often involve complex workflows where similar interactions are performed across different records and sections.&lt;/p&gt;

&lt;p&gt;This becomes even more interesting when introducing AI into test automation.&lt;/p&gt;

&lt;h2&gt;
  
  
  The challenge with generic locator generation
&lt;/h2&gt;

&lt;p&gt;Playwright provides several powerful ways to identify elements, including accessible roles, names, text and other locator strategies.&lt;/p&gt;

&lt;p&gt;For example:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;button&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Edit&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;click&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This works when the target button is unique.&lt;/p&gt;

&lt;p&gt;But what happens when the same button appears in multiple sections of the page?&lt;/p&gt;

&lt;p&gt;Playwright uses strictness checks for operations that require a single element. If a locator matches multiple elements, it throws a strict mode violation rather than arbitrarily selecting one.&lt;/p&gt;

&lt;p&gt;One possible workaround is positional selection using &lt;code&gt;nth()&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;button&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Edit&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;
&lt;span class="p"&gt;}).&lt;/span&gt;&lt;span class="nf"&gt;nth&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;click&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Although this can work, it introduces a dependency on element ordering. If the page structure changes or another matching element is introduced, the locator may target the wrong element.&lt;/p&gt;

&lt;p&gt;The problem is not necessarily the locator strategy itself. It is the lack of sufficient context to distinguish between elements.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Finding an element is one thing. Identifying the correct element within its intended context is another.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  A practical challenge in Salesforce Lightning
&lt;/h2&gt;

&lt;p&gt;Consider a Salesforce workflow involving parent and child cases.&lt;/p&gt;

&lt;p&gt;Both records may follow a similar status-update process. The automation needs to open the relevant case, click Edit, locate the Status field, open its dropdown and select the required value, such as In Progress, Approved or Rejected.&lt;/p&gt;

&lt;p&gt;Although the interaction pattern is similar, the automation must operate on the correct record.&lt;/p&gt;

&lt;p&gt;For example, updating the parent case should not accidentally update the child case, and vice versa.&lt;/p&gt;

&lt;p&gt;This is where locator identification becomes important.&lt;/p&gt;

&lt;p&gt;When using AI-driven automation, the model needs to inspect the available page information and identify the relevant elements for each interaction.&lt;/p&gt;

&lt;p&gt;Depending on the framework implementation, it may also need to generate or retrieve locator information as the workflow progresses.&lt;/p&gt;

&lt;p&gt;Sometimes, the AI fails to identify the expected element or misses the relevant locator information. The interaction then fails, potentially causing the entire test case to fail.&lt;/p&gt;

&lt;p&gt;The underlying issue is not that Playwright cannot interact with these elements. It is that the AI must consistently identify the correct element within the correct application context.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why AI-driven automation introduces another challenge
&lt;/h2&gt;

&lt;p&gt;AI-assisted automation adds another dimension to locator identification.&lt;/p&gt;

&lt;p&gt;An LLM can inspect page information and attempt to identify the target element based on the context it receives.&lt;/p&gt;

&lt;p&gt;However, when that information contains numerous elements with similar descriptions, the model must distinguish between them and understand their relationships.&lt;/p&gt;

&lt;p&gt;For complex enterprise applications, providing an entire page representation can introduce a significant amount of information that may not be relevant to the requested interaction.&lt;/p&gt;

&lt;p&gt;The model needs to determine:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Which section or record contains the intended element?&lt;/li&gt;
&lt;li&gt;What is the relationship between the parent and child elements?&lt;/li&gt;
&lt;li&gt;Are there multiple elements with identical or similar accessible names?&lt;/li&gt;
&lt;li&gt;Which element belongs to the requested business context?&lt;/li&gt;
&lt;li&gt;Is the identified element relevant to the current step of the workflow?&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;For example, locating a Status field is not enough if the page contains multiple records or sections with similar fields.&lt;/p&gt;

&lt;p&gt;The model must first understand which case is being updated and then identify the Status field belonging to that case.&lt;/p&gt;

&lt;p&gt;This makes context an important part of AI-driven locator generation.&lt;/p&gt;

&lt;h2&gt;
  
  
  What if we changed the way we provide context?
&lt;/h2&gt;

&lt;p&gt;Instead of asking AI to identify an element across the entire page, what if we first helped it understand the relevant section?&lt;/p&gt;

&lt;p&gt;Imagine a page containing multiple customer records, each with its own Edit button.&lt;/p&gt;

&lt;p&gt;Rather than identifying an Edit button globally, we can first identify the relevant customer container and then locate the button within that container.&lt;/p&gt;

&lt;p&gt;This is the principle behind scoped locator strategies.&lt;/p&gt;

&lt;p&gt;Playwright supports this through locator chaining and filtering:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight typescript"&gt;&lt;code&gt;&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;customerSection&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;
  &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByTestId&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;customer-card&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
  &lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
    &lt;span class="na"&gt;has&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nx"&gt;page&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;heading&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
      &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Customer A&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
      &lt;span class="na"&gt;exact&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
    &lt;span class="p"&gt;})&lt;/span&gt;
  &lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="kd"&gt;const&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;customerSection&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;getByRole&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;button&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="s1"&gt;Edit&lt;/span&gt;&lt;span class="dl"&gt;'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
  &lt;span class="na"&gt;exact&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="p"&gt;});&lt;/span&gt;

&lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="nx"&gt;editButton&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;click&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Here, the first locator identifies the relevant customer container. The second locator searches for the Edit button within that container.&lt;/p&gt;

&lt;p&gt;This establishes a parent-child relationship between the locators rather than searching for the button across the entire page.&lt;/p&gt;

&lt;p&gt;The example uses illustrative test IDs and headings; actual selectors depend on the application's markup and accessibility structure.&lt;/p&gt;

&lt;p&gt;We should still validate that the parent and child locators resolve to exactly one intended element.&lt;/p&gt;

&lt;p&gt;Scoped locators provide a way to reduce ambiguity by limiting element identification to the relevant application context.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where does this leave AI-driven automation?
&lt;/h2&gt;

&lt;p&gt;Scoped locators help address element ambiguity, but how can we make an AI agent understand which scope it should use in the first place?&lt;/p&gt;

&lt;p&gt;Can we improve AI-generated locator identification by providing structured accessibility information and explicit contextual instructions instead of asking the model to search through an entire page representation?&lt;/p&gt;

&lt;p&gt;What if the AI could first identify the relevant scope and then search for the requested element only within that context?&lt;/p&gt;

&lt;p&gt;This is the approach I want to explore in this series.&lt;/p&gt;

&lt;p&gt;In the next article, I'll look at Playwright's Accessibility Tree, how semantic information can be represented in a structured format, and why it can provide useful context for AI-driven browser automation.&lt;br&gt;
What's next?&lt;br&gt;
This is the first article in a series exploring context-aware AI-driven locator generation with Playwright.&lt;/p&gt;

&lt;p&gt;In the upcoming articles, I'll explore Playwright's Accessibility Tree and how structured accessibility information, natural-language instructions and scoping techniques can help AI identify elements more reliably.&lt;/p&gt;

&lt;p&gt;Stay tuned for Part 2!&lt;/p&gt;

&lt;h1&gt;
  
  
  playwright #javaScript #Typescript #CI/CD #AI #AIBasedAutomation
&lt;/h1&gt;

</description>
      <category>automation</category>
      <category>testing</category>
      <category>webdev</category>
    </item>
  </channel>
</rss>
