Automating Shadow DOM with Selenium WebDriver

How to drive web elements inside a shadow root on a web page

This article is included in my “How to in Selenium WebDriver” series.

A Shadow DOM is a self-contained web component in a web page. Google uses shadow DOMs extensively (link to their Shadow DOM introduction). By default, a standard automation driver cannot drive web elements inside a shadow DOM.

The latest Selenium WebDriver v4 supports Shadow Roots (HTML elements for a shadow DOM). In this article, I will show a quick and easy way to drive web elements inside a Shadow DOM using Selenium WebDriver.

An Example

The task: Add a new list item to the Editable List of the demo Fiddle site.

A typical Selenium script looks like this:

# add new list element
driver.find_element(:xpath, "//input[@class='add-new-list-item-input']").send_keys("Buy milk and eggs")

driver.find_element(:xpath, "//button[@class='add-new-list-item-input']").click

# verify new element is there
expect(page_text).to include("Buy milk and eggs")

But it won’t work on the demo site. Why? The editable list is inside a Shadow DOM and inaccessible — for now.

Finding the Shadow DOM

Open Inspect Element in Chrome (or equivalent for other browsers).

Looking at the page, there is a stand-out tag: <luigi-wc-2f77632f6c6973742e6a73> (highlighted in the screenshot below).

Underneath it is one element, which is how you can identify it. It is a bit like a frame (it has a title tag), except Selenium cannot find any elements within it.

Testing in a Shadow DOM

The approach:

1. Find the shadow root element

Go to the shadow root’s parent tag and get it using Javascript.

Note: XPaths do not work in Shadow DOM finding, so instead, use Tag names or CSS Selectors.

For our example, the shadow root tag is in the format luigi-wc- followed by a hex.

# regex this if you wish
shadow_element_wrapper_tag = "luigi-wc-2f77632f6c6973742e6a73"
elem = driver.find_element(:tag_name, shadow_element_wrapper_tag)

# get the shadow root using Javascript
shadow_root = driver.execute_script("return arguments[0].shadowRoot", elem)

The hex is hardcoded on this page. For dynamic ones, we can use Ruby scripting to extract them using regular expressions (see below).

Use execute_script to retrieve the shadow root and save it to a variable, shadow_root .

Update (2022–10–22): with new Selenium version, don’t need use execute_script .

shadow_root = elem.shadow_root

2. Using the shadow root as a driver to interact with elements inside it.

We now use shadow_root the same way as driver for the web elements inside it.

input_elem = shadow_root.find_element(:tag_name, "input")
input_elem.send_keys("Mario here I come!")
shadow_root.find_elements(:tag_name, "button").last.click

To verify the text outside the shadow root, switch back to the driver .

expect(driver.find_element(:tag, "body").text).to include("...")

And that’s how you can access shadow roots for automated tests in Selenium WebDriver.

Complete Script

# retrieve using regex for a dynamic shadow root
 def retrieve_shadow_root
    elem = driver.find_elements(:xpath, "//div[contains(@class, 'wcContainer svelte-')]").first
    
    elem_html = driver.execute_script("return arguments[0].outerHTML;", elem)
    # puts elem_html
    if elem_html =~ /<luigi-wc-([\d\w]+)>/
      shadow_element_wrapper_tag_name = "luigi-wc-" + $1
      elem = driver.find_element(:xpath, "//div[contains(@class, 'wcContainer svelte-')]/#{shadow_element_wrapper_tag_name}")
      return driver.execute_script("return arguments[0].shadowRoot", elem)
    end 
    return nil
end

shadow_root = retrieve_shadow_root()

input_elem = shadow_root.find_element(:tag_name, "input")    input_elem.send_keys("Mario, here I come")

button_elems = shadow_root.find_elements(:tag_name, "button")   add_button = button_elems.last    
add_button.click

The above works, but it won’t be easy to maintain. The refactored (based on Maintainable Automated Test Design) version can be accessed on this GitHub repository (link).

The test case looks like this.

it "Add and remove from list" do
  click_nav_web_component
  click_sub_nav("WC Editable List")

  editable_list_page = EditableListPage.new(driver)
  editable_list_page.enter_new_item("Mario, here I come!")
  editable_list_page.click_add_button()

  last_item_text = editable_list_page.last_item
  expect(last_item_text).to eq("Mario, here I come!")
  editable_list_page.click_remove_button()
  sleep 0.5
  last_item_text = editable_list_page.last_item
  expect(last_item_text).not_to eq("Mario, here I come!")
end

The shadow-root handling is encapsulated in the page classes, which makes the test scripts easier to read, and more importantly, easier to maintain.

Happy testing!