Selenium 4 Chrome DevTools Examples

A few Selenium CDP examples in the Ruby language

This article is included in my “How to in Selenium WebDriver” series.

Selenium WebDriver 4 supports Chrome DevTools Protocol (CDP), enabling a debugger inside Chromium-based browsers. By the way, Cypress and Puppeteer are based on CDP. CDP has some issues and limitations, not surprisingly, it is a debugging protocol after all.

In this article, I will show some DevTools examples with the new Selenium 4.

Table of Contents:
· General
  ∘ Navigate to a URL
  ∘ Print to PDF
  ∘ Get browser Client Size
  ∘ Get browser info
  ∘ Execute CDP Command
· Event
  ∘ Console log messages
· Network
  ∘ Basic Authentication
· Emulate
  ∘ GEO Location
· Page
  ∘ Download file
  ∘ Navigation history
· Summary

While Selenium 4 supports CDP, it is not included by default. You need to install a separate gem ( ruby library):

> gem install — no-document selenium-devtools
Successfully installed selenium-devtools-0.94.0
1 gem installed

You will see the first limitation, CDP is associated with a particular version of Chrome. For example, I got the following error after the Chrome browser self-updated to v95.

Failure/Error: driver.devtools.page.enable
LoadError:
 cannot load such file — selenium/devtools/v95
 # ./spec/ch27_chrome_devtools_spec.rb:20

The message was quite clear, updating selenium-devtools gem should fix it. However, the “selenium-devtools-0.95” is not released yet.

General

Navigate to a URL

No difference from driver.get .

driver.devtools.page.navigate(url: “https://travel.agileway.net")
expect(driver.title).to eq(“Agile Travel”)

Print to PDF

Printing a web page to PDF via DevTools works only in headless mode. By the way, Selenium v4 already has save_print_page for this.

driver.get("https://whenwise.com")
data = driver.devtools.page.print_to_pdf
require "base64"
File.open("/tmp/a.pdf","wb").write(Base64.decode64(data["result"]["data"]))

Get browser window Size

Return the size of the browser window, excluding the address bar and others.

layout_metrics = driver.devtools.page.get_layout_metrics["result"]["layoutViewport"]
browser_client_width = layout_metrics["clientWidth"]
browser_client_height= layout_metrics["clientHeight"]
puts("Client Size: #{browser_client_width} x #{browser_client_height}") 
# => 1024 x 720

Get browser info

Return the info about the browser window.

browser_version_obj = driver.devtools.browser.get_version
puts browser_version_obj.inspect

Sample output:

{"id"=>3, "result"=>{"protocolVersion"=>"1.3", "product"=>"Chrome/94.0.4606.81",
 "revision"=>"@5a03c5f1033171d5ee1671d219a59e29cf75e054", 
 "userAgent"=>"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36", 
 "jsVersion"=>"9.4.146.21"}, "sessionId"=>"E7EC8D0B49DB2CA25B41840670175D06"}

Execute CDP Command

selenium-devtools is a library to communicate CDP with commands. You can send these commands directly, like the one below.

 driver.devtools.send_cmd(“Network.clearBrowserCache”)

The CDP commands are organized in a set of categories, for example, here is the command list for NetWork.

Event

CDP can ‘monitor’ browser events.

Console log messages

The scripts below will save every console event to a list logs that we defined.

logs = []
driver.on_log_event(:console) { |log| logs.push(log) }

Later, when log entries are added to the browser console. We can verify them.

driver.execute_script(“console.log(‘I am a wise tester’)”)
expect(logs.last.args).to eq([“I am a wise tester”])
puts logs.inspect
Here is a dump of the console log entries. 
 
```
[#<Selenium::WebDriver::DevTools::ConsoleEvent:0x00007fac0520c098 @type=:log, 
@timestamp=2021–10–18 11:58:34 1598475/2097152 +1000, 
@args=[“[Companion]”, “Service worker registered!”]>,
 #<Selenium::WebDriver::DevTools::ConsoleEvent:0x00007fac0526cb28 @type=:lo0, 
 @timestamp=2021–10–18 11:58:36 180343/524288 +1000, @args=[“I am a wise tester”]>] 

Network

We can modify network requests, such as HTTP headers, with CDP.

Basic Authentication

You have seen two recipes for handling basic authentication in Selenium WebDriver. Here is another one with CDP, by injecting an authorization header into the browser request.

driver.devtools.network.enable # must have enable network
require “base64”
# need to strip “\n”, otherwise return ‘Invalid header value’
basic_auth = Base64.encode64(“test:test”).strip 
headers = { “Authorization” => “Basic “ + basic_auth }
driver.devtools.send_cmd(“Network.setExtraHTTPHeaders”, headers: headers)driver.get(“http://browserspy.dk/password-ok.php")
driver.find_element(:link_text, “Test again”) # pass basic auth

Emulate

CDP provides a set of commands to emulate the client’s devices, locations and event network conditions.

GEO Location

coordinates = { latitude: 35.689487,
                longitude: 139.691706,
                accuracy: 100 }
driver.execute_cdp("Emulation.setGeolocationOverride", coordinates)
driver.get("https://my-location.org")
try_for(6) { expect(driver.find_element(:id, "address").text).to include("Tokyo") }
coordinates ={ latitude: -33.8947, longitude: 151.08374, accuracy: 100 }
driver.devtools.send_cmd("Emulation.setGeolocationOverride", coordinates)
driver.get("https://my-location.org")
try_for(6) { 
  expect(driver.find_element(:id, "address").text).to include("NSW") 
}

Please note that the above scripts work on Windows at the time of writing, but not macOS.

Page

CDP offers some extra automation features on a web page.

Download file

We can tell Selenium to download files to a specified folder, using Chrome preferences. With CDP, the syntax is easier.

driver.devtools.page.enable
dl_dir = RUBY_PLATFORM =~ /mingw/ ? "C:/Temp" : "/tmp"
driver.devtools.page.set_download_behavior(behavior: "allow",  download_path: dl_dir)
driver.get "http://zhimin.com/books/pwta"
driver.find_element(:link_text, "Download").click
downloaded_file = File.join(download_dir, "practical-web-test-automation-sample.pdf")
try_for(40, 5) { expect(File.exists?(downloaded_file)).to eq(true) }

Navigation History

Get a list of the pages that you have visited.

driver.devtools.page.enable
driver.devtools.page.reset_navigation_history # has no effect
driver.get("https://travel.agileway.net")
driver.get("https://whenwise.agileway.net")
history_sites = driver.devtools.page.get_navigation_history["result"]["entries"].collect{ |x| x["url"] }
expect(history_sites[0]).to eq("data:,")
expect(history_sites.last).to eq("https://whenwise.agileway.net/")
expect(history_sites[-2]).to eq("https://travel.agileway.net/login") # after redirect
puts history_sites.inspect

Summary

CDP is quite complex and relatively new, there are very limited examples. I came up with the above by reading its code on Github. Many of DevTools features are still classified as “experimental”. Selenium WebDriver is a feature-complete, that enabled Facebook’s ‘release twice a day’ back in 2013. I achieved pretty much the same, and I don’t remember any particular limitations.

I am not against CDP, as certain operations probably will be easier with CDP. However, I strongly recommend readers focus on the core Selenium WebDriver first. After all, CDP is a debugging protocol for Chrome.

Please check out my book for more examples like the above: “Selenium WebDriver Recipes in Ruby”.