Summary

The article critiques the claim that Playwright has a superior native auto-waiting mechanism over Selenium, arguing that such a feature is unnecessary and can lead to confusion and inefficiencies in test automation.

Abstract

The author continues a series of articles correcting misconceptions about Playwright's advantages over Selenium, focusing on the claim that Playwright possesses a native auto-waiting mechanism. The article asserts that this auto-waiting concept, similar to one previously marketed by Cypress, is misleading and does not align with the principles of end-user testing. Instead, the author advocates for a more targeted waiting approach, akin to manual testing practices, where waiting is only applied when necessary. The author provides a Selenium WebDriver test script example using Ruby to demonstrate a reliable and efficient waiting strategy that aligns with the actual needs of test steps, particularly for AJAX operations like a credit card payment process. The article also touches on the author's preference for Ruby over JavaScript for test automation and questions the existence of an equivalent waiting approach in JavaScript.

Opinions

The author believes that the concept of "auto-waiting" in Playwright is more of a marketing term than a practical feature, leading to unnecessary complexity and potential side effects in test scripts.
The author disagrees with the idea that generally applied waiting times in Playwright are beneficial, emphasizing that they deviate from the end-user testing perspective and can result in unreliable test automation scripts.
The author has a strong preference for Ruby in test automation, considering it the best scripting language for end-to-end test automation, and suggests that those dissatisfied with their current language should consider switching to Ruby.
The author implies that the Playwright configuration file's timeout settings are inappropriate for all test scenarios, as they do not reflect the nuanced waiting requirements of real-world test cases.
The author promotes the use of a custom try_for function in Ruby as a superior alternative to Playwright's auto-waiting, highlighting its effectiveness in maintaining a daily automated regression test suite of over 200 tests.
The author challenges the notion that JavaScript is the only viable language for modern test automation, stating that the choice of language should be based on the effectiveness of the waiting strategy and the reliability of the test scripts, rather than language preferences.
The author suggests that the need for a configuration file in test automation is unnecessary, based on personal experience spanning 17 years, and hints at an upcoming article on the subject.

Correcting Wrong ‘Playwright’s Advantage over Selenium” Part 3: “Playwright has Native Auto-Waiting Mechanism”

A wrong approach that brings confusion.

Let’s continue to correct the third wrong claim in this YouTube video, “Playwright vs Selenium: What Advantages Make Playwright the Winner in Automation Testing Battle 🏆”.

This article series:

“Playwright is Modern and Faster than Selenium” 😒
“Playwright has Parallel Execution Support” 👎🏽
“Playwright has Native Auto-Waiting Mechanism” 👎🏽
“Playwright has a native test runner” 👎🏽 (coming soon)
“Playwright has native HTML Reporters” 👎🏽
“Playwright Features Can be Configured in one Configuration” 👎🏽
“Playwright supports a range of Testing Types, e.g. API Testing, Component Testing, …” 👎🏽
“Playwright ARIA locator support” 👎🏽
“Playwright UI Mode, CodeGen, Debugging support.” 👎🏽

Table of Contents: · Claim 3. “Playwright has Native Auto-Waiting Mechanism” · Playwright waiting is generally applied · My Selenium Test Script for the Payment Test · FAQ

Claim 3. “Playwright has Native Auto-Waiting Mechanism”

This so-called “Auto-Waiting” is no good, as it can cause confusion unnecessarily. Remind you that Cypress has a similar thing, and its marketing team sang for it for quite some time. Did it actually win over Selenium? No, Cypress.io is dying.

“Auto-Waiting” is more like a marketing term (not sure invented by whom? Maybe Cypress marketing team?). It is, in fact, a kind of fake auto-waiting. I have written three articles on this topic:

Playwright’s Auto-Waiting is Wrong
Waiting Strategies for Test Steps in Web Test Automation (Medium’s curators recommended this as a ‘Boosted’)
Auto-Waiting in Web Test Automation Clarified

The above explained, from a technical perspective,

why ‘auto-waiting’ is unnecessary and causes problems
Selenium WebDriver waiting is sufficient, maybe a bit verbose
A simple solution to overcome that minor ‘verbose’ issue.

In this article, I will clarify from a manual testing (or end-user) perspective. After all, Web test automation is a form of end-user testing.

The heading of World Quality Report 2018–2019, https://www.capgemini.com/au-en/service/world-quality-report-2018-19/

It is better illustrated with an example. Assume a manual tester (or business analyst or end-user) is about to test a credit card payment page on a dynamic site (i.e. with AJAX).

There are 7 steps in his test design (the happy path):

Select the card type: Visa or Master
Enter the holder's name
Enter card number
Select expiry month
Select expiry year
Click the ‘Pay Now’ button
Verify the payment success (i.e. checking receipt number)

The ‘Pay now’ is an AJAX operation, i.e., the payment result (needed for Step 7) is not returned immediately. Now, consider the waiting requirement in the test design.

Steps 1–5: No waiting is required at all.

Step 6: Need to wait after performing the step.

Step 7: Wait! (some testers will write the maximum waiting time is allowed)

See, only one step requires waiting.

A Test Automation Engineer’s job is to convert the above test design into reliable test automation scripts.

Playwright waiting is generally applied

In the Playwright configuration file, playwright.config.ts , there is an entry timeouts . (see Playwright official doc)

// Maximum time expect() should wait for the condition to be met.
timeout: 5000,

Because this is generally applied, it is wrong. Why? See the test design.

Some Playwright fans might argue, “Even so, because test execution is fast, checking every condition is OK”. I disagree; it deviated from the thinking of end-user testing. Furthermore, that approach brings side effects (see my articles listed earlier).

What would you say if I conformed to the original test design (wait once) and the test scripts ran super reliably with little effort?

My Selenium Test Script for the Payment Test

The payment test script below is in raw Selenium WebDriver + RSpec.

it "[5] Book flight with payment" do
  # ... navigate to the payment page
  payment_page = PaymentPage.new(driver)
  payment_page.select_card_type("master")
  payment_page.enter_holder_name("Bob the Tester")
  payment_page.enter_card_number("4242424242424242")
  payment_page.enter_expiry_month("04")
  payment_page.enter_expiry_year("2024")
  payment_page.click_pay_now
  # wait up to 6 seconds for the booking number
  try_for(6) { expect(driver.page_source).to include("Booking number")}
  puts("booking number: " + driver.find_element(:id, 'booking_number').text)
end

The try_for is a function I created using Ruby Block.

By the way, I never had the need for a configuration file in 17 years of Test automation (using Watir and Selenium WebDriver). There will soon be an article on this topic.

FAQ

1. Your solution is cool but only available in Ruby. Our project is using JavaScript.

Firstly, We are discussing auto-waiting, independent of languages. Choosing a good waiting approach or being fixated with JavaScript is your personal option. I prioritize outcomes over technical preferences. The outcome I mean here is daily automated E2E (via UI) regression testing.

As long as the approach works well (e.g. being able to maintain execution of a suite consisting of 200+ tests daily as regression testing), it is fine. But if not, consider switching to Ruby, in my opinion, the best test automation scripting language.

2. Is there an equivalent (to your approach) in JS?

Maybe. I am not a JS guru, but I doubt it. This is really a question for your JS guru in the team. This is a general Ruby language feature, not specific for test automation.

3. Is your approach tied to your TestWise tool?

Not at all. TestWise IDE supports raw open-source automation frameworks like Selenium, Appium and Playwright, in a few languages.

The try_for is a Ruby function (in a test_helper) with dozens of lines, that’s all.

 # Try the operation up to specified timeout (in seconds), and sleep given interval (in seconds).
  # Error will be ignored until timeout
  # Example
  #    try_for { click_link('waiting')}
  #    try_for(10, 2) { click_button('Search' } # try to click the 'Search' button upto 10 seconds, try every 2 seconds
  #    try_for { click_button('Search' }
  def try_for(timeout = $testwise_polling_timeout, polling_interval = $testwise_polling_interval || 1, &block)
    start_time = Time.now

    last_error = nil
    until (duration = Time.now - start_time) > timeout
      begin
        yield
        last_error = nil
        return true
      rescue ArgumentError => ae
        last_error = ae
      rescue RSpec::Expectations::ExpectationNotMetError => ree
        last_error = ree
      rescue => e
        last_error = e
      end
      sleep polling_interval
    end

    raise "Timeout after #{duration.to_i} seconds with error: #{last_error}." if last_error
    raise "T

Quickly inserting into the test script in TestWise is also a generic feature known as Snippets in TextMate or Live Templates in JetBrains IDEs. A competent software engineer should also implement it in Visual Studio Code.