The author critically examines the claim that Playwright is superior to Selenium in terms of modernity and speed due to its clean architecture.
Abstract
The author critically examines the claim made in a YouTube video that Playwright is a superior web test automation framework compared to Selenium. The author argues that the claim that Playwright is modern and faster than Selenium due to its clean architecture is wrong. The author provides evidence to support this argument, including the fact that new does not necessarily mean better, and that software testers may not care about the architecture underneath. The author also points out that while Playwright may be faster in some cases, this has not been proven with a solid implementation. The author concludes by advising testers to focus on the job at hand rather than hypes.
Bullet points
The author critically examines the claim that Playwright is superior to Selenium in terms of modernity and speed due to its clean architecture.
The author argues that new does not necessarily mean better, and that software testers may not care about the architecture underneath.
The author points out that while Playwright may be faster in some cases, this has not been proven with a solid implementation.
The author advises testers to focus on the job at hand rather than hypes.
Correct Wrong “Playwright’s Advantage over Selenium” Part 1: “Playwright is Modern and Faster than Selenium”
New ≠ Modern ≠ Better. Faster? I haven’t seen a truly successful Playwright implementation with a large suite yet.
How could this youtuber got it all wrong? Yes, it is very possible, because real automated engineers extremely rare. (Patrick Copeland, Google Senior Engineering Director, referred them as “Gold”)
The first question in Playwright fans’ minds is, “Zhimin, you are being subjective”. How many statements on the Internet are not subjective, including the “Playwright is Modern and Faster than Selenium”, then?
Unlike many others, I will provide objective proof to support my statements.
1. I do know a bit about Playwright
Many readers know I mostly do web test automation using raw Selenium WebDriver + RSpec (since 2011). I have used and refine other frameworks as well, including Playwright. My daughter used Playwright for her intern job, where she crushed the so-called ‘best senior test automation engineer’ there (an interesting story).
Also, I have rescued failed Playwright test automation (upgrading to raw Selenium WebDriver).
2. I added Playwright support in my testing IDE (TestWise) and award-winning CT server (BuildWise)
I am an old-school engineer. If necessary, I would dive deep (really deep) for new hyped technology before commenting on it. In this case, I added Playwright support to my tools: TestWise and BuildWise. This also shows that I am not totally against Playwright; it is just not as good as raw Selenium WebDriver.
Check out the following articles (by my daughter).
This is just one of my Selenium suites that enabled me to do “Daily Production Releases” for all my web apps since 2012.
4. Playwright is winning over Cypress, but not Selenium WebDriver.
There is a clear trend that Playwright is taking over Cypress (my prediction in this article, 5 months ago. One tech blogger disagreed, I used the reference he mentioned to prove him wrong). The most recent two independent surveys yielded a similar result:
The “old” Selenium WebDriver leads by a huge margin.
Claim 1, “Playwright is Modern and Faster than Selenium, due to its Clean Architecture“. Wrong!
1. New ≠ Modern ≠ Better.
Communists claimed their system was “Newer” and “Modern” too.
Playwright is newer, but it does not necessarily mean “Modern” or “Better”. Several of the following- already-failed automation frameworks could/did claim that, too.
2. “Clean Architecture”? Do we testers really care?
By the way, the author also got the architecture name wrong.
To my knowledge, there is no “Chrome Driver Protocol” 😱. He might mean “Chrome DevTools Protocol” (CDP). If so, Selenium v4 supports it too.
In terms of architecture, Selenium WebDriver clearly wins. Why? It conforms with the WebDriver standard, designed and refined by top experts for years. W3C sets the standards for all web technologies, and all browser vendors support WebDriver. How can any other top that?
Do software testers really need to care about the architecture underneath? I don’t think so. I consider myself a highly technical, hands-on test automation engineer (100X and international award-winning programmer) and have reviewed dozens of test automation frameworks/tools. When working as an automation tester at jobs, the thought of a test automation framework architecture never crossed my mind.
For example, to click a link using Selenium WebDriver,
driver.find_element(:link, "TestWise").click
Do I need to care about the JSON wired protocol WebDriver uses (underneath)? When authoring books on Selenium WebDriver, I did, but not for scripting steps.
For web test automation, knowing a framework conforms to W3C standards and major browser vendors support it is good enough for me.
“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.
My advice to testers: focus on the job at hand, not hypes.
3. “Playwright faster”, maybe. But I haven’t yet seen a solid Playwright implementation to support that.
Running a handful of tests might show Playwright slightly faster; the difference is minor, though (as revealed by several benchmarks, don’t add parallel execution, which I will explain in a separate article). Remember, Cypress claimed that, too.
The Cypress claim provided no proof (test suite and timing). Various benchmark tests, like this one, showed Cypress is the slowest among Selenium, Cypress and Playwright. So “much, much faster” is a joke.
For individual End-to-end test case execution, nearly ALL web automation frameworks are FAST enough. After all, the majority of E2E test execution time is on the app itself.
Please note that a fast execution on a few sample tests (which I refer to as demo mode) does not mean it will still be fast for executing a large suite. Also, don’t confuse Playwright parallel mode here (which will be covered in a later dedicated article).
“Because of what it (CDP) was designed to do, it is a chatty protocol, and not a user-facing API that allows you to introspect into the browser. Worse, introduce a network hop between the test and the browser, this chattiness leads to slower tests as the network latency increases.” — Simon Stewart, 2020–11–06
We care about the execution speed of a large suite, say 200 End-to-End User Story Level tests. However, I have never witnessed a single case of OK-level implementation of Cypress or Playwright test automation. My definition of OK level (2nd of AgileWay Continuous Testing Grading): 50 User story level UI tests that run daily and, on most days, remain valid. That’s not a big ask, right? I haven’t seen one using Cypress or Playwright. The successful ones were all using raw Selenium WebDriver.
In terms of speed, check out my article, “Playwright vs Selenium Speed Comparison”. For individual test execution, the difference is minor (here is another benchmark). For real web test automation, with a large test suite, you need to run them in a good CT server on a parallel testing lab.