avatarRodrigo de Brito de Oliveira Cabral

Summarize

Playwright vs. Cypress — A Detailed Analysis

Please note that this article is a translation of the original. The original article can be found here.

A brief comparative article between the two most popular tools for interface testing or end-to-end (E2E) testing. Who will come out on top in this showdown? Playwright or Cypress?

Introduction

Exploring the choice between Playwright and Cypress becomes crucial for development teams aiming for efficient automation. This article will provide a detailed analysis, highlighting the distinct features of each tool and assisting in decision-making to optimize functional testing processes.

Playwright: Developed by Microsoft, this tool offers a comprehensive approach to interface testing automation. Its ability to interact with browsers across multiple platforms and the built-in feature suite stand out as notable advantages.

Cypress: Known for its simplicity and effectiveness, Cypress is a JavaScript-centric interface testing automation tool. Its unique architecture and ease of integration make Cypress a popular choice for developers seeking efficiency.

Who is more popular?

Playwright vs Cypress — npm downloads

The chart provides a comparative analysis of downloads over the past two years, as recorded on npm, between Playwright and Cypress. Throughout this period, we observed the performance and popularity of these two testing platforms. We noticed that Cypress is more popular than Playwright; however, since the beginning of the year, Playwright has shown steady growth, while Cypress remains stable.

Differences and Similarities

Similarities:

1 — Both are Open-source;

2 — Support Multiple Browsers;

3 — Generate test evidence;

4 — Integrated Debugging Tools;

5 — Active Community.

Differences:

1 — Playwright has multilingual support, while Cypress supports only JavaScript;

2 — Playwright supports parallelism;

3 — Playwright supports “multi-tabs” and “frames”;

4 — While Playwright waits until the element appears or reaches the timeout using the “wait” function, Cypress makes multiple attempts to find the element until it finds it or reaches the timeout.

Practical Analysis: Playwright vs Cypress

Up to this point, dear reader, in my opinion, we can say that the competition is tied. Now, we will conduct a practical test where we will compare some points, namely:

1 — Ease of environment setup;

2 — Development simplicity;

3 — Execution time for a simple scenario in “headless”;

4 — Execution time for 10 scenarios in “headless”.

Ease of environment setup

The test will be executed in a development container (dev-container), but in a WSL2 environment. Both will use Node.js version 20.10 and npm version 10.2.4.

Code for Playwright’s dev-container:

{
    "name": "Playwright-Artigo",
    "image": "mcr.microsoft.com/playwright:next",
    "preCreateCommand": "hwclock --hctosys || true", 
    "postCreateCommand": "apt-get update && apt-get install -y software-properties-common && curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - && add-apt-repository \"deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable\" && apt-get install -y docker-ce-cli && curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash && export NVM_DIR=\"$([ -z \"${XDG_CONFIG_HOME-}\" ] && printf %s \"${HOME}/.nvm\" || printf %s \"${XDG_CONFIG_HOME}/nvm\")\" && [ -s \"$NVM_DIR/nvm.sh\" ] && . \"$NVM_DIR/nvm.sh\" && nvm install 20.10",
    "postStartCommand": "npm install -g npm@latest && npm install && npx playwright install",
    "settings": {
        "terminal.intgreated.shell.linux": "/bin/bash"
    },
    "runArgs": [
        "-v",
        "/var/run/docker.sock:/var/run/docker.sock"
    ]
}

Code for Cypress’ dev-container:

{
    "name":"Cypress-Artigo",
    "image": "mcr.microsoft.com/devcontainers/javascript-node",
    "forwardPorts": [3000],
    "settings": {
      "terminal.intgreated.shell.linux": "/bin/bash"
  },
    "customizations": {
      // Configure properties specific to VS Code.
      "vscode": {
        // Add the IDs of extensions you want installed when the container is created.
        "extensions": ["streetsidesoftware.code-spell-checker"]
      }
    },
    "preCreateCommand": "hwclock --hctosys || true",
    "postCreateCommand":"sudo apt-get update && sudo apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2 libxtst6 xauth xvfb -y",
    "postStartCommand": "sudo npm install -g [email protected] && sudo npm install && ./node_modules/.bin/cypress install"
  }

During the research for the creation and configuration of DevContainers, I noticed the lack of documentation regarding the Cypress DevContainer. When there was some documentation, certain bugs were encountered with no anticipated solutions. Even when a solution was available, it often didn’t work. On the other hand, for the Playwright DevContainer, it was easily found, perhaps because it is a technology created by Microsoft itself, or because the community recognized its utility in DevContainer creation.

After setting up the environment, let’s move on to repository configuration. I will follow the best practices of each framework. However, this time, and only this time, I will have to abandon the “pageObject” to take full advantage of each framework.

While the “pageObject” brings various benefits such as better code understanding, function reuse, and easier maintenance, it comes with a downside, namely a decrease in test performance. This is a topic for another article.

Validation Test of Amazon’s Home Screen

First, we will analyze the performance of the frameworks when running a simple test, where we validate the existence of the header on Amazon’s home screen.

In this test, we will only check if the items can be found and are visible on the screen. We will use the screen definition “1920 x 1080” in both frameworks. For better validation, we will use the same query to locate the element.

Cypress:

Presentation of the test in Cypress

Playwright:

Presentation of the test in Playwright

We can observe that in the test with only one scenario, the execution by Cypress took about 6 seconds, while the same test executed by Playwright took exactly 4 seconds. The difference between the competitors is small.

The research and usage in both cases were straightforward and almost identical. This was to be expected since both employ a fairly similar and, we can say, generic element search concept.

Now, we will move on to the final test. I will run 10 scenarios using both Playwright and Cypress. The scenarios will be the same, with one observation: I will use Playwright in two ways, with and without parallelism, to compare it with Cypress’s performance.

Final Test

Now, we will execute 10 different tests. In the first stage, it will be performed with Cypress and Playwright without parallelism. Finally, we will use Playwright with parallelism enabled.

We will use this site: www.amazon.com.br

And these scenarios are:

1 — Validar tela principal da Amazon

2 — Validar tela “Venda na amazon”

3 — Pesquisar produto “Coleção Lendas do Batman Ed. 48 — o Cavaleiro das Trevas” com sucesso

4 — Pesquisar produto “Batman: A Piada Mortal: Edição Absoluta” com sucesso

5 — Pesquisar produto “DC Graphic Novels. Superman/Batman. Supergirl” com sucesso

6 — Validar carrinho de compras

7 — Adicionar o produto “DC Graphic Novels. Superman/Batman. Supergirl” no carrinho com sucesso

8 — Adicionar o produto “Batman: A Piada Mortal: Edição Absoluta” no carrinho com sucesso

9 — Remover “DC Graphic Novels. Superman/Batman. Supergirl” do carrinho com sucesso

10 — Remover “Batman: A Piada Mortal: Edição Absoluta” do carrinho com sucesso

Playwright parallelism off:

Executing 10 scenarios using Playwright without parallelism

Playwright parallelism on:

Executing 10 scenarios using Playwright with parallelism

Cypress:

Executing 10 scenarios using Cypress

Result of the “dispute”

During the development of test scripts, we noticed a greater versatility in element searching with Playwright compared to Cypress. Not only can we use a variety of native methods in Playwright, but it also demonstrates greater effectiveness in certain functions embedded in the code. On the other hand, with Cypress, the search became more restricted, although this doesn’t hinder agile and precise development. However, for some searches and methods, a deeper understanding of the tool and perhaps the use of non-native libraries may be necessary.

In terms of performance, there’s a noticeable difference between Cypress and Playwright. Even when compared without fully utilizing Playwright’s potential, especially its parallelism, a distinct gap is evident. The total execution time for Playwright was 48 seconds, whereas Cypress took approximately 150 seconds, or around 2 minutes and 50 seconds. When executed with parallelism, Playwright took just over 31 seconds, highlighting an even better performance.

Conclusion

Deciding which technology is the best can be a bit tricky; it’s easier to say which one fits better into your workflow or work environment. As a wise developer I had the pleasure of working with over these past 6 years used to say:

“The best tech is the one that keeps the paychecks coming.” — Autor desconhecido

Totally, I’m on the same page as him. The best testing framework is the one that pays the bills. However, in the context of test automation, it’s crucial to consider the specific project demands and the team’s skills. Playwright, with its roots in Puppeteer, proves to be a solid choice for more complex and demanding systems, delivering outstanding performance. On the flip side, Cypress shines in smaller-scale scenarios, bringing simplicity and efficiency to less intricate projects. The key is to weigh the nuances of each framework based on individual needs, ensuring a choice that aligns with the goals and characteristics of software development.

Wrapping up with the hope of having positively contributed. I invite everyone to check out two must-read articles: a guide to kickstarting projects with Playwright, available here, and another dedicated to standardizing your projects, whether with Playwright or Cypress, using the effective “page object” methodology, accessible through this link. Find out how to take your automation projects to a whole new level!

Sources and Useful Links

Fast and reliable end-to-end testing for modern web apps | Playwright

Download | Node.js (Node.js.org)

LinkedIn: Rodrigo Cabral | LinkedIn

GitHub: RodrigoOBC (Rodrigo de Brito de Oliveira Cabral) · GitHub

The original article: Playwright vs. Cypress — Uma Análise Detalhada | by Rodrigo de Brito de Oliveira Cabral | Dec, 2023 | Medium

Playwrights
Cypress
Test Automation
QA
Test
Recommended from ReadMedium