Playwright vs Cypress: Our Experience with Automation Testing Frameworks

In a previous blog post on testing methodologies, we discussed different approaches to define the baseline of our automation projects. We are inspired towards the development of a customized testing library to support all of them: from our GraphQL API Composabase to internal testing projects. With this in mind, we decided to probe Playwright and Cypress.

What is Playwright? 

Playwright is a flexible tool for end-to-end testing automation in web applications providing a wide range of features and multi browser/language support. It allows us to interact with web pages in complex manners, such as clicking on elements, sending text to input fields, scrolling, waiting for elements to appear, etc. With Playwright, we can also simulate the use of mobile devices and test in different environments to ensure compatibility and user experience. Playwright is designed to be fast and reliable, allowing the writing of robust and efficient tests.

In our team, we were able to run several tests in parallel obtaining the necessary documentation in a variety of presentations: screenshots, videos, timelines, and component’s pick locators. 

Playwright provides an UI with the following elements:

1. Timeline View.
2. Pick Locator.
3. Test Filters. 
4. Actions.
5. DOM Snapshot.

10 More Relevant Playwright Characteristics

1. Multi-browser support (Chrome, Edge, Firefox, and Safari).
2. Multi-language support (JavaScript, Python, Java, and C#).
3. Provides us with assertions making our tests faster.
4. Pick Locator Tool simplifies element location.
5. Relevant reports providing status (pass, fail, flaky) and specific information of each test case.
6. Parallel test execution with up to four workers/processes running simultaneously.
7. Easily accessible documentation.
8. Speed and stability.
9. Detailed error capturing.
10. Interaction with mobile devices.

5 Disadvantages of Playwright 

1. Few educational resources.
2. Documentation under development.
3. Initial learning curve.
4. Limited compatibility with legacy browsers.
5. Doesn’t support native mobile apps.

What is Cypress?

Cypress is an open source end-to-end testing framework for modern web applications. It allows us to quickly write, execute, and search some issues in automated tests efficiently.

With Cypress, we can simulate user interaction with the web application. This is very helpful since we can observe how the application behaves under different situations: we can click on buttons, fill out forms, navigate through different pages, accessing data in various forms. 

Additionally, Cypress includes commands for making assertions and checking the state of the application during tests. Cypress also provides a real-time graphical user interface where we can see what is happening in our application as the tests are running. 

At the end of these tests, we are given reports on the results, including execution times, successful and failed tests. This helps us to evaluate the quality of the application and to obtain parameters for improving execution time if required and for identifying areas for improvement.

10 More Relevant Cypress Characteristics

1. Easy setup and use.
2. Multi-browser support (Chrome, Edge, and Firefox).
3. Visibility during execution.
4. Large community and support.
5. Continuous integration and continuous delivery.
6. End-to-end testing capability.
7. Powerful selector and debugging capabilities.
8. Reports expected results to stakeholders.
9. Ensures consistency in test cases across browsers, providing stability for the projects we support.
10. Efficiently documents with the best possible developer experience.

5 Disadvantages of Cypress

1. Limited interaction with elements outside the DOM.
2. No support for multiple windows or tabs.
3. Only for web applications.
4. Limited compatibility.
5. Specific environment requirements.

Differences Between Playwright and Cypress

In the following table we can see a comparison we made focusing on some key features of both frameworks. This comparison allowed us to understand which testing framework was more suitable for us.

It is important to say that as a team, we already have had some experience using Cypress in previous projects developing tests to support some main flows to increase the efficiency of the QA team within the organization.

Our Experience Using Cypress

The development of automated tests was inspired by the Composabase project, which in general terms allows us to unify different data sources in a single GraphQL endpoint. During the process, the development team has been constantly asking for validation, as well as some other repetitive and time consuming tasks for our manual testers.

Using Cypress as our first approach was useful to learn the basics of automation and to practice how to access the components from the page code. We recommend the use of Cypress as a good practice for getting started with this.

With this in mind, we developed three flows. In the next table, we list the issues we faced during the implementation:

Our Experience Using Playwright

After facing these issues and challenges we decided to explore another testing framework called Playwright, with the purpose of obtaining better performance. As we switched, we started using Typescript, these changes helped us to understand that we needed to prioritize adaptation to different programming languages according to the project needs. We decided to use Playwright and with that, compare it with Cypress, not only for the features that this framework offers but also because of the developer experience it provides. We concluded that working with Playwright is more suitable for our projects because of features such as parallelization using multiple workers, development support using the pick locator, and programming language flexibility among others. 

Playwright vs. Cypress: Is There a Winner? 

During our QA practice we decided to primarily use Playwright as our testing framework while maintaining a flexible approach that allowed us to continue supporting and extending our work with Cypress. It’s worth mentioning that these frameworks are not mutually exclusive. 

When we began using Playwright, we observed a notable increase in execution speed compared to Cypress, even while continuing to work with the same four default workers. Another significant advantage we observed was the availability of the pick locator in the UI, which allows us to access web components more easily.

For our latest automation work, we used TypeScript as it aligns with most of the projects we support. Playwright not only supports this language but also allows us to use a variety of languages depending on project needs, business rules, and team experience, providing more flexibility and enhancing the developer experience.

Playwright and Cypress are both powerful tools for end-to-end testing automation in web applications, each with their unique strengths, the use of each one of them must be motivated by your team’s characteristics and the project features.

Testing Framework Allies: How Playwright and Cypress Can Work Together

When utilizing either Playwright or Cypress for automated test development, we follow a structured process, including the following steps:

Requirements Definition:
As contractors, we typically receive specifications from our organization, outlining the requirements for the upcoming tests. These specifications provide critical details such as the testing environment, scope, effort estimation, and programming language, along with additional context to guide the development team.

Understanding the Problem:
With the requirements in hand, it's essential to fully comprehend the problem before proceeding. A thorough understanding prevents rework, avoids exceeding effort estimations, and ensures that the final solution aligns with the initial scope.

Ticket Generation:
Once the problem is clearly understood, we create a corresponding ticket that includes all necessary information for both the organization and the contractor. This ticket serves as a detailed guide for developers, outlining the test flow, data requirements, states, and any other relevant considerations or implementation details.

Test Development:
Using the information from the ticket, the assigned developer begins coding to meet the defined expectations. The quality of the code and adherence to estimates largely depend on the developer’s experience.

Assuring Test Quality:
Before submitting the new code, we ensure that it meets the agreed-upon quality standards, leveraging a centralized repository for automated tests. Depending on the test's nature, we may generate unit tests or use mock data, such as test pages, API calls, and stubs, to cover both positive and negative scenarios.

Implementation:
Once approved, the developed test is included in the test suite. Depending on the chosen framework(s), we may create or update custom commands to ensure our tests run effectively and deliver accurate results in real scenarios.

In the end declaring a definitive "winner" between Playwright and Cypress would be an oversimplification. Both frameworks have proven to be powerful allies in our QA process, each offering distinct advantages that cater to different project needs and team dynamics. While Playwright has become our primary tool due to its speed, flexibility, and broad language support, Cypress remains an invaluable resource, especially in scenarios where its strengths—like real-time UI interaction and extensive documentation—shine.

Rather than choosing one over the other, the real power lies in understanding the strengths of each framework and applying them where they can add the most value. In our experience, leveraging both Playwright and Cypress in a complementary manner has enabled us to deliver robust, reliable, and efficient testing solutions across a wide range of projects.

Ultimately, the best testing framework is the one that aligns with your team's skills, the project’s requirements, and the specific challenges you need to overcome. By keeping an open mind and a flexible approach, you can harness the full potential of both Playwright and Cypress, turning potential competitors into cooperative partners in your automation process.