How to Succeed in E2E Test Automation within a Team Already on a Wrong Path?
Refuse to be ordinary. Advice to Motiated QA Engineers on Making Impact.
This is an article requested by readers who are frustrated with their teams’ current wrong test automation framework, language or tools but fear to challenge.
In TIST 2010 and 2011 testing conferences. One attendant caught my attention. He sat in the same position in the front row for all my four speaking sessions. Then I asked him, “What do you think (my topics)?”.
He answered, “I really liked it and believe it is the right way to do E2E test automation.”
I continued with another question, “Will you try to apply those practices at your work?”
He shook his head with a shy smile, “I don’t have the confidence”.
I kept thinking over this conversation on the flight back. True, an existing test automation approach (Most were wrong. Otherwise, it would have worked because E2E testing for web apps has been unchanged for nearly two decades) means endorsed by the senior management or architect; how dare an individual test engineer challenge that?
Long-time readers know I have worked as a test automation engineer on various projects for a number of years. In other words, I experienced this situation a lot, almost at every client project. This article shared my approach and experience on this matter.
Table of Contents: · Real End-to-End Test Automation is Extremely Rare · You Can’t Win Arguments with Engineers/Managers who Have Never Experienced Real E2E Test Automation. ∘ 1. “We should use the coding language for E2E test scripts!” ∘ 2. “Ruby (or X) is a very slow language compared to Java”. ∘ 3. “Our company adopts BDD, so you must write E2E test scripts using Given-When-Then syntax”. ∘ 4. “You must get E2E tests running in our Jenkins or Bamboo CI Server”. · Strategy: Human Factors ∘ 1. You can only prove others wrong with results, solid results, gradually ∘ 2. Acknowledge when you’re unable to perform their suggested tasks and pass the ball back. · Strategy: Technical ∘ 1. Don’t point out the wrongness of the current test automation framework or approach. Play along, but do the correct implementation on side tasks as Plan B. ∘ 2. Select a proven formula. Especially the framework is well accepted. ∘ 3. Become really good at hands-on E2E Test Automation, by practice. ∘ 4. Find a Mentor, Optional but strongly recommended. ∘ 5. Don’t OverPromise, Slow and Steady wins the game.
Real End-to-End Test Automation is Extremely Rare
First of all, we should understand the game. Long-time readers of my articles have seen a number of quotes (from software giants and world-renowned authorities) on how hard it is to implement real E2E Test Automation as regression testing. Check out the quotes in this Boosted article, “A Tale of a Deceptive End-to-End Test Automation Engineer”.
Also, my articles:
- Definition of End-to-End Test Automation Success
- Showcase a 500+ End-to-End (via UI) Test Suite: E2E Test Automation is Surely Feasible for Large/Complex Apps
- “Daily Production Releases” Clarified
You Can’t Win Arguments with Engineers/Managers who Have Never Experienced Real E2E Test Automation.
Proposing a new test automation approach (i.e., correcting others’ wrongness) is usually in vain, and it most likely leads to meaningless and relationship-hurting debates.
Those senior engineers/architects can pick many topics that are wrong for E2E Test Automation, but almost impossible to convince them by words. Here, I will show some examples.
1. “We should use the coding language for E2E test scripts!” ❌
E2E Test Scripts are meant to be written from an end-user perspective; therefore, taking the developer’s stand is clearly wrong. However, you don’t want to argue with developers or architects unless you are blessed by senior management.
2. “Ruby (or X) is a very slow language compared to Java” ❌
Again, this is a developer mindset that applies to E2E Testing, wrong! The raw language execution speed matters a little in the context of E2E test execution. In fact, compiled languages often turned out to be slower overall. See the below article.
Above all, we call test scripts for a good reason.
3. “Our company adopts BDD, so you must write E2E test scripts using Given-When-Then syntax” ❌
A software architect typically says this, and of course, it is very wrong. The reason: the tremendous maintainable effort for the middle English-language interpretation tier.
“Unmaintabile or Invalid Automated E2E Test Scripts are Useless.” — Zhimin Zhan
4. “You must get E2E tests running in our Jenkins or Bamboo CI Server” ❌
Jenkins, Bamboo and TeamCity are CI servers. They are built to execute unit and integration testing. Executing Automated E2E UI Tests (regularly) is a totally different game.
…
With all the above, if you bring your proposal (despite being correct) to a meeting, surely, you will lose the arguments, or worse, maybe your job.
In the documentary “The Pixar Story”, John Lasseter (creator of Toy Story) told a story of being fired by Disney. He was adamant about advancing digital art forms (absolutely correct) instead of traditional animation, while the Walt Disney Animation Studios wanted to stay with those traditional methods. The old animators felt threatened, many young artists either quited or were fired. After his huge success at Pixar, in 2006, John Lasseter returned to Disney as the cheif creative officer of Pixar and Water Disney Animation Sutdios.
Strategy: Human Factors
Software development is a human activity.
1. You can only prove others wrong with results, solid results, gradually
The approach is just what John Lasseter did, but he achieved the results outside. For you, the task is to achieve the results while remaining at the project. This may sound hard, but it is not due to the nature of testing.
Testing is verified by the visible results. If you can get solid results (like this one) and keep the face of others (despite the fact they were completely wrong), most would accept for the following reasons:
- The results that they couldn’t achieve. Focus on daily execution as regression testing.
- They dare not challenge you for fear of exposing they really know nothing about Test Automation and Continuous Testing. However, some of them will sabotage though.
- Getting support from the senior management. Check out the benefits of E2E Test Automation & Continuous Testing for Executives and Managers.
- Get support from your team members, such as business analysts, manual testers, the project manager and some developers. Check out the benefits of E2E Test Automation & Continuous Testing for Business Analysts, Customers, Managers, Testers and Developers.
“Don’t forget, others don’t like faking either. They are not your enemy, simply incompetent in E2E test Automation. You can’t blame them for that, it is a failure of Uni Education in Computer Science. They would like a way out, too, but understandably remain in doubt until seeing solid proof.”
2. Acknowledge when you’re unable to perform their suggested tasks and pass the ball back.
Once you gain some success, people will start to show some (or reignite) interest with different motives, such as sharing some glory or sabotage.
For example, a senior test engineer might suggest transforming your RSpec tests to Cucumber. From my experience, the best answer is: “I would be struggling with maintaining 50 cucumber E2E tests for the app under active development (change frequently). I am comfortable with 500+ User-Story-Level E2E tests in RSpec”. Maybe you can also add, “If you can achieve that, I would be more than happy to learn from you”. Trust me, they will leave you alone and never mention it again. Why? The ball has been passed to him/her.
This approach is effective in various scenarios, like when faced with the question, “Should we switch from running tests on BuildWise to Jenkins?”. A good answer will be, “I know little about Jenkins; you are a Jenkins expert…”
One of my mentees, working at a large company, had renewed their TestWise license for approximately three years. During that time, several new testing team leaders proposed different test automation frameworks or languages. His response was always, “That’s great. If you take charge and can consistently execute these 100 tests (pointing to TestWise, essentially raw Selenium + RSpec tests), it’s fine for me.” Subsequently, those testing leaders ceased to trouble him regarding Test Automation.
Strategy: Technical
Prepare yourself well before the battle, as it might affect your career. Conducting a self-assessment first, aiming for Level 3 of AgileWay Continuous Testing Grading (or at least Level 2 + access to a good mentor).
1. Don’t point out the wrongness of the current test automation framework or approach. Play along, but do the correct implementation as a side task (Plan B).
Despite many senior software engineers/architects know little about E2E Test Automation, they don’t want to admit that. They thought, “Testing is easy” (very wrong, see quotes). They usually would NOT take suggestions from an individual engineer. So, let them fail. Frankly, unless there is a new wise IT executive, you cannot do much about it.
Let’s say, unfortunately, you had to work with Cypress (decided by a testing manager or test architect). You know that its limitations (ridiculous to claim as a web test automation framework) would impede further testing certain pages in the target application. Keep your mouth shut, and play along just like your peers.
Instead, start to develop a set of test scripts (for the same test scenarios) in a proven formula, such as this one, but not limited to. If you truly mastered a certain level of E2E Test Automation or have access to a real mentor, this task is a lot easier than you thought, for the reasons below:
- The test design is done.
- The test data is provided.
- The test steps (from an end user’s perspective) are neutral. In other words, it is more or less a mechanical conversion. Moreover, the locators (the main work) in the steps have been decided and verified.
- If using the Page object Model, the top tier of test scripts looks the same anyway, regardless of test automation frameworks or scripting languages.
Therefore, it is not 2X work to implement another set of E2E test scripts; it is more like just 10% more. As a matter of fact, I commonly did my version (raw Selenium + RSpec in TestWise IDE) first and converted it to the team’s current wrong automation framework. This made me much more efficient than my peers in the assigned work.
2. Select a proven formula. Especially the framework is well accepted.
For Web Test Automation, raw Selenium WebDriver; For Mobile or Desktop Test Automation, raw Appium. The reason: they are based on W3C’s WebDriver standard and supported by major vendors.
You can find plenty of proof and support online.
“Facebook is released twice a day, and keeping up this pace is at the heart of our culture. With this release pace, automated testing with Selenium is crucial to making sure everything works before being released.” — DAMIEN SERENI, Engineering Director at Facebook, at Selenium 2013 conference.
Don’t propose a hyped new automation framework unless you are 100% confident. I have seen a number of people make this silly mistake, which affects their reputation or even their jobs.
Please note that selecting Selenium for web testing does not mean you will face challenges, just fewer. There are plenty of QA engineers who have failed on Selenium badly (due to incompetence).
In terms of productive tools that others are not familiar with, such as TestWise, you don’t need to show others before reaching a milestone. As I advised my daughter: “You develop/run tests in TestWise for yourself; When demonstrating to others, run tests in VS Code. (test scripts are in free Selenium WebDriver, which are independent of tools)”. In other words, introduce your personal productive tools gradually.
3. Become really good at hands-on E2E Test Automation, by practice.
Long-time readers of my books or blog articles know it is not hard to reach Level 2 of AgileWay Continuous Testing Grading (~50 tests, better than 90%) if you choose the right automation framework, test syntax framework, testing tool and CT server.
Check out my Selenium Training Workbook series.
4. Find a Mentor, Optional but strongly recommended.
Most likely, you are about to accomplish a major achievement that your company has consistently failed for nearly two decades. However, there are individuals who may hope for your failure. In essence, your reputation or career could be at stake.
It is wise to engage a real Test Automation Coach/Mentor that you truly trust. As you navigate through various challenges, having the support of a mentor with prior experience can be immensely advantageous.
Many Software Engineers (in Test) lack the culture to pay for professional mentorship (but they will do so for their children’s piano lessons and tennis coaching). With modern video conferencing tools, access to a real test automation expert at a global scale is very affordable. Some of my former colleagues and even former mentees have encountered difficulties with their test automation attempts in new companies. They contacted me for free suggestions, but without formal engagement (better protection for both parties), my help is limited to only general advice.
5. Don’t OverPromise, Slow and Steady wins the game.
A common mistake is that over-promise. The maintenance effort for the whole test suite grows quickly (and nearly exponentially) with an increasing number of tests. So, don’t make yourself a fool when many of the developed tests fail here and there.
35-Word Functional Test Automation Strategy: Write 1+ Automated End-to-End Test for a user story; Add it to the regression suite; Run the regression suite daily and keep them valid! Starting on the first day and then every day!
Will my above advice work for you? I believe so, and a lot really is up to you (you are responsible for your actions, not advisors). Anyway, that’s the same advice I gave to my daughter. You might find the story of her first intern experience interesting.
The majority of QA Engineers, I believe, would continue unwillingly working on a doomed test automation toolset. That’s OK; they are making a living, and no one can criticise that. However, I would say it is a pity, though, as Real E2E Test Automation is so much fun and fulfilling. Most (if not all) of my professional achievements (such as international award-winning developer, and Micro-ISV with passive income) are rooted in mastering E2E Test Automation.
Related reading:
- My eBooks: - Practical Web Test Automation with Selenium WebDriver - Practical Continuous Testing: make Agile/DevOps real
- Why Software Testing is Not Effective in Most Software Teams?
- Selenium WebDriver is Still the Best Web Test Automation Framework in 2024
- 30-minute Test Automation Coaching for $1
- An Inspiring Story of My Highly Successful Mentee: Transitioning from a casual Non-IT worker to a Test Automation Engineer Earning a Six-Figure Salary in Just Seven Months
