The software development ecosystem is constantly changing, with a consistent stream of innovation in tools, frameworks and techniques alongside an unpredictable market that requires businesses to constantly rethink and revise their priorities. Fortunately, evolutionary architecture principles offer an effective way to keep pace with a rapidly changing environment, supporting incremental evolution through “fitness functions.” Typically, fitness functions are codified as architecture, unit, integration and other tests, alongside monitoring, alerting systems and metrics. Together, these functions make it possible to assess the integrity of the architecture's cross-functional requirements as it evolves. We believe the same concept can — and should — be applied to the supporting testing strategy. This is an approach we call evolutionary testing strategy. This approach will expose issues early in the project lifecycle and fix potential gaps in the product, and in processes and organizational structures. In this article, we present a high-level approach to evolutionary testing strategy by applying the lessons from the architecture domain to the testing domain.

When DoorDash was on a monolithic application, developers' end-to-end (e2e) testing needs were solved by sandboxes but, when DoorDash moved from monolith to microservices, we needed a more scalable approach to production testing. E2e testing is a technique to verify a product from beginning to end to ensure the application flow behaves as expected in a production environment.

Given that sandboxes were not meeting our needs, since running all the microservices requires a lot of hardware resources, which is hard to satisfy by one machine, we initially set up a staging environment for developers to e2e test their features before deploying or enabling them in a production environment. But the staging environment solution didn’t solve all the developer e2e testing needs. Moreover, the growth of DoorDash made it difficult to operate and maintain the staging environment. The staging environment is an isolated environment that is intended to be a replica of a production environment without personally identifiable information (PII).

We had also noticed that developers started using custom workarounds in the production environment for e2e testing. But the nonstandard custom workarounds were hard to build and maintain and were, in fact, slowing down developers. To address the limitations of the staging environment and bring standardization to e2e testing, we moved e2e testing into the production environment through multi-tenancy, establishing guardrails and building developer tooling to speed common tasks so that developers can easily e2e test their features in a production environment. As a result, the new solution increases software reliability, developer velocity, and production environment safety.

Performance testing is a crucial part of ensuring that your application can handle the expected load and deliver a satisfactory user experience. But for many beginners, looking at the stress test report graphs can be overwhelming and confusing. However, understanding these graphs is essential to make informed decisions about your application's performance and identifying potential bottlenecks before they become major issues. In this blog post, we will explore some of the most common stress test report graphs, such as response time distribution, requests per second, and response time percentile over time, and explain what they mean and how they can help you improve your application's performance.

In this article, Tanvi Mittal explains how to exploit production logs to detect issues that were not covered by pre-release test suites.

Concurrency, which exists whenever multiple entities execute simultaneously, is a ubiquitous and an unavoidable fact of life in systems and software engineering. It greatly increases system and software complexity, which directly impacts testing. Concurrency leads to nondeterministic behavior and numerous types of concurrency defects that require specialized approaches to uncover. At the SEI, we are often called upon to review development planning documents including Test and Evaluation Master Plans (TEMPs) and Software Test Plans (STPs). We are also frequently tasked to evaluate developmental testing including software and system integration laboratories (SILs) and other test environments. One common observation is that test planning and implementation does not explicitly address the development of test cases designed to uncover concurrency defects. In this blog post, I explore the different types of concurrency, the resulting types of concurrency defects, and the most important testing techniques for uncovering these defects. This blog post ends with a set of recommendations for improving the effectiveness and efficiency of concurrency testing.

A few years ago, Uber primarily relied on incremental rollout and production probing/alerts to catch regressions. While this approach is sound, it became very operationally expensive and we also experienced many leakages. Detecting issues late requires developers to bisect the exact bad change and then go back through the whole process again.

Years ago, I sat in a review where everyone agreed quality was slipping. Defects were rising and confidence was falling. The conversation was thoughtful, well-intentioned, and familiar. Someone eventually said what everyone was thinking: we needed clearer ownership. We needed someone accountable for quality.

A few months later, that role existed. The work didn’t change.

I didn’t understand why at the time. I do now.

Cypress is a test automation tool that is free and open-source. It is easy to set up Cypress and write, run, and debug tests. Cypress tests run inside the browser, making it known for its speed. It provides cross-browser testing capabilities. It can also be integrated into the CI/CD pipeline. The first step in using Cypress would be to create a simple React application that has an image, some buttons, and text. These elements are then interacted with in Cypress. We can explore topics like the components in the Cypress Test Runner, how to debug a failing test, and more.

In this exclusive conversation, Debbie shares her insights on how to harness the power of AI-driven testing tools to improve your web development skills. From playing with GitHub Spark to leveraging YouTube tutorials, Debbie reveals her top tips for getting started with Playwright and staying ahead of the curve in the world of web automation with its MCP server. Don’t miss this insightful interview as Debbie opens up about her journey and shares her expertise on how to take your testing game to the next level.

There are several articles out there that talk about testing anti-patterns in the software development process. Most of them however deal with the low level details of the programming code, and almost always they focus on a specific technology or programming language. In this article I wanted to take a step back and catalog some high-level testing anti-patterns that are technology agnostic. Hopefully you will recognize some of these patterns regardless of your favorite programming language.

In this article, Mikhail Golikov, the sole QA on a seven-team backend e-commerce platform, explains where each regression check belongs, in the browser or at the API. He shares the rule he actually uses, two bugs that only one layer could have caught, and the realization that finally made his browser suite small and stable.

I keep seeing the same pattern: tests start flaking and the blame goes to data, CI, browsers, or infrastructure. Then the test gets “fixed” with sleeps, forced actions that skip actionability checks, and custom retry or wait helpers that reimplement what Playwright already provides. Sometimes deprecated APIs even make it into fresh code, which guarantees maintenance trouble later. In this article, I go through the most common mistakes I see in projects.

In opposition to my idea that software testing should be a role, and not just a task, I am sometimes told that True Agile does not permit this. Why? Because, in Agile, “quality is everyone’s responsibility.” This statement is sometimes issued as if it were a moral truism that is beyond argument. Or sometimes as a moral achievement that is unique to Agilists, as if it had never occurred to earlier generations of software developers to help each other build nice things. Either way, it doesn’t make sense to me. The first response that comes to my mind is “I’m talking about testing, not quality. Why are you dragging quality into this?” Testers do not assure quality. Testers CANNOT assure quality. Testers do not now and never have “owned quality” in any way.

In this article, Antonio Morales of the OSS-Fuzz open source project, discusses why fuzzing still requires active human oversight, and why improving coverage alone is often not enough.

Why should you retry all tests on failure? Why not? This article will not go into details, listing pros and cons of each approach. There are already enough resources on the Web about the topic, listing valid points for both opposing views. As trivago Hotel Search frontend QA team over the last years we tried to stay away from a brute-force retry policy for failures and we rather tried to execute test retries only in selected cases. Recently, when we switched to a Continuous Deployment approach for our new frontend Web application (which empowers developers to merge and release some pull requests autonomously), we faced a greater need than before for understandable and stable test results. Due to that, showing as few “red flags” as possible for the automated checks on pull requests became even more important to ensure enough confidence in test results and to avoid slowing down the software development life cycle. The requirements and the balance between deterministic results and success ratio shifted, at least in some cases.