Testing in Production Explained

Testing in Production is the kind of testing done by running your tests in a live production environment before you release the software to the end-user. It's an important step not just for developers, but for businesses too. Testing in production helps with continuous delivery, which means that companies can release new code changes on a daily basis with better quality.

Why Test in Production?

There are several reasons why developers should test their code in production. Modern systems have many dependencies and edge cases that are difficult to reproduce in development or staging environments. Without testing features in production you ultimately risk compromising the trust built with your users.

Historically, companies have tried to make sure that the software they produce passes end to end tests and integration testing with synthetic monitoring before it is released to real users. However, when a new product is released, it can be riddled with unexpected errors as no environment can completely replicate your production environment.

Unexpected errors can arise, but by using the best practice of testing your production code thoroughly through a test run before launching to end-users, your company can avoid struggling with these errors by learning from your test results.

The Value Of Testing In Production‍

When testing new functionality, it's a good practice to do so in the same environment that your users will experience. Development and staging environment configurations and data may differ from production, thus making it difficult to accurately test. If you test in production, you'll have confidence knowing your users will experience the same functionality that was verified by your team.

By deploying more often, you enable your team to react quickly to customer demand and make changes as needed. Frequent deployments go hand-in-hand with feature flag driven development. Done right, this means a developer doesn’t need to worry about their work being fully tested or complete before deploying, as long as it's controlled by a feature flag.

Testing in production allows teams to see how a change reacts with other parts of the system. It tests for scalability, stability, performance, and functionality. For example, if a team wanted to launch a new user interface they could first enable the interface for the internal team in production. If the changes are passing validation, they could then release to real customers with a canary test or A/B testing to collect data on whether or not the new UI is better than the old one.

The Stages Of Releasing Code

When people say software has been deployed, it means it is running on your company’s servers. The deployment process is the team’s way of installing the new version of your service’s code onto production infrastructure. In this stage, your code is ready to handle traffic and user data but may not be getting any.

The next step is a rollout. When we release a feature, it is live and active. When we say a version of a service is released, we mean that it is responsible for serving production traffic. To do this, the service needs to be ready to take over for the previous one.

The best way to run a release process is through it being automated and incremental. Likewise, rollbacks of services should also be done automatically by ensuring that the error rates or request rates are matching a baseline of patterns for your application.

If the release of the new service went well and production traffic is being served without any issues, we can call the release successful. This stage requires monitoring your software to ensure it continues to perform as expected.

How Feature Flags Play A Part

Developing new software can be tricky. Feature flags are an incredible tool for engineering teams. When activated, new features can be released to only a small percentage of the site's traffic. This is useful for making sure that it's fully developed and working correctly before being released to all customers.

A team could also use feature flags to test out new changes in a live environment without risking breaking core functionality. Feature flags allow you to easily monitor the effectiveness of your new release and verify that new features are working as expected.