5 Steps to a Continuous Deployment Culture

Deploying software into production so that your users can access it is one of the key areas of software development. Whether you are a two-person start-up using the latest tools and frameworks, a small business integrating multiple SaaS applications, or an enterprise business with 40 years of legacy systems to maintain, the work your development team isn’t fulfilled until it’s in the hands of users.
To do this well requires a commitment to it. Not just by a few, but by everyone. From the top of the management team to the newest developer, everyone needs to understand that the commitment to Continuous Deployment is critical to the ongoing success of the company. Too many teams are unable to deploy quickly to production; they haven’t invested in the people, processes, and most importantly, the culture of their company.
Those companies that invest in making it easy for their developers to get their work into production are able to adapt quickly to changing markets, fix security vulnerabilities, address bugs reported by their users, and make their development teams happier.
Continuous Deployment is a Culture
You don’t just do continuous deployment; you live it. Everything your team does aligns with it. It has to be part of your culture, but it can’t happen by default; it has to be by design.
You can’t get to Continuous Deployment (CD) overnight. Trying to do so will create a broken process, resentment by the team, and most likely a broken application. If you are just getting started with a CI/CD pipeline, or if you have been running one for years, you need a plan to get to CD.
This article will outline five steps that a team, department, or organization can take to move the culture to one that embraces change through CD.
1. State it as a goal and talk about it all the time. Align it with a team or company value.
2. Start with continuous delivery. Deploy once a week, then twice, then daily.
3. Fix what is preventing continuous deployment. Whether it’s a slow build pipeline, flaky tests, or a hard-to-manage local development environment, invest in your team’s workflow. Do whatever it takes to remove the barriers to making it continuous.
4. Collect DORA metrics and show them at every meeting
5. Engage with the rest of the company and tell everyone you can that “we release to production multiple times a day.”
With these steps, teams have gone from deploying once every three months to multiple times per day, from being leery of continuous deployment to it being just part of what they do.
You too can do CD. You have to make it part of the culture.
State it as a Goal
CD is part of the culture not only within the development team but the entire company.
Many teams are stuck in their current release process because they don’t know a path forward, are leery of this kind of change, or can’t see that it will make any impact. They release their software every 3 months, instead of every hour. Individual developers will have ideas on how to get out of the current situation, but it’s the role of leadership to bring this change to the team.
To get to CD, that goal needs to be started in a way that everyone on the team can understand and that they can make an impact upon.
1. State it as a goal and talk about it all the time. Align it with a team or company value.
You can’t get to CD without first painting the picture for the team. Deploying code should be done without ceremony or notice. The goal is that each aspect of the build pipeline should work without failing or giving false positives. This way, the team and the company have trust in the process.
A developer’s job is to write code and merge it to main, and that’s the last they see or hear of it. Now, this is obviously an overstatement of what a developer’s job is when it comes to taking care of the application or components they are responsible for, but if you set this as a goal, then when it’s working, your developers will have time to focus on features and bugs and not infrastructure.
With these two ideas as your vision, you need to talk about it all the time. You need to bring it up in every meeting: Architecture review; how do we automate this to do it continuously? Sprint planning; can we feature flags this new work to release it right now? Retros; can we fix those flaky tests?
By talking about it all the time, the questions about what, why, and how will come out. You’ll need to address each of these questions in turn. They will also set your strategy and resourcing to implement this change.
Culture is either by default or by design. Moving to CD is a change in culture and will require buy-in across the team and the company. So in order to get there, you’ll need to design the culture around the change. For this type of shift, pick one of your company or team’s values that aligns with CD. This will help you frame why the work is going on in terms that anyone in the company can understand.
If the team has concerns around the stability of the application, dedicate more time to writing automated end-to-end tests and implement feature flags to turn code on and off. If issues with failed builds or deployments arise, then invest in refactoring the build pipeline and invest in more infrastructure. If you believe it can’t be done in your industry or technology stack, then bring in other development leaders in your industry or technology who have done this. As a leader, you’ll get pushback, but keep to the vision and start to adjust the culture.
Make sure you spend the time bringing the thought leaders, senior technologists, product managers, and business owners onboard with your vision. For some companies or teams, this will take a week or two; for larger enterprises, this can take months or years. If it’s the latter, then find a digital transformation project and become a part of it. They will be looking for ways to bring change to the company, and CD and the value it brings is certainly that kind of change.
It’s easy to just think about only the development team caring about CD, but actually, it’s everyone. Make sure that your product managers know you’ll be able to address issues quickly, get new features in the hands of users early, and increase the stability of the applications. For your business owners, they will be able to get bugs fixed quickly, know that you can rapidly address changes in business, and create a better experience for the users. You are embarking on creating value for the company, so bring everyone onboard with you. People will doubt that it can be done without major issues or impact, but enrolling everyone into this change will help in the long run.
By setting the vision of deploying multiple times per day without ceremony, talking about CD all the time, and aligning it with one of your company’s values, you can start the journey. However, that’s the easy part; the following steps are the actual work.
Start with Continues Delivery
Now that you have set the stage for your team with the vision of CD, it’s time to put that vision to the test.
2. Start with continuous delivery. Deploy once a week, then twice, then daily.
You most likely are already doing continuous delivery. How does that differ from continuous deployment? Well, in continuous delivery, someone needs to do something to make it go live. It can be as simple as pressing a button on a webpage or invoking a GitHub action. But with CD, it happens automatically. After code is approved, it is merged into version control, it is built, tested, and deployed to production.
The reason to get started with continuous delivery is that your team needs to get into the cadence of delivering code often and automatically. Your systems need to be improved to allow for the appropriate amount of guardrails so that buggy code doesn’t make it to production.
It isn’t going to be easy, and you are going to cause some issues with your application. It is essential to learn from each of these mistakes and then apply those learnings to getting it right the next time.
The first step is to automate the deployment process and remove the humans from getting the code into production. Look at everything, from builds to tests to deploying the code into production; it all must be automated.
The next thing to do is to put in place a rollback. You must know that if something goes monumentally wrong, you can quickly return to the previous working version.
This process also helps determine what in your processes won’t work for CD. For example, if you make breaking changes in your database or API schemas, you need to stop doing that and move to using non-breaking changes.
Now that you can deploy your code with nothing more than a single action and can roll back, all the issues with this process will come to light. It is an excellent time to implement some DORA metrics, primarily deployment frequency and lead time for change. These need to be visible to everyone and talked about often. More on that later.
The more work you put into automating the release process, the more often you can do continuous delivery.
The next step is to change the cadence of releases to once a sprint, once a week, and eventually once a day. Since you talk about the vision in all your meetings, you’ll see where your process prevents continuous delivery from happening.
You must speed up the build process and harden those flakey tests. More on that in the next step.
Fix What is Broken
Now some of the most challenging work begins, getting rid of all the roadblocks that are preventing continuous deployment.
3. Fix what is preventing continuous deployment. Whether it’s a slow build pipeline, flaky tests, or a hard-to-manage local development environment, invest in your team’s workflow. Do whatever it takes to remove the barriers to making it happen.
The biggest complaint from developers is that their productivity is negatively impacted by things they perceive as outside their control. Such as flaky tests or a flaky test environment. Slow or under-resourced build servers. A local development environment that doesn’t work for them.
These are all valid complaints but are entirely within the control of your team. Too much focus on getting the next release or feature out, your team is missing the forest for the trees. If you want to speed up something, you have to invest the time and effort in finding where it’s not working and fixing that.
If your team doesn’t believe that they are in control of these things, then this is the time to roll up your sleeves and work on the code directly. This isn’t a time for you to build a new feature, but to take on something small, such as a bug, and go through the process of fixing it and releasing it to production. Each step of the way, you’ll live the pain your team feels.
Since you are talking about CD all the time, have experienced the problems with the process, and now doing continuous delivery, give your team the time and space to fix the things that make their life difficult. Have them fix the flaky tests. Invest in larger machines for the build process. Tear down and rebuild from scratch the local development environment. You’ve seen where the pain is firsthand; go out and help them fix it.
Start to look at your DORA metrics for deployment frequency and lead time. Dig into why the numbers are what they are. Gather the data on build times, determine which tests fail the most, and run the development environment as your developers do. Be the one who identifies these issues in your retrospectives, pull out the wins, and identify the remaining bottlenecks. Once that is done, switch over to doing CD and see how it goes.
Collection DORA Metrics
Now, as development manager, are you leading the charge to get to CD? It’s time to reap its benefits.
4. Collect DORA metrics and show them at every meeting.
The act of actually collecting DORA metrics means you are doing a lot of things right.
The best metric to start with is Deployment Frequency. Since you are already doing CD, you can add a step to your process to log successful deployments. Deployment Frequency is a canary metric; a slowdown in deployment frequency indicates something broken in your development process.
The next one to tackle is Lead Time for Change, how long it takes for a commit to get deployed to production. This takes a little more work to extract this from version control, but the data is in there so you can get it out. There are some nuances to exactly what commit to use; the first commit in a branch is a good one. This metric is not designed to show how long it takes for a developer to build a feature but to measure how much time it takes for that feature, once completed, to get into production; the time it takes to build, test, review, and finally deploy. Lead Time for Change will show you where the bottlenecks are in your process and will give time back to your developers to build software.
Change Failure Rate measures the percentage of deployments that cause a failure in production. The question you need to ask yourself before you start to collect this metric is “what is a failure in production?”. Is it the site or app crashing, a single service being unavailable, or a reduction in a service level agreement (SLA) metric? This metric requires the input of other teams who manage the application, deal with outages, and customer support issues.
Then there is Mean Time to Recovery (MTTR), how long it takes for the application to recover from a failure. Again, it is critical to know what a failure is, and then what a recovery looks like. MTTR and Change Failure Rate processes are very closely related to each other; once you have one, you’ll get the other.
Now that you have these metrics, it’s time to talk about them all the time. Every meeting you have, you need to show them. In the early days, ask team members to define each one as a way to reinforce what they are and how they are collected. Most importantly, these are not measurements of individual developers or the team; they are a measurement of your process. They reflect upon you. If they are not where you want them to be, you are the one to fix them.
With the metrics understood by your team, you can now set goals to improve them. You can experiment with changing your processes and see how they impact each metric. With DORA metrics, you can create a path to improvement.
Engage the Organization
With the DORA metrics, you now have time to let everyone know what your team is capable of.
5. Engage with the rest of the company and tell everyone you can that “we release to production multiple times a day.”
Most development teams do not do CD; they bundle code into a release and then deploy it to their customers. Instead, they deploy once a week, once a month, or, more likely, once a quarter. After following the five steps I laid out, you should now be deploying to production multiple times per day. Of course, there are bumps along the way, but you now have a software development process based on automation and, most importantly, trust. You are now doing what most engineering teams can’t or won’t do, so let everyone inside your company know.
By being able to deploy at any time, you are in a far better position to fix bugs that creep into the application. You can use more advanced development techniques, such as Feature Flags and real-time debugging. You can also now address security issues in your underlying frameworks in hours, not weeks.
From your team’s perspective, they now have a build pipeline that runs efficiently and is maintained. Your tests test what they need to and don’t return false positives. Their collaboration is higher as they can see changes in their code quickly and work together to bring new features to your users.
Conclusion
Following the 5 steps outlined here will get you to being able to do CD and along the way help embed it into your team’s and company’s culture. It is not something that can be forgotten about but will need continuous validation, input, and updates from the team, but just like with writing tests to ensure code runs as expected in the future, maintaining a solid CD pipeline will allow you to deploy code whenever you need to.