‘Tis the season…for code freeze!
Although the prevailing Agile wisdom (and much of the research, for example, see the 2016 State of DevOps Report) tells us code freezes are bad, they are a reality for many folks in a variety of industries. If you are faced with a code freeze, what should you do about it? Is there any way to transition away from code freezes? How can you ease the pain of a code freeze? And potentially even worse, the pain of exiting a code freeze? These are just some of the questions we set out to answer on our code freeze panel at deliver:Agile Live! this month. I was lucky enough to be joined by my friends Sarah Alsanifar, Rob Myers, and Michele Guthrie, all experienced members of our community and enthusiastic advocates for Agile technical practices.
Here are the major insights we had through the course of the panel:
There are at least two commonly used definitions of code freeze
Feature Freeze
A feature freeze is a period of time when all feature development stops. Development is entirely devoted to stability work. Deployment of stability-related changes is allowed. I have also heard this referred to as a pause on value demand and a focus on failure demand. That seems to have a pretty negative connotation, but that might be desirable sometimes.
Deployment Freeze
A deployment freeze is a period where all deployments to production are paused. Development may or may not continue, but no changes are permitted in production. Realistically, this never actually occurs. Ops will likely have to make changes in production to keep everything running. Emergency deployments will happen (but usually with many approvals and a high degree of scrutiny). It’s also worth noting — *not* changing is itself a change from normal operation. Everyone on the panel seemed to have a story about the negative effects of making no changes, e.g. never rebooting machines, never spinning instances down and back up again, etc.
Code freezes exist due to a lack of confidence in change
The good news is that you can use the code freeze time to make some improvements that can drive confidence.
- Do you practice trunk-based development and/or avoid long-lived branches?
- Do you have a comprehensive test suite that runs before deployments to any environment?
- Do you have a deployment pipeline that deploys to all environments in an automated fashion, encouraging small changes and decreasing the likelihood of human error in deployments?
- Do you have any kind of synthetic monitoring?
- Are you using canary releases and automated rollbacks?
- Are you practicing chaos engineering and/or continuous verification?
We want to enable small changes (decrease the cost of each change) and increase confidence in each change. Each of the panelists has had great luck with making this change-enabling work a focus during a code freeze (instead of continuing on feature work). This work paves the way for reducing or eliminating code freezes the next time around. If you’re not sure which change enabling-work to prioritize first, you may find Accelerate by Dr Nicole Forsgren et al has some research on which practices correlate most with high performance.
Do something besides development
Long-lived branches are not going to do you any favors when the freeze lifts and neither is a glut of unreleased code sitting in your staging environment. Rather than continuing to do feature development during a code freeze, our panel of experts suggests doing non-development activities. Especially now during the pandemic, some team-building activities could be a welcome reprieve. Additionally, team-building activities can build confidence and trust, which are critical when pursuing a culture of more frequent production deployments.
Cut yourself some slack
While code freezes are painful, lifting the code freeze is usually even more painful. All the teams/products/services that have been under code freeze now have a backlog of changes to deploy, and many will be tempted (or coerced) to deploy those changes right away. The first weeks of deployments after an extended code freeze are often riddled with incidents. To absorb the volatility, all our panelists highly suggested scheduling little to no work for the weeks after a long code freeze. Leave plenty of capacity available for issues, bugs, expedite requests, and incident calls.
Lead with curiosity and respect
All our panelists agreed the most important part of navigating any sticky situation in a sociotechnical system is leading with curiosity and respect. Instead of leading with the dismissive attitude “all code freezes are stupid,” which is all too prevalent in the Agile community, get curious about why code freezes are practiced in your organization. Understand what the actual needs are. Come up with a plan for addressing those needs through modern Agile and DevOps technical practices. Instead of criticizing an implementation or technical practice, have respect for history. Don’t criticize a monolith citing the advantages of microservices when that trusty old monolith has been going strong and powering your business for decades. Instead, ask how that legacy of success can be best served moving forward.
If you are interested in more context or specifics on these insights and other great tips and tricks, feel free to check out the full recording and transcript. We hope to see you next month for our end of year celebration.
