What Are Feature Flags?
Feature flags are a software development tool that allow you to enable or disable a feature without modifying the source code or performing a new deployment. Other names for feature flags include feature toggles, release toggles, feature flippers, and remote configurations.
Feature flags are typically used in conditional statements (e.g., if-then-else) to determine which code path will be executed. Their power lies in the ability to determine at runtime which functionality is executed and to make these decisions at the user or request level, not just globally. With feature flags, you can deploy new features “turned off” so they are not initially visible to users or executed by back-end processes. When ready, you can selectively turn these features on for specific subsets of users and environments.
Decoupling Deploy From Release
Historically, taking a release live was accomplished by deploying code to a production environment. From that moment onward, the new release was live for all users. The risk of disrupting production by introducing unstable code was high, so most teams would gather up multiple features into large releases that might happen weekly, monthly, or even once a quarter. Testing all of these changes together in an environment that accurately represented production was challenging.
Some organizations invested heavily in complex test environments, but there was always a great deal of anxiety over the actual production release, which led to less frequent releases and a tendency to perform those releases late at night or on weekends when few users would be impacted. If a problem did “leak” through to production, typically, the entire release would be rolled back and all features, whether broken or not, would be forced back into a dev, integration test, and wait-for-next-release loop.
This process was painful and led to the velocity of development teams growing slower over time with each new release. Cloud solutions in particular could not afford the stability risks and slow pace of innovation that this traditional way of releasing trapped other teams in. They adopted feature flags as a way to ship new functionality to production environments and ultimately to their end users with less dependency on cross-team coordination or gathering up changes into large-scale releases.
Feature flags remove risk and delay from the traditional deployment workflow by decoupling deploy from release. Now code can be part of a production deployment but initially configured to be “off.” Since it does not execute, it is not released. That makes deployment a low risk proposition of simply moving bits into place. The higher-risk release phase can be done any time after that, and it can be done selectively, at the individual feature level. Feature flags do more than decouple deploy from feature release; they also decouple separate teams from each other by escaping the “all or nothing” release pattern of the past. Feature flags provide a new level of control and safety that breaks the traditional tradeoff between “speed or safety”. Feature flags allow teams to increase speed and safety at the same time by encouraging smaller, more frequent deployments that each contain fewer changes and by making it possible to enable and disable individual features without a new build/test/deploy loop. DORA Metrics, the four key measures of high-performing engineering teams introduced by DevOps Research Associates, are all positively impacted by adopting feature flags to your workflow.
How Feature Flags Change Software Development
Feature flags dramatically change the way development teams build and deliver new code by separating deploy from release, rolling out new features to a targeted subset of users at runtime, and generating metrics during rollouts. When you use feature flags, you tilt the odds in your favor that you can move faster and achieve better outcomes. Here are some examples:
Trunk-based development
With the ability to turn work in progress “off” but still have it in your main branch or trunk, you no longer need to create, sync, and merge multiple long-lived feature branches. Even though git and GitHub have made merging multiple changes in your codebase easier, long-lived branches still cause frequent merge conflicts. The longer feature branches live, the more difficult it can be to debug those merge conflicts. Feature flags enable trunk-based development, where teams work on a single main branch rather than multiple separate feature branches for each specific feature. Short-lived feature branches may still be used to support effective code review via pull requests, but the key is that the separation of work in progress from production code is handled by the abstraction of flags, not by separate files in the codebase. TrunkDev, as it is often called, removes the complexity of merges and enables teams to move faster by allowing them to deploy code changes continuously. For a comparison of gitflow and trunk-based development, have a look at A Complete Guide to Trunk-Based Development.
Progressive delivery: test and release life cycle with less risk and earlier learning
Controlling who has access to a feature allows you to implement your development and testing life cycle with fewer environments and fewer copies of code. The same deployment can be released first to the dev team, then to additional testers or stakeholders, then to early adopters, beta testers or design partners. Early iterations may cycle through the first part of this list, while later ones continue on to your entire user population.
Synchronize releases with marketing campaigns while you sleep
New functionality deployed behind a feature flag today can be scheduled to be released next week, without requiring a redeployment or any other actions by the engineering team. Instead of waiting up to deploy and verify a release moments before a marketing campaign is supposed to launch, you deploy and test in production during daytime hours and use feature flags to turn on the release as you sleep. No more hurried, tired work troubleshooting a failed deployment while the world waits to see your latest efforts.
Remove the stress and risk associated with software releases
When you switch from deployment as the way to release to flipping on a feature flag, you remove much of the stress of releasing. This is because the deployment becomes a non-event, and the release is done with a simple switch that can be turned on when you are ready and just as easily turned off if needed. When incidents do occur, you resolve them without the time and coordination required for a multi-step rollback or hotfix. Your team’s mean time to remediate (MTTR), a key DORA metric, will be a small fraction of what it would otherwise be. And by improving the psychological safety of releases, your team will be inclined to do them more often.
Observe the performance and behavior of each feature before all users see it
Initial approaches to feature flagging, such as LaunchDarkly, focused solely on mastering control: Turn things on; turn things off; do that at scale; and do that faster. Granular and instant control was a vast improvement, but eventually, teams discovered that gradual rollouts could do a much better job “limiting the blast radius” of problems if the metrics used to observe them were computed at the feature flag level, comparing requests with a feature “on” to requests with the feature “off” rather than just looking at global application metrics. While adopting any progressive delivery technique adds some safety and learning, feature-level observability goes well beyond what blue/green deployments, canary releases, and basic feature flag rollouts can do. With a feature data platform, you finally have an in-built way to “watch your back” as you release your changes and move on to the next task. Look at Four Shades of Progressive Delivery if you want to compare the value of blue/green, canary releases, feature flag rollouts alone, and feature flags with metrics computed at the individual flag level.
Feature flags are unlike other flags in software delivery
Feature flags are evaluated at runtime and can be changed instantly without changing code, operating a continuous delivery pipeline, or restarting servers. They also allow more precise targeting and control, controlling whether code is executed at the level of each separate request based on straightforward and remotely updated rules. This differentiates them from configuration files or command-line arguments used at build time or service launch. If you or a colleague ever wrestle with this distinction, have a look at the 90-second video in this blog: What’s The Difference Between Feature Flags and Other “Flags” in Software Engineering?
Who Uses Feature Flags?
Feature flags were adopted first by development teams in large tech companies as part of continuous integration, continuous delivery (CI/CD), and daily, hourly, or more frequent code deployments. Now, even teams with only four customer releases a year use them because of the internal benefits of frequent code pushes and short-lived feature branches (or no feature branches at all). Developers are always the first to implement feature flags, since they require implementation in code.
As more developers in organizations of all sizes embraced feature flags, a marketplace of feature management solutions has emerged. That created an incentive to build more intuitive interfaces, better documentation, and features such as role-based access controls that in-house or “DIY” systems lacked.
Now, developers can implement flags not just to make their own jobs more efficient, but to offload tasks to teams outside of engineering entirely so they can focus on new features. Examples are adding and removing beta test participants, updating in-app announcement text and images, running A/B tests, and coordinating releases with marketing campaigns.
- QA, designers, product managers, and other stakeholders can perform testing in production to validate changes without needing a developer or a separate environment.
- Sales, Customer Success, and Support can add and remove entitlements for specific customers without a custom deployment.
- Product Management can manage beta program participation right in the production environment.
- Product and Marketing can iterate A/B tests without code changes, a practice that’s often known as dynamic configuration.
Creating, operating, and tracking feature flags with a feature management platform makes feature flags a standard and easily understood part of getting things done throughout the software delivery lifecycle.
Feature Flag Use Cases and Benefits
Feature flags make it easier to:
- Reduce the surface area of bugs by using progressive delivery instead of big-bang releases to all users
- Simplify QA and beta testing by testing in production environments, targeting specific users without exposing new features to end users until ready.
- Streamline launch coordination and the sequencing of complex product launches
- Avoid code freezes
- Perform safer technology migrations with the feature parity testing pattern
- Split up a monolith gradually (not as a rip and replace or big bang) using the strangler fig pattern
- Target specific accounts or other granular segments with different experiences from the same codebase
- Change features dynamically in production (text copy, pricing, and other aspects of user experience) without burning software development time to code, test, deploy, and release each time
- Manage multi-channel touch points, coordinating release and exposure of new features across web, mobile, and partner platforms by simply toggling flags
How Are Feature Flags Implemented?
Many developers start using feature flags via a configuration file, command-line argument, or environmental variable, where new features are listed as either being “on” or “off,” and the executable acts on the config file, argument, or environmental variable when it is launched or restarted. This is fine for hiding work in progress and running new features in test environments, but it doesn’t accomplish the decoupling of deploy and release, and it can’t selectively expose new features to a subset of your user base during runtime.
More advanced homegrown solutions implement flags in a database, either as columns in a user table or name:value pairs in a user object. This can accomplish control at the user-level, but it doesn’t scale well as each new feature requires schema changes to the user object. It also isn’t likely to support the instant “kill switch” use case or more powerful targeting, such as “all users on the west coast” or “new users without module x in their package” very well. These shortcomings can be overcome by investing in building more advanced functionality, but it’s challenging to prioritize building and operating a sophisticated feature flag as a service solution for your developer over building new features in your roadmap for the user base, so it seldom happens.
The good news is that as feature flagging has become more common, a few open-source feature flag management solutions have become available. Most open-source solutions typically allow for user segmentation, targeting, and controlled feature rollouts and include some form of management dashboard, but the more full-featured ones all force some challenging decisions: Should you purchase their paid version to get timely support? If you choose open-source because the option to make your own customizations is attractive, do you put yourself in an odd position somewhere between purchasing a supported solution and being responsible for a homegrown one?
The most common implementation teams adopt once they discover the value of feature flags is to upgrade to a commercial feature flag platform. These have the advantage of well-supported SDKs for common languages, real-time streaming of updates, rich APIs, built-in feature-level observability that can also be used for A/B testing when needed, high availability delivered by redundant infrastructure and dedicated operations teams, and constant investments in usability, most of which you are unlikely to accomplish on your own.
Feature Flags and Technical Debt
What happens when you have too many flags? Worse still, what happens when your code base is filled with old code hidden behind flags that have effectively turned it off even though it still remains in the code? This is called flag debt, and it’s a form of technical debt that can quickly build up over time but just as easily be avoided by simple hygiene. The key is to see most flags as temporary – as nothing more than a tool to speed up the development process safely. Since they are temporary, removing them isn’t “extra work” but rather part of the typical lifecycle that makes them valuable.
Here are some tips for avoiding flag debt:
- Use well-structured self-evident names for feature flags, so other developers will know why they are there
- If you don’t include the project or task id in the flag name, make it easy to “look up” a feature flag by name to see who its owner is and when it was last modified.
- Place flag decisions at natural points of abstraction. Instead of placing them throughout a module, put them in the code that calls the module and have it decide whether to call the old version or the new one. That way, the flag, and the old module can simply be removed and replaced with a direct call to the new module without any complex editing.
- Create a Jira ticket (or whatever task ticketing system you use) to remove a flag as part of the workflow you use for creating it.
Looking for More Resources?
Here are some additional resources on feature flags, feature management, and progressive delivery: