5 Common Challenges When Using Feature Flags
We've talked about the good, now let's talk about the bad. There's still good news though: when it comes to Feature Flags, even the "bad" is easily resolvable.
Using Feature Flags comes with a lot of upside. We’ve discussed a lot of that in The Benefits of Feature Flags in Software Delivery and 5 Feature Flag Use Cases You May Not Have Thought Of. However, like any change to the way you’re working, you might find some challenges with Feature Flags that you weren’t planning ahead for.
Feature Flags are ultimately easy to efficiently manage, but they do require some new processes and conversations that are worth knowing about ahead of time. We’re going to take a look here at five very common issues that we recommend you consider up front when adopting feature flags - and how to address them.
Challenge #1: Over time, Flags Add Up.
“We don’t know which to get rid of and nobody owns the problem!”
While we are working to make Harness Feature Flags much more capable at solving this problem over time, there are plenty of easy easy ways for a team to improve this today.
There are really two questions to keep track of:
- Who owns a flag?
- Is a flag still in use?
You can do this with flag name and description templates, spreadsheets, wiki pages, or any other tracking document. Make sure whoever the ultimate responsible party will be - whether that’s PM, engineering, ops, or program management - has a regular cadence to review their flags and update the status accordingly.
Challenge #2: We Never Get Around to Removing Flags From the Code.
Regardless of who owns the flag, engineering will ultimately have to be the one to remove it.
When thinking through the issue of stale flags lingering forever, the two important questions are:
- When is a flag ready to be removed?
- When should engineering actually remove it?
If you are responsible for maintaining the status of the flags (described above), then you simply need to let engineering know when the flag has served its purpose. Sometimes, this also involves getting their input - do they feel comfortable running the feature without a flag to kill it off yet?
Once the flag status has been communicated to engineering (or decided on with them), teams will next wrestle with where to find the time to deprecate. Is it tech debt, bugs, scheduled feature work, or something else?
The answer can be any of these things, as long as your team is explicit. Generally, the sooner you get a stale flag out, the better - because these things have momentum. Set aside one story per sprint to clean up flags, or one sprint per quarter, or some other clearly blocked-off time acknowledging that flags are an important part of your process and that cleaning them up is an important part of working with flags.
Challenge #3 - Flag Names Are Heavily Engineering-Focused
Sometimes, when engineers create flags, names like “OLD_SWITCH_ULNG” make perfect sense to them. But, when it comes time for a PM or someone else to start using the flag for external access, they’re lost. Do we want “OLD_SWITCH_ULNG” or “OLD_SWITCH_ULNG_BE” - or both?
PMs can work with engineers to decide on naming templates up front, to help with this process. Some good ground rules are:
- The flag name should describe what changes with the flag.
- There should always be a useful, customer-impact-describing flag description.
- Describe state in the flag name (i.e. append “Prod Ready” or something else to the flag as status changes).
With a few simple agreements, it can become very easy to avoid this common naming pitfall that limits the use of flags across the organization.
Challenge #4 - Who Turns Flags On For Customers?
Let me describe a scenario for you that definitely never happened here, earlier in our flag journey:
- An account executive is excited to offer a beta feature to a customer, and the customer agrees.
- The account executive asks the sales engineer to turn the feature on.
- The sales engineer asks the support rep to turn the feature on.
- The support rep asks the PM to turn the feature on.
- The PM doesn’t know which flag to use, so he interrupts engineering and asks them to turn the feature on.
One of the big wins with flags is empowering other folks across the organization, but in this kind of scenario, you will end up distracting almost every possible person for a change that ultimately takes only a few seconds.
To avoid this, your main stakeholders - support, account teams, eng, and PMs - should all agree on who at the company will be the primary resource for turning things on and off for customers. At Harness, we rely on account teams primarily, with PMs supporting when needed. Sometimes, for betas being led more closely by PMs, we will invert that process a bit.
Some flags, it should be noted, don’t actually impact customers in a visible or direct way. A strictly backend change, such as an API refactor, does not need to get rolled out per-customer, it needs only to be tested across a cohort for reliability before being turned on globally. For flags like these, you may have a somewhat alternative process where engineering owns the flag end-to-end.
Challenge #5 - How Do We Keep Our Product Environment Secure While Using Feature Flags?
This question often comes up pretty late in the process. Product and engineering teams are brought into using flags and are moving forward, and then finally, a security person takes a look. They see that a lot more people can now potentially make customer-facing changes in production instantly, without the heavy technical lift of a deployment or a rollback. If they haven’t worked with feature flags before, they can get alarmed.
Fortunately, Harness Feature Flags has a comprehensive RBAC model, making it easy to control who should have access to which environments. This means the only thing you need to sort out internally is who controls access and what teams should have that access.
One common scenario is that all of engineering has unlimited access up to prod, but in prod, only a subset of engineering, such as managers or ops folks, have access. Prod flags instead are largely controlled by PMs and account teams, with either program management or ops overseeing the access configuration.
Having a strict outline like this in place, with someone clearly responsible for onboarding and auditing user access and configuration, is critical to keeping everything compliant and secure while using feature flags.
Despite how long some of these answers may be, you might notice that for the most part they are all one answer: communication!
Flags represent a new way of working, which means a change to ways your team has previously done things. These changes are largely not technical, though, instead coming down to who does what and when.
With good communication and planning internally, almost all the new challenges introduced by working with feature flags for the first time can be easily overcome.
Want to read more on feature flags? Here’s a piece on Best Practices When Migrating Feature Flag Tools. And, keep your eyes peeled on our Resources page - we’ll be coming out for a free, ungated Feature Flags eBook soon so you can say goodbye to pesky feature flags challenges!