Add an API for feature flags

I love everything about this proposal! Thanks for the fabulous idea, clear write up, and simple solution. This would be a huge win. I'm sure if I spent some time triaging issues I've worked on that are still not committed, I could add another 5-10 examples to the list. 😅

Do we always require until? Couldn't we use this for long-lived features that want a "killswitch"? There are good reasons for the update manager killswitch to be in settings land, but I could see other things like that which might want to use this, even if they're not slated for removal.

Thanks,
-Derek

Not sure about these being in state because that's not deployable, means either making people go to the UI after a deployment, or write update paths themselves to toggle flags.

Could we have a single config object that stores the status of all the flags maybe?

larowlan → credited alexpott → .

Discussed the design with @alexpott and @catch and concluded we need to think about the storage a bit more.

I'll come up with some pros and cons for each of three models

* state
* settings
* config

there may be a hybrid approach too.

p.s. There was also talk (in Slack) of a CR link field of some kind.

FWIW I would love if this avoided the config API entirely.

This sounds awesome!

Is it reasonable to integrate that with PluginManagerInterface such that each plugin is treated as a feature which is on by default but could be turned off, if certain sites don't want some of them being available? Examples could be field types, widgets, actions, and all the others.

Have been a bit concerned about how much API/storage we end up with for this, and might have a way to avoid that - using modules.

Each feature flag is a module. The module doesn't contain the runtime code for the feature, that goes in core (where it's going to stay) but is inside a moduleExists() check.

The module is as empty as possible - .info.yml, maybe GenericTest and short help.

For features enabled by default - add an update to install the new module.

In the next major, we make the module obsolete and remove the moduleExists() checks.

It means more modules in the modules UI, but on the other hand, no new admin page etc.

we have one feature flag for linkset endpoint (decoupled menus) in a config file with true/false, that would be a good first candidate to try it out? although it's not like it's a "preview" feature so might not apply here.

Yep, agree with @catch that modules seem like a suitable mechanism already, and if we know the code is going to end up in core (or a core module) we can just use moduleExists() for the flag check, and then remove it later. Is there anything that the proposed flags can do that this won't cover?

Sounds like a great approach, which issue should we use as a test-case?

modules interact with the config system by way of core.extension though? Which means we'd need an upgrade path to disable it too. Why would using modules be beneficial over state?

So for example 📌 Combine field storage and field instance forms Fixed broke a couple of contrib modules in 10.2.

If we used a module, then field UI would check ::moduleExists() and then use either the new or old form structures/UI.

If a site has a contrib module that's not updated for the new structure yet, let's assume they find the issue before they deploy to production, they can uninstall the feature module and the old code path will run - and deploy the update to production without having to then also set something in state.

If it's only found on production, they can still hotfix it by uninstalling the module on production, and then do the same locally and export config.

So modules are good because they are tied to config? Making it easier to deploy? Just trying to understand the rationale.

Modules also have the downside of once we remove the flag, the module itself needs to remain for some amount of time because uninstalling a module that doesn't exist on disk throws a warning/error?

So modules are good because they are tied to config? Making it easier to deploy? Just trying to understand the rationale.

Yes if it's not in config (or $settings, or a container parameter) it can't be deployed, and if it's not deployable, it requires a custom update or manual intervention as part of the deployment.

Modules also have the downside of once we remove the flag, the module itself needs to remain for some amount of time because uninstalling a module that doesn't exist on disk throws a warning/error?

Yes this is true, but we have the obsolete process for this, so while it takes a long time to get rid of them again, it's an easy/low effort task to do so, just one that's spread out.

Hi,

I have a different scenario we've started implementing just in the browser -- feature-flags that can be enabled in the browser with a GET param.

We've done this so far for two features:

- Enabling a "control" for toggling dark mode and growing/shrinking text
- Disabling a headless widget so we can more easily test a fallback experience

To do this, I just added some JS at the top of the page to look for a particular query param, and if found/set, store in the browser LocalStorage. Also a remove value. Then, if the localstorage value is enabled, apply a class or do something different in the JS bootstrapping of the widget.

So I was thinking of generalizing this, and found this issue.

Main point: I think there are different scopes to consider for feature flags, and obviously these have different implementations for the developers -- and I wanted to add this browser scope to the list. I'm thinking the base scopes here should include:

- Site-wide -- config and/or state to enable site-wide
- User -- user can enable a feature individually
- Some sort of plugin along the lines of @jurgenhaas mentions, that might be activated through ECA or similar -- which could be extended to support individual groups, roles, etc
- Browser - sessionless -- this can only work client-side, because otherwise it would have an impact on caching
- Browser - cookie/session-based

... a "feature flag" at its base would be one of these types, with a service making it easy to check if a feature is active, and perhaps a lightweight Javascript version for client-side flags.

Then, the feature flag system could provide plugins for reacting to flags -- condition plugins for use in blocks, something to wrap the Javascript attach() behavior, a default CSS class to apply to the body if a feature is active, an ECA condition (not in core, obviously, but if there's a plugin that could be provided)...

Just spit-balling here, expanding the scope slightly, perhaps in a direction that could go in contrib for part of...

Is this relevant?

📌 Replace "Expose all fields as blocks to Layout Builder" configuration with feature flag Active Here's a chance to use this API

What I'd like to discuss here is:

1. How should these modules be named? We should try to standardise on something
2. How should we communicate what the module (or feature flag) is used for? I.e should we use .info.yml description, a README, a hook help? Or something else

Very much in favor of this!

Back at DrupalCon Lille, @lauriii and I were talking about 📌 Combine field storage and field instance forms Fixed and I floated this exact idea as a way for us to reduce the risk associated with large-scale UI changes while still allowing us to move and iterate at pace.

A great example of how we could implement this is by following the patterns that web browsers like Chromium (and all of those built on it) use for feature flags. A screen that lists all of them and allows enabling/disabling them.

I fully support the idea of making this deployable, whether that's in configuration or in settings.

I would be in favor of this having a UI, because as noted, we could then use this for UI changes which can be turned on for sites without the need to update configuration or settings. We should though communicate clearly that these feature flags are for experimental features, and think carefully about where we place links to such a page.

Overall though, big +1 to this!

While commenting on #3431164-7: [meta] Improve the "Expose all fields as blocks to Layout Builder" feature → , it made me realize that we need to be really clear about how feature flags should be used, to quote myself from over there:

When I think feature flags though, I'm thinking things which are experimental and might be merged in some day, because that's how web browsers like Chromium treat them, I think that'll mean a lot of people will already have that expectation. What we have here though doesn't quite fit that expectation, where we've added this option not as something which might one day be "on by default" but almost the opposite, to remove something which was causing problems.

So we need to be mindful of those existing expectations from people before going ahead and using them. We should have a clear governance process/documentation for when it's appropriate to use feature flags.

I did find myself needing a new feature flag just this week -- for a decoupled website that is getting a design refresh on the front end. I implemented it as a custom condition that I applied to two different blocks, each associated with a different library getting sent to the browser.

The new front end in this case depends upon new fields we needed to deploy so the site owners could populate content and preview it before going live.

So it does seem like there are 3 parts to making a feature flag work:

1. A service that indicates whether a particular feature flag is active or not
2. Code that is controlled by a specific feature flag -- a condition plugin makes this easy ,to show or hide a block, to enable/disable something, etc
3. Mechanisms for enabling/disabling a feature flag.

For #3, having something that can be enabled/disabled as an anonymous user is crucial for the scenarios I've been doing so far. I ended up using a query argument to toggle a feature on or off, and then set a cookie to persist this status across requests.

I think this is an important scenario, to allow testing in production.

Also need to have ways to toggle feature flags per user, per role, site-wide, and perhaps to log places in code where each flag is used, so the code can be cleaned up when the feature is live for everyone if desired...

because uninstalling a module that doesn't exist on disk throws a warning/error

It's core, we can bypass uninstall validate for feature flag modules and just nuke them from config storage and K-V. Smartsheet updates do have a helper for nuking modules like that, it does work.

We can bypass uninstall validate for feature flag modules and just nuke them from config storage and K-V. Smartsheet updates do have a helper for nuking modules like that, it does work.

I would be in favor of this having a UI, because as noted, we could then use this for UI changes which can be turned on for sites without the need to update configuration or settings. We should though communicate clearly that these feature flags are for experimental features, and think carefully about where we place links to such a page.

I think to power both these features we probably want to support an additional key in info.yml files to flag this as a feature flag module.

That might be the only API we need here, along with some policy about what the module needs to contain as a minimum which I think is likely:

* A GenericTest
* A help topic/hook_help
* An info file with a decent description about what it does

Thoughts? If folks agree I'll update the issue summary to document what we want to do here.

I don't think the additional key in the info.yml file should prevent us from actually using this for real-world cases we already have.

We can bypass uninstall validate for feature flag modules and just nuke them from config storage and K-V. Smartsheet updates do have a helper for nuking modules like that, it does work.

We'd need to store this key somewhere if we were going to rely on it for uninstall-without-module-in-filesystem, currently .info.yml is only in cache. If we can do that easily, it would save keeping obsolete modules around for entire major release cycles though - although I wonder if we would want to also use this flag for some deprecated modules too?

I don't think the additional key in the info.yml file should prevent us from actually using this for real-world cases we already have.

Yes agreed. 📌 Replace "Expose all fields as blocks to Layout Builder" configuration with feature flag Active shows it's considerably less intrusive than what we're currently doing, we can build on top of that.

Yes, K-V is where I would store this, we already store module version there, put the "this module is safe to uninstall" flag there as well. Please don't restrict to feature flags, there are more than a few modules which know , just by the subject matter that it will never need an uninstall hook. The system already automatically deletes config belonging to a module being uninstalled, there's more than a few that needs nothing more.

So maybe I'm conflating things here, but to me having a module enabled/disabled is not a feature flag.

When I'm thinking of feature flags, I'm thinking of this: https://launchdarkly.com/blog/what-are-feature-flags/ ... specifically NOT config. Disabling a module is a config change, it's not a feature flag that can be done for some users and not others.

I think for 📌 Replace "Expose all fields as blocks to Layout Builder" configuration with feature flag Active having a module to enable/disable functionality seems like a reasonable approach -- but can we please not call it a feature flag? If we do, we're going to confuse a whole lotta devops folks that have an entirely different definition, when they need to work with Drupal.

@freelock I think the previous comments in this issue make a good case for using modules, and while it is slightly different to the traditional idea of a feature flag, we can add wrapping APIs in the future and other niceties mentioned above to make it clear which modules are considered feature flags.

In essence a FF is a concept that we are allowed to implement however we like. We've gone through other ideas here already, but modules give us a deployable method for these flags without having to reinvent yet-another-system to store this information.

From your link:

Feature flags are NOT configuration files

Generally, when you use configuration files to change the behavior of your software, you have to manually go into a file, change an environment variable, and then push the change through your deployment pipeline. But feature flags—in a platform like LaunchDarkly—evaluate in runtime. You flip a switch, and the change occurs instantly without you needing to redeploy, restart, or wait.

While, yes, technically this is Drupal configuration - it's not a configuration file in the traditional sense. We can still enable/disable modules directly in production (which obviously isn't recommended). It's up to the project maintainers to determine how changes propagate.

Personally, even with a traditional FF, I'd still be promoting that through environments and would not be toggling directly in production.

If you have any other ideas how these should/could be implemented by all means let us know :)

Can feature flags get their own package in the .info.yml - ie. not Core. This would make them easy to find / ignore on the module UI and also easy to report on etc...

I really like the idea of marking a module uninstallable without the code. It would make something much simpler. I think we should open a separate issue to discuss that feature. I think the core reasons why a module can not be uninstalled without it being there are:

• hook_uninstall
• hook_schema
• Provides an entity type

Thinking about what modules might be doing in hook_module_preuninstall is a bit complex because that's about other module reacting to a module being uninstalled before the module has been uninstalled... not sure about the ramifications of that.

I'd pretty interested in expanding this out to Contrib as well, so I'm not sure if we should be removing the Core package.

This might be expanding the scope of this a little too much, but what if we added another key for feature flags to the info.yml? Then in the modules page in a follow up we could filter them out based on the new key and have them in their own dedicated section?

Not something I feel super strongly about, especially if it will delay the issue moving forward. Just trying to think of options for both Core and Contrib.

I was looking for a way to use feature flags within Drupal and stumbled upon this issue. I'm a bit surprised seeing modules being proposed, but I'm probably missing something obvious.

If we want the on/off state of feature flags to be deploy-able (I'm still trying to grasp exactly why - especially looking at tools like LaunchDarkly - in which feature flags are meant to be evaluated during runtime), why is a module preferred over just a true/false configuration setting? What benefit does it give us?

why is a module preferred over just a true/false configuration setting?

Generally we're proposing to use this for things that are eventually supposed to be enabled permanently, but initially enabled via a flag so that sites have control over when the change is introduced.

Adding configuration boolean values would work for that, but it requires a lot of boilerplate and process that modules don't:

- you have to add the configuration schema, and an update path to set the default 'off' state of the flag
- you have to add a form somewhere so site owners can change the flag. This would either require a central 'feature flags' page or it would be dotted around the UI.
- when the feature is eventually enabled by default, it requires another schema change and another update path to remove the configuration.

This is compared to modules which are off (and therefore don't affect config at all) by default, can have names, descriptions and help text, already have a UI for installing and uninstalling. We still need the update path/process for uninstalling the module and removing it when the feature flag is removed, but that's covered by the obsolete module process and can potentially be simplified in the future.

We should consider migrating the existing linkset_endpoint setting from system.feature_flags.yml to a feature flag module.