Managing Shared Configuration Part 4: Configuration Alters

This is the fourth installment in a series presenting work on shared configuration that comes out of the Drutopia initiative. To catch up, see Part 1, Configuration Providers, Part 2, Configuration Snapshots, and Part 3, Respecting Customizations.

In the next installment we'll start to pull it all together, showing how all the pieces covered so far fit into a solution for merging in configuration updates. But first there's one more piece to add. In this installment we'll be looking at creating and updating configuration alters.

First off, what is a configuration alter?

An alter is an addition, deletion, or change to a piece of configuration.

When we're producing a several packages of shared configuration - what are often called feature modules - sooner or later we need a particular feature to modify configuration that was provided by another feature.

Say I'm producing a Drupal distribution that includes two different features: an event feature and a location feature. Any given site might install the event feature, or the location feature, or both, or neither. (A big part of the point of having distinct features is to make them optional and independent.)

In this example, say the event feature provides an 'event' content type with fields including a required 'date' field. The description of the event content type is: "An event takes place at a specified time." The location feature provides a 'location' content type.

But if I have both events and locations, there's a relationship between them. Events take place at a location. So on a site with both the event and the location features installed, the event content type should get an additional required field, "Venue", that's a reference to a location. When that happens, the description of the original event content type should change accordingly. Now it should read: "An event takes place at a specified time and place."

To make this happen, the location feature (or, possibly, a third feature) is going to have to alter the event feature's configuration.

Drupal's configuration data is exported in YML format. Here's a snippet of what the event content type would look like as provided by the event feature:

name: Event type: event description: 'An event takes place at a specified time.'

We only want to change one property, description. Here's the result we want after an alter:

name: Event type: event description: 'An event takes place at a specified time and place.'

While it looks innocuously simple, even this rudimentary alter turns out to present thorny challenges when it comes to producing and updating packages of shared configuration.

Overrides and alters in Drupal core

There are several ways that configuration changes, overrides, and alters are supported in Drupal core. We'll review them before digging into specifics.

Optional configuration

Extensions can provide optional configuration: configuration items that are installed only when certain conditions are met. In our example, the event venue field could work well as optional configuration provided by the location feature. This optional venue field will be installed only when the event feature is installed and its event content created. We'll see below that the task isn't quite so easily accomplished, but getting the field in place is at least a start.

Optional configuration helps with the use case of extending one module's configuration in another module, but doesn't do alters.

Installation profile configuration overrides

If you've ever tried to install a Drupal module that provides configuration that already exists on your site, you'll be familiar with the often perplexing error that Drupal core throws to the effect that configuration items provided by the module "already exist in the active configuration". This error occurs because regular modules and themes can't re-provide configuration that's already provided by another extension. But installation profiles can. For example, if you put a customized version of the site.info.yml file in an installation profile's config/install directory, that file will be installed on the site rather than the version provided by core's system module. For more on configuration overrides in installation profiles, see this blog post.

Configuration overrides in installation profiles are useful, but have key limitations:

• They're only an option in the installation profile.
• They only work for a full override of a configuration item, not for a selective alter.

Programmatic changes to configuration

It's possible to make programmatic changes to configuration in custom code--for example, in an update function. That's what core does in the Standard installation profile's install function. For example, code in that function sets the site's home page, a property of the site.info configuration item.

That works for one-off changes. But:

• As alters multiply, so does the amount of custom code.
• Such changes can't be worked into an automated configuration update workflow. For example, the Configuration Provider module introduced in part 1 of this series has no way of guessing what arbitrary changes may have been made in custom code.

Configuration override system

Drupal 8 introduced a system to override configuration from modules--see the relevant documentation on drupal.org.

In this system, an override is a change to configuration that's dynamically applied on top of the site's active configuration. Overrides are a good fit for cases where a change needs to be made only under certain conditions. The classic use case in core is language-specific configuration overrides--see the drupal.org documentation on translating site interfaces. If you've ever switched languages on a Drupal 8 site and seen a field label change to the new language, you've seen configuration overrides in action. User interface translation fits the model of a conditional change because the text presented to users needs to be changed only if the site is multilingual and the interface language is other than the site's default language.

Configuration overrides are also used in the contributed Domain Access module. There, the need is to customize configuration per domain.

There's a contributed Configuration Override module that allows modules to provide configuration overrides alongside the other configuration they provide.

But for our purposes it won't work for alters to be layered on top of the active configuration. Instead, we need them to behave just like regular extension-provided configuration. For example, we want the resulting configuration to be editable. Configuration overrides aren't a good fit.

Overriding configuration in settings.php

A final way core supports altering configuration is in the settings.php file used to provide and customize a site's settings--see the relevant documentation on drupal.org. For more on overriding configuration in settings.php, see this blog post.

Again, while useful for other purposes, this approach doesn't help with feature-provided configuration since (a) it has the same disadvantages as module-provided configuration overrides and (b) we can't rely on site admins to manually edit their settings.php files.

Challenges of Drupal 8

In the intro we looked at an example of configuration altering: changing a content type's description. But in Drupal 8 the need for alters runs much deeper.

Shared configuration was the original use case in Drupal for exportable configuration. Drawing on innovation by Earl Miles and others in the Views module, major work took place in and around the Features module. The original announcement of the Features module in 2009 laid out the program:

A feature is a collection of Drupal entities that, taken together, satisfy a certain use-case.

And:

individual feature modules don’t script site building or use update scripts to maintain themselves. Instead, we take advantage of the powerful exportables concept pioneered by Views to allow all of your feature’s site building to live in code (exported views, contexts, imagecache presets, node types, etc.) but also be overridden and in the database.

Because work focused squarely on the Features use case, exportables were structured from the start to facilitate stand-alone configuration packages. To take two key examples:

• A user role can have multiple permissions assigned to it. Typically, a site will have a small number of user roles and each separate feature will need to provide permissions to one or all of those roles. In Drupal 7, exportables were structured to make this easy, by exporting user permissions separately from roles.
• A content type has multiple fields. A fairly common need is for one feature to add a field to another--like in our example of a location feature adding a field to events. As well as the field structure itself, a field typically needs to be displayed. In Drupal 7, the export of a field (then termed a field instance) included its presentation information for multiple displays (view modes).

But the focus on configuration staging in Drupal 8 meant that exportables were rewritten seemingly without reference to the use case of shared configuration packages.

Contrary to Drupal 7, in Drupal 8:

• Permissions are exported as part of the role entity. This means that this quintessentially basic requirement of stand-alone configuration packages - adding a permission to a site-wide user role - requires complex customization via alters. Various workarounds have emerged in contrib. Two examples are Config Role Split and Role mixins / Subroles. The Features module in Drupal 8 includes a related workaround: permissions are optionally stripped on user role export, since lumping them into a single feature would break the ability to produce stand-alone features.
• With field configuration, we're no better off. The relevant configuration entities include display information for all fields in a single entity, meaning a feature adding a field to another feature's content type can't just export its own configuration. Instead it must perform complex alters on multiple entities provided by the other feature. How? See this Drutopoia documentation page for the messy details of one approach.

Because of regressions in Drupal 8 compared to previous versions, alters are needed even for some of the most basic and common tasks of building features.

Modules for configuration alters

Fortunately, there are at least two contributed modules that provide support for module-provided configuration alters: Config Actions and Configuration Rewrite. The Update Helper module provides similar functionality, though in that case linked specifically to updates. In Drutopia we've chosen to work with Config Actions since it provides a full set of tools for adding, deleting, and changing configuration properties along with extensive documentation.

Using Config Actions, a config action is a manually edited file that you place in a module's config/actions directory. The file contains just the additions, deletions, or changes you want to make.

Here's a simple example. To accomplish the alter from the intro - changing the description value of the event content type - we would start by creating a file with the same name as the item we're altering, node.type.event.yml, and putting it in the location feature module's config/actions directory. The file's content would be:

plugin: change path: ['description'] value: 'An event takes place at a specified time and place.'

This says:

• Use the change plugin (a plugin determines what type of action to apply).
• Make the change the description property.
• Set the new value to read 'An event takes place at a specified time and place.'

Add the file, install the event feature, then install the location feature, and voila, we have the altered event content type description just like we wanted.

But what if a given site installs the location feature first, and only later adds the event feature? Sadly, in that case, we'd be out of luck. By default config actions (and the config rewrites supported by the Configuration Rewrite module) are applied only when a module is first installed. To address that issue, we might have to pull our alter into a bridging feature: a feature that provides the integration between two other features. If we make the bridging feature require both the event and the location feature, we can ensure the event content type exists when the alter is applied.

For our use case of importing configuration updates from features, alters present a series of additional challenges that don't have such easy workarounds.

Altering provided configuration

In Part 1 of this series we looked at the Configuration Provider module, which answers the question: what configuration is available to be updated? That module includes support for the configuration models supported by Drupal core: required configuration in config/install and optional configuration in config/optional.

Support for config actions is accomplished in Config Actions Provider. As noted on its Drupal.org project page:

Config Actions Provider integrates the Config Actions module with Configuration Provider, making it possible to apply config actions during configuration updates.

Concretely, Config Actions Provider provides a config_provider plugin that accepts configuration provided by other providers (such as required and optional) and applies any available alters to it.

The Nitty Gritty

For those interested in the messy details....

Alters raise the issue of ownership. Which module "owns" the snapshot of a configuration item? Typically, we'd say a piece of configuration is owned by the module that provides it. But with alters the resulting configuration structure is the sum result of input provided by multiple sources.

Also, the order in which alters are applied matters. If two different modules have changed the same configuration property, only the second change will be in effect.

In Config Actions Provider and related modules, we assign ownership to the module that originally provided the configuration. For example, when we're listing available configuration updates by providing module, any alter-provided changes are assigned to the module that provided the original item.

To emulate the altering that happens at install time, alters are applied in module dependency order--the same order that modules are installed in. Even if we're loading only a single module's available configuration, we iterate through all installed modules, since any of them could provide an alter for this module's configuration.

If determining what alters would be applied to configuration as provided sounds tricky enough, it's really nothing compared to the challenges of applying alters to configuration snapshots.

In Part 2 of this series we looked at the Configuration Snapshot module which provides the ability to take snapshots of configuration as provided by installed extensions. A snapshot should answer the question: what was the state of this configuration as provided when it was last installed or updated?

Introducing configuration alters makes that question a lot harder to answer.

Recall that alters are applied when the module providing them is installed. This means that a snapshot of the configuration as installed or updated has to include all alters that were actually applied, but not any that may have been added since installation or update. Say as a module author I add a new alter to an existing module. New sites that install that module will get the alter, but sites on which the module is already installed won't. For our purposes, what should happen in this case is:

• The alter should show up in the data returned by Configuration Provider on what's available to be installed or updated.
• The alter should not be included in the snapshot.

That way, when the currently-available configuration as provided is compared with the previous snapshot, the alter will show up as an available update.

In our Drutopia work, much of the handling of snapshotting for configuration alters has gone into the Configuration Synchronizer module, which we'll introduce in the next installment in this series.

Integration for snapshotting config actions is in the Config Actions Provider module. There we do pretty much the inverse of what we do for provision. Rather than iterating through all modules and applying their alters to the provided configuration, as we do for provision, for snapshotting we iterate through all existing extension snapshots and apply to the snapshots any alters provided by the extensions we've just installed. That way, when a module is installed, any alters it provides are added to existing snapshots.

Potential enhancements

The core issue Add configuration transformer API proposes an approach that dispatches and subscribes to events (not the kind in the hypothetical event feature discussed above!) to assemble available configuration. If that API is added to core, it could open the way to use a parallel approach for config alters.

Related core issues

• Provide an API for altering extension provided configuration.
• Review and selectively refactor configuration entity structure to facilitate shared configuration packages.
• Pull permission assignments out of role config entity.
• Pull field view mode and form mode configuration into separate entities.

Next up

Stay tuned for the next post in this series: Updating From Extensions. In that installment, we'll start to pull together everything we've covered so far: configuration providers, snapshots, merges, and alters.