Managing Shared Configuration Part 5: Updating from Extensions

This is the fifth installment in a series presenting work on shared configuration that comes out of the Drutopia initiative and related efforts. To catch up, see Part 1, Configuration Providers, Part 2, Configuration Snapshots, Part 3, Respecting Customizations, and Part 4, Configuration Alters.

In this installment we'll start to pull it all together.

Paraphrasing a bit from Part 1, we described a key problem this way:

How can I update my site so that I have all the latest configuration changes from a distribution--while still retaining any customizations I made?

In Part 1 we mentioned Fabian Bircher's widely used Configuration Split module and its enabling API module, Config Filter, returning in Part 3 to give a more detailed introduction. In Part 2, we summarized the API and accompanying user interface that core provides for staging configuration. Here we'll take a deep dive into how we can merge in configuration updates from installed extensions through an approach that's built on Config Filter and closely parallels core's configuration staging.

Config Filter and core's configuration staging

From Part 2 in this series:

Using Drupal core, you can stage configuration between different environments or versions of a given site--say, from a development environment to a testing one. For details of how this works, see the Configuration Management section of the Drupal 8 handbook. The Synchronizing Configuration Versions section of the Drupal 8 User Guide also reviews this functionality.

When you stage configuration, core reads the configuration to import from a configuration storage (see the overview on storages in Part 1). Core uses a storage provided by the service, which by default serves up files from a specified directory.

At its most basic, configuration staging is a two-step process:

  • Export configuration from one environment - say, a development ("dev") environment - and copy them to the sync directory on another instance of the site--say, a live environment.
  • Run core's configuration synchronization. All going well, your live site now has configuration that's identical to what was exported from dev.

But what if you don't want live to be exactly the same as dev?

That's where Config Filter comes in. Any module can provide one or more filters. Configuration Split allows site admins to create configuration "splits" that are applied as filters on top of core's sync directory. For details, see the module's documentation and the blog post Configuration Split: Managing Drupal 8 Configuration for Different Environments.

Config Filter makes Drupal core's configuration staging workflow a lot more flexible. But its API is not limited to the staging workflow. A config filter can apply to any storage; core's is only the default.

So, using Config Filter, the way is open for a contributed module to mirror core's configuration staging workflow, but providing a framework for merging in updates from extensions rather than staging configuration between environments.

Config Distro and Configuration Synchronizer

And that's exactly what the Configuration Distro module does.

Configuration Distro provides a filtered storage, that wraps core's active configuration storage. Config Filter plugins using that storage will apply their changes to the site's active configuration--just what you need to bring in updates from extensions. It then provides a UI directly parallel to core's configuration staging UI.

And it stops there, for reasons explained in the blog post announcing Configuration Distro:

If you only install Config Distro, the import screen eternally shows that there is nothing to import. This is because the module only provides the framework for this to work, ie the UI and the drush command. The module is not opinionated on how or what should be updated. All the complexity can be addressed by a distribution maintainer with the help of a ConfigFilter plugin (the same way Config Split and Config Ignore work). One such module is Config Sync [Configuration Synchronizer]. All the complexity of finding out what configuration a module has originally shipped with, what it does now and whether a user has changed the originally installed configuration is left to Config Sync and its dependencies.

Much of the heavy lifting for "all the complexity" is accomplished through the various modules profiled in this series so far, including Configuration Provider, Config Merge, and Configuration Snapshot. But there's plenty left for Configuration Synchronizer to do ;)

A config snapshot for each extension

The Configuration Snapshot module makes it possible to take snapshots, but leaves the logic of when and why to other modules. In Configuration Synchronizer, we take config snapshots and apply alters to existing snapshots when extensions are installed. We also subscribe to an event, ConfigDistroEvents::IMPORT, that the Config Distro module dispatches on completing a configuration import. This allows us to refresh the snapshot for every extension that had updates imported.

Updates = Providers - Snapshots

Determining what updates are available for each extension involves three steps:

  • First, consult Configuration Provider to find out what configuration is currently provided.
  • Next, load our configuration snapshot for the extension via Configuration Snapshot.
  • Finally, compare the two (using core's StorageComparer) to determine what's been changed or added.

A config filter for each extension

Just as we have a snapshot for each extension that provides configuration, we provide a corresponding configuration filter.

We need a filter for every extension that has available updates--a list that may change any time we update to a new version of an extension or run the Config Distro updates. So we follow a pattern from Drupal core that allows us to dynamically declare our filter plugins: plugin derivatives. An example of a plugin deriver is, BlockContent, used in core to make a block available for each piece of block content. For a detailed backgrounder, see this Tutorial on Using Drupal 8 Plugin Derivatives Effectively.

In our plugin deriver, SyncFilterDeriver, we iterate through the list of available updates by extension and add a plugin for each.

Update modes

In Configuration Synchronizer we provide two different modes for bringing in configuration updates.

The default update mode is merge. When the merge mode is selected, the configuration filter for a given extension merges in changes via a call to the relevant method in Config Merge. This way, any customization done to the site's active configuration is retained.

But in certain cases you might not want to retain customization. So a second update mode is available: reset. When the reset mode is selected, the active configuration is reset to what's currently provided by the extension. This mode is equivalent to what was called "reverting" in the 7.x version of the Features module.

Customizing the update form

Config Distro provides a form for configuration updates that's pretty much identical to the "Configuration synchronization" form provided by Drupal core. But in Configuration Synchronizer we support a separate update per extension. So we extend the Config Distro form to provide a lot more information as well as options.

For each extension, we list details of what updates are available, showing:

  • The type of update (new items vs. changed ones).
  • The type of configuration.
  • The configuration item names.

Clicking on or selecting an individual item's name will load a diff for that item.

And we provide a checkbox to select that extension's updates.

In an "Advanced" section, we offer a select to choose the update mode, "Merge" or "Reset".

Config Distro Ignore

The merge and reset update modes are useful, but there may be cases where you want to combine the two. For example, you might want to reset almost all configuration on your site to the state that currently ships with a distribution, but still retain a select few customizations. To cover that case, Config Distro ships with a sub-module.

Using Config Distro Ignore, you can specify particular configuration items that should be ignored (left untouched) when you bring in changes. Once you have done so, if you then set the update mode to "Reset" and run an import, those specific items should be retained while all others are reset.

Potential enhancements

If there's one thing that's probably obvious by this point, it's that managing configuration updates from contributed modules is complex!

A major lack in Configuration Synchronizer and its Drutopia-sponsored dependencies is test coverage.

Support for Drush has not yet caught up with what's in the module's UI; see the feature request Add Drush command for the "reset" update mode.

Another area for improvement is handling of configuration that's been deleted from the module that originally provided it; see Optionally delete config entities when they're deleted from code.

Next up

Stay tuned for the next post in this series: Packaging Configuration.