DevOps NetworkDevOpsNetwork
Join Free
Build it yourself

Deep config merger with override rules

mediumConfig and dependencies

Problem statement

Most infra tools build their final config from layers: built-in defaults, then a config file, then environment-specific values, then command-line flags. Helm values files and Kustomize overlays work on the same idea. Write the function that merges one layer onto another.

API (Java: ConfigMerger.merge(Map<String, Object> base, Map<String, Object> override) and mergeAll(Map... layers))

◈ DIAGRAM
merge(base: dict, override: dict) -> dict
merge_all(*layers: dict) -> dict # lowest priority first

Values are JSON-like: dicts, lists, strings, numbers, booleans and None.

Rules for each key in override

  1. If both sides are dicts, merge them recursively.
  2. A value of None (JSON null) deletes the key from the result.
  3. A key ending in +, like "features+": ["b"], appends its list to the base list under the name without the + (features). A missing base key counts as an empty list. If either side is not a list, raise ValueError("<key> expects a list").
  4. Anything else, including lists and type changes such as a dict replaced by a string, replaces the base value.

Also

  • Neither input may be modified, and the result must not share any dict or list with the inputs: changing the result later must not change base.
  • Key order: base keys keep their order, and new keys from override follow in override order.
  • merge_all(a, b, c) equals merge(merge(merge({}, a), b), c).

Outputs are printed as JSON.

Examples

Example 1

Input: base = {"server": {"port": 8080, "tls": {"enabled": false}}, "features": ["a"], "debug": true, "replicas": 2} override = {"server": {"tls": {"enabled": true, "cert": "/etc/tls.pem"}}, "features+": ["b"], "debug": null, "replicas": 3, "region": "eu-west-1"} merge(base, override) base["server"]["tls"] base["features"]

Output: {"server": {"port": 8080, "tls": {"enabled": true, "cert": "/etc/tls.pem"}}, "features": ["a", "b"], "replicas": 3, "region": "eu-west-1"} {"enabled": false} ["a"]

Explanation: port survives because server is merged, not replaced. debug is deleted by null. base is unchanged afterwards.

Example 2

Input: defaults = {"log": {"level": "info", "format": "json"}, "workers": 4} env = {"log": {"level": "warn"}, "workers": 8} cli = {"workers": null, "log": {"format": "text"}} merge_all(defaults, env, cli) merge({"db": {"host": "10.0.0.5"}}, {"db": "sqlite:///tmp/dev.db"}) merge({"features": "a"}, {"features+": ["b"]})

Output: {"log": {"level": "warn", "format": "text"}} {"db": "sqlite:///tmp/dev.db"} error: features+ expects a list

Explanation: Each layer overrides only what it names. A scalar may replace a whole dict. Appending to a string is an error, not a silent replace.

Hints

Approach

Optimal

A recursive function over override's keys:

  1. Start result as a deep copy of base. This preserves base key order and guarantees no shared mutable objects.
  2. For each (key, value) in override:
    • key ends in +: look up the base list under key[:-1] (empty if absent), check both sides are lists, and store current + copy(value) as a new list.
    • value is None: remove the key if present.
    • Both value and result[key] are dicts: result[key] = merge(result[key], value).
    • Otherwise: result[key] = deepcopy(value).
  3. merge_all folds the layers left to right starting from {}.

New keys land at the end of the dict in override order because Python dicts (and Java's LinkedHashMap) keep insertion order. The work is proportional to the total size of both inputs because of the copies. Skipping the copies would be faster, but aliasing bugs in config code tend to show up far from where they were caused.

ComplexityTime O(size of base + size of override)Space O(size of result)
Python
import copy
def merge(base, override):
"""Deep-merge override onto base. Returns a new dict; neither input is modified."""
result = {k: copy.deepcopy(v) for k, v in base.items()}
for key, value in override.items():
if key.endswith("+"): # "plugins+": [...] appends
target = key[:-1]
current = result.get(target, [])
if not isinstance(current, list) or not isinstance(value, list):
raise ValueError(f"{key} expects a list")
result[target] = current + copy.deepcopy(value)
elif value is None: # null deletes the key
result.pop(key, None)
elif isinstance(value, dict) and isinstance(result.get(key), dict):
result[key] = merge(result[key], value) # both dicts: recurse
else: # scalars, lists, type changes: replace
result[key] = copy.deepcopy(value)
return result
def merge_all(*layers):
"""Lowest priority first, e.g. merge_all(defaults, env_file, cli_flags)."""
result = {}
for layer in layers:
result = merge(result, layer)
return result

Follow-up questions

  • Track provenance: for each leaf value in the result, which layer set it?
  • Add a - suffix that removes listed items from a base list.
  • Merge lists of dicts by a key field (name), the way Kubernetes merges container lists.

Frequently asked questions

Because there is no single right answer. Should the override's second element replace the base's second, or be added? Tools that try (Kustomize strategic merge patches match list items by a key such as name) need a schema to do it. Replacing by default, with an explicit append syntax, is predictable. Helm replaces lists entirely, which is why people often turn lists into maps keyed by name in values files.

In Go the config is map[string]any after json.Unmarshal or yaml.Unmarshal, and the merge type-switches on map[string]any, []any and scalars. Watch out for YAML libraries that produce map[any]any for nested maps. Go maps are unordered, so if order matters (for stable diffs), sort keys when writing out. Production tools also record which layer each final value came from, which turns "why is replicas 3?" from an hour's search into one command, and validate the merged result against a schema.

Layered config is a daily reality: Helm values, Terraform variable files, Ansible group_vars, twelve-factor environment overrides. The question tests recursion, careful edge-case rules, and not mutating inputs, a classic source of "it only breaks on the second deploy" bugs.