Skip to content

perf: carry a source's group vars onto the target's groups - #215

Merged
blaipr merged 1 commit into
mainfrom
feat/enricher-merges-group-vars
Sep 1, 2026
Merged

perf: carry a source's group vars onto the target's groups#215
blaipr merged 1 commit into
mainfrom
feat/enricher-merges-group-vars

Conversation

@blaipr

@blaipr blaipr commented Sep 1, 2026

Copy link
Copy Markdown
Member

Follows #211 and #212. Same principle as #211, applied one layer up.

The problem

#212 taught a declarative merge to find a field the source declares on a group — but it resolved that onto each host and wrote it into hostvars. A value shared by 780 machines cost 780 entries: exactly the duplication #211 had removed from the static-inventory connector in the same release, rebuilt in the enricher.

The change

Merge it onto the target's group of the same name instead. One copy, and the consumer resolves it with its own precedence — all, then more specific groups, then host vars, then extra_vars.

The source declares what a group means; the target decides who is in it. Only variables cross, so a shared group name still cannot pull one source's hosts into another — the property that makes an enricher usable where widening an endpoint's source_ids is not.

all needs no matching name, because in Ansible it means every host.

The trap this could have shipped with

merge_and_filter drops a group with neither hosts nor children even when it carries vars. An all group created with vars alone would be pruned on the way out: the enricher would report success, the dataset would hold the values, and the rendered inventory would contain nothing. Silent.

So the merged all carries the target's hostnames, and enriched_group_vars_reach_the_rendered_inventory asserts on the rendered endpoint rather than the cache. Removing the host list turns it red.

Also

  • merge_group_var_fields is additive key by key, so two enrichers on one group cannot erase each other, and it never rewrites membership the target owns.
  • It invalidates the serialized dataset, like every other mutation. Tested on its own, because the enricher calls merge_hostvar_fields straight afterwards and that would mask a missing invalidation — the integration test genuinely could not tell.
  • A field declared on a host is still copied onto that host: per-host data stays per host.
  • fields filters both paths; a group the target does not have is skipped.

Compatibility

A consumer reading _meta.hostvars for a value declared on a group now finds it on the group. Through Ansible — or anything that resolves an inventory — the per-host answer is unchanged.

Verification

cargo fmt --check clean, cargo clippy --all-targets -- -D warnings clean, 583 tests passing. Six new, each red before the change.

Beyond the suite, the release binary was run against a real 1097-host inventory (305 vars on all) enriching a real 780-host target:

all.vars:   {"backups_role":"avamar","cmdb_role":"device42","monitoring_role":"opsview"}
all.hosts:  780 hostnames
groups:     219 rendered
hostvars:   780 hosts, 0 carrying cmdb_role     ← no per-host duplication

--check-config also passes against all five real deployment configs, including the one with 20 sources, 6 enrichers and 5 endpoints.

0.25.0 taught a declarative merge to find a field the source declares on a
group, and then wrote the result onto every member of it. A value shared
by 780 machines cost 780 entries -- the duplication the same release had
just removed from the static-inventory connector, rebuilt one layer up.

Merge it onto the target's group of the same name instead: one copy, and
the consumer resolves it with its own precedence. The source declares what
a group means, the target decides who is in it, and only variables cross.

`all` needs no matching name, because in Ansible it means every host. The
merged `all` carries the target's hostnames, because an endpoint drops a
group with neither hosts nor children -- without them the variables would
be rendered away silently, which is the failure the new integration test
exists to catch.

A field declared on a HOST is still copied onto that host: that data is
genuinely per host. fields filters both paths, and a group the target does
not have is skipped.

merge_group_var_fields is additive key by key, so two enrichers on one
group cannot erase each other, and it never rewrites membership the target
owns. It invalidates the serialized dataset like every other mutation --
tested on its own, because the enricher calls merge_hostvar_fields right
after and that would mask a missing invalidation.
@blaipr
blaipr merged commit 9cf3c7f into main Sep 1, 2026
6 checks passed
@blaipr
blaipr deleted the feat/enricher-merges-group-vars branch September 1, 2026 15:24
@blaipr blaipr mentioned this pull request Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant