perf: carry a source's group vars onto the target's groups - #215
Merged
Conversation
0.25.0 taught a declarative merge to find a field the source declares on a group, and then wrote the result onto every member of it. A value shared by 780 machines cost 780 entries -- the duplication the same release had just removed from the static-inventory connector, rebuilt one layer up. Merge it onto the target's group of the same name instead: one copy, and the consumer resolves it with its own precedence. The source declares what a group means, the target decides who is in it, and only variables cross. `all` needs no matching name, because in Ansible it means every host. The merged `all` carries the target's hostnames, because an endpoint drops a group with neither hosts nor children -- without them the variables would be rendered away silently, which is the failure the new integration test exists to catch. A field declared on a HOST is still copied onto that host: that data is genuinely per host. fields filters both paths, and a group the target does not have is skipped. merge_group_var_fields is additive key by key, so two enrichers on one group cannot erase each other, and it never rewrites membership the target owns. It invalidates the serialized dataset like every other mutation -- tested on its own, because the enricher calls merge_hostvar_fields right after and that would mask a missing invalidation.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follows #211 and #212. Same principle as #211, applied one layer up.
The problem
#212 taught a declarative merge to find a field the source declares on a group — but it resolved that onto each host and wrote it into
hostvars. A value shared by 780 machines cost 780 entries: exactly the duplication #211 had removed from the static-inventory connector in the same release, rebuilt in the enricher.The change
Merge it onto the target's group of the same name instead. One copy, and the consumer resolves it with its own precedence —
all, then more specific groups, then host vars, thenextra_vars.The source declares what a group means; the target decides who is in it. Only variables cross, so a shared group name still cannot pull one source's hosts into another — the property that makes an enricher usable where widening an endpoint's
source_idsis not.allneeds no matching name, because in Ansible it means every host.The trap this could have shipped with
merge_and_filterdrops a group with neither hosts nor children even when it carries vars. Anallgroup created with vars alone would be pruned on the way out: the enricher would report success, the dataset would hold the values, and the rendered inventory would contain nothing. Silent.So the merged
allcarries the target's hostnames, andenriched_group_vars_reach_the_rendered_inventoryasserts on the rendered endpoint rather than the cache. Removing the host list turns it red.Also
merge_group_var_fieldsis additive key by key, so two enrichers on one group cannot erase each other, and it never rewrites membership the target owns.merge_hostvar_fieldsstraight afterwards and that would mask a missing invalidation — the integration test genuinely could not tell.fieldsfilters both paths; a group the target does not have is skipped.Compatibility
A consumer reading
_meta.hostvarsfor a value declared on a group now finds it on the group. Through Ansible — or anything that resolves an inventory — the per-host answer is unchanged.Verification
cargo fmt --checkclean,cargo clippy --all-targets -- -D warningsclean, 583 tests passing. Six new, each red before the change.Beyond the suite, the release binary was run against a real 1097-host inventory (305 vars on
all) enriching a real 780-host target:--check-configalso passes against all five real deployment configs, including the one with 20 sources, 6 enrichers and 5 endpoints.