Label file_test_mod2 as the solution - #49
Merged
Merged
Conversation
It holds the ground truth, but "Test mod2" reads like just another input. Also regenerate the README, which had drifted from the api yamls for unrelated reasons.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Describe your changes
file_test_mod2.yamlis the ground truth this whole benchmark is scored against, and the one file a method must never see. Its label was:which reads like a fourth ordinary input, sitting next to "Train mod1" / "Train mod2" / "Test mod1". Relabelled to "Solution" and added a description saying who receives it -- the metrics and the control methods, nobody else. That separation is the thing keeping the benchmark honest, so it's worth stating rather than leaving to be inferred from
run_benchmark'sfromState.No functional change; the file name and the argument names stay as they are.
This also regenerates
README.md, which is why the diff is bigger than the change. The README had drifted from the api yamls for unrelated reasons -- the author table gained columns, the mermaid node ordering changed, and Marius was added as a contributor in #10 -- so a regeneration was overdue anyway. I held it back from the earlier api PRs (#28, #29, #30) to keep those diffs readable; this is the catch-up.Part of a series of PRs coming out of a pre-run review of the benchmark.
Checklist before requesting a review
I have performed a self-review of my code
Check the correct box. Does this PR contain:
Proposed changes are described in the CHANGELOG.md
CI Tests succeed and look good!