Add GenerateAll command to bulk generate tests
#368
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related to #191, #194, and #207.
I was originally troubleshooting the leadling newline on all of our generated test suites. That turned out to be occurring because the generator was adding that line to separate the test suite from a version header at the top. However, the version header wasn't being generated any more. In the process of that, I realized we needed a GenerateAll command to bulk-generate test suites for all implemented practice exercises. This would help test additional fixes and tweaks across the entire track.
Once this command was implemented, I was able to test parsing the tests.toml files directly to filter out tests that shouldn't be included, finding two tests in
anagramthat should have been removed. I also discovered the occasional lack of a separating newline in execute blocks was at the boundaries of each group of test cases so that's handled as a result here and newlines are inserted properly between groups. Finally, nulls in the canonical data are remapped tov:nullinstead of an empty string which is important forflatten-array.Using this command, I'll begin regenerating the test suites across the track. There are a few like
dnd-characterthat require a lot of manual cleanup, but they'll be dealt with separately. Other exercises need a closer look on how to best present long lists since I manually put those across multiple lines and it would be good to keep that behavior rather than creating long one-liner lists. That'll likely be part of the next set of tweaks to the test generator, but what I have is probably enough for now.