SUGGESTION: Improve defaults/README to give performance tips #456
Description
By default, running llvm-cov with a decently tested project of medium size (~10k LOC, ~20-30 integration tests) will take upwards of 3-4 minutes and generate potentially 100GB of files. It was so bad, it filled 250GB of empty space on my C drive when running it.
Digging through the settings, issues on the github, and experimenting a lot, I've brought it down to 700MB of output, and running in ~30 seconds instead. This mainly came from the following env var:
LLVM_PROFILE_FILE_NAME="myproject-%m.profraw"
Since each integration test was generating a ton of files, the duplication across profraws was insane, and caused both the test running and report generation to take way longer than it should. I don't know why a better file name like this isn't default, with how big the speedup is, and with what seems like no drawbacks.
More documentation on clean and clean --workspace would be nice, I noticed that it was deleting the .fingerprint files for workspace members, which I didn't want (I don't know why Cargo shouldn't just handle it).
Another thing I've noticed is report generation takes super long, running all my tests and merging the reports takes ~5 seconds, but the report always takes ~20 seconds no matter the report type. After a bit of debugging, I figure that this is heavily inflated by workspace members and tests/ files, since every one is its own binary (causing another exe to be created, and forcing llvm to read the entire binary again and again, and not just the workspace's code, which is compounded even worse by llvm-cov only having parallel file writes, the running is synchronous. By manually calling it myself I reduced it to ~10 seconds, and parallelizing it by running it once for every binary and merging it resulted in ~3 seconds total. Coverage was almost exactly the same (included the integration tests themselves on accident), but implementing the parallelization and merging in llvm-cov itself would be really nice for speed.
Overall, something to communicate these optimizations, and potentially making them defaults or integrating them in some way, would be really helpful so people don't have to spend a lot of time digging through github issues and debugging to figure them out.