another attempt for more ca problems reproduction

[cdietrich]

No description provided.

[github-actions]

Test Results

  8 076 files  ±0    8 076 suites  ±0   3h 47m 25s ⏱️ + 16m 39s
 43 241 tests ±0   42 655 ✅ ±0    586 💤 ±0  0 ❌ ±0 
212 460 runs  ±0  209 529 ✅ ±0  2 931 💤 ±0  0 ❌ ±0 

Results for commit 464e510. ± Comparison against base commit 043223e.

♻️ This comment has been updated with latest results.

[cdietrich]

@LorenzoBettini wdyt about this holzhammer change

[LorenzoBettini]

I seem to understand that this does not fix the problem for good. The last but one commit still has a failure, hasn't it?

Feel free to merge, but I think this just increases chances to avoid flakiness due to delays...

It might still be good to extract some methods into our testing library or maybe add some of the wait conditions in waitForBuild itself.

[cdietrich]

@LorenzoBettini well i had 20 green runs on ci and the one failing was not in the content assist test
which failure are you referring too

and no i dont have time to refactor. i need to use that time to sleep :(

[LorenzoBettini]

OK, please feel free to merge. And thanks for all the work. :)

As I said, all the operations about waiting and so on don't make sense to me. I mean, I don't understand them; I only think they delay the execution, reducing the likelihood of flakiness.

Retrying the search in the JDT index, after specifying that we want the index to be ready, also doesn't make sense: if we need to retry the query, it means there's a bug in JDT.

I think Eclipse, JDT, PDE just don't provide reliable synchronization mechanisms for testing purposes and do tons of things asynchronously. Unfortunately, in our IDE tests, we "stress" all these mechanisms a lot ;)

cc @szarnekow

[cdietrich]

so the alternative would be to delete all flaky tests

[LorenzoBettini]

so the alternative would be to delete all flaky tests

No: what I suggested in the ticket was to re-enable the surefire rerun option ONLY in test projects that exhibit flakiness.

But, as I said, let's keep this and merge it, since it contains interesting code and experiments that might be used in the future in the testing library.

In the future, we could also think of a more involved waitForBuildAndAssert(Runnable r) that

• calls the original waitForBuild
• calls the runnable in a try catch catching a possible AssertionError
• in case, retry after some delay and for a limited number of times.

Including some additional utilities like waitForBuildAndAssertNoErrors, which calls waitForBuildAndAssert(Runnable r) passing a lambda that checks there are no errors in the workspace.