chore: fix test caching #6470

Job	Run time
Format check	39s
Linter	2m 28s
Unit tests (windows-latest)	2m 25s
Unit tests (macos-latest)	1m 6s
Integration tests (windows-latest)	1m 36s
Integration tests (macos-latest)	46s
E2E tests (windows-latest)	16m 6s
Integration tests (ubuntu-latest)	44s
E2E tests (ubuntu-latest)	8m 17s
E2E tests (macos-latest)	11m 19s
Build	1m 24s
Unit tests (ubuntu-latest)	1m 11s
	48m 1s

Provide feedback