Testing with Angular Component Harnesses?

[bjmc]

I'm new to Angular and trying to understand where component harnesses fit into the testing landscape.

Would this be something worth addressing in this book? Or is it out of scope?

[molily]

Great question! It's definitely in the scope of the book, but there are reasons why testing harnesses are not covered yet.

When I wrote the better part of the book, Angular Material already provided harnesses for their components. And you could create your own test harnesses using Material's ComponentHarness. Since then, the Angular team has promoted the concept more broadly. Still I haven't seen that much adoption in the wider Angular community that is not tied to Material.

The idea of harnesses make sense from the viewpoint of a generic, reusable component library like Material. If you use Material components in my component, you cannot write a pure unit test and mock these dependencies easily. So you need to write some form of integration test. Then you face the problem of interacting with these third-party components. You need to make assumptions about the internals, reaching deep into the component's DOM. People then tend to find nested elements by classes, like .mat-datepicker-input, which is an anti-pattern. Such tests rely on implementation details of a third-party library and are prone to fail.

Test harnesses solve this problem by abstracting the interaction. For example, in my test, I'd like to write the logic "when the user enters the date 8/8/23 using the time picker, I want the component to do X". And not simulate the clicks, key presses and other events that lead to 8/8/23 being selected.

This abstraction seemed odd to me 2-3 years ago but I couldn't exactly pinpoint why back then. That's why I did not include a chapter on test harnesses. (Also I did not see much community adoption.)

I think today I have a clearer picture: Whether the harness abstraction make sense in a particular project depends on the testing philosophy and abstraction level you choose for your tests.

Today I'm more and more in favor of the testing principles from the Testing Library with the slogan "The more your tests resemble the way your software is used, the more confidence they can give you." The goal is also to write high-level, abstract tests that do not test implementation details. But the approach is very different from harnesses.

With the Testing Library, you interact with the document on the HTML level as it is presented to the user: Text, buttons, links, form fields, elements with certain ARIA roles. Tests should deal with DOM nodes rather than component instances. Quite like end-to-end tests which by nature know nothing about Angular and components.

The Testing Library recommends to find elements by accessible name, field label, text content and ARIA role. Because this is what matters to the user eventually. (In the book, I tried to follow the core principles but have mostly used test ids for querying.)

I think this is a very healthy way of testing because it focuses on the output while promoting semantic markup and basic accessibility practices. And it's independent from UI libraries and how you organize your Angular components.

So how does the Testing Library approach perform when interacting with third-party components? I think I works quite well since established component libraries like Material produce an accessible HTML DOM. Even complex components boil down to form fields and buttons with proper labels and ARIA roles.

Of course, when the library changes its internals, such tests need to be adapted. Proponents of the Testing Library principles would say: That's expected behavior, the tests should fail when the relevant HTML structure (labels, accessible names, buttons, roles etc.) change. Because this is the public, user-facing interface.

But it's still tedious to adapt many tests only because Material fixed an ARIA role or similar. For these cases, I would write simple helper functions that perform the interactions with the widget. Looking at Material today, it's pretty straight-forward to interact with the widgets via buttons, labelled inputs and such. When Material is updated, only these testing helpers need to be updated.

Today I would recommend to follow the Testing Library principles first and test components on this abstraction level. I think this already solves some problems that test harnesses try to address.

I'm sure there are interactions that should better be performed using the provided test harnesses. The Material guide mentions the asynchronity of certain actions – for example, the user enters text, the widget loads data and re-renders asynchronously. This is indeed cumbersome to solve with Angular's async testing tools like fakeAsync/tick, whenStable etc.

Of course, there are more tools in the tool belt. There is still mocking with ng-mocks. And there are also component tests in browser-based testing frameworks like Cypress and Playwright where asynchronity is solved by auto-waiting.

In summary, there are different testing philosophies. They are all valid and viable with pros and cons. Harnesses imply testing on a level of component instances. Today I'd default to test against the HTML DOM and the accessibility tree, as the (Angular) Testing Library promotes it.

[bjmc]

Thank you for such a thoughtful and detailed response. I'm going to have to chew on this a bit as we define our testing practices for this project.