@@ -162,6 +162,8 @@ ATHENA_NON_EXISTENT_AFFILIATE=non-existent-affiliate-id (default:
162162thisaffiliatedoesnotexist123) - this is used to test error handling.
163163ATHENA_NON_PERMITTED_AFFILIATE=non-permitted-affiliate-id (default:
164164thisaffiliatedoesnothaveathenaenabled) - this is used to test error handling.
165+ ATHENA_E2E_TESTCASE_DIR=test-case-directory (default: integrator_sample) - this is the test case directory to use for the e2e tests.
166+ See E2E Tests section below for more details.
165167```
166168
167169Then run the functional tests with:
@@ -170,8 +172,18 @@ Then run the functional tests with:
170172pytest -m functional
171173```
172174
173- To exclude the e2e tests, which require usage of the live classifier and
174- therefore are unsuitable for regular development runs, use:
175+ #### E2E Tests
176+
177+ The e2e tests assert that the API returns some expected _ scores_ rather than
178+ exercising different API paths. As such, they are dependent on the classifier
179+ that you are calling through the API. Right now, there are 2 types of
180+ classifier, benign and live. By default, the tests will run the
181+ ` integrator_sample ` test set, which uses the live classifier. If you wish to
182+ use the benign classifier instead, you may set the ` ATHENA_E2E_TESTCASE_DIR `
183+ environment variable to ` benign_model ` .
184+
185+ Alternatively, you may disable these tests altogether, by excluding tests that
186+ have the ` e2e ` marker, something like this:
175187
176188``` bash
177189pytest -m ' functional and not e2e'
0 commit comments