Skip to content

Commit b8f36a3

Browse files
author
anna-singleton-resolver
committed
doc: update docs to include information about e2e testing
1 parent f5c7ebc commit b8f36a3

1 file changed

Lines changed: 14 additions & 2 deletions

File tree

README.md

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,8 @@ ATHENA_NON_EXISTENT_AFFILIATE=non-existent-affiliate-id (default:
162162
thisaffiliatedoesnotexist123) - this is used to test error handling.
163163
ATHENA_NON_PERMITTED_AFFILIATE=non-permitted-affiliate-id (default:
164164
thisaffiliatedoesnothaveathenaenabled) - this is used to test error handling.
165+
ATHENA_E2E_TESTCASE_DIR=test-case-directory (default: integrator_sample) - this is the test case directory to use for the e2e tests.
166+
See E2E Tests section below for more details.
165167
```
166168

167169
Then run the functional tests with:
@@ -170,8 +172,18 @@ Then run the functional tests with:
170172
pytest -m functional
171173
```
172174

173-
To exclude the e2e tests, which require usage of the live classifier and
174-
therefore are unsuitable for regular development runs, use:
175+
#### E2E Tests
176+
177+
The e2e tests assert that the API returns some expected _scores_ rather than
178+
exercising different API paths. As such, they are dependent on the classifier
179+
that you are calling through the API. Right now, there are 2 types of
180+
classifier, benign and live. By default, the tests will run the
181+
`integrator_sample` test set, which uses the live classifier. If you wish to
182+
use the benign classifier instead, you may set the `ATHENA_E2E_TESTCASE_DIR`
183+
environment variable to `benign_model`.
184+
185+
Alternatively, you may disable these tests altogether, by excluding tests that
186+
have the `e2e` marker, something like this:
175187

176188
```bash
177189
pytest -m 'functional and not e2e'

0 commit comments

Comments
 (0)