Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 22 additions & 3 deletions getting-started/ci-setup/nx.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,10 +97,21 @@ This is showcased in the `e2e-03` nx project and the `project.json` file is slig
"projectType": "application",
"sourceRoot": "apps/e2e-03/src",
"targets": {
"or8n-discover": {
"executor": "nx:run-commands",
"options": {
"cwd": "apps/e2e-03",
"commands": [
{
"command": "npx pwc-p discover --pwc-discovery-file tests.txt"
}
]
}
},
"or8n": {
"executor": "nx:run-commands",
"options": {
"config": "apps/e2e-03/playwright.config.ts",
"cwd": "apps/e2e-03",
"commands": [
{
"command": "npx pwc-p run"
Expand All @@ -115,8 +126,9 @@ This is showcased in the `e2e-03` nx project and the `project.json` file is slig
Key differences with other nx projects in the example repository:

* `executor` property has the value `nx:run-commands`.
* A `command` property is added within `options`. This executes `npx pwc-p run`. See [playwright-orchestration.md](../../guides/ci-optimization/playwright-orchestration.md "mention").
* A `command` property is added within `options`. This executes `npx pwc-p run` for orchestration. See [playwright-orchestration.md](../../guides/ci-optimization/playwright-orchestration.md "mention").
* Reporter config in `playwright.config` file is not needed
* Orchestration runs the full suite by default — use `or8n-discover` first to filter tests

To locally execute orchestration nx project:

Expand All @@ -127,7 +139,14 @@ CURRENTS_CI_BUILD_ID=unique-id \
nx run-many -t or8n
```

This Github Action yaml file executes the Playwright tests distributed in three shards:
Or with discovery (for filtered runs):

```bash
CURRENTS_RECORD_KEY=recordkey \
CURRENTS_PROJECT_ID=projectid \
CURRENTS_CI_BUILD_ID=unique-id \
nx run-many -t or8n-discover -- --grep @smoke && nx run-many -t or8n -- --pwc-discovery-file tests.txt
```

```yaml
name: Run or8n Tests
Expand Down
50 changes: 28 additions & 22 deletions guides/ci-optimization/playwright-orchestration.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,17 +14,15 @@ Orchestration helps decrease the duration of Playwright tests in CI pipelines. R

At a high level:

* create a list of tests for orchestration using 
* establishes an orchestration session with Currents servers
* assigns work to available CI machines using historical execution and timing data
* records results to Currents for troubleshooting and analysis
* create a list of tests for orchestration using `pwc-p discover`
* establish an orchestration session with Currents servers
* assign work to available CI machines using historical execution and timing data
* record results to Currents for troubleshooting and analysis

`@currents/playwright` includes a CLI command `pwc-p` that implements Playwright Orchestration.


`@currents/playwright` includes a CLI command `pwc-p` that implements Playwright Orchestration.

1. `pwc-p discover` — runs Playwright in discovery mode and writes the list of tests to be orchestrated to a file.
2. `pwc-p run` — initiates the orchestrated execution based on the previous step.
1. [`pwc-p discover`](../../resources/reporters/currents-playwright/pwc-p-discover.md) — runs Playwright in discovery mode and writes the list of tests to be orchestrated to a file.
2. [`pwc-p run`](../../resources/reporters/currents-playwright/pwc-p-run.md) — initiates the orchestrated execution based on the previous step.

## Setup

Expand All @@ -34,19 +32,22 @@ Install `@currents/playwright`
npm i @currents/playwright@latest
```

* Run `pwc-p discover` to create a discovery file with tests selected for orchestration, you can apply the same filters and CLI arguments as for `playwright` command.
* Run `pwc-p run` with `--pwc-discovery-file` to execute the orchestration. Run `pwc-p run --help` to see the available flags or refer to [currents-playwright](../../resources/reporters/currents-playwright/ "mention").
* Run `pwc-p discover` to create a discovery file with tests selected for orchestration, you can apply the same filters and CLI arguments as for `playwright` command.
* Run `pwc-p run` with `--pwc-discovery-file` to execute the orchestration. Run `pwc-p run --help` to see the available flags or refer to [currents-playwright](../../resources/reporters/currents-playwright/ "mention").

{% code overflow="wrap" lineNumbers="true" %}

```bash
npx pwc-p discover --pwc-discovery-file <discovery-path> [...filters]
npx pwc-p run --pwc-discovery-file <discovery-path> --key <currents-record-key> --project-id <currents-project-id> --ci-build-id <ci-build-id>
```

{% endcode %}

A successfully created orchestration prints an output similar to this.

{% code overflow="wrap" %}

```bash
$ npx pwc-p run --key *secret* --project-id WeZwSj --ci-build-id example-001
🚀 Starting orchestration session...
Expand All @@ -62,10 +63,11 @@ $ npx pwc-p run --key *secret* --project-id WeZwSj --ci-build-id example-001
🌐 Run URL: https://app.currents.dev/run/4aefcb5cd3bb5c89
#...
```

{% endcode %}

{% hint style="info" %}
Currents automatically balances tests between all available CI machines, which makes `--shard` redundant — it should be removed.
Currents automatically balances tests between all available CI machines, which makes `--shard` redundant — it should be removed.

Read more about [ci-build-id.md](../parallelization-guide/ci-build-id.md "mention") and [reporting-strategy.md](../parallelization-guide/reporting-strategy.md "mention").
{% endhint %}
Expand All @@ -74,21 +76,23 @@ Read more about [ci-build-id.md](../parallelization-guide/ci-build-id.md "mentio

`pwc-p discover` is required when you want to explicitly select tests for orchestration — for example:

* Filtering tests by tag: `--grep / -g @smoke`
* Filtering tests by last run outcome: `--last-failed` (see [#re-running-only-failed-tests](playwright-orchestration.md#re-running-only-failed-tests "mention"))
* Filtering tests by tag: `--grep / -g @smoke`
* Filtering tests by last run outcome: `--last-failed` (see [#re-running-only-failed-tests](playwright-orchestration.md#re-running-only-failed-tests "mention"))
* Filtering tests by Playwright project: `--project chromium`
* Explicit spec file location: `playwright test <spec-file-path>`

Apply the desired arguments and parameters as if you are running `playwright` command, for example:

{% code overflow="wrap" lineNumbers="true" %}

```sh
# Create discovery file with filters applied
npx pwc-p discover --pwc-discovery-file ./test-list --grep @smoke --project chromium

# Use the discovery file as an input for orchestration
npx pwc-p run --pwc-discovery-file ./test-list --key currents-record-key --project-id currents-project-id --ci-build-id ci-build-id
```

{% endcode %}

Omitting discovery stage selects **all** tests for orchestration.
Expand All @@ -100,7 +104,7 @@ Omitting discovery stage selects **all** tests for orchestration.

## Discovery output

Creates a discover file at a destination, the use the file as an input for `pwc-p run` command
Creates a discovery file at a destination, then use the file as an input for `pwc-p run` command

```bash
npx pwc-p discover --pwc-discovery-file tests.txt --grep @smoke --project frontend
Expand Down Expand Up @@ -138,7 +142,7 @@ Add tags on the recorded run (no discovery step):
npx pwc-p run --key <record-key> --project-id <project-id> --ci-build-id <ci-build-id> --tag tagA --tag tagB
```

Run `pwc-p discover --help` or `pwc-p run --help` to see the list of supported flags.
Run `pwc-p discover --help` or `pwc-p run --help` to see the list of supported flags.

## Orchestration in CI

Expand All @@ -161,12 +165,14 @@ Provider-specific orchestration examples are being updated for `discover` and `r

`pwc-p run` automatically injects Currents reporter [currents-playwright](../../resources/reporters/currents-playwright/ "mention") into Playwright, replacing all other reporters configured in `playwright.config.ts`. To add additional reporters use one of the two options.

#### **Add reporters as a CLI parameter.**
#### **Add reporters as a CLI parameter.**

{% code overflow="wrap" %}

```bash
pwc-p run --key <record-key> --project-id <id> --ci-build-id <build-id> --reporter="./myreporter/my-awesome-reporter.ts"
```

{% endcode %}

#### **Manual Configuration**
Expand Down Expand Up @@ -195,22 +201,22 @@ import { currentsReporter } from "@currents/playwright";
import { PlaywrightTestConfig } from "@playwright/test";

const config: PlaywrightTestConfig = {
reporter: [
currentsReporter(),
],
reporter: [currentsReporter()],

// ... rest of playwright configuration
}
};
```

* **Optional:** Update the `pwc-p run` CLI command

`pwc-p run` reads all the configuration from `currents.config.ts` — no need to use CLI params.

{% code overflow="wrap" %}
```

```bash
pwc-p run -- [...playwright-cli-params]
```

{% endcode %}

### Merging Fragmented Reports
Expand Down
25 changes: 12 additions & 13 deletions guides/troubleshooting-playwright.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,13 @@ icon: square-question

# Troubleshooting Playwright

If you are experiencing issues with using `@currents/playwright`, enable debug mode to collect more information about the failure and submit a support request via our support channels.&#x20;
If you are experiencing issues with using `@currents/playwright`, enable debug mode to collect more information about the failure and submit a support request via our support channels.&#x20;

{% hint style="info" %}
<mark style="color:yellow;">**TL;DR share the following information**</mark>

Collect environment information



* Package Versions
* Currents Run ID or Dashboard URL associated with the issue
* Screenshots or video recordings if applicable
Expand All @@ -29,7 +27,7 @@ Collect and share the debug logs
* `npx pwc --pwc-debug=full ...` OR
* `npx pwc-p --pwc-debug=remote ...` OR
* `DEBUG=currents* playwright test ...`
{% endhint %}
{% endhint %}

### 1. Collecting Environment Information

Expand Down Expand Up @@ -61,31 +59,32 @@ To enable uploading the debug logs:
* For `pwc-p` run `pwc-p --pwc-debug=full` OR
* Set environment variable `DEBUG=currents* playwright test...`



`--pwc-debug` enables uploading the debug logs to our servers (see [#pwc-debug-boolean-or-remote-or-full](../../resources/reporters/currents-playwright/pwc-p-orchestration.md#pwc-debug-boolean-or-remote-or-full "mention"):
`--pwc-debug` enables uploading the debug logs to our servers (see [pwc-p run](../../resources/reporters/currents-playwright/pwc-p-run.md#debugging--output) documentation):

* `remote` uploads the debug logs to Currents servers;
* `full` prints the logs to stdout and also upload them to Currents.



For example:

{% tabs %}
{% tab title="pwc" %}

```bash
npx pwc --pwc-debug=full ...
```
npx pwc --pwc-debug=full ...
```

{% endtab %}

{% tab title="pwc-p" %}

```
npx pwc-p --pwc-debug=full ...
npx pwc-p --pwc-debug=full ...
```

{% endtab %}

{% tab title="Environment variable" %}

```
# on Linux
CURRENTS_PROJECT_ID=PROJECT_ID \ // the projectId from https://app.currents.dev
Expand All @@ -105,10 +104,10 @@ set CURRENTS_CI_BUILD_ID=unique_build_id
## - the run the command
npx playwright test ...
```

{% endtab %}
{% endtabs %}

When enabled, the debug logs will be uploaded to Currents servers and a confirmation message will be shown after the run's completion, for example:

<figure><img src="../../.gitbook/assets/currents-2023-12-11-15.56.04@2x.png" alt=""><figcaption><p>Remote debug logs notification example</p></figcaption></figure>

8 changes: 5 additions & 3 deletions resources/reporters/currents-playwright/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,19 +97,21 @@ Using `pwc` command overrides the reporters configured in `playwright.config.ts`

Use `pwc-p run` to orchestrate the full suite. When applying Playwright filters, run `pwc-p discover` first and pass the same discovery file to `pwc-p run`.

**Full suite (discovery optional):**

```bash
npx pwc-p run --key <record-key> --project-id <project-id> --ci-build-id <ci-build-id>
```

Filter tests, then orchestrate the discovered list:
**Filtered tests (discovery required):**

```bash
npx pwc-p discover --pwc-discovery-file tests.txt --grep @smoke
npx pwc-p run --key <record-key> --project-id <project-id> --ci-build-id <ci-build-id> --pwc-discovery-file tests.txt
```

- `pwc-p run` reads configuration from `currents.config.ts`
- Playwright filter flags belong on `discover`; runtime flags (`-j`, `--timeout`) belong on `run`
* `pwc-p run` reads configuration from `currents.config.ts`
* Playwright filter flags belong on `discover`; runtime flags (`-j`, `--timeout`) belong on `run`

### Manually Add Currents Reporter

Expand Down
Loading