|
| 1 | +# Cross-Package E2E Tests |
| 2 | + |
| 3 | +> **See also:** [Root TESTING.md](../TESTING.md) for an overview of testing across the entire monorepo. |
| 4 | +
|
| 5 | +## Overview |
| 6 | + |
| 7 | +This directory contains end-to-end tests that span multiple packages, specifically testing the complete login flow: |
| 8 | + |
| 9 | +``` |
| 10 | +CLI (Terminal) → Web Browser → GitHub OAuth → Callback → CLI (Authenticated) |
| 11 | +``` |
| 12 | + |
| 13 | +These are the most comprehensive tests in the monorepo, verifying the entire authentication journey a real user would experience. |
| 14 | + |
| 15 | +## Prerequisites |
| 16 | + |
| 17 | +1. **Docker** must be running (for test database) |
| 18 | +2. **SDK** must be built: |
| 19 | + ```bash |
| 20 | + cd sdk && bun run build |
| 21 | + ``` |
| 22 | +3. **Playwright browsers** must be installed: |
| 23 | + ```bash |
| 24 | + cd e2e && bun run install:browsers |
| 25 | + ``` |
| 26 | +4. **GitHub test account credentials** must be configured (see below) |
| 27 | + |
| 28 | +## GitHub Test Account Setup |
| 29 | + |
| 30 | +These tests require a real GitHub account for OAuth testing. We recommend creating a dedicated test account: |
| 31 | + |
| 32 | +1. Create a new GitHub account for testing (e.g., `codebuff-e2e-test@example.com`) |
| 33 | +2. If 2FA is enabled (recommended for security), get the TOTP secret: |
| 34 | + - Go to GitHub Settings → Password and authentication → Two-factor authentication |
| 35 | + - When setting up, click "Can't scan? Enter setup key" instead of scanning QR code |
| 36 | + - Copy the base32 secret key (e.g., `JBSWY3DPEHPK3PXP`) |
| 37 | +3. Set the following environment variables: |
| 38 | + |
| 39 | +```bash |
| 40 | +export GH_TEST_EMAIL="your-test-account@example.com" |
| 41 | +export GH_TEST_PASSWORD="your-test-password" |
| 42 | +export GH_TEST_TOTP_SECRET="your-base32-totp-secret" # Only if 2FA is enabled |
| 43 | +``` |
| 44 | + |
| 45 | +## Architecture |
| 46 | + |
| 47 | +### File-based IPC for Login URL |
| 48 | + |
| 49 | +The tests use file-based IPC to reliably capture the login URL from the CLI: |
| 50 | + |
| 51 | +1. Test creates a unique coordination file path and passes it to CLI via `CODEBUFF_E2E_URL_FILE` |
| 52 | +2. When CLI generates a login URL, it writes `{status: 'ready', loginUrl: '...'}` to the file |
| 53 | +3. Test polls the file instead of parsing TUI output (which is unreliable) |
| 54 | +4. On error, CLI writes `{status: 'error', error: '...'}` for clear test failures |
| 55 | + |
| 56 | +This approach is more robust than text pattern matching because: |
| 57 | +- It's unaffected by TUI rendering, ANSI codes, or terminal buffer management |
| 58 | +- Errors are explicit and debuggable |
| 59 | +- The file can be inspected after test failures |
| 60 | + |
| 61 | +## Running Tests |
| 62 | + |
| 63 | +```bash |
| 64 | +cd e2e |
| 65 | + |
| 66 | +# Run all tests |
| 67 | +bun run test |
| 68 | + |
| 69 | +# Run with UI mode (interactive debugging) |
| 70 | +bun run test:ui |
| 71 | + |
| 72 | +# Run in headed mode (see the browser) |
| 73 | +bun run test:headed |
| 74 | + |
| 75 | +# Debug mode (step through) |
| 76 | +bun run test:debug |
| 77 | +``` |
| 78 | + |
| 79 | +## Test Structure |
| 80 | + |
| 81 | +``` |
| 82 | +e2e/ |
| 83 | +├── fixtures/ |
| 84 | +│ ├── cli-session.ts # CLI terminal emulation with tuistory |
| 85 | +│ ├── infra.ts # Docker database + web server setup |
| 86 | +│ ├── oauth-helpers.ts # GitHub OAuth automation |
| 87 | +│ └── test-context.ts # Combined test fixtures |
| 88 | +├── flows/ |
| 89 | +│ └── login-flow.spec.ts # Main login flow tests |
| 90 | +├── utils/ |
| 91 | +│ ├── env.ts # Environment variable management |
| 92 | +│ └── totp.ts # TOTP code generation for 2FA |
| 93 | +├── package.json |
| 94 | +├── playwright.config.ts |
| 95 | +├── tsconfig.json |
| 96 | +└── README.md |
| 97 | +``` |
| 98 | + |
| 99 | +## How It Works |
| 100 | + |
| 101 | +### Infrastructure |
| 102 | + |
| 103 | +- Each test suite spins up an isolated Docker container with PostgreSQL |
| 104 | +- A Next.js web server is started pointing to the test database |
| 105 | +- Dynamic ports are used to avoid conflicts (DB: 5433+, Web: 3100+) |
| 106 | + |
| 107 | +### CLI Session |
| 108 | + |
| 109 | +- CLI is launched via `tuistory` (terminal emulator) |
| 110 | +- `CODEBUFF_E2E_NO_BROWSER=true` makes CLI print login URLs instead of opening browser |
| 111 | +- Test captures the URL and uses Playwright to complete OAuth |
| 112 | + |
| 113 | +### OAuth Flow |
| 114 | + |
| 115 | +1. CLI requests login code from `/api/auth/cli/code` |
| 116 | +2. CLI prints login URL with `[E2E_LOGIN_URL]` prefix |
| 117 | +3. Playwright navigates to the URL |
| 118 | +4. Playwright fills GitHub credentials and handles 2FA |
| 119 | +5. After OAuth callback, CLI detects the session via polling |
| 120 | + |
| 121 | +## CI/CD |
| 122 | + |
| 123 | +These tests run: |
| 124 | +- **Nightly** via scheduled workflow (to avoid OAuth rate limits) |
| 125 | +- **On-demand** via `workflow_dispatch` |
| 126 | + |
| 127 | +### Required Secrets |
| 128 | +- `GH_TEST_EMAIL` - Email for GitHub test account |
| 129 | +- `GH_TEST_PASSWORD` - Password for GitHub test account |
| 130 | + |
| 131 | +### System Dependencies (installed automatically in CI) |
| 132 | +- `postgresql-client` - For database seeding (`psql`) |
| 133 | +- `lsof` - For port availability checking |
| 134 | +- Playwright browser dependencies (installed via `--with-deps` flag) |
| 135 | + |
| 136 | +## Troubleshooting |
| 137 | + |
| 138 | +### Tests timeout waiting for login URL |
| 139 | + |
| 140 | +- Check that `CODEBUFF_E2E_NO_BROWSER` is being respected by CLI |
| 141 | +- Verify the CLI is reaching the login prompt |
| 142 | + |
| 143 | +### OAuth fails with "rate limited" |
| 144 | + |
| 145 | +- GitHub rate limits OAuth attempts |
| 146 | +- Wait 15-30 minutes and try again |
| 147 | +- Consider using a different test account |
| 148 | + |
| 149 | +### 2FA code is rejected |
| 150 | + |
| 151 | +- Ensure system clock is accurate (TOTP is time-sensitive) |
| 152 | +- Verify the TOTP secret is correct (base32 encoded) |
| 153 | + |
| 154 | +### Orphaned containers |
| 155 | + |
| 156 | +If tests fail and leave Docker containers running: |
| 157 | + |
| 158 | +```bash |
| 159 | +docker ps -aq --filter 'name=manicode-e2e' | xargs -r docker rm -f |
| 160 | +``` |
| 161 | + |
| 162 | +## Adding New Tests |
| 163 | + |
| 164 | +```typescript |
| 165 | +import { test, expect } from '../fixtures/test-context' |
| 166 | + |
| 167 | +test.describe('E2E: My New Flow', () => { |
| 168 | + test('my test', async ({ page, e2eContext }) => { |
| 169 | + const { createCLISession, completeOAuth } = e2eContext |
| 170 | + |
| 171 | + // Launch CLI |
| 172 | + const cli = await createCLISession() |
| 173 | + |
| 174 | + // Complete login if needed |
| 175 | + await cli.waitForText(/login/i, { timeout: 30000 }) |
| 176 | + await cli.press('enter') |
| 177 | + const loginUrl = await cli.waitForLoginUrl() |
| 178 | + await completeOAuth(page, loginUrl) |
| 179 | + |
| 180 | + // Test your flow |
| 181 | + await cli.type('/your-command') |
| 182 | + await cli.waitForText(/expected output/i) |
| 183 | + |
| 184 | + expect(await cli.text()).toContain('expected') |
| 185 | + }) |
| 186 | +}) |
| 187 | +``` |
0 commit comments