Add Playwright tests

[mraible]

Add Comprehensive Playwright E2E Tests for Foundry Apps

This PR introduces a comprehensive end-to-end testing framework for Foundry applications using Playwright, with a focus on maintainability, reliability, and developer experience.

🎯 What This Adds

Core Test Coverage

• App Lifecycle Testing: Install, verify, and uninstall Foundry apps
• Navigation Testing: Complex multi-step navigation through Falcon UI
• UI Extension Verification: Validate custom UI extensions are working
• Environment Awareness: Works in both CI (pre-deployed apps) and local (manual deployment) environments

Robust Architecture

• Centralized Configuration: Single source of truth for all test settings
• Structured Logging: Step-by-step execution tracking with performance metrics
• Smart Waiting: Intelligent waits replace hard-coded timeouts
• Retry Logic: Exponential backoff for flaky operations
• Base Page Pattern: Eliminates code duplication across page objects

🏗️ Architecture Overview

e2e/
├── src/
│   ├── config/
│   │   └── TestConfig.ts          # Centralized configuration management
│   ├── utils/
│   │   ├── Logger.ts              # Structured logging service
│   │   └── SmartWaiter.ts         # Smart waiting & retry utilities
│   ├── pages/
│   │   ├── BasePage.ts            # Base class for all page objects
│   │   ├── FoundryHomePage.ts     # Foundry home page interactions
│   │   ├── AppCatalogPage.ts      # App installation/management  
│   │   └── EndpointDetectionsPage.ts # Navigation & UI extension testing
│   └── fixtures.ts                # Playwright fixtures with dependency injection
└── tests/
    └── foundry.spec.ts            # Main test suite

🚀 CI/CD Integration

The tests are fully integrated with GitHub Actions:

• Automatic App Deployment: Uses Foundry CLI to deploy test apps
• Environment Setup: Configures authentication and dependencies
• Test Execution: Runs all tests with proper reporting
• Cleanup: Automatically removes test apps after completion
• Artifact Collection: Saves test reports and screenshots

📊 Test Results

Local Execution

🔧 Test Configuration:
  Environment: Local
  Base URL: https://falcon.us-2.crowdstrike.com
  App Name: foundry-quickstart
  Default Timeout: 30000ms
  Retry Attempts: 3

🧭 [1] FoundryHomePage: Navigate to /foundry/home
⚡ Navigate to FoundryHomePage completed in 5.48s
✅ Foundry home page loaded successfully

[... 22 total logged steps ...]

✅ 7 passed (1.2m)

CI Execution

• ✅ All tests pass consistently
• ⚡ ~2m43s total execution time
• 📊 Structured JSON logging for analysis
• 🔍 Comprehensive error reporting

🔧 How to Use This in Other Foundry Apps

1. Copy the Test Framework

Copy the entire e2e/ directory structure to your Foundry app:

# Copy the framework
cp -r e2e/ /path/to/your-foundry-app/

# Install dependencies  
cd /path/to/your-foundry-app/e2e
npm install

2. Configure for Your App

Update the configuration in .env:

# Required: Your app details
APP_NAME=your-app-name

# Required: Falcon credentials
# NOTE: For CrowdStrike internal engineers - these are NOT your Okta credentials
# Use the shared test credentials provided by the Foundry team
# For external customers and partners - use your standard Falcon console credentials
# If using SSO, you may need to create a user that supports password + MFA authentication
# Contact CrowdStrike Support if you need assistance setting this up
FALCON_USERNAME=your-falcon-username
FALCON_PASSWORD=your-falcon-password  
FALCON_AUTH_SECRET=your-mfa-secret

# Optional: Customize endpoints
FALCON_BASE_URL=https://falcon.us-2.crowdstrike.com

3. Adapt Page Objects

Extend or modify page objects for your specific UI:

// Create custom page objects
export class YourCustomPage extends BasePage {
  constructor(page: Page) {
    super(page, 'YourCustomPage');
  }

  protected getPagePath(): string {
    return '/your-custom-path';
  }

  protected async verifyPageLoaded(): Promise<void> {
    await this.waiter.waitForVisible(
      this.page.getByText('Your Page Identifier'),
      { description: 'Your page loaded' }
    );
  }

  async performCustomAction(): Promise<void> {
    this.logger.step('Perform custom action');
    
    await this.smartClick(
      this.page.getByRole('button', { name: 'Custom Button' }),
      'Custom action button'
    );
    
    this.logger.success('Custom action completed');
  }
}

4. Write Your Tests

Create tests using the established patterns:

import { test, expect } from '../src/fixtures';

test.describe('Your App Tests', () => {
  test('should perform custom workflow', async ({ yourCustomPage, appName }) => {
    await yourCustomPage.goto();
    await yourCustomPage.performCustomAction();
    // Add your specific assertions
  });
});

5. Update CI Configuration

Adapt the GitHub Actions workflow (.github/workflows/main.yml):

- name: Prepare app manifest
  run: |
    # Update for your app structure
    yq -i '.name = .name + "-${{ github.actor }}-" + "'$(date +"%Y%m%d%H%M")'"' manifest.yml
    APP_NAME=$(yq '.name' manifest.yml)
    echo "APP_NAME=$APP_NAME" >> $GITHUB_ENV

6. Customize for Your UI Extensions

If your app has UI extensions, update the verification logic:

async verifyYourUIExtension(): Promise<boolean> {
  this.logger.step('Verify your custom UI extension');
  
  return RetryHandler.withPlaywrightRetry(
    async () => {
      // Look for your specific UI extension elements
      const yourElement = this.page.locator('[data-your-extension]');
      
      if (await this.elementExists(yourElement, 8000)) {
        this.logger.success('Your UI extension is working!');
        return true;
      }
      
      // Add fallback strategies specific to your extension
      return false;
    },
    'Verify your UI extension'
  );
}

7. Configure Navigation Paths

Update navigation if your app integrates with different Falcon sections:

async navigateToYourSection(): Promise<void> {
  return this.withTiming(
    async () => {
      // Navigate to your app's section in Falcon
      await this.navigateToPath('/foundry/home');
      
      await this.smartClick(
        this.page.getByRole('button', { name: 'Menu', exact: true }),
        'Menu button'
      );
      
      // Navigate to your specific section
      await this.smartClick(
        this.page.getByRole('link', { name: 'Your Section' }),
        'Your section link'
      );
      
      await this.verifyPageLoaded();
    },
    'Navigate to your section'
  );
}

🛠️ Framework Benefits

For Development Teams

• Consistent Patterns: Standardized approach across all Foundry apps
• Rich Debugging: Step-by-step logging with performance metrics
• Reliable Tests: Smart waits and retry logic reduce flakiness
• Easy Maintenance: DRY principles with reusable components

For CI/CD

• Environment Awareness: Automatically adapts to CI vs local execution
• Comprehensive Reporting: Detailed logs and artifact collection
• Fast Feedback: Efficient test execution with parallel capabilities
• Zero Setup: Ready-to-use GitHub Actions integration

For Quality Assurance

• Full Coverage: End-to-end user journey validation
• Real Browser Testing: Actual user interaction simulation
• Visual Regression: Screenshot capture for debugging
• Cross-Environment: Consistent behavior across environments

📋 Requirements

• Node.js: LTS version 22+ (currently 22.18.0)
• Foundry CLI: For app deployment in CI
• Falcon Access: Valid credentials with appropriate permissions
• GitHub Secrets: Configure required environment variables

🤝 Contributing

When extending this framework:

1. Follow Patterns: Use existing page object and logging patterns
2. Add Tests: Ensure new functionality is tested
3. Update Docs: Document any new configuration options
4. Test Locally: Verify changes work in both local and CI environments

---

This testing framework provides a solid foundation for reliable E2E testing of any Foundry application. The modular architecture ensures maintainability and scalability as your app grows.

[dbutterfield77]

If the test fails or the pipeline breaks for reasons outside this runners control how can we ensure these apps are uninstalled? One approach would be to create a cronjob (outside of Playwright) to query for apps and do these kind of actions as a backup. You could look at https://playwright.dev/docs/test-global-setup-teardown and add this project as a dependency.

[mraible]

This app will have a unique name (with the GitHub username and a timestamp) each time it's deployed, so this shouldn't be an issue. However, I realize there could be problems when using the same name for workflows and functions in multiple apps. I can address that in other apps as I add e2e tests to them.

For now, these are all being deployed/installed in my CID on US-2, so the burden will be on me to clean them up if things fail. I think it's easy to add this later if necessary.