Add Support for Firebolt Core

.cursor/rules/architecture/RULE.md

@@ -0,0 +1,77 @@
+---
+alwaysApply: false
+---
+# Firebolt Node.js SDK Architecture Rules
+
+## Version Architecture (Critical)
+
+**V1 (Legacy)**: Username/password auth → `ConnectionV1`, `QueryFormatterV1`, `DatabaseServiceV1`, `EngineServiceV1`
+**V2 (Current)**: Service account auth (client_id/secret) → `ConnectionV2`, `QueryFormatterV2`, `DatabaseServiceV2`, `EngineServiceV2`
+**Core (Self-Hosted)**: `FireboltCore()` auth → `ConnectionCore`, `QueryFormatterV2`, no ResourceManager, no async queries, no transactions
+
+Version selected in `makeConnection()` based on auth type. **Always support all versions** unless explicitly version-specific feature.
+
+## Core Patterns
+
+**Dependency Injection**: Factory functions (`FireboltClient()`, `ResourceClient()`) accept `logger` and `httpClient`, return configured instances. Context object passes dependencies.
+
+**Abstract Base Classes**: `Connection` (base) → `ConnectionV1`/`ConnectionV2`. `QueryFormatter` (base) → `QueryFormatterV1`/`QueryFormatterV2`. Base classes contain shared logic.
+
+**Statement Types**:
+- `Statement`: `execute()` → `fetchResult()` (in-memory) or `streamResult()` (in-memory stream, not true streaming)
+- `StreamStatement`: `executeStream()` → true server-side streaming
+- `AsyncStatement`: `executeAsync()` → returns token, no immediate data
+
+## Query Execution Flow
+
+1. `prepareQuery()`: Format with `QueryFormatter` (handles `?`, `:name`, or `$1`/$2` for server-side)
+2. `getRequestUrl()`: Add query params and settings
+3. `executeQuery()`: POST to engine endpoint
+4. `processHeaders()`: Update session parameters from response headers
+5. Parse JSON, handle errors via `throwErrorIfErrorBody()`
+6. Return appropriate Statement type
+
+## Parameter Management
+
+Connection maintains session parameters:
+- **Immutable**: `database`, `account_id`, `output_format` (cannot be removed)
+- **Mutable**: Updated via `SET` statements or response headers
+- **Server headers**: `Firebolt-Update-Parameters`, `Firebolt-Update-Endpoint`, `Firebolt-Reset-Session`, `Firebolt-Remove-Parameters`
+
+## Authentication & Caching
+
+**Managed Firebolt**: `Authenticator` handles OAuth tokens with thread-safe caching (read/write locks via `rwlock`). Cache key: `{clientId, secret, apiEndpoint}`. Tokens expire at 50% of actual expiry for safety. Disable with `useCache: false`.
+
+**Firebolt Core**: `CoreAuthenticator` provides no-op authentication (no tokens, no caching). Core connections don't require authentication.
+
+## Engine Endpoint Resolution
+
+**V2**: Get system engine URL → connect → `USE DATABASE` → `USE ENGINE` (if specified)
+**V1**: Resolve account ID → get engine URL by database/engine → direct connection
+**Core**: `engineEndpoint` must be provided explicitly in connection options (no resolution needed)
+
+## Error Handling
+
+Use `CompositeError` for multiple errors. Custom errors: `AccountNotFoundError`, `AuthenticationError`, `ApiError` (from `src/common/errors.ts`).
+
+## Important Details
+
+- **Prepared statements**: `native` (default, client-side `?`/`:name`) vs `fb_numeric` (server-side `$1`/`$2`)
+- **Transactions**: `begin()`, `commit()`, `rollback()` on Connection (state per connection).
+- **Async queries**: `async: true` setting → token for `isAsyncQueryRunning()`, `isAsyncQuerySuccessful()`, `cancelAsyncQuery()`. **Not supported in Core** - methods throw errors.
+- **ResourceManager**: Available for V1/V2 managed connections. **Not available in Core** - accessing `resourceManager` throws error.
+- **Result hydration**: SQL types → JS types (dates, BigNumber for large ints, normalization)
+- **Connection cleanup**: `destroy()` aborts all active requests
+- **Response formats**: `JSON_COMPACT` (default), `JSON`, `JSON_LINES`
+
+## Development Rules
+
+1. Support V1, V2, and Core unless version-specific feature
+2. Use abstract base classes for shared functionality (`Connection`, `QueryFormatter`, `Authenticator`)
+3. Keep DI pattern for testability
+4. Use custom error types from `src/common/errors.ts`
+5. Maintain separate V1/V2/Core test suites
+6. V1 is legacy; prioritize V2 for new features
+7. **Core limitations**: No ResourceManager, no async queries, no transactions. Use `CoreAuthenticator` for no-op auth. `engineEndpoint` required.
+8. **Type guards**: Use `"type" in auth && auth.type === "firebolt-core"` to detect Core connections
+

.env.example

@@ -1,6 +1,6 @@
-FIREBOLT_USERNAME=
-FIREBOLT_PASSWORD=
+FIREBOLT_CLIENT_ID=
+FIREBOLT_CLIENT_SECRET=
+FIREBOLT_ACCOUNT=
 FIREBOLT_DATABASE=
 FIREBOLT_ENGINE_NAME=
-FIREBOLT_ENGINE_ENDPOINT=
-FIREBOLT_API_ENDPOINT=
+FIREBOLT_CORE_ENDPOINT="http://localhost:3473"

.github/actions/run-core-integration-tests/action.yml

@@ -0,0 +1,33 @@
+name: 'Run Core Integration Tests'
+description: 'Setup, run, and teardown Firebolt Core integration tests'
+inputs:
+  database:
+    description: 'Database name for Firebolt Core'
+    required: false
+    default: 'firebolt'
+  endpoint:
+    description: 'Firebolt Core endpoint'
+    required: false
+    default: 'http://127.0.0.1:3473'
+runs:
+  using: 'composite'
+  steps:
+    - name: Setup Firebolt Core
+      id: setup-core
+      uses: ./.github/actions/setup-firebolt-core
+      with:
+        database: ${{ inputs.database }}
+        endpoint: ${{ inputs.endpoint }}
+
+    - name: Run Core integration tests
+      shell: bash
+      env:
+        FIREBOLT_DATABASE: ${{ steps.setup-core.outputs.database }}
+        FIREBOLT_CORE_ENDPOINT: ${{ steps.setup-core.outputs.endpoint }}
+      run: |
+        npm run test:ci integration/core
+
+    - name: Teardown Firebolt Core
+      if: always()
+      uses: ./.github/actions/teardown-firebolt-core
+

.github/actions/setup-firebolt-core/action.yml

@@ -0,0 +1,96 @@
+name: 'Setup Firebolt Core'
+description: 'Start Firebolt Core in Docker and wait for it to be ready'
+inputs:
+  database:
+    description: 'Database name for Firebolt Core'
+    required: false
+    default: 'firebolt'
+  endpoint:
+    description: 'Firebolt Core endpoint'
+    required: false
+    default: 'http://127.0.0.1:3473'
+outputs:
+  database:
+    description: 'Database name'
+    value: ${{ inputs.database }}
+  endpoint:
+    description: 'Firebolt Core endpoint'
+    value: ${{ inputs.endpoint }}
+runs:
+  using: 'composite'
+  steps:
+    - name: Create firebolt-core-data directory
+      shell: bash
+      run: |
+        mkdir -p ./firebolt-core-data
+        # Firebolt Core runs as user 1111:1111, so we need to set ownership
+        sudo chown -R 1111:1111 ./firebolt-core-data
+        sudo chmod 755 ./firebolt-core-data
+
+    - name: Start Firebolt Core
+      shell: bash
+      run: |
+        docker run -d --rm --name firebolt-core \
+          --ulimit memlock=8589934592:8589934592 \
+          --security-opt seccomp=unconfined \
+          -p 127.0.0.1:3473:3473 \
+          -v $(pwd)/firebolt-core-data:/firebolt-core/volume \
+          ghcr.io/firebolt-db/firebolt-core:preview-rc
+
+    - name: Wait for Firebolt Core to be ready
+      shell: bash
+      run: |
+        echo "Waiting for Firebolt Core to be ready..."
+        timeout=30
+        elapsed=0
+        while [ $elapsed -lt $timeout ]; do
+          # Try to connect and execute a simple query
+          response=$(curl -s -w "\n%{http_code}" -X POST "${{ inputs.endpoint }}/" \
+            --data-binary "SELECT 1" 2>&1) || response=""
+          http_code=$(echo "$response" | tail -n1)
+
+          if [ "$http_code" = "200" ]; then
+            echo "Firebolt Core is ready!"
+            break
+          fi
+          echo "Waiting... ($elapsed/$timeout seconds) - HTTP $http_code"
+          sleep 3
+          elapsed=$((elapsed + 3))
+        done
+
+        if [ $elapsed -ge $timeout ]; then
+          echo "Firebolt Core failed to start within $timeout seconds"
+          echo "Container logs:"
+          docker logs firebolt-core || true
+          echo "Container status:"
+          docker ps -a | grep firebolt-core || true
+          exit 1
+        fi
+
+        # Give Core a bit more time to fully initialize
+        echo "Core is responding, waiting additional 5 seconds for full initialization..."
+        sleep 5
+
+    - name: Create database if it doesn't exist
+      shell: bash
+      run: |
+        echo "Creating database '${{ inputs.database }}' ..."
+        # Create the database using a SQL command
+        response=$(curl -s -w "\n%{http_code}" -X POST "${{ inputs.endpoint }}/?database=default" \
+          --data-binary "CREATE DATABASE IF NOT EXISTS \"${{ inputs.database }}\"" 2>&1)
+        http_code=$(echo "$response" | tail -n1)
+
+        if [ "$http_code" = "200" ]; then
+          echo "Database '${{ inputs.database }}' created or already exists"
+        else
+          echo "Warning: Database creation returned HTTP $http_code"
+          echo "Response: $response"
+          exit 1
+        fi
+
+    - name: Set outputs
+      shell: bash
+      run: |
+        echo "database=${{ inputs.database }}" >> $GITHUB_OUTPUT
+        echo "endpoint=${{ inputs.endpoint }}" >> $GITHUB_OUTPUT
+

.github/actions/teardown-firebolt-core/action.yml

@@ -0,0 +1,9 @@
+name: 'Teardown Firebolt Core'
+description: 'Stop Firebolt Core Docker container'
+runs:
+  using: 'composite'
+  steps:
+    - name: Stop Firebolt Core
+      shell: bash
+      run: docker stop firebolt-core || true
+

.github/workflows/integration-tests-core.yaml

@@ -0,0 +1,51 @@
+name: Run integration tests for Core
+on:
+  workflow_dispatch:
+  workflow_call:
+    inputs:
+      token:
+        description: 'GitHub token if called from another workflow'
+        required: false
+        type: string
+
+jobs:
+  core-tests:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Check out code
+        uses: actions/checkout@v4
+
+      - name: Set up node.js
+        uses: actions/setup-node@v6
+        with:
+          node-version: '24'
+
+      - name: Install dependencies
+        run: npm install
+
+      - name: Run Core integration tests
+        uses: ./.github/actions/run-core-integration-tests
+
+  allure-report:
+    needs: core-tests
+    runs-on: ubuntu-latest
+    if: always() && github.repository == 'firebolt-db/firebolt-node-sdk'
+    steps:
+      # Need to pull the pages branch in order to fetch the previous runs
+      # Only run Allure reporting on the main repository, not forks
+      - name: Get Allure history
+        uses: actions/checkout@v4
+        continue-on-error: true
+        with:
+            ref: gh-pages
+            path: gh-pages
+
+      - name: Allure Report
+        uses: firebolt-db/action-allure-report@v1
+        with:
+            github-key: ${{ inputs.token || secrets.GITHUB_TOKEN }}
+            test-type: integration-core
+            allure-dir: allure-results
+            pages-branch: gh-pages
+            repository-name: firebolt-node-sdk
+

.github/workflows/integration-tests-v2.yaml

@@ -57,17 +57,18 @@ jobs:
           npm run test:ci integration/v2
 
       # Need to pull the pages branch in order to fetch the previous runs
+      # Only run Allure reporting on the main repository, not forks
       - name: Get Allure history
         uses: actions/checkout@v6
-        if: always()
+        if: always() && github.repository == 'firebolt-db/firebolt-node-sdk'
         continue-on-error: true
         with:
             ref: gh-pages
             path: gh-pages
 
       - name: Allure Report
         uses: firebolt-db/action-allure-report@v1
-        if: always()
+        if: always() && github.repository == 'firebolt-db/firebolt-node-sdk'
         with:
             github-key: ${{ inputs.token || secrets.GITHUB_TOKEN }}
             test-type: integration

.github/workflows/integration-tests.yaml

@@ -29,4 +29,6 @@ jobs:
     secrets:
       FIREBOLT_CLIENT_ID_STG_NEW_IDN: ${{ secrets.FIREBOLT_CLIENT_ID_STG_NEW_IDN }}
       FIREBOLT_CLIENT_SECRET_STG_NEW_IDN: ${{ secrets.FIREBOLT_CLIENT_SECRET_STG_NEW_IDN }}
+  integration-test-core:
+    uses: ./.github/workflows/integration-tests-core.yaml
 

.github/workflows/nightly.yaml

@@ -45,9 +45,12 @@ jobs:
         run: |
           npm run test:ci integration/v2
 
+      - name: Run Core integration tests
+        if: matrix.os == 'ubuntu-latest'
+        uses: ./.github/actions/run-core-integration-tests
+
       - name: Slack Notify of failure
         if: failure()
-        id: slack
         uses: firebolt-db/action-slack-nightly-notify@v1
         with:
           os: ${{ matrix.os }}