Upgrade to upstream quickwit main; point tantivy dep at upgrade/tantivy-0.26

.claude/skills/bump-tantivy/SKILL.md

@@ -0,0 +1,83 @@
+---
+name: bump-tantivy
+description: Bump tantivy to the latest commit on main branch, fix compilation issues, and open a PR
+disable-model-invocation: true
+---
+
+# Bump Tantivy
+
+Follow these steps to bump tantivy to its latest version:
+
+## Step 1: Check that we are on the main branch
+
+Run: `git branch --show-current`
+
+If the current branch is not `main`, abort and ask the user to switch to the main branch first.
+
+## Step 2: Ensure main is up to date
+
+Run: `git pull origin main`
+
+This ensures we're working from the latest code.
+
+## Step 3: Get the latest tantivy SHA
+
+Run: `gh api repos/quickwit-oss/tantivy/commits/main --jq '.sha'`
+
+Extract the first 7 characters as the short SHA.
+
+## Step 4: Update Cargo.toml
+
+Edit `quickwit/Cargo.toml` and update the `rev` field in the tantivy dependency to the new short SHA.
+
+The line looks like:
+```toml
+tantivy = { git = "https://github.com/quickwit-oss/tantivy/", rev = "XXXXXXX", ... }
+```
+
+## Step 5: Run cargo check and fix compilation errors
+
+Run `cargo check` in the `quickwit` directory to verify compilation.
+
+If there are compilation errors:
+- If the fix is straightforward (simple API changes, renames, etc.), fix them without asking
+- If the fix is complex or unclear, ask the user before proceeding
+
+Repeat until cargo check passes.
+
+## Step 6: Format code
+
+Run `make fmt` from the `quickwit/` directory to format the code.
+
+## Step 7: Update licenses
+
+Run `make update-licenses` from the `quickwit/` directory, then move the generated file:
+```
+mv quickwit/LICENSE-3rdparty.csv ./LICENSE-3rdparty.csv
+```
+
+## Step 8: Create a new branch
+
+Get the git username: `git config user.name | tr ' ' '-' | tr '[:upper:]' '[:lower:]'`
+
+Get today's date: `date +%Y-%m-%d`
+
+Create and checkout a new branch named: `{username}/bump-tantivy-{date}`
+
+Example: `paul/bump-tantivy-2024-03-15`
+
+## Step 9: Commit changes
+
+Stage all modified files and create a commit with message:
+```
+Bump tantivy to {short-sha}
+```
+
+## Step 10: Push and open a PR
+
+Push the branch and open a PR using:
+```
+gh pr create --title "Bump tantivy to {short-sha}" --body "Updates tantivy dependency to the latest commit on main."
+```
+
+Report the PR URL to the user when complete.

.claude/skills/fix-clippy/SKILL.md

@@ -0,0 +1,26 @@
+---
+name: fix-clippy
+description: Fix all clippy lint warnings in the project
+---
+
+# Fix Clippy
+
+Clippy issues are **warnings**, not errors. Never grep for `error` when looking for clippy issues.
+
+## Step 1: Auto-fix
+
+Run `make fix` to automatically fix clippy warnings:
+
+```
+make fix
+```
+
+## Step 2: Fix remaining warnings manually
+
+Check for remaining warnings that couldn't be auto-fixed:
+
+```
+cargo clippy --tests 2>&1 | grep "^warning:" | sort -u
+```
+
+For each remaining warning, find the exact location and fix it manually.

.claude/skills/fmt/SKILL.md

@@ -0,0 +1,23 @@
+---
+name: fmt
+description: Run `make fmt` to check the code format.
+---
+
+# Format Check
+
+Run `make fmt` from the `quickwit/` subdirectory to check code formatting:
+
+```
+cd /Users/paul.masurel/git/quickwit/quickwit && make fmt
+```
+
+This command checks:
+1. Rust code formatting
+2. License headers
+3. Log format policy (no trailing punctuation, no uppercase first character)
+
+If there are log format issues, fix them by:
+- Making the first character lowercase
+- Removing trailing punctuation (periods, exclamation marks, etc.)
+
+Fix any issues found and re-run until clean.

.claude/skills/rationalize-deps/SKILL.md

@@ -0,0 +1,125 @@
+---
+name: rationalize-deps
+description: Analyze Cargo.toml dependencies and attempt to remove unused features to reduce compile times and binary size
+---
+
+# Rationalize Dependencies
+
+This skill analyzes Cargo.toml dependencies to identify and remove unused features.
+
+## Overview
+
+Many crates enable features by default that may not be needed. This skill:
+1. Identifies dependencies with default features enabled
+2. Tests if `default-features = false` works
+3. Identifies which specific features are actually needed
+4. Verifies compilation after changes
+
+## Step 1: Identify the target
+
+Ask the user which crate(s) to analyze:
+- A specific crate name (e.g., "tokio", "serde")
+- A specific workspace member (e.g., "quickwit-search")
+- "all" to scan the entire workspace
+
+## Step 2: Analyze current dependencies
+
+For the workspace Cargo.toml (`quickwit/Cargo.toml`), list dependencies that:
+- Do NOT have `default-features = false`
+- Have default features that might be unnecessary
+
+Run: `cargo tree -p <crate> -f "{p} {f}" --edges features` to see what features are actually used.
+
+## Step 3: For each candidate dependency
+
+### 3a: Check the crate's default features
+
+Look up the crate on crates.io or check its Cargo.toml to understand:
+- What features are enabled by default
+- What each feature provides
+
+Use: `cargo metadata --format-version=1 | jq '.packages[] | select(.name == "<crate>") | .features'`
+
+### 3b: Try disabling default features
+
+Modify the dependency in `quickwit/Cargo.toml`:
+
+From:
+```toml
+some-crate = { version = "1.0" }
+```
+
+To:
+```toml
+some-crate = { version = "1.0", default-features = false }
+```
+
+### 3c: Run cargo check
+
+Run: `cargo check --workspace` (or target specific packages for faster feedback)
+
+If compilation fails:
+1. Read the error messages to identify which features are needed
+2. Add only the required features explicitly:
+   ```toml
+   some-crate = { version = "1.0", default-features = false, features = ["needed-feature"] }
+   ```
+3. Re-run cargo check
+
+### 3d: Binary search for minimal features
+
+If there are many default features, use binary search:
+1. Start with no features
+2. If it fails, add half the default features
+3. Continue until you find the minimal set
+
+## Step 4: Document findings
+
+For each dependency analyzed, report:
+- Original configuration
+- New configuration (if changed)
+- Features that were removed
+- Any features that are required
+
+## Step 5: Verify full build
+
+After all changes, run:
+```bash
+cargo check --workspace --all-targets
+cargo test --workspace --no-run
+```
+
+## Common Patterns
+
+### Serde
+Often only needs `derive`:
+```toml
+serde = { version = "1.0", default-features = false, features = ["derive", "std"] }
+```
+
+### Tokio
+Identify which runtime features are actually used:
+```toml
+tokio = { version = "1.0", default-features = false, features = ["rt-multi-thread", "macros", "sync"] }
+```
+
+### Reqwest
+Often doesn't need all TLS backends:
+```toml
+reqwest = { version = "0.11", default-features = false, features = ["rustls-tls", "json"] }
+```
+
+## Rollback
+
+If changes cause issues:
+```bash
+git checkout quickwit/Cargo.toml
+cargo check --workspace
+```
+
+## Tips
+
+- Start with large crates that have many default features (tokio, reqwest, hyper)
+- Use `cargo bloat --crates` to identify large dependencies
+- Check `cargo tree -d` for duplicate dependencies that might indicate feature conflicts
+- Some features are needed only for tests - consider using `[dev-dependencies]` features

.claude/skills/simple-pr/SKILL.md

@@ -0,0 +1,60 @@
+---
+name: simple-pr
+description: Create a simple PR from staged changes with an auto-generated commit message
+disable-model-invocation: true
+---
+
+# Simple PR
+
+Follow these steps to create a simple PR from staged changes:
+
+## Step 1: Check workspace state
+
+Run: `git status`
+
+Verify that all changes have been staged (no unstaged changes). If there are unstaged changes, abort and ask the user to stage their changes first with `git add`.
+
+Also verify that we are on the `main` branch. If not, abort and ask the user to switch to main first.
+
+## Step 2: Ensure main is up to date
+
+Run: `git pull origin main`
+
+This ensures we're working from the latest code.
+
+## Step 3: Review staged changes
+
+Run: `git diff --cached`
+
+Review the staged changes to understand what the PR will contain.
+
+## Step 4: Generate commit message
+
+Based on the staged changes, generate a concise commit message (1-2 sentences) that describes the "why" rather than the "what".
+
+Display the proposed commit message to the user and ask for confirmation before proceeding.
+
+## Step 5: Create a new branch
+
+Get the git username: `git config user.name | tr ' ' '-' | tr '[:upper:]' '[:lower:]'`
+
+Create a short, descriptive branch name based on the changes (e.g., `fix-typo-in-readme`, `add-retry-logic`, `update-deps`).
+
+Create and checkout the branch: `git checkout -b {username}/{short-descriptive-name}`
+
+## Step 6: Commit changes
+
+Commit with the message from step 3:
+```
+git commit -m "{commit-message}"
+```
+
+## Step 7: Push and open a PR
+
+Push the branch and open a PR:
+```
+git push -u origin {branch-name}
+gh pr create --title "{commit-message-title}" --body "{longer-description-if-needed}"
+```
+
+Report the PR URL to the user when complete.

.devcontainer/devcontainer.json

@@ -1,6 +1,6 @@
 {
     "name": "Quickwit",
-    "image": "mcr.microsoft.com/devcontainers/rust:latest",
+    "image": "mcr.microsoft.com/devcontainers/rust:bookworm",
     "customizations": {
         "codespaces": {
             "openFiles": [
@@ -33,7 +33,7 @@
             "moby": true
         },
         "ghcr.io/devcontainers/features/node:1": {
-            "version": "18"
+            "version": "24"
         },
         "ghcr.io/devcontainers/features/aws-cli:1": {},
         "ghcr.io/devcontainers-contrib/features/protoc:1": {}

.github/actions/cargo-build-macos-binary/action.yml

@@ -17,7 +17,7 @@ runs:
       shell: bash
     - uses: actions/setup-node@v3
       with:
-        node-version: 20
+        node-version: 24
         cache: "yarn"
         cache-dependency-path: quickwit/quickwit-ui/yarn.lock
     - run: yarn global add node-gyp
@@ -36,9 +36,9 @@ runs:
       working-directory: ./quickwit
     - name: Retrieve and export commit date, hash, and tags
       run: |
-          echo "QW_COMMIT_DATE=$(TZ=UTC0 git log -1 --format=%cd --date=format-local:%Y-%m-%dT%H:%M:%SZ)" >> $GITHUB_ENV
-          echo "QW_COMMIT_HASH=$(git rev-parse HEAD)" >> $GITHUB_ENV
-          echo "QW_COMMIT_TAGS=$(git tag --points-at HEAD | tr '\n' ',')" >> $GITHUB_ENV
+        echo "QW_COMMIT_DATE=$(TZ=UTC0 git log -1 --format=%cd --date=format-local:%Y-%m-%dT%H:%M:%SZ)" >> $GITHUB_ENV
+        echo "QW_COMMIT_HASH=$(git rev-parse HEAD)" >> $GITHUB_ENV
+        echo "QW_COMMIT_TAGS=$(git tag --points-at HEAD | tr '\n' ',')" >> $GITHUB_ENV
       shell: bash
     - name: Build binary
       run: cargo build --release --features release-macos-feature-vendored-set --target ${{ matrix.target }} --bin quickwit
@@ -60,7 +60,7 @@ runs:
         path: ./${{ env.ASSET_FULL_NAME }}.tar.gz
         retention-days: 3
     - name: Deploy archive to GitHub release
-      uses: quickwit-inc/upload-to-github-release@v1
+      uses: quickwit-inc/upload-to-github-release@9b2c40fba23bf8dea05b7d2eece24cbc95d4a190
       env:
         GITHUB_TOKEN: ${{ inputs.token }}
       with: