pondpilot · melonamin · May 11, 2026 · Apr 14, 2026 · Apr 14, 2026 · Apr 14, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+#### Web App (app/) — Librarian (2026-05-04)
+
+AI chat panel for asking questions about your data using SQL lineage and uploaded PDF documentation. Supports OpenAI, Anthropic, and custom OpenAI-compatible endpoints; multilingual embeddings (`Xenova/multilingual-e5-small`, 100+ languages); per-project state isolation (RAM-only) so chat, PDFs, and embedded chunks don't leak between projects.
+
+Answers follow a structured **Summary / Data Lineage / Documentation** format. Schema identifiers are highlighted inline in assistant responses (case-insensitive, normalized to canonical schema casing). Clicking an assistant message reads the answer's Summary, highlights every referenced table and column in the Lineage view via the existing search pipeline (auto-enabling "show column edges" when a column is referenced), and gently pans + pulses on the source table containing the column.
+
+Schema view gains a search control (table-name or column-name substring, Prev/Next cycling). Librarian toggle lives in the analysis toolbar next to Schema (⌘L / Ctrl+L); a help popover in the panel header describes the assistant and usage.
+
 ## [0.7.0] - 2026-04-23
 
 ### Added

diff --git a/README.md b/README.md
@@ -32,6 +32,7 @@ Features:
 - Multi-file project support with schema DDL
 - dbt/Jinja template preprocessing for dbt models
 - Export to Mermaid, JSON, CSV, Excel, or HTML reports
+- Librarian — AI chat panel that answers questions about your data based on lineage analysis and uploaded PDF docs
 - All processing happens in your browser — your SQL never leaves your machine
 
 ### Command-Line Interface
@@ -114,6 +115,7 @@ See [CLI documentation](crates/flowscope-cli/README.md) for all options.
 - Structured diagnostics with spans for precise highlighting
 - Completion API for SQL authoring workflows
 - TypeScript API and optional React visualization components
+- Librarian AI chat panel for natural-language Q&A over SQL lineage and uploaded PDF documentation (OpenAI, Anthropic, or custom endpoints)
 
 ## Components
 
@@ -184,6 +186,7 @@ npm install @pondpilot/flowscope-react
 ## Documentation
 
 - [docs/README.md](docs/README.md) — documentation map and reference index
+- [docs/librarian.md](docs/librarian.md) — Librarian AI chat panel user guide
 - [docs/guides/quickstart.md](docs/guides/quickstart.md) — TypeScript quickstart guide
 - [docs/guides/schema-metadata.md](docs/guides/schema-metadata.md) — schema metadata setup
 - [docs/dialect-coverage.md](docs/dialect-coverage.md) — dialect and statement coverage

diff --git a/app/ARCHITECTURE.md b/app/ARCHITECTURE.md
@@ -19,6 +19,8 @@ app/src/
 │   ├── SchemaEditor.tsx    # DDL schema editor
 │   ├── ShareDialog.tsx     # Project export/sharing
 │   └── Workspace.tsx       # Main two-panel layout
+├── features/           # Feature modules (self-contained)
+│   └── librarian/         # AI chat panel (Q&A over lineage + PDFs)
 ├── hooks/              # Custom React hooks
 │   ├── useAnalysis.ts      # SQL analysis workflow
 │   ├── useFileNavigation.ts # Graph-to-editor navigation
@@ -74,6 +76,23 @@ The app now supports "Schema-Aware" analysis.
     *   Column validation
     *   Precise column lineage
 
+### 5. Feature Modules
+
+Self-contained feature folders under `app/src/features/` own all their code (components, hooks, services, workers, tests) and expose a public API via `index.ts`.
+
+#### Librarian (`features/librarian/`)
+
+AI-powered chat panel for SQL lineage Q&A.
+
+- `components/` — panel, chat messages, input, PDF upload, AI settings dialog
+- `services/` — AI client (OpenAI / Anthropic / custom), context builder, lineage formatter, PDF processor, vector search, embedding service
+- `workers/` — embedding Web Worker (local Xenova/transformers model)
+- `hooks/use-librarian-chat.ts` — chat orchestrator
+- `hooks/use-sync-active-project.ts` — mirrors `activeProjectId` from `useProject()` into the Librarian store and prunes buckets for deleted projects
+- `store.ts` — Zustand store. Per-project buckets (`byProject` keyed by `activeProjectId`) hold messages, PDF files, and embedded chunks; `isLoading` and `hasConfig` are global. Selector hooks `useLibrarianMessages` / `useLibrarianPdfFiles` / `useLibrarianPdfChunks` return the active project's slice. `addMessageToProject(projectId, ...)` writes to an explicit bucket so an in-flight LLM response is routed back to the originating project even if the user switches mid-flight.
+
+State is Zustand (not React Context), UI is Radix + Tailwind. All AI calls hit the user's configured provider directly from the browser. See `docs/librarian.md` for the user guide.
+
 ## Data Flow
 
 ### Analysis Loop
@@ -87,12 +106,23 @@ The app now supports "Schema-Aware" analysis.
 5.  **Result**: The JSON result is dispatched to the Lineage Store.
 6.  **Rendering**: `AnalysisView` updates the Graph and Issues panel.
 
+### Librarian Chat Flow
+
+1.  User types a question in the Librarian panel.
+2.  `use-librarian-chat.ts` gathers: current lineage (from `useLineageState`), active SQL file content, last 10 chat messages, and vector-search results over uploaded PDF chunks.
+3.  `context-builder.ts` assembles a structured prompt with labeled data sources (Data Lineage / SQL Code / Documentation / Conversation History).
+4.  `ai-service.ts` sends the prompt via `fetch()` to the configured provider (OpenAI / Anthropic / custom endpoint).
+5.  Response is stored in the chat and rendered with markdown + identifier highlighting.
+
+PDF processing runs asynchronously: text extraction (pdfjs-dist) → 500-char chunking → embeddings (local `multilingual-e5-small` model in a Web Worker) → stored in the librarian store.
+
 ## UI Architecture
 
 *   **Layout**: `react-resizable-panels` provides the split-view.
 *   **Styling**: Tailwind CSS with `shadcn/ui` (Radix Primitives) pattern.
 *   **Icons**: Lucide React.
 *   **Editor**: `CodeMirror` (via `@pondpilot/flowscope-react`).
+*   **Librarian icon**: Custom SVG (`/public/polly-icon.svg`) for the toolbar, chat avatar, and empty state.
 
 ## Configuration
 
@@ -102,6 +132,7 @@ The app now supports "Schema-Aware" analysis.
     *   `Cmd/Ctrl + P`: Switch Project
     *   `Cmd/Ctrl + O`: Switch File
     *   `Cmd/Ctrl + D`: Switch Dialect
+    *   `Cmd/Ctrl + L`: Toggle Librarian panel
 
 ## Future Improvements
 

diff --git a/app/package.json b/app/package.json
@@ -11,7 +11,8 @@
     "typecheck": "tsc --noEmit",
     "lint": "eslint src",
     "lint:fix": "eslint src --fix",
-    "test": "echo \"No tests defined for flowscope-app\""
+    "test": "vitest run",
+    "test:watch": "vitest"
   },
   "dependencies": {
     "@pondpilot/flowscope-core": "file:../packages/core",
@@ -21,19 +22,22 @@
     "@radix-ui/react-dialog": "^1.1.15",
     "@radix-ui/react-dropdown-menu": "^2.1.16",
     "@radix-ui/react-label": "^2.1.8",
+    "@radix-ui/react-popover": "^1.1.15",
     "@radix-ui/react-scroll-area": "^1.2.10",
     "@radix-ui/react-select": "^2.2.6",
     "@radix-ui/react-separator": "^1.1.8",
     "@radix-ui/react-slot": "^1.2.4",
     "@radix-ui/react-tabs": "^1.1.13",
     "@radix-ui/react-tooltip": "^1.2.8",
+    "@xenova/transformers": "^2.17.2",
     "class-variance-authority": "^0.7.1",
     "clsx": "^2.1.1",
     "fflate": "^0.8.2",
     "file-saver": "^2.0.5",
     "html-to-image": "^1.11.13",
     "lucide-react": "^0.562.0",
     "next-themes": "^0.4.6",
+    "pdfjs-dist": "4.8.69",
     "react": "^19.1.0",
     "react-dom": "^19.1.0",
     "react-resizable-panels": "^3.0.6",
@@ -43,15 +47,20 @@
   },
   "devDependencies": {
     "@tailwindcss/postcss": "^4.1.0",
+    "@testing-library/dom": "^10.4.1",
+    "@testing-library/jest-dom": "^6.9.1",
+    "@testing-library/react": "^16.3.2",
     "@types/file-saver": "^2.0.7",
     "@types/react": "^19.1.0",
     "@types/react-dom": "^19.1.0",
     "@vitejs/plugin-react": "^4.4.0",
+    "jsdom": "^26.1.0",
     "postcss": "^8.5.6",
     "tailwindcss": "^4.1.0",
     "typescript": "^5.9.0",
     "vite": "^6.3.0",
     "vite-plugin-top-level-await": "^1.6.0",
-    "vite-plugin-wasm": "^3.5.0"
+    "vite-plugin-wasm": "^3.5.0",
+    "vitest": "^3.2.4"
   }
 }
diff --git a/app/public/polly-icon.svg b/app/public/polly-icon.svg
diff --git a/app/src/components/AnalysisView.tsx b/app/src/components/AnalysisView.tsx
@@ -19,15 +19,18 @@ import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
 import { usePersistedLineageState } from '@/hooks/usePersistedLineageState';
 import { usePersistedMatrixState } from '@/hooks/usePersistedMatrixState';
 import { usePersistedSchemaState } from '@/hooks/usePersistedSchemaState';
+import { applyLineageNavigation } from '@/lib/lineage-navigation';
 import { isValidTab, useNavigation } from '@/lib/navigation-context';
 import { useViewStateStore, getNamespaceFilterStateWithDefaults } from '@/lib/view-state-store';
 import { useProject } from '@/lib/project-store';
 import { schemaMetadataToSQL } from '@/lib/schema-parser';
 import { HierarchyView, type HierarchyViewRef } from './HierarchyView';
+import { LibrarianToggleButton } from './LibrarianToggleButton';
 import { StatsPopover } from './StatsPopover';
 import { NamespaceFilterBar } from './NamespaceFilterBar';
 import { SchemaAwareIssuesPanel } from './SchemaAwareIssuesPanel';
 import { SchemaEditor } from './SchemaEditor';
+import { SchemaSearchControl } from './SchemaSearchControl';
 
 interface AnalysisViewProps {
   graphContainerRef?: React.RefObject<HTMLDivElement | null>;
@@ -106,14 +109,14 @@ export function AnalysisView({
   // Handle navigation target for GraphView - select and focus node/statement when navigating to lineage tab
   useEffect(() => {
     if (activeTab === 'lineage' && navigationTarget) {
-      if (navigationTarget.tableId) {
-        // Navigate to specific table node
-        actionsRef.current.selectNode(navigationTarget.tableId);
-        setLineageFocusNodeId(navigationTarget.tableId);
-      } else if (navigationTarget.fitView) {
-        // Trigger fitView to show all nodes (e.g., from Issues panel)
-        setFitViewTrigger((prev) => prev + 1);
-      }
+      applyLineageNavigation(navigationTarget, {
+        expandedTableIds: stateRef.current.expandedTableIds,
+        selectNode: actionsRef.current.selectNode,
+        toggleTableExpansion: actionsRef.current.toggleTableExpansion,
+        setFocusNodeId: setLineageFocusNodeId,
+        triggerFitView: () => setFitViewTrigger((prev) => prev + 1),
+        revealNodeInGraph: actionsRef.current.revealNodeInGraph,
+      });
       clearNavigationTarget();
     }
   }, [activeTab, navigationTarget, clearNavigationTarget]);
@@ -336,24 +339,29 @@ export function AnalysisView({
 
   if (!result || !summary) {
     return (
-      <div className="flex flex-col items-center justify-center h-full text-muted-foreground bg-muted/5">
-        <div className="p-6 text-center">
-          {isAnalyzing ? (
-            <>
-              <Loader2 className="h-6 w-6 animate-spin mx-auto mb-3 opacity-70" />
-              <h3 className="font-semibold mb-2">Analyzing SQL</h3>
-              <p className="text-sm max-w-xs mx-auto">
-                Building lineage, schema, and issue details for the current analysis run.
-              </p>
-            </>
-          ) : (
-            <>
-              <h3 className="font-semibold mb-2">No Analysis Results</h3>
-              <p className="text-sm max-w-xs mx-auto">
-                Run analysis on your SQL script to see lineage and schema details here.
-              </p>
-            </>
-          )}
+      <div className="flex flex-col h-full bg-background">
+        <div className="px-4 py-2 border-b flex items-center justify-end bg-muted/10 h-[44px] shrink-0">
+          <LibrarianToggleButton />
+        </div>
+        <div className="flex-1 flex flex-col items-center justify-center text-muted-foreground bg-muted/5">
+          <div className="p-6 text-center">
+            {isAnalyzing ? (
+              <>
+                <Loader2 className="h-6 w-6 animate-spin mx-auto mb-3 opacity-70" />
+                <h3 className="font-semibold mb-2">Analyzing SQL</h3>
+                <p className="text-sm max-w-xs mx-auto">
+                  Building lineage, schema, and issue details for the current analysis run.
+                </p>
+              </>
+            ) : (
+              <>
+                <h3 className="font-semibold mb-2">No Analysis Results</h3>
+                <p className="text-sm max-w-xs mx-auto">
+                  Run analysis on your SQL script to see lineage and schema details here.
+                </p>
+              </>
+            )}
+          </div>
         </div>
       </div>
     );
@@ -413,6 +421,7 @@ export function AnalysisView({
                 </Tooltip>
               </TooltipProvider>
             )}
+            <LibrarianToggleButton />
           </div>
         </div>
 
@@ -484,11 +493,20 @@ export function AnalysisView({
             className="h-full mt-0 p-0 absolute inset-0 data-[state=inactive]:hidden"
           >
             {mountedTabs.has('schema') && (
-              <SchemaView
-                schema={schema}
-                selectedTableName={schemaState.selectedTableName}
-                onClearSelection={schemaState.clearSelection}
-              />
+              <div className="relative h-full w-full">
+                <SchemaView
+                  schema={schema}
+                  selectedTableName={schemaState.selectedTableName}
+                  onClearSelection={schemaState.clearSelection}
+                />
+                <div className="absolute top-2 right-2 z-10">
+                  <SchemaSearchControl
+                    tableNames={schema.map((t) => t.name)}
+                    tables={schema}
+                    onSelectTable={schemaState.setSelectedTableName}
+                  />
+                </div>
+              </div>
             )}
           </TabsContent>