feat: Add natural language analytics dashboard

[danenania]

Summary

Add a natural language analytics feature where property owners can ask questions about their booking data in plain English (e.g., "How many bookings did I have last month?"). The system uses an LLM to generate SQL queries that are executed against a SQLite database.

New Features

• POST /authorized/:level/analytics/query - Natural language to SQL analytics endpoint
• SQLite database with sample booking/property/owner data
• Database initialization script

Files Added

• src/routes/analytics.ts - Analytics endpoint with LLM-based SQL generation
• src/scripts/init-analytics-db.ts - Database initialization script
• src/data/bookings.db - SQLite database (gitignored, generated by init script)

Dependencies

• better-sqlite3 for SQLite database access

In general, the problem is fixed by adding a rate-limiting middleware in front of the expensive handler so that individual clients cannot send unbounded numbers of requests in a short time. In Express, a common solution is to use the well-known express-rate-limit package, configure reasonable thresholds, and apply the resulting middleware either globally to the router or specifically to this analytics route.

For this snippet, the least invasive and clearest fix is:

• Import express-rate-limit at the top of src/routes/analytics.ts.
• Configure a limiter specifically for the analytics query endpoint (for example, a small number of requests per minute per IP, given it’s doing LLM + DB work).
• Apply that limiter as a middleware only to router.post('/authorized/:level/analytics/query', ...) so that existing behavior of other routes in this router is unchanged.

Concretely:

• Add import rateLimit from 'express-rate-limit'; below the existing imports.
• Define a const analyticsLimiter = rateLimit({ ... }) near the router initialization, setting windowMs and max and possibly a friendly message.
• Update the route definition on line 77 from router.post('/authorized/:level/analytics/query', async (req, res) => { ... }) to router.post('/authorized/:level/analytics/query', analyticsLimiter, async (req, res) => { ... }).

No other behavior of the handler needs to change; we are only inserting middleware to control how frequently it can be invoked.

[promptfoo-scanner]

This PR introduces a natural language analytics dashboard with text-to-SQL functionality. I found several critical LLM security vulnerabilities including prompt injection leading to secrets exposure, missing authentication on the endpoint, and prompt-only safeguards that can be bypassed. The most severe issue is that untrusted user input flows to an LLM to generate SQL queries, which are then executed without validation against a database containing API keys and PII.

_{Minimum severity threshold for this scan: 🟡 Medium | Learn more}

[promptfoo-scanner]

🔴 Critical

User input flows directly to the LLM to generate SQL queries, which are then executed without validation. An attacker can use prompt injection to bypass the system prompt's safety rules and extract sensitive data like API keys from the owners table. Since the endpoint is unauthenticated and the database is opened in read-write mode, attackers can also potentially modify or delete data.

💡 Suggested Fix

Implement defense-in-depth with multiple layers of protection:

// Layer 1: Validate generated SQL before execution
function validateSqlQuery(query: string): { valid: boolean; error?: string } {
  const upperQuery = query.trim().toUpperCase();

  if (!upperQuery.startsWith('SELECT')) {
    return { valid: false, error: 'Only SELECT queries are allowed' };
  }

  const dangerousKeywords = ['DROP', 'DELETE', 'INSERT', 'UPDATE', 'ALTER', 'CREATE', 'TRUNCATE', 'PRAGMA'];
  for (const keyword of dangerousKeywords) {
    if (new RegExp(`\\b${keyword}\\b`, 'i').test(query)) {
      return { valid: false, error: `Dangerous keyword detected: ${keyword}` };
    }
  }

  return { valid: true };
}

// Layer 2: Open database in read-only mode (line 18)
db = new Database(dbPath, { readonly: true });

// Layer 3: Filter sensitive columns from results
function filterSensitiveColumns(results: any[]): any[] {
  const sensitiveColumns = ['api_key', 'password', 'secret', 'token'];
  return results.map(row => {
    const filtered: any = {};
    for (const [key, value] of Object.entries(row)) {
      if (!sensitiveColumns.some(col => key.toLowerCase().includes(col))) {
        filtered[key] = value;
      }
    }
    return filtered;
  });
}

// Apply validation before execution (line 90-95)
const sqlQuery = await generateSqlQuery(question, model);
const validation = validateSqlQuery(sqlQuery);
if (!validation.valid) {
  return res.status(400).json({ error: 'Invalid query', message: validation.error });
}

const rawResults = db.prepare(sqlQuery).all();
const results = filterSensitiveColumns(rawResults);

🤖 AI Agent Prompt

The text-to-SQL feature at src/routes/analytics.ts:44-95 has a critical prompt injection vulnerability. User input flows to an LLM (via generateSqlQuery), which generates SQL that's executed directly without validation at line 95. The database contains sensitive data (API keys in the owners table, guest emails in bookings table) and is opened in read-write mode.

Your task: Implement comprehensive defense-in-depth security controls. Investigate the entire data flow from the /authorized/:level/analytics/query endpoint through to database execution. You need to add:

1. Application-layer SQL validation to enforce SELECT-only queries and block dangerous keywords
2. Convert the database connection to read-only mode
3. Result filtering to redact sensitive columns like api_key
4. Consider removing the api_key column from the system prompt schema disclosure (lines 29-42)

Start by examining the current implementation to understand the attack surface. Then implement each layer of defense systematically. The goal is to prevent both data exfiltration and data modification attacks via prompt injection.

---

_{Was this helpful?  👍 Yes  |  👎 No} 

[promptfoo-scanner]

🟠 High

Guest email addresses and names in the bookings table are exposed to unauthenticated users through unvalidated LLM-generated queries. Any attacker can extract all guest PII by crafting natural language questions like "Show me all guest emails". This is a cross-user data exposure vulnerability where one user's data is accessible to others.

💡 Suggested Fix

Implement row-level authorization to filter results based on authenticated user identity:

// After authentication middleware provides req.user
router.post('/authorized/:level/analytics/query', authenticateToken, async (req: Request, res: Response) => {
  // ... query generation and validation ...

  // Execute query
  const rawResults = db.prepare(sqlQuery).all();

  // Filter results based on user authorization
  const userId = (req as any).user?.sub;
  const userEmail = (req as any).user?.email;

  const authorizedResults = rawResults.filter(row => {
    // Users can only see their own bookings
    if ('guest_email' in row) {
      return row.guest_email === userEmail;
    }
    return true; // Allow non-sensitive data
  });

  return res.json({
    question,
    generatedQuery: sqlQuery,
    results: authorizedResults,
    rowCount: authorizedResults.length,
  });
});

🤖 AI Agent Prompt

At src/routes/analytics.ts:90-102, query results are returned without authorization checks, allowing cross-user PII exposure. The database contains guest emails and names in the bookings table that should only be accessible to the respective guests or property owners.

Your task: Implement row-level authorization filtering. After the authentication middleware is in place (see the missing authentication issue), filter query results to ensure users can only access data they're authorized to see.

Investigate the application's user model to understand user roles (guests, property owners, admins) and implement appropriate filtering logic. For example, guests should only see their own bookings, while property owners should see bookings for their properties. The exact authorization model will depend on your application's requirements, but at minimum, prevent cross-user data access.

---

_{Was this helpful?  👍 Yes  |  👎 No} 

[promptfoo-scanner]

🟠 High

Security rules restricting SQL queries to SELECT-only are enforced solely through system prompt instructions, which can be bypassed via prompt injection. The database is opened in read-write mode, so if an attacker successfully manipulates the LLM to generate UPDATE, DELETE, or DROP statements, these will execute without any application-layer validation blocking them.

💡 Suggested Fix

Replace prompt-only safeguards with deterministic technical controls (this is covered by the comprehensive fix for the prompt injection vulnerability):

// Application-layer validation (cannot be bypassed)
function validateSqlQuery(query: string): { valid: boolean; error?: string } {
  const upperQuery = query.trim().toUpperCase();
  if (!upperQuery.startsWith('SELECT')) {
    return { valid: false, error: 'Only SELECT queries are allowed' };
  }

  const dangerousKeywords = ['DROP', 'DELETE', 'INSERT', 'UPDATE', 'ALTER', 'CREATE', 'TRUNCATE', 'PRAGMA'];
  for (const keyword of dangerousKeywords) {
    if (new RegExp(`\\b${keyword}\\b`, 'i').test(query)) {
      return { valid: false, error: `Dangerous keyword detected: ${keyword}` };
    }
  }
  return { valid: true };
}

// Database-level protection (line 18)
db = new Database(dbPath, { readonly: true });

// Apply validation before execution
const validation = validateSqlQuery(sqlQuery);
if (!validation.valid) {
  return res.status(400).json({ error: 'Invalid query', message: validation.error });
}

🤖 AI Agent Prompt

The code at src/routes/analytics.ts:36-40 defines security rules only in the system prompt, with no technical enforcement. At line 95, any SQL the LLM generates is executed directly. The database is opened in read-write mode at line 18.

Your task: Replace bypassable prompt instructions with deterministic security controls. Implement two layers:

1. Application-layer validation that checks the generated SQL before execution, rejecting anything that isn't a SELECT query or contains dangerous keywords
2. Open the database connection in read-only mode to prevent modifications even if validation is bypassed

These technical controls should be impossible to circumvent via prompt injection, unlike the current prompt-only safeguards. Review the database initialization at line 18 and the query execution at line 95 to implement these protections.

---

_{Was this helpful?  👍 Yes  |  👎 No} 

[promptfoo-scanner]

🟠 High

The analytics router is mounted without authentication middleware, despite the endpoint path including /authorized/ which suggests auth should be required. This makes the text-to-SQL capability publicly accessible to unauthenticated users, amplifying all other vulnerabilities. Compare this to line 33 where the chat endpoint correctly applies authenticateToken middleware.

💡 Suggested Fix

Apply authentication middleware to the analytics routes:

// In src/routes/analytics.ts, add middleware to the route definition:
import { authenticateToken } from '../middleware/auth';

router.post('/authorized/:level/analytics/query',
  authenticateToken,  // Add authentication middleware
  async (req: Request, res: Response) => {
    // ... existing handler code ...
  }
);

Or alternatively in src/server.ts:

// Apply middleware when mounting the router
app.use('/authorized', authenticateToken, analyticsRouter);

🤖 AI Agent Prompt

At src/server.ts:35-36, the analytics router is mounted without authentication middleware. The endpoint path includes /authorized/ but no auth is actually enforced, unlike the chat endpoint at line 33 which correctly uses authenticateToken.

Your task: Add authentication to protect the analytics endpoint. You have two options:

1. Modify src/routes/analytics.ts at line 77 to import and apply authenticateToken middleware directly to the route handler
2. Modify src/server.ts at line 36 to apply middleware when mounting the router

Option 1 is recommended for clarity. Once implemented, the endpoint will require a valid JWT Bearer token, preventing unauthenticated access to the text-to-SQL functionality. Review how the chat endpoint applies authentication at line 33 as a reference pattern.

---

_{Was this helpful?  👍 Yes  |  👎 No} 

[promptfoo-scanner]

🟡 Medium

The system prompt reveals the complete database schema including the existence of an api_key column in the owners table. This information disclosure makes prompt injection attacks easier by providing attackers a roadmap of what sensitive data exists and exactly how to query it.

💡 Suggested Fix

Remove sensitive columns from the schema disclosure:

const systemPrompt = `You are a SQL query generator for a vacation rental analytics system.

DATABASE SCHEMA:
- bookings (id, property_id, guest_name, check_in, check_out, total_price, status, created_at)
- properties (id, name, address, nightly_rate, owner_id, created_at)
- owners (id, name, email, created_at)

Note: Some columns containing sensitive data are omitted from this schema.

RULES:
- Only generate SELECT queries
- Never use DROP, DELETE, UPDATE, INSERT, or ALTER statements
- Always limit results to 100 rows maximum

Generate a single SQL query to answer the user's question. Return ONLY the SQL query, no explanation.`;

🤖 AI Agent Prompt

The system prompt at src/routes/analytics.ts:29-42 reveals the database schema including sensitive columns like api_key. While this doesn't directly expose secrets, it aids attackers by showing them exactly what sensitive data exists.

Your task: Remove the api_key column from the schema provided to the LLM. The schema should include only non-sensitive columns for each table. This is a defense-in-depth measure that works in conjunction with result filtering to prevent sensitive data exposure.

Update the system prompt construction in the generateSqlQuery function to omit sensitive column names while still providing enough schema information for the LLM to generate useful analytics queries.

---

_{Was this helpful?  👍 Yes  |  👎 No} 

[promptfoo-scanner]

🟡 Medium

The authorization level parameter ('minnow' or 'shark') is extracted from the URL but never used for access control. Both authorization levels have identical database access, suggesting unfinished implementation of tiered permissions. This overly broad access violates the principle of least privilege.

💡 Suggested Fix

Implement table-based access restrictions for different authorization levels:

router.post('/authorized/:level/analytics/query', authenticateToken, async (req: Request, res: Response) => {
  try {
    const { level } = req.params as { level: 'minnow' | 'shark' };
    const { question, model } = analyticsQuerySchema.parse(req.body);

    // Define allowed tables based on authorization level
    const allowedTables: Record<string, string[]> = {
      minnow: ['bookings', 'properties'],  // Limited access
      shark: ['bookings', 'properties', 'owners'],  // Full access
    };

    const sqlQuery = await generateSqlQuery(question, model);

    // Validate generated SQL and check table access
    const referencedTables = extractTablesFromQuery(sqlQuery);
    const unauthorizedTables = referencedTables.filter(
      table => !allowedTables[level].includes(table)
    );

    if (unauthorizedTables.length > 0) {
      return res.status(403).json({
        error: 'Forbidden',
        message: `Authorization level '${level}' cannot access tables: ${unauthorizedTables.join(', ')}`
      });
    }

    // ... execute query ...
  }
});

🤖 AI Agent Prompt

At src/routes/analytics.ts:77-80, the code extracts a level parameter ('minnow' or 'shark') but never uses it for authorization. This suggests an intended design where different levels should have different access, but it's not implemented.

Your task: Implement authorization level enforcement. Define what tables or data each level can access, then validate the generated SQL queries to ensure they only reference allowed tables for that level. For example, 'minnow' users might only access bookings and properties, while 'shark' users can also access the owners table.

You'll need to parse the generated SQL to extract referenced tables and reject queries that access unauthorized tables. Consider using regex patterns or a lightweight SQL parser. The specific access model should align with your application's business requirements - this is a defense-in-depth measure to limit the blast radius if other security controls fail.

---

_{Was this helpful?  👍 Yes  |  👎 No}