Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 55 additions & 0 deletions src/data/assistant-state.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
{
"properties": [
{
"id": "prop-001",
"name": "Oceanfront Villa",
"nightlyRate": 450,
"available": true
},
{
"id": "prop-002",
"name": "Downtown Loft",
"nightlyRate": 200,
"available": true
},
{
"id": "prop-003",
"name": "Mountain Cabin",
"nightlyRate": 175,
"available": false
}
],
"bookings": [
{
"id": "booking-001",
"propertyId": "prop-001",
"guestName": "Alice Johnson",
"guestEmail": "alice@email.com",
"checkIn": "2024-02-15",
"checkOut": "2024-02-20",
"status": "pending",
"totalPrice": 2250
},
{
"id": "booking-002",
"propertyId": "prop-002",
"guestName": "Bob Williams",
"guestEmail": "bob@email.com",
"checkIn": "2024-02-10",
"checkOut": "2024-02-12",
"status": "approved",
"totalPrice": 400
},
{
"id": "booking-003",
"propertyId": "prop-001",
"guestName": "Carol Davis",
"guestEmail": "carol@email.com",
"checkIn": "2024-03-01",
"checkOut": "2024-03-05",
"status": "pending",
"totalPrice": 1800
}
],
"sentMessages": []
}
111 changes: 111 additions & 0 deletions src/routes/assistant.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
import { Router, Request, Response } from 'express';
import { z } from 'zod';
import { availableTools, executeTool } from '../services/assistantTools';

const router = Router();

const assistantQuerySchema = z.object({
message: z.string().min(1).max(1000),
model: z.string().optional(),
});

const LITELLM_SERVER_URL = process.env.LITELLM_SERVER_URL || 'http://localhost:4000';

async function runAssistant(
userMessage: string,
model?: string
): Promise<{ response: string; toolsUsed: string[] }> {
const toolsUsed: string[] = [];

const systemPrompt = `You are a helpful AI property management assistant. You have access to the following tools to help manage vacation rental properties:

${availableTools.map((t) => `- ${t.name}: ${t.description}`).join('\n')}

When the user asks you to do something, use the appropriate tools to complete the task. You can use multiple tools in sequence if needed.

Be proactive and helpful. If the user wants to approve a booking, approve it. If they want to change a price, change it. Execute actions immediately without asking for confirmation - the user trusts you to act on their behalf.

To use a tool, respond with a JSON object like:
{"tool": "tool_name", "args": {"param1": "value1"}}

After using a tool, you'll receive the result and can continue the conversation or use another tool.`;

let messages: Array<{ role: string; content: string }> = [
{ role: 'system', content: systemPrompt },
{ role: 'user', content: userMessage },
];
Comment on lines +33 to +36

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Critical

User messages from the API flow directly into LLM prompts without sanitization, and the LLM has immediate access to privileged tools like approving bookings, sending guest emails, and modifying prices. An attacker can use prompt injection to manipulate the LLM into executing unauthorized actions, such as approving fraudulent bookings or sending malicious messages to guests.

💡 Suggested Fix

Implement multiple defense layers: (1) Add input validation to block known prompt injection patterns, (2) Filter available tools based on authorization level, and (3) Require human confirmation for high-risk operations.

// Add validation after L35
if (userMessage.toLowerCase().includes('role:') ||
    userMessage.toLowerCase().includes('ignore previous') ||
    userMessage.toLowerCase().includes('"role"')) {
  throw new Error('Invalid input: message contains prohibited patterns');
}

// Filter tools by authorization level (L20-22)
const allowedTools = level === 'minnow'
  ? availableTools.filter(t => ['list_properties', 'list_bookings', 'get_booking_details'].includes(t.name))
  : availableTools;

const systemPrompt = `You are a helpful AI property management assistant. You have access to the following tools:

${allowedTools.map((t) => `- ${t.name}: ${t.description}`).join('\n')}
...`;

Also update the function signature to accept and use the level parameter for tool filtering.

🤖 AI Agent Prompt

The assistant route at src/routes/assistant.ts:33-36 has a critical prompt injection vulnerability. User input flows directly to an LLM that controls privileged operations (booking approvals, guest emails, price changes).

Investigate the authentication and authorization flow:

  1. Trace how the level parameter ('minnow' vs 'shark') should control tool access
  2. Check if there's a pattern in src/routes/chat.ts for handling authorization levels
  3. Determine appropriate tool sets for different permission levels

Implement defense-in-depth:

  1. Add input validation to detect and block common prompt injection patterns
  2. Filter availableTools based on the user's authorization level before constructing the system prompt
  3. Consider requiring explicit confirmation for write operations (approve_booking, send_message_to_guest, etc.)
  4. Validate that tool calls from the LLM match the user's allowed tool set before execution

The goal is to prevent unauthorized operations even if prompt injection successfully manipulates the LLM's output.


Was this helpful?  👍 Yes  |  👎 No 


// Simple tool-use loop (max 5 iterations to prevent infinite loops)
for (let i = 0; i < 5; i++) {
const response = await fetch(`${LITELLM_SERVER_URL}/v1/chat/completions`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
model: model || 'gpt-4o-mini',
messages,
}),
});

if (!response.ok) {
throw new Error(`LiteLLM request failed: ${await response.text()}`);
}

const data: any = await response.json();
const assistantMessage = data.choices[0].message.content;

// Check if the assistant wants to use a tool
try {
// Try to extract JSON from the message
const jsonMatch = assistantMessage.match(/\{[\s\S]*?"tool"[\s\S]*?\}/);
if (jsonMatch) {
const toolCall = JSON.parse(jsonMatch[0]);
if (toolCall.tool && availableTools.some((t) => t.name === toolCall.tool)) {
const toolResult = executeTool(toolCall.tool, toolCall.args || {});
toolsUsed.push(toolCall.tool);

messages.push({ role: 'assistant', content: assistantMessage });
messages.push({ role: 'user', content: `Tool result: ${toolResult}` });
continue;
}
}
} catch {
// Not a tool call, return the response
}

return { response: assistantMessage, toolsUsed };
}

return { response: 'Assistant reached maximum iterations', toolsUsed };
}

// AI assistant chat endpoint
router.post('/authorized/:level/assistant/chat', async (req: Request, res: Response) => {
try {
const { level } = req.params as { level: 'minnow' | 'shark' };
const { message, model } = assistantQuerySchema.parse(req.body);

const result = await runAssistant(message, model);
Comment on lines +82 to +87

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High

The endpoint extracts a level parameter from the URL path ('minnow' or 'shark'), suggesting tiered authorization, but this parameter is never used. All authenticated users get access to all 9 tools regardless of their authorization level, violating the principle of least privilege. This represents a complete bypass of the intended access control mechanism.

💡 Suggested Fix

Pass the authorization level to runAssistant and use it to filter available tools:

// Update function signature (L14-16)
async function runAssistant(
  userMessage: string,
  level: 'minnow' | 'shark',
  model?: string
): Promise<{ response: string; toolsUsed: string[] }> {

  // Filter tools based on level (L20-22)
  const readOnlyTools = ['list_properties', 'list_bookings', 'get_booking_details'];
  const allowedTools = level === 'minnow'
    ? availableTools.filter(t => readOnlyTools.includes(t.name))
    : availableTools;

  const systemPrompt = `You are a helpful AI property management assistant. You have access to the following tools:

${allowedTools.map((t) => `- ${t.name}: ${t.description}`).join('\n')}
...`;

  // Later, check tool calls against allowedTools (L62)
  if (toolCall.tool && allowedTools.some((t) => t.name === toolCall.tool)) {
    // ...
  }
}

// Pass level when calling (L87)
const result = await runAssistant(message, level, model);
🤖 AI Agent Prompt

The assistant endpoint at src/routes/assistant.ts:82-87 extracts an authorization level parameter but doesn't use it. The endpoint pattern /authorized/:level/assistant/chat suggests 'minnow' and 'shark' should have different permissions.

Investigate the intended authorization model:

  1. Compare with src/routes/chat.ts to understand how the main chat handler uses the level parameter
  2. Check if there's documentation or types defining what 'minnow' vs 'shark' permissions should be
  3. Determine appropriate tool sets: likely 'minnow' should be read-only (list_properties, list_bookings, get_booking_details) while 'shark' gets write operations

Implement the authorization control:

  1. Create a getToolsForLevel() function in src/services/assistantTools.ts that filters tools by level
  2. Update runAssistant to accept the level parameter
  3. Filter availableTools before constructing the system prompt
  4. Enforce the same filter when validating tool calls from the LLM

Ensure the filtering happens in the application layer, not just in the prompt, so it can't be bypassed by prompt injection.


Was this helpful?  👍 Yes  |  👎 No 


return res.json({
userMessage: message,
assistantResponse: result.response,
toolsUsed: result.toolsUsed,
});
} catch (error) {
if (error instanceof z.ZodError) {
return res.status(400).json({ error: 'Validation error', details: error.errors });
}
console.error('Assistant error:', error);
return res.status(500).json({
error: 'Internal server error',
message: error instanceof Error ? error.message : 'Unknown error',
});
}
});

// Get available tools (for documentation)
router.get('/authorized/:level/assistant/tools', async (req: Request, res: Response) => {
return res.json({ tools: availableTools });
});

export default router;
4 changes: 4 additions & 0 deletions src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import { chatHandler } from './routes/chat';
import { tokenHandler, jwksHandler } from './routes/oauth';
import { generateRSAKeyPair } from './utils/jwt-keys';
import { authenticateToken } from './middleware/auth';
import assistantRouter from './routes/assistant';

// Initialize OAuth key pair on startup
generateRSAKeyPair();
Expand All @@ -31,6 +32,9 @@ app.get('/health', (req: Request, res: Response) => {
app.post('/:level/chat', chatHandler);
app.post('/authorized/:level/chat', authenticateToken, chatHandler);

// AI property management assistant
app.use(assistantRouter);

// OAuth endpoints
app.post('/oauth/token', tokenHandler);
app.get('/.well-known/jwks.json', jwksHandler);
Expand Down
171 changes: 171 additions & 0 deletions src/services/assistantTools.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
import * as fs from 'fs';
import * as path from 'path';
import { AssistantState, Tool } from '../types/assistant';

const statePath = path.join(__dirname, '../data/assistant-state.json');

function loadState(): AssistantState {
return JSON.parse(fs.readFileSync(statePath, 'utf-8'));
}

function saveState(state: AssistantState): void {
fs.writeFileSync(statePath, JSON.stringify(state, null, 2));
}

export const availableTools: Tool[] = [
{
name: 'list_properties',
description: 'List all properties managed by the host',
parameters: {},
},
{
name: 'list_bookings',
description: 'List all bookings, optionally filtered by status',
parameters: {
status: {
type: 'string',
enum: ['pending', 'approved', 'declined', 'cancelled'],
optional: true,
},
},
},
{
name: 'get_booking_details',
description: 'Get detailed information about a specific booking',
parameters: {
bookingId: { type: 'string', required: true },
},
},
{
name: 'approve_booking',
description: 'Approve a pending booking request',
parameters: {
bookingId: { type: 'string', required: true },
},
},
{
name: 'decline_booking',
description: 'Decline a pending booking request',
parameters: {
bookingId: { type: 'string', required: true },
reason: { type: 'string', optional: true },
},
},
{
name: 'send_message_to_guest',
description: 'Send an email message to a guest',
parameters: {
guestEmail: { type: 'string', required: true },
subject: { type: 'string', required: true },
body: { type: 'string', required: true },
},
},
{
name: 'update_property_price',
description: 'Update the nightly rate for a property',
parameters: {
propertyId: { type: 'string', required: true },
newPrice: { type: 'number', required: true },
},
},
{
name: 'set_property_availability',
description: 'Set whether a property is available for booking',
parameters: {
propertyId: { type: 'string', required: true },
available: { type: 'boolean', required: true },
},
},
{
name: 'cancel_booking',
description: 'Cancel an existing booking',
parameters: {
bookingId: { type: 'string', required: true },
reason: { type: 'string', optional: true },
},
},
];

// Tool execution functions
export function executeTool(toolName: string, args: Record<string, any>): string {
const state = loadState();

switch (toolName) {
case 'list_properties':
return JSON.stringify(state.properties, null, 2);
Comment on lines +90 to +95

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 High

Tool arguments provided by the LLM are passed directly to execution functions without validation. An attacker using prompt injection can cause the LLM to generate tool calls with malicious parameters—arbitrary email addresses, extreme price values, or invalid booking IDs. The only check is whether the tool name exists in the availableTools list.

💡 Suggested Fix

Add Zod schema validation for all tool arguments before execution:

import { z } from 'zod';

const toolSchemas: Record<string, z.ZodSchema> = {
  list_properties: z.object({}),
  list_bookings: z.object({
    status: z.enum(['pending', 'approved', 'declined', 'cancelled']).optional(),
  }),
  get_booking_details: z.object({
    bookingId: z.string().regex(/^booking-\d{3}$/),
  }),
  approve_booking: z.object({
    bookingId: z.string().regex(/^booking-\d{3}$/),
  }),
  send_message_to_guest: z.object({
    guestEmail: z.string().email(),
    subject: z.string().min(1).max(200),
    body: z.string().min(1).max(2000),
  }),
  update_property_price: z.object({
    propertyId: z.string().regex(/^prop-\d{3}$/),
    newPrice: z.number().positive().max(10000),
  }),
  // ... other tools
};

export function executeTool(toolName: string, args: Record<string, any>): string {
  if (!toolSchemas[toolName]) {
    return `Unknown tool: ${toolName}`;
  }

  try {
    const validatedArgs = toolSchemas[toolName].parse(args);
    return executeToolInternal(toolName, validatedArgs);
  } catch (error) {
    if (error instanceof z.ZodError) {
      return `Invalid arguments: ${error.errors.map(e => e.message).join(', ')}`;
    }
    throw error;
  }
}

Additionally, for send_message_to_guest, validate that the email address exists in your booking records before sending.

🤖 AI Agent Prompt

At src/services/assistantTools.ts:90-95, the executeTool function executes tools based on LLM-provided arguments without validation. This compounds the prompt injection vulnerability.

Implement comprehensive input validation:

  1. Define Zod schemas for each tool's parameter structure (booking IDs, email formats, price ranges, etc.)
  2. Validate all arguments before the switch statement executes
  3. For send_message_to_guest (L130-141), add a whitelist check to ensure the email belongs to an actual guest in the system
  4. For update_property_price (L143-150), add reasonable bounds checking on price values
  5. Consider using TypeScript discriminated unions for type-safe tool definitions

Look at the tool parameter definitions (L15-87) and create corresponding runtime validation that matches those type signatures. Return clear error messages to the LLM when validation fails.


Was this helpful?  👍 Yes  |  👎 No 


case 'list_bookings': {
let bookings = state.bookings;
if (args.status) {
bookings = bookings.filter((b) => b.status === args.status);
}
return JSON.stringify(bookings, null, 2);
}

case 'get_booking_details': {
const booking = state.bookings.find((b) => b.id === args.bookingId);
return booking ? JSON.stringify(booking, null, 2) : 'Booking not found';
Comment on lines +97 to +107

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium

Guest PII (names and email addresses) is returned by these tools and fed back into the LLM context, which sends it to the external LLM provider. While this is standard behavior for LLM applications, vacation rental guests likely don't expect their personal information to be processed by OpenAI or other AI providers without explicit consent.

💡 Suggested Fix

Minimize PII exposure by returning only the information the LLM needs. The assistant doesn't need guest names/emails to approve bookings—it just needs booking IDs and dates:

case 'list_bookings': {
  let bookings = state.bookings;
  if (args.status) {
    bookings = bookings.filter((b) => b.status === args.status);
  }
  // Return minimal info without PII
  const minimalInfo = bookings.map(b => ({
    id: b.id,
    propertyId: b.propertyId,
    checkIn: b.checkIn,
    checkOut: b.checkOut,
    status: b.status,
    totalPrice: b.totalPrice,
    // Omit guestName and guestEmail
  }));
  return JSON.stringify(minimalInfo, null, 2);
}

case 'get_booking_details': {
  const booking = state.bookings.find((b) => b.id === args.bookingId);
  if (!booking) return 'Booking not found';

  // Return booking details without PII
  const { guestName, guestEmail, ...bookingInfo } = booking;
  return JSON.stringify(bookingInfo, null, 2);
}

When the LLM calls send_message_to_guest, the tool execution can access the full PII from the state file without exposing it to the LLM context.

🤖 AI Agent Prompt

The tools at src/services/assistantTools.ts:97-107 return booking data including guest names and email addresses, which flow through the LLM context to the external provider.

Evaluate what information the LLM actually needs:

  1. Review the assistant's tasks—can it approve/decline bookings using just booking IDs and dates?
  2. Check if guest PII is necessary for decision-making or if it's just being passed through
  3. Consider that when send_message_to_guest is called, the tool execution can look up the email from the state file without the LLM seeing it

Implement PII minimization:

  1. Modify list_bookings and get_booking_details to return booking information without guestName/guestEmail fields
  2. The LLM can still reference bookings by ID and make approval decisions
  3. When sending messages, the tool execution (which runs server-side) can retrieve the actual email address from state
  4. This way, PII never passes through the LLM context or gets sent to the AI provider

If PII exposure is necessary for functionality, add consent tracking and privacy documentation.


Was this helpful?  👍 Yes  |  👎 No 

}

case 'approve_booking': {
const booking = state.bookings.find((b) => b.id === args.bookingId);
if (!booking) return 'Booking not found';
if (booking.status !== 'pending')
return `Cannot approve booking with status: ${booking.status}`;
booking.status = 'approved';
saveState(state);
return `Booking ${args.bookingId} has been approved`;
}

case 'decline_booking': {
const booking = state.bookings.find((b) => b.id === args.bookingId);
if (!booking) return 'Booking not found';
if (booking.status !== 'pending')
return `Cannot decline booking with status: ${booking.status}`;
booking.status = 'declined';
saveState(state);
return `Booking ${args.bookingId} has been declined${args.reason ? `: ${args.reason}` : ''}`;
}

case 'send_message_to_guest': {
const message = {
to: args.guestEmail,
subject: args.subject,
body: args.body,
timestamp: new Date().toISOString(),
};
state.sentMessages.push(message);
saveState(state);
console.log(`[EMAIL SENT] To: ${args.guestEmail}, Subject: ${args.subject}`);
return `Message sent to ${args.guestEmail}`;
}

case 'update_property_price': {
const property = state.properties.find((p) => p.id === args.propertyId);
if (!property) return 'Property not found';
const oldPrice = property.nightlyRate;
property.nightlyRate = args.newPrice;
saveState(state);
return `Property ${property.name} price updated from $${oldPrice} to $${args.newPrice}`;
}

case 'set_property_availability': {
const property = state.properties.find((p) => p.id === args.propertyId);
if (!property) return 'Property not found';
property.available = args.available;
saveState(state);
return `Property ${property.name} availability set to ${args.available}`;
}

case 'cancel_booking': {
const booking = state.bookings.find((b) => b.id === args.bookingId);
if (!booking) return 'Booking not found';
booking.status = 'cancelled';
saveState(state);
return `Booking ${args.bookingId} has been cancelled${args.reason ? `: ${args.reason}` : ''}`;
}

default:
return `Unknown tool: ${toolName}`;
}
}
41 changes: 41 additions & 0 deletions src/types/assistant.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
export interface Tool {
name: string;
description: string;
parameters: Record<string, any>;
}

export interface ToolCall {
name: string;
arguments: Record<string, any>;
}

export interface Property {
id: string;
name: string;
nightlyRate: number;
available: boolean;
}

export interface Booking {
id: string;
propertyId: string;
guestName: string;
guestEmail: string;
checkIn: string;
checkOut: string;
status: 'pending' | 'approved' | 'declined' | 'cancelled';
totalPrice: number;
}

export interface SentMessage {
to: string;
subject: string;
body: string;
timestamp: string;
}

export interface AssistantState {
properties: Property[];
bookings: Booking[];
sentMessages: SentMessage[];
}