Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions examples/drive-thru/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,49 @@
A complete drive-thru ordering system demonstrating interactive voice agents for food ordering with database integration and order management.

For setup instructions and more details, see the [main examples README](../README.md).

## Overview

This example simulates a fast food drive-thru. It is split across three files: `database.py` contains the menu and formats it as system prompt text, `order.py` holds Pydantic models for the three order types, and `agent.py` defines `DriveThruAgent` with dynamically built ordering tools.

The full menu is loaded once per session and injected directly into the agent's instructions, so the LLM has menu context without needing to call a tool.

### Menu Loading

At the start of each session, `new_userdata()` queries `FakeDB` for all item categories (drinks, combos, Happy Meals, regulars, sauces) and stores them in the `Userdata` dataclass alongside a fresh `OrderState`.
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/drive-thru/agent.py#L382-L399
`DriveThruAgent.__init__` then formats each category using `menu_instructions()` and concatenates the results with `COMMON_INSTRUCTIONS` to build the full system prompt. This means the LLM sees the entire menu from the first turn and can answer questions or suggest items without any tool calls.
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/drive-thru/agent.py#L55-L83
### Dynamic Tool Building

The three ordering tools are constructed by `build_combo_order_tool`, `build_happy_order_tool`, and `build_regular_order_tool`. Each method closes over the relevant item lists and injects their IDs as the `enum` constraint in the tool's JSON schema.

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/drive-thru/agent.py#L85-L119

This restricts the LLM to known IDs at the schema layer before any runtime logic runs. `ToolError` handles the cases that can't be caught statically — for example, when a drink has multiple available sizes and the customer hasn't specified one yet, the tool raises a `ToolError` prompting the agent to ask for clarification before retrying.

### Order Types

`order.py` defines three Pydantic models: `OrderedCombo`, `OrderedHappy`, and `OrderedRegular` . A discriminated union `OrderedItem` is also defined. Each ordered item receives a random short `order_id` on creation via `order_uid()`.

`OrderState` stores the current cart as a `dict[str, OrderedItem]` keyed by `order_id`, which the `remove_order_item` and `list_order_items` tools use to look up or modify existing items.
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/drive-thru/order.py#L45-L56

### Managing the Order

Two tools handle cart management:

- `list_order_items` returns all current cart items with their `order_id`s. The agent is instructed to call this first when modifying or removing an item whose `order_id` is unknown.
- `remove_order_item` removes one or more items by `order_id`. Modifications (e.g., upsizing fries) are done by removing the old item and re-adding it with the new parameters.

`max_tool_steps=10` is set on the session to give the agent enough budget to call `list_order_items` followed by `remove_order_item` in a single turn when needed.

### Background Audio

`BackgroundAudioPlayer` plays an ambient drive-thru noise track (`bg_noise.mp3`) throughout the session to set the scene.
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/drive-thru/agent.py#L438-L443

### STT Tuning

The STT model is also initialized with `keyterm` hints for McDonald's brand names (e.g., `"Big Mac"`, `"McFlurry"`, `"McCrispy"`) to improve transcription accuracy.
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/drive-thru/agent.py#L415-L430
37 changes: 37 additions & 0 deletions examples/frontdesk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,40 @@
A front desk agent demonstrating customer service with calendar integration and appointment management.

For setup instructions and more details, see the [main examples README](../README.md).

## Overview

In this example, you will be able to schedule appointments (optionally with cal.com's API if `CAL_API_KEY` is set) and evaluate the agent's performance using `JudgeGroup`. The session will always begin with the agent saying "Hello, I can help you schedule an appointment."

### Scheduling appointments

The LLM will call list_available_slots before `schedule_appointment`, since `slot_id` is a required argument.

`list_available_slots` will return slots like:

```bash
ST_abc123 - Saturday, January 1, 2000 at 14:00 PDT (in 5 days)
```

The slots are also cached as a lookup table for `schedule_appointment`.

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/frontdesk/frontdesk_agent.py#L184


This prevents the LLM from passing a hallucinated answer.

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/frontdesk/frontdesk_agent.py#L94-L95


The user's email is then collected via `GetEmailTask()`. If the agent is interrupted after the task completes, `schedule_appointment` is aborted before an API call is made to book the slot. After the task, the function is uninterruptible.

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/frontdesk/frontdesk_agent.py#L97-L119


### Evaluations

After the session ends, we use a `JudgeGroup` with pre-built judges to score the conversation.

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/frontdesk/frontdesk_agent.py#L200-L214

When the success criteria for an agent is clear, using judges can complete the evaluation by measuring the performance quality.
2 changes: 1 addition & 1 deletion examples/frontdesk/frontdesk_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ def __init__(self, *, timezone: str) -> None:
self._slots_map: dict[str, AvailableSlot] = {}

async def on_enter(self) -> None:
await self.session.say("hello, I can help you to schedule an appointment")
await self.session.say("Hello, I can help you schedule an appointment!")

@function_tool
async def schedule_appointment(
Expand Down
46 changes: 46 additions & 0 deletions examples/healthcare/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Healthcare Example

A full healhcare assistant providing secure appointment management, billing handling, and lab result retrieval.

For setup instructions and more details, see the [main examples README](https://github.com/livekit/agents/blob/main/examples/README.md).

## Overview

The healthcare agent utilizes a variety of `AgentTasks` to achieve structured workflows to collect information. This example is modality-agnostic, where users can interact via text or voice and switch seamlessly. If the conversation heads out of the scope of the agent, the user will be transfered to a human.

### Profile Authentication

Before any sensitive information is queried, the user must go through an authentication process. This process will only occur once per call. If the user provides a name and birthday existing in the database, the process is fast-forwarded. This is possible via task completion callbacks:
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/healthcare/healthcare_agent.py#L574-L588
Otherwise, the user will also be asked for their phone number and insurance provider
to create a profile.

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/healthcare/healthcare_agent.py#L590-L641

After the profile is created, the agent will be given a tool to update the patient's record as needed.

### Appointment Management

Function tools are added dynamically, so the LLM cannot hallucinate parameters or call tools prematurely. The user is given options for compatible doctors based on their insurance:

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/healthcare/healthcare_agent.py#L306-L333

After the doctor is chosen, the appointment scheduling tool is built dynamically with the availabilities.
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/healthcare/healthcare_agent.py#L734-L780

Finally, the visit reason is collected, and once confirmed the database will be updated accordingly (the doctor's availability will be removed).

Users are also able to modify existing appointments. If the user wishes to reschedule an appointment, the appointment is canceled and `ScheduleAppointmentTask` is reused.

https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/healthcare/healthcare_agent.py#L506-L543

### Billing Handling

`GetCreditCardTask()` is showcased here. The user's details are verified, and a balance is generated and connected to their profile.
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/healthcare/healthcare_agent.py#L734-L748

### Lab Result Retrieval

We use OpenAI's provider tool, `FileSearch`, to read through an uploaded lab report (mock_checkup_report.pdf).
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/healthcare/healthcare_agent.py#L734-L780

57 changes: 55 additions & 2 deletions examples/survey/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,58 @@
# Survey Example
# Survey Agent

Example showing how to build an automated voice survey agent.
Screen a candidate for a software engineer role to see if they meet the prerequisites and are an overall good fit. The responses, summary, and evaluation will be written to a CSV file.

For setup instructions and more details, see the [main examples README](../README.md).

## Overview

The flow of this agent is flexibly structured, where the specified sequence is maintained but the user is able to regress to a previously visited task if needed. This is possible via `TaskGroup`, which is set up here: https://github.com/livekit/agents/blob/f8efe436afe2470104ce7587f1d89ae383ed619e/examples/survey/survey_agent.py#L285-L315

### IntroTask
This stage facilitates introductions and collects the candidate’s name.

### GetEmailTask
This task is built in to our framework. By default, it can collect and update emails and mark when a user doesn’t want to give their email. If the input modality is audio, emails are confirmed before the task is marked as complete. See the [docs for GetEmailTask](https://docs.livekit.io/agents/prebuilt/tasks/get-email/).

### CommuteTask
This stage collects whether or not the candidate can commute to the office and their method of transportation. We define the possible commute methods here:
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/survey/survey_agent.py#L32

And we pass this to a function tool like so:
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/survey/survey_agent.py#L231-L237

### ExperienceTask
This stage collects the candidate’s years of experience and a short description of their professional career. It follows a structure similar to `IntroTask` and `CommuteTask`.

### BehavioralTask
For some tasks, you might not want a structured flow of questions. In this stage, we are collecting the candidate’s strengths, weaknesses, and work style. This task incrementally collects answers in no particular order. This allows for a more natural conversation.

After the candidate answers one of the questions, `self._check_completion()` is called to check if all 3 fields (`”strengths”`, `“weaknesses”`, `“work_style”`) have been collected. If so, then `BehavioralTask` is marked as complete. If not, then the agent will continue prompting for the rest of the answers.

In practice, this would ensure variability among candidates’ experiences.

### Closing out
Once the interview is concluded and TaskGroup is completed, we extract the summary message (the last inserted message):

```python
summary = self.chat_ctx.items[-1]
```

And we generate a candidate evaluation based off of the summary:

```python
evaluation = await evaluate_candidate(llm_model=self.session.llm, summary=summary)
```

The session LLM evaluates the candidate from the given summary:
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/survey/survey_agent.py#L76-L98


Finally, the agent hangs up and you can find the results, summary, and evaluation in `results.csv`!

### Disqualification
In each stage after the first, the candidate may be disqualified for unsatisfactory answers or for refusing to answer. We create a function tool that will be passed to the tasks:
https://github.com/livekit/agents/blob/8283a5a5c9863a07bcf030ee90e8ab780e1e569b/examples/survey/survey_agent.py#L101-L118

The candidate will be informed of the interview ending, and then the session will shut down.

Loading