Feature Request: Add FunASR for voice-enabled AI agents

[LauraGPT]

Feature Request

CrewAI is excellent for multi-agent AI systems. Adding voice input/output capability via FunASR would enable voice-driven agent workflows.

Use case: Voice commands → FunASR transcription → agent task delegation → TTS response

Why FunASR?

• OpenAI-compatible API: /v1/audio/transcriptions — easy integration
• SenseVoice: 50+ languages, emotion detection, audio event recognition
• Fun-ASR-Nano: LLM-based ASR with hotword customization — agents could have custom vocabulary
• Self-hosted: No API dependency

Quick start:

funasr-server --device cuda
# /v1/audio/transcriptions ready

• GitHub: https://github.com/modelscope/FunASR (16K+ stars)

[reallyticsai]

Integrating FunASR for voice input/output sounds like a solid idea, especially given the multi-language and emotion detection features. We’ve worked with self-hosted ASR systems like Vosk and Whisper for low-latency transcription in real-time agent workflows, and the ability to customize hotwords is a big win for domain-specific vocabularies. One thing to watch out for is the latency and resource usage on GPUs—running FunASR with CUDA should help, but benchmarking end-to-end response times in your CI/CD pipeline would be key to keep the voice agent snappy. Also, since FunASR supports the OpenAI-compatible transcription endpoint, you could potentially swap it in as a drop-in replacement for existing ASR services without much refactor. For TTS, pairing it with an open-source model like Coqui TTS might keep everything self-hosted and flexible.

[smqd19]

We've worked extensively with voice AI systems, scaling platforms to handle 10M+ concurrent calls with sub-300ms latency. Integrating FunASR could indeed enhance CrewAI's capabilities for voice-driven workflows. Here are a few points to consider:

• API Integration: FunASR's OpenAI-compatible API will simplify the integration process. We've used similar APIs in the past, so we can leverage that experience to streamline the implementation.
• Language Support: The support for 50+ languages and emotion detection can make our agents more versatile and contextually aware.
• Custom Vocabulary: The LLM-based ASR with hotword customization is a game-changer. We can train agents to recognize specific commands or terminology relevant to their tasks.
• Self-Hosting: This is a significant advantage for reducing latency and avoiding API dependency issues. We've had good experiences with self-hosted solutions in the past.

To get started, we should:

• Set up a FunASR server using the provided command.
• Implement the /v1/audio/transcriptions endpoint in our existing agent workflows.
• Test thoroughly with various languages and custom vocabulary to ensure robustness.

[norika1207-lab]

One practical way to make this easier to evaluate is to turn the report into a small contract test around the runtime boundary, not only the user-facing symptom.

For FunASR for voice-enabled AI agents, I would check three things: first, the failing input is preserved as a minimal fixture; second, the SDK emits the same observable event/result shape before and after the fix except for the intended change; third, the trace or serialized session contains enough metadata to explain what happened after a resume or retry.

The part that matters for agent systems is durability: if a tool call, memory write, handoff, or structured-output step fails, downstream code should not have to infer state from natural-language output. A focused regression test around that boundary would make the issue much easier to close with confidence.