Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@ uv pip install pip && uv run mypy --install-types --non-interactive \
-p livekit.plugins.sarvam \
-p livekit.plugins.inworld \
-p livekit.plugins.simli \
-p livekit.plugins.anam
-p livekit.plugins.anam \
-p livekit.plugins.inyaai \
Comment on lines +95 to +96
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify the correct plugin module names in livekit-plugins/

echo "=== Listing livekit-plugins directories ==="
ls -la livekit-plugins/ 2>/dev/null | head -20

echo -e "\n=== Searching for plugin directories matching 'anam', 'inyaai', or 'gnani' ==="
fd -type d "(anam|inyaai|gnani)" livekit-plugins/

echo -e "\n=== Checking for pyproject.toml or setup.py in potential plugin dirs ==="
fd -type f "(pyproject.toml|setup.py)" livekit-plugins/ | grep -E "(anam|inyaai|gnani)" | head -20

echo -e "\n=== Contents of plugin __init__.py files ==="
fd -type f "^__init__.py$" livekit-plugins/livekit-plugins-anam --exec echo "File: {}" \; --exec head -5 {} \;
fd -type f "^__init__.py$" livekit-plugins/livekit-plugins-gnani --exec echo "File: {}" \; --exec head -5 {} \;

Repository: livekit/agents

Length of output: 2358


🏁 Script executed:

#!/bin/bash
# Better search for plugin packages

echo "=== All plugin directories in livekit-plugins/ ==="
ls -1 livekit-plugins/ | grep -E "(anam|inyaai|gnani)" 

echo -e "\n=== Full list of livekit-plugins directories (first 60) ==="
ls -1 livekit-plugins/

echo -e "\n=== Search for 'inyaai' anywhere in repo ==="
grep -r "inyaai" . --include="*.md" --include="*.py" --include="*.toml" 2>/dev/null | head -20

echo -e "\n=== Search for 'gnani' anywhere in repo ==="
grep -r "gnani" . --include="*.md" --include="*.py" --include="*.toml" 2>/dev/null | head -20

echo -e "\n=== View CONTRIBUTING.md around lines 95-96 ==="
sed -n '90,100p' CONTRIBUTING.md

Repository: livekit/agents

Length of output: 3735


🏁 Script executed:

#!/bin/bash
# Check gnani plugin structure and module definition

echo "=== Contents of livekit-plugins-gnani/livekit/plugins/gnani/__init__.py ==="
cat livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/__init__.py

echo -e "\n=== Check if there are any aliases or inyaai references in gnani ==="
grep -r "inyaai" livekit-plugins/livekit-plugins-gnani/ 2>/dev/null

echo -e "\n=== Check test_plugin_inyaai_stt.py imports and assertions ==="
cat tests/test_plugin_inyaai_stt.py | head -30

Repository: livekit/agents

Length of output: 2433


Change livekit.plugins.inyaai to livekit.plugins.gnani.

The plugin module is livekit.plugins.gnani (from the livekit-plugins-gnani package), not livekit.plugins.inyaai. This reference will cause mypy to fail. Note that tests/test_plugin_inyaai_stt.py also has the incorrect import path and will need to be updated.

🤖 Prompt for AI Agents
In `@CONTRIBUTING.md` around lines 95 - 96, Update the incorrect plugin module
name `livekit.plugins.inyaai` to `livekit.plugins.gnani` wherever referenced
(e.g., the two lines in CONTRIBUTING.md and the import in
tests/test_plugin_inyaai_stt.py); change the import path in the test from `from
livekit.plugins.inyaai...` to `from livekit.plugins.gnani...` and update any
test identifiers or docstrings that reference "inyaai" so they match the gnani
package naming.

```

### Linting
Expand Down
135 changes: 135 additions & 0 deletions livekit-plugins/livekit-plugins-gnani/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
# LiveKit Gnani Plugin

This plugin provides Speech-to-Text functionality for LiveKit Agents using Gnani's (Vachana) multilingual STT engine with strong support for Indian languages.

## Installation

```bash
pip install livekit-plugins-gnani
```

## Features

- **Multilingual Support**: Transcribe audio in multiple Indian languages including Hindi, Tamil, Telugu, Kannada, Bengali, Gujarati, Marathi, Malayalam, Punjabi, and English (Indian accent)
- **Code-Switching**: Support for mixed language conversations (e.g., English-Hindi)
- **High-Quality Transcription**: Optimized for Indian accents and languages
- **Simple REST API**: Easy to integrate non-streaming transcription

## Supported Languages

- `en-IN` - English (India)
- `hi-IN` - Hindi
- `gu-IN` - Gujarati
- `ta-IN` - Tamil
- `kn-IN` - Kannada
- `te-IN` - Telugu
- `mr-IN` - Marathi
- `bn-IN` - Bengali
- `ml-IN` - Malayalam
- `pa-IN` - Punjabi
- `en-IN,hi-IN` - English-Hindi (code-switching)

## Configuration

You'll need to set up your Gnani credentials. You can do this via environment variables:

```bash
export GNANI_API_KEY="your-api-key"
export GNANI_ORG_ID="your-organization-id"
export GNANI_USER_ID="your-user-id"
```

Or pass them directly when initializing the STT instance.

## Usage

### Basic Usage

```python
from livekit.agents import AutoSubscribe, JobContext, WorkerOptions, cli, llm
from livekit.plugins import gnani

async def entrypoint(ctx: JobContext):
# Initialize Gnani STT
stt_instance = gnani.STT(
language="hi-IN", # Hindi
)

# Use with voice assistant
assistant = VoiceAssistant(
vad=silero.VAD.load(),
stt=stt_instance,
llm=llm.openai.LLM(),
tts=tts.openai.TTS(),
)

await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
assistant.start(ctx.room)

if __name__ == "__main__":
cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
```

### With Explicit Credentials

```python
from livekit.plugins import gnani

stt_instance = gnani.STT(
language="en-IN",
api_key="your-api-key",
organization_id="your-org-id",
user_id="your-user-id",
)
```

### Code-Switching (Multiple Languages)

```python
from livekit.plugins import gnani

# Support English-Hindi code-switching
stt_instance = gnani.STT(
language="en-IN,hi-IN",
)
```

### Direct Recognition

```python
from livekit.plugins import gnani
from livekit import rtc

stt_instance = gnani.STT(language="ta-IN") # Tamil

# Recognize from audio buffer
event = await stt_instance.recognize(buffer=audio_frames)
print(f"Transcript: {event.alternatives[0].text}")
```

## API Details

The Gnani STT plugin uses the Gnani Vachana API v3 endpoint:

- **Endpoint**: `https://api.vachana.ai/stt/v3`
- **Method**: POST (multipart/form-data)
- **Authentication**: Via API headers
- **Audio Formats**: WAV, MP3, FLAC, OGG, M4A
- **Sample Rates**: 8 kHz – 44.1 kHz (mono recommended)
- **Max Duration**: Up to 60 seconds per request

## Limitations

- **Non-Streaming**: This plugin uses the REST API which does not support streaming recognition. For streaming use cases, consider using `StreamAdapter` with a VAD.
- **Audio Duration**: Maximum 60 seconds per request
- **No Interim Results**: Only final transcripts are returned

## Links

- [LiveKit Documentation](https://docs.livekit.io)
- [Gnani Vachana Platform](https://vachana.ai)
- [GitHub Repository](https://github.com/livekit/agents)

## License

Apache License 2.0
2 changes: 2 additions & 0 deletions livekit-plugins/livekit-plugins-gnani/livekit/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Namespace package
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Namespace package
__path__ = __import__("pkgutil").extend_path(__path__, __name__)
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Copyright 2025 LiveKit, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Gnani (Vachana) plugin for LiveKit Agents

This plugin provides Speech-to-Text functionality using Gnani's multilingual STT engine,
with support for Indian languages.

See https://vachana.ai for more information.
"""

from .stt import STT
from .version import __version__

__all__ = ["STT", "__version__"]


from livekit.agents import Plugin

from .log import logger


class GnaniPlugin(Plugin):
def __init__(self) -> None:
super().__init__(__name__, __version__, __package__, logger)


Plugin.register_plugin(GnaniPlugin())

# Cleanup docs of unexported modules
_module = dir()
NOT_IN_ALL = [m for m in _module if m not in __all__]

__pdoc__ = {}

for n in NOT_IN_ALL:
__pdoc__[n] = False
17 changes: 17 additions & 0 deletions livekit-plugins/livekit-plugins-gnani/livekit/plugins/gnani/log.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Copyright 2025 LiveKit, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import logging

logger = logging.getLogger("livekit.plugins.gnani")
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Copyright 2025 LiveKit, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

from typing import Literal

GnaniLanguages = Literal[
"en-IN", # English (India)
"hi-IN", # Hindi
"gu-IN", # Gujarati
"ta-IN", # Tamil
"kn-IN", # Kannada
"te-IN", # Telugu
"mr-IN", # Marathi
"bn-IN", # Bengali
"ml-IN", # Malayalam
"pa-IN", # Punjabi
"en-IN,hi-IN", # English-Hindi (code-switching)
]
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Marker file for PEP 561
Loading