context-window

Smart context window management for LLM conversations. Automatically handles token counting, message pruning and context optimization.

Installation

pip install -r requirements.txt

Quick Start

from contextwindow import ContextManager

ctx = ContextManager(max_tokens=8000)

ctx.add_message("user", "Hello, how are you?")
ctx.add_message("assistant", "I'm doing well, thanks!")
ctx.add_message("user", "Can you help me with Python?")

# Get messages that fit in context window
messages = ctx.get_messages()

Features

• Accurate token counting using tiktoken
• Automatic message pruning when context exceeds limit
• Sliding window with configurable overlap
• System message preservation
• Summarization triggers for long conversations

API Reference

ContextManager

manager = ContextManager(
    max_tokens: int = 8000,
    model: str = "claude-3-sonnet",
    preserve_system: bool = True,
    summarize_threshold: float = 0.8
)

Methods

• add_message(role: str, content: str) - Add message to context
• get_messages() -> List[Dict] - Get messages within token limit
• count_tokens() -> int - Get current token count
• clear() - Clear all messages
• summarize() -> str - Generate summary of conversation

Configuration

# Customize token limits per model
ctx = ContextManager(
    max_tokens=16000,
    model="claude-3-opus"
)

Examples

Sliding window conversation

ctx = ContextManager(max_tokens=4000)

for message in long_conversation:
    ctx.add_message(message["role"], message["content"])

# Automatically prunes old messages
current = ctx.get_messages()

With summarization

ctx = ContextManager(max_tokens=8000, summarize_threshold=0.9)

# When context reaches 90%, triggers summarization
ctx.add_message("user", "...")
if ctx.should_summarize():
    summary = ctx.summarize()
    ctx.clear()
    ctx.add_message("system", f"Previous conversation summary: {summary}")

License

MIT