AI Agents with AI Toolkit: Design. Build. Evaluate.

[juliamuiruri4]

Welcome to the AI Toolkit discussion hub! 🎉

This is a dedicated space for our community to collaborate, learn, and explore building intelligent agents with the AI Toolkit for VS Code - a powerful extension that makes AI agent development efficient, intuitive, and native to your coding workflow.

Use This Discussion For:

• 💬 AI Toolkit Help – Extension setup, Model Catalog navigation, Playground exploration, agent creation
• 🏗️ Agent Architecture – Designing agents with MCP tools, system prompt engineering, variable management
• 🔧 Tool Integration – Creating custom MCP servers, integrating specialized tools, connecting external APIs
• 📊 Evaluation Strategies – Building evaluation metrics, measuring agent performance, prompt optimization

Getting Started:

New to AI Toolkit? Start here:

• Design & Build an AI Agent Livestream
• AI Toolkit Quest: Comprehensive overview of building the Cora Customer Support Agent

Community Guidelines:

• Be respectful and inclusive
• Search existing threads before posting duplicates
• Share code snippets and error logs when asking for help
• Help others! Your expertise matters

Have a question, project or insight to share? Start a new reply or thread below!

Discussion host:

[Nyrok]

One thing that's helped me a lot with agent design and evaluation: treating the system prompt as typed semantic blocks rather than one flat string.

When you define separate blocks for role, constraints, output format, and examples, you can swap one block out and rerun evaluation without touching the others. That makes it much easier to identify what change actually moved the needle. Free-form prompts make this hard because everything is tangled.

I built flompt (https://flompt.dev) for this exact workflow: visual canvas that decomposes prompts into 12 typed blocks and compiles to Claude-optimized XML. Open-source: github.com/Nyrok/flompt