Seng/agent migration (#327)

* Update webhook URLs in sync-docs workflows to new endpoint

* new doc-agents repo

* Convert doc-agents from submodule to regular directory

* test

* fix query failure

* Add Documentation Q&A Agent

- Implemented a new Documentation Q&A agent that answers questions about Agentuity documentation using Retrieval-Augmented Generation (RAG).
- Created supporting files including agent logic, prompt handling, document retrieval, and type definitions.
- Added API route for the agent to handle requests and return structured responses.
- Updated the frontend to interact with the new agent, allowing users to ask questions about documentation directly.

* Refactor agent structure and add authentication middleware

- Removed the 'hello' agent and its associated files from the project.
- Updated API routes to remove references to the 'hello' agent.
- Introduced a new middleware for bearer token authentication to enhance security for API routes.
- Updated TypeScript configuration to include paths for middleware.

* Update agent configuration and refactor build logic

- Changed agentBaseUrl to use a local development URL for easier testing.
- Refactored buildAgentConfig to accept an endpoint instead of an agent ID.
- Updated getAgentQaConfig and getAgentPulseConfig to use specific API endpoints.
- Improved error messages for agent configuration validation.

* Update CLI commands to use bunx for execution

- Replaced instances of `agentuity` with `bunx @agentuity/cli` in various skill documentation files to standardize command execution.
- Updated the README to reflect the new total skills count and generation date.
- Removed the `agentuity-cli-upgrade` skill from the documentation as it is no longer applicable.

* Refactor API services and remove environment validation

- Replaced environment variable validation with direct API service calls for agent configuration.
- Updated API routes to utilize new service methods for querying agents and managing tutorial states.
- Refactored session management to improve error handling and streamline KV store interactions.
- Introduced a new title generation service for session titles based on conversation history.
- Removed deprecated environment validation logic and associated files to simplify the codebase.

* Update environment variables and enhance KV service initialization

- Added new environment variables: AGENT_BEARER_TOKEN and AGENTUITY_REGION to .env.example for improved configuration.
- Updated KV service initialization to require AGENTUITY_REGION alongside AGENTUITY_SDK_KEY for better error handling.
- Refactored logger implementation to utilize createLogger from the agentuity server package for improved logging capabilities.

* Enhance agent pulse functionality and improve tutorial state management

- Updated the agent pulse API to streamline message handling and response streaming.
- Introduced a new state manager to handle tutorial actions and fetch complete tutorial step data.
- Refactored the streaming processor to format AI responses as SSE-compatible chunks.
- Improved error handling and logging throughout the agent pulse services.
- Added tutorial data handling to provide users with relevant content during interactions.

* Implement session management API and refactor session handling

- Introduced a new API for managing user sessions, including endpoints for creating, retrieving, updating, and deleting sessions.
- Added message handling within sessions to allow users to append messages to existing sessions.
- Implemented cookie-based authentication middleware to secure session endpoints.
- Refactored session persistence logic to streamline data storage and retrieval.
- Enhanced error handling and logging for improved debugging and user feedback.

* Rename sync job to full-sync in GitHub Actions workflow for clarity

* Remove pull_request trigger from full docs sync workflow in GitHub Actions

* Enhance middleware and API authentication for Agentuity integration

- Implemented middleware to handle user session management and proxy API requests to the Agentuity backend with bearer token authentication.
- Updated session and agent pulse API routes to utilize bearer token authentication alongside cookie authentication.
- Refactored API rewrites to be managed within middleware, improving configuration clarity.
- Added cookie management for user sessions and ensured proper error handling for missing configuration.
- Updated documentation and comments for clarity on authentication mechanisms.

* Update .env.example to remove duplicate AGENT_BEARER_TOKEN entry

* Remove deprecated files and update package dependencies

- Deleted obsolete files from the agent-docs directory, including configuration, documentation, and source files.
- Updated package.json to reflect the latest dependencies for the project.

* Update authentication mechanism in workflows and API route

- Replaced AGENTUITY_TOKEN with AGENT_BEARER_TOKEN in GitHub Actions workflows for consistency.
- Enhanced API route to include bearer token authentication middleware for improved security.

* Update environment configuration and clean up logging in agent pulse

- Modified .env.example files to clarify usage of AGENT_BEARER_TOKEN and AGENTUITY_SDK_KEY.
- Removed unnecessary timestamp fields from message schema and related API routes, ensuring server-side timestamps are generated consistently.
- Cleaned up logging statements in agent pulse files to reduce verbosity and improve clarity.

* Refactor agent pulse and document QA components

- Updated streaming processor and tools for improved event handling.
- Enhanced types for agent pulse and document QA to include additional properties.
- Cleaned up code in the document QA agent and retriever for better readability and maintainability.
- Adjusted session persistence logic to streamline message handling within sessions.

Author: afterrburn

.env.example

@@ -1,6 +1,6 @@
 # Agent Configuration
 AGENT_BASE_URL=http://127.0.0.1:3500
-AGENT_BEARER_TOKEN=
+AGENTUITY_REGION=use
 
-# KV-store is hitting Agentuity Service API, this can be found in `agent-docs` after running `agentuity dev`
-AGENTUITY_API_KEY=
+# This is for Bearer token auth
+AGENT_BEARER_TOKEN=

.github/workflows/sync-docs-full.yml

@@ -4,7 +4,7 @@ on:
   workflow_dispatch:
 
 jobs:
-  sync:
+  full-sync:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
@@ -19,9 +19,9 @@ jobs:
 
       - name: Build and send payload
         env:
-          AGENTUITY_TOKEN: ${{ secrets.AGENTUITY_TOKEN }}
+          AGENT_BEARER_TOKEN: ${{ secrets.AGENT_BEARER_TOKEN }}
         run: |
           set -euo pipefail
           cat all-files.txt | \
             ./bin/build-payload.sh "${{ github.repository }}" full | \
-            ./bin/send-webhook.sh "https://agentuity.ai/webhook/f61d5ce9d6ed85695cc992c55ccdc2a6" "Bearer $AGENTUITY_TOKEN"
+            ./bin/send-webhook.sh "https://p0f83a312791b60ff.agentuity.run/api/process-docs" "Bearer $AGENT_BEARER_TOKEN"

.github/workflows/sync-docs.yml

@@ -62,9 +62,9 @@ jobs:
 
       - name: Build and send payload
         env:
-          AGENTUITY_TOKEN: ${{ secrets.AGENTUITY_TOKEN }}
+          AGENT_BEARER_TOKEN: ${{ secrets.AGENT_BEARER_TOKEN }}
         run: |
           set -euo pipefail
           cat changed-files.txt | \
             ./bin/build-payload.sh "${{ github.repository }}" incremental | \
-            ./bin/send-webhook.sh "https://agentuity.ai/webhook/f61d5ce9d6ed85695cc992c55ccdc2a6" "Bearer $AGENTUITY_TOKEN"
+            ./bin/send-webhook.sh "https://p0f83a312791b60ff.agentuity.run/api/process-docs" "Bearer $AGENT_BEARER_TOKEN"

README.md

@@ -37,6 +37,19 @@ To make the search feature work, you must set up `.env.local` with the following
 4. **Update `AGENTUITY_API_KEY`:**
    If you are a contributor from outside the Agentuity organization, ensure that you update the `AGENTUITY_API_KEY` in your `.env.local` file with your specific SDK key from the `agentuity dev` run.
 
+## GitHub Actions Secrets
+
+The following secrets need to be configured in GitHub repository settings for the CI/CD workflows to function:
+
+| Secret | Description |
+|--------|-------------|
+| `AGENT_BEARER_TOKEN` | Bearer token for authenticating with the `process-docs` API endpoint. This must match the `AGENT_BEARER_TOKEN` environment variable configured in your Agentuity deployment. |
+
+To set up:
+1. Go to your GitHub repository → Settings → Secrets and variables → Actions
+2. Click "New repository secret"
+3. Add `AGENT_BEARER_TOKEN` with the same value configured in the Agentuity environment
+
 ## Running Docs Application
 
 ```bash