Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
124732a
Merge pull request #94 from GeekInTheNorth/main
GeekInTheNorth Aug 7, 2025
2c88431
Create tool to get robots configurations.
GeekInTheNorth Aug 20, 2025
70411b6
Update the discovery endpoint
GeekInTheNorth Aug 20, 2025
3efe102
Correct issue with Static Action not responding as desired.
GeekInTheNorth Aug 20, 2025
4e1042f
Update Json Property Names
GeekInTheNorth Aug 21, 2025
0faf1e9
Handle opal tools having relative paths.
GeekInTheNorth Aug 21, 2025
9626dd7
add initial controllers and token management
GeekInTheNorth Aug 21, 2025
82dfac4
Add in repository and basic crud operations for tokens
GeekInTheNorth Aug 21, 2025
b0972ea
Enforce bearer token and scope restrictions.
GeekInTheNorth Aug 24, 2025
eb0943f
Fix filtering of the robots-txt-configurations endpoint.
GeekInTheNorth Aug 24, 2025
2a89914
Add Save tool
GeekInTheNorth Aug 24, 2025
f8a820a
Update tokens so they can create/read/delete only.
GeekInTheNorth Aug 25, 2025
8513c15
Correct permissions for the discover api
GeekInTheNorth Aug 26, 2025
8e39222
Update the content of the opal token management screen
GeekInTheNorth Aug 28, 2025
5ddbd89
Add endpoints for editing llms.txt Content
GeekInTheNorth Sep 6, 2025
0efcc85
Add functionality for managing llms.txt content in opal
GeekInTheNorth Sep 9, 2025
7f91c32
Update Opal Tools to be API Tokens and adjust the UI of the API descr…
GeekInTheNorth Sep 10, 2025
b54af3e
Modify Opal Get content APIs to return data for each host.
GeekInTheNorth Sep 11, 2025
8c40755
Handle creating of content where a robots or llms content cannot be p…
GeekInTheNorth Sep 14, 2025
854306d
Remove development token
GeekInTheNorth Sep 14, 2025
84ffe04
Improve unit test coverage.
GeekInTheNorth Sep 14, 2025
1ea5cc5
Add additional unit tests.
GeekInTheNorth Sep 15, 2025
3f41468
Update src/Stott.Optimizely.RobotsHandler/Opal/OpalLlmsApiController.cs
GeekInTheNorth Sep 15, 2025
58c22af
Fix unit tests
GeekInTheNorth Sep 15, 2025
77875c2
Update all packages to the latest versions.
GeekInTheNorth Sep 15, 2025
1ed8243
Add in buttons for copying the endpoint urls.
GeekInTheNorth Sep 15, 2025
04c5f1a
Merge pull request #95 from GeekInTheNorth/feature/opal-tools
GeekInTheNorth Sep 15, 2025
14d90cf
Hash Opal Tokens for the APIs.
GeekInTheNorth Sep 16, 2025
55a8b5e
Fix deterministic issues with the salt
GeekInTheNorth Sep 16, 2025
ce4c7d5
Fix tests for salting hashes
GeekInTheNorth Sep 16, 2025
68ff391
Fix PR issues
GeekInTheNorth Sep 16, 2025
827ceb0
Update redirect rules in the sample project
GeekInTheNorth Sep 16, 2025
668a25e
Merge pull request #96 from GeekInTheNorth/feature/hash_opal_tokens
GeekInTheNorth Sep 16, 2025
eb95cc8
Package up as version 6.0.0
GeekInTheNorth Sep 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions Opal/Discovery.http
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
###
# Opal Discovery API Tests
# Tests for the /stott.robotshandler/opal/discovery/ endpoint
###

# Variables - Change these as needed
@baseUrl = https://localhost:44344
@apiPath = /stott.robotshandler/opal

# Test 1: Basic Discovery API call (IIS Express)
GET {{baseUrl}}{{apiPath}}/discovery/
Accept: application/json
Content-Type: application/json

###

# Test 2: Discovery API with query parameters (should still work even if not used)
GET {{baseUrl}}{{apiPath}}/discovery/?test=true
Accept: application/json
Content-Type: application/json

###

# Test 3: Discovery API with different Accept headers
GET {{baseUrl}}{{apiPath}}/discovery/
Accept: */*
Content-Type: application/json

###

# Test 4: Discovery API without Content-Type header
GET {{baseUrl}}{{apiPath}}/discovery/
Accept: application/json

###

# Expected Response Structure:
# The API should return a JSON object with the following structure:
# {
# "functions": [
# {
# "name": "getrobottxtconfigurations",
# "description": "Get a collection of robot.txt configurations optionally filtered by host name.",
# "parameters": [...],
# "endpoint": "/tools/get-robot-txt-configurations/",
# "httpMethod": "POST"
# },
# {
# "name": "saverobottxtconfigurations",
# "description": "Saves robots.txt content for a specific Id or Host Name...",
# "parameters": [...],
# "endpoint": "/tools/save-robot-txt-configuration/",
# "httpMethod": "POST"
# },
# {
# "name": "getllmstxtconfigurations",
# "description": "Get a collection of llms.txt configurations optionally filtered by host name.",
# "parameters": [...],
# "endpoint": "/tools/get-llms-txt-configurations/",
# "httpMethod": "POST"
# },
# {
# "name": "savellmstxtconfigurations",
# "description": "Saves llms.txt content for a specific Id or Host Name...",
# "parameters": [...],
# "endpoint": "/tools/save-llms-txt-configuration/",
# "httpMethod": "POST"
# }
# ]
# }
270 changes: 270 additions & 0 deletions Opal/Llms.http
Original file line number Diff line number Diff line change
@@ -0,0 +1,270 @@
###
# Opal LLMS API Tests
# Tests for the OpalLlmsApiController endpoints
# LLMS.TXT is used to provide information about Large Language Models and their usage
###

# Variables - Change these as needed
@baseUrl = https://localhost:44344
@apiPath = /stott.robotshandler/opal
@token = put-bearer-token-here
@testHostName = localhost:44344

# For production testing, uncomment these instead:
# @baseUrl = https://test-cms.stott.pro
# @testHostName = example.com

###

### Get LLMS Txt Configurations - All configurations
POST {{baseUrl}}{{apiPath}}/tools/get-llms-txt-configurations/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {}
}

###

### Get LLMS Txt Configurations - Filter by hostname
POST {{baseUrl}}{{apiPath}}/tools/get-llms-txt-configurations/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "{{testHostName}}"
}
}

###

### Get LLMS Txt Configurations - Filter by example.com
POST {{baseUrl}}{{apiPath}}/tools/get-llms-txt-configurations/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "example.com"
}
}

###

### Get LLMS Txt Configurations - Discovery route
POST {{baseUrl}}{{apiPath}}/discovery/tools/get-llms-txt-configurations/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "{{testHostName}}"
}
}

###

### Save LLMS Txt Configuration - By LlmsId
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"llmsId": "d994532b-02df-42dd-93ef-18bee7c3346f",
"llmsTxtContent": "# LLMS.TXT\n\n# Model Information\nmodel-name: GPT-4\nmodel-version: 2024-03\ntraining-data-cutoff: 2023-04\n\n# Usage Guidelines\nallowed-uses: content-generation, code-assistance, research\nrestricted-uses: personal-data-processing, medical-diagnosis\n\n# Contact Information\ncontact: ai-team@example.com\nwebsite: https://example.com/ai-policy"
}
}

###

### Save LLMS Txt Configuration - By HostName
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "{{testHostName}}",
"llmsTxtContent": "# LLMS.TXT - AI Model Information\n\n# Primary Model\nmodel-name: Claude-3\nmodel-version: Opus\ntraining-data-cutoff: 2024-02\n\n# Capabilities\ncapabilities: text-generation, analysis, coding, creative-writing\n\n# Limitations\nlimitations: no-real-time-data, no-internet-access, knowledge-cutoff-applies\n\n# Usage Policy\ncommercial-use: allowed\ndata-retention: 30-days\n\n# Contact\ncontact: support@newsite.example.com"
}
}

###

### Save LLMS Txt Configuration - Discovery route
POST {{baseUrl}}{{apiPath}}/discovery/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "{{testHostName}}",
"llmsTxtContent": "# LLMS.TXT - Test Environment\nDiscovery Route Example\n# Development Models\nmodel-name: GPT-3.5-Turbo\nmodel-version: test\nenvironment: development\n\n# Test Configuration\npurpose: testing-and-development\ndata-persistence: none\n\n# Restrictions\nproduction-use: forbidden\nexternal-access: blocked\n\n# Contact\ncontact: dev-team@test.example.com"
}
}

###

### Save LLMS Txt Configuration - Minimal llms.txt
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "{{testHostName}}",
"llmsTxtContent": "# LLMS.TXT\nmodel-name: Basic-LLM\ncontact: info@minimal.example.com"
}
}

###

### Save LLMS Txt Configuration - Enterprise llms.txt
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "{{testHostName}}",
"llmsTxtContent": "# LLMS.TXT - Enterprise Configuration\n\n# Model Infrastructure\nmodel-provider: OpenAI\nmodel-deployment: azure-openai-service\nmodel-region: east-us\n\n# Security\ndata-encryption: AES-256\nnetwork-isolation: private-endpoint\nauthentication: azure-ad\n\n# Compliance\ncertifications: SOC2-Type2, ISO27001\ndata-residency: US\naudit-logging: enabled\n\n# Governance\ndata-classification: confidential\nretention-policy: 7-years\ndeletion-policy: secure-wipe\n\n# Monitoring\nlogging: comprehensive\nmetrics: performance-security\nalerting: 24x7-soc\n\n# Business Continuity\nbackup: geo-redundant\ndisaster-recovery: rpo-1hour-rto-4hours\n\n# Contact Information\nsecurity-contact: security@enterprise.example.com\ndpo-contact: privacy@enterprise.example.com\nincident-response: incident@enterprise.example.com"
}
}

###

### Error Test - Save without bearer token
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json

{
"parameters": {
"hostName": "error-test.example.com"
}
}

###

### Error Test - Save with incorrect bearer token
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer wrong-token

{
"parameters": {
"hostName": "error-test.example.com"
}
}

###

### Error Test - Save without llmsTxtContent
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"hostName": "error-test.example.com"
}
}

###

### Error Test - Save with invalid LlmsId
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"llmsId": "99999999-9999-9999-9999-999999999999",
"llmsTxtContent": "# LLMS.TXT\nmodel-name: Test-Model\ncontact: test@example.com"
}
}

###

### Error Test - Save without LlmsId or HostName
POST {{baseUrl}}{{apiPath}}/tools/save-llms-txt-configuration/
Accept: application/json
Content-Type: application/json
Authorization: Bearer {{token}}

{
"parameters": {
"llmsTxtContent": "# LLMS.TXT\nmodel-name: Orphan-Model\ncontact: orphan@example.com"
}
}

###

# Expected Response Formats:
#
# GET configurations (all):
# [
# {
# "id": "guid",
# "siteName": "string",
# "isDefaultForSite": boolean,
# "specificHost": "string",
# "llmsContent": "string"
# }
# ]
#
# GET configurations (single):
# {
# "id": "guid",
# "siteName": "string",
# "isDefaultForSite": boolean,
# "specificHost": "string",
# "llmsContent": "string"
# }
#
# SAVE configurations (success):
# {
# "success": true,
# "message": "string",
# "data": {
# "id": "guid",
# "siteId": "guid",
# "siteName": "string",
# "specificHost": "string",
# "llmsContent": "string"
# }
# }
#
# SAVE configurations (error):
# {
# "success": false,
# "message": "error description"
# }
#
# LLMS.TXT Format Information:
# LLMS.TXT is a standard for documenting AI model usage and policies.
# Common fields include:
# - model-name: Name of the AI model
# - model-version: Version or variant
# - training-data-cutoff: Knowledge cutoff date
# - capabilities: What the model can do
# - limitations: What the model cannot do
# - usage-policy: How the model should be used
# - data-retention: How long data is kept
# - contact: Contact information for queries
# - compliance: Regulatory compliance information
Loading
Loading