Changed: Move some files around for namespacing server

Author: delner

README.md

@@ -367,7 +367,7 @@ See examples: [eval.rb](./examples/eval.rb), [dataset.rb](./examples/eval/datase
 
 Run evaluations from the Braintrust web UI against code in your own application.
 
-**Run as a Rack app**
+#### Run as a Rack app
 
 Define evaluators, pass them to the dev server, and start serving:
 
@@ -395,11 +395,38 @@ run Braintrust::Server::Rack.app(
 )
 ```
 
+Add your Rack server to your Gemfile:
+
+```ruby
+gem "rack"
+gem "puma" # recommended
+```
+
+Then start the server:
+
 ```bash
 bundle exec rackup eval_server.ru -p 8300 -o 0.0.0.0
 ```
 
-**Run as a Rails engine**
+See example: [server/eval.ru](./examples/server/eval.ru)
+
+**Custom evaluators**
+
+Evaluators can also be defined as subclasses:
+
+```ruby
+class FoodClassifier < Braintrust::Eval::Evaluator
+  def task
+    ->(input:) { classify(input) }
+  end
+
+  def scorers
+    [Braintrust::Scorer.new("exact_match") { |expected:, output:| output == expected ? 1.0 : 0.0 }]
+  end
+end
+```
+
+#### Run as a Rails engine
 
 Use the Rails engine when your evaluators live inside an existing Rails app and you want to mount the Braintrust eval server into that application.
 
@@ -433,29 +460,16 @@ end
 
 The generator writes `config/initializers/braintrust_server.rb`, where you can review or customize the slug-to-evaluator mapping it discovers from `app/evaluators/**/*.rb` and `evaluators/**/*.rb`.
 
-**Developing locally**
-
-If you want to skip authentication on incoming eval requests while developing locally, set `config.auth = :none` in `config/initializers/braintrust_server.rb`.
-
-That only disables authentication on requests into your Rails app. Any outgoing Braintrust API calls still require normal Braintrust credentials such as `BRAINTRUST_API_KEY`.
-
 See example: [contrib/rails/eval.rb](./examples/contrib/rails/eval.rb)
 
-**Custom evaluators**
+**Developing locally**
 
-Evaluators can also be defined as subclasses:
+If you want to skip authentication on incoming eval requests while developing locally:
 
-```ruby
-class FoodClassifier < Braintrust::Eval::Evaluator
-  def task
-    ->(input:) { classify(input) }
-  end
+- **For Rack**: Pass `auth: :none` to `Braintrust::Server::Rack.app(...)`
+- **For Rails**: Set `config.auth = :none` in `config/initializers/braintrust_server.rb`
 
-  def scorers
-    [Braintrust::Scorer.new("exact_match") { |expected:, output:| output == expected ? 1.0 : 0.0 }]
-  end
-end
-```
+*NOTE: Setting `:none` disables authentication on incoming requests into your server; executing evals requires a `BRAINTRUST_API_KEY` to fetch resources.*
 
 **Supported web servers**
 
@@ -468,14 +482,7 @@ The dev server requires the `rack` gem and a Rack-compatible web server.
 | [Passenger](https://www.phusionpassenger.com/) | 6.x               |                                      |
 | [WEBrick](https://github.com/ruby/webrick)     | Not supported     | Does not support server-sent events. |
 
-Add your chosen server to your Gemfile:
-
-```ruby
-gem "rack"
-gem "puma" # recommended
-```
-
-See examples: [server/eval.ru](./examples/server/eval.ru), [contrib/rails/eval.rb](./examples/contrib/rails/eval.rb)
+See examples: [server/eval.ru](./examples/server/eval.ru), 
 
 ## Documentation
 

examples/README.md

@@ -33,7 +33,7 @@ BRAINTRUST_DEBUG=true ruby examples/login/login_basic.rb
 ### Dev Server Examples
 
 - **`server/eval.ru`**: Set up a dev server for remote evals — define evaluators (subclass or inline) and serve them via a Rack app. Start with: `bundle exec appraisal server rackup examples/server/eval.ru -p 8300 -o 0.0.0.0`
-- **`contrib/rails/eval.rb`**: Mount the dev server as a Rails engine, define evaluator classes under `app/evaluators/`, and generate `config/initializers/braintrust_server.rb` with `bin/rails generate braintrust:eval_server`
+- **`contrib/rails/eval.rb`**: Mount the dev server as a Rails engine, define evaluator classes under `app/evaluators/`, and generate `config/initializers/braintrust_server.rb` with `bin/rails generate braintrust:server`
 
 ## Coming Soon
 

examples/contrib/rails/eval.rb

@@ -5,7 +5,7 @@
 # This file shows one conventional setup for the Braintrust eval server in Rails:
 #   1. Define evaluator classes under app/evaluators/
 #   2. Generate the initializer with:
-#        bin/rails generate braintrust:eval_server
+#        bin/rails generate braintrust:server
 #   3. Mount the engine in config/routes.rb
 #
 # Requirements:
@@ -31,11 +31,11 @@ def scorers
 # config/initializers/braintrust_server.rb
 # ---------------------------------------------------------------------------
 
-# Generated by: bin/rails generate braintrust:eval_server
+# Generated by: bin/rails generate braintrust:server
 #
-# require "braintrust/server/rails"
+# require "braintrust/contrib/rails/server"
 #
-# Braintrust::Contrib::Rails::Engine.configure do |config|
+# Braintrust::Contrib::Rails::Server::Engine.configure do |config|
 #   config.evaluators = {
 #     "my-classifier" => MyClassifier.new
 #   }
@@ -51,7 +51,7 @@ def scorers
 # ---------------------------------------------------------------------------
 
 # Rails.application.routes.draw do
-#   mount Braintrust::Contrib::Rails::Engine, at: "/braintrust"
+#   mount Braintrust::Contrib::Rails::Server::Engine, at: "/braintrust"
 # end
 
 puts "Braintrust Rails Engine example — see comments for usage"

lib/braintrust/contrib/rails/application_controller.rb

@@ -0,0 +1,20 @@
+# frozen_string_literal: true
+
+begin
+  require "action_controller"
+  require "rails/engine"
+rescue LoadError
+  raise LoadError,
+    "Rails (actionpack + railties) is required for the Braintrust Rails server engine. " \
+    "Add `gem 'rails'` or `gem 'actionpack'` and `gem 'railties'` to your Gemfile."
+end
+
+require "json"
+require_relative "../../eval"
+require_relative "../../server/sse"
+require_relative "../../server/auth/no_auth"
+require_relative "../../server/auth/clerk_token"
+require_relative "../../server/middleware/cors"
+require_relative "../../server/services/list_service"
+require_relative "../../server/services/eval_service"
+require_relative "server/engine"

lib/braintrust/contrib/rails/engine.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+module Braintrust
+  module Contrib
+    module Rails
+      module Server
+        class ApplicationController < ActionController::API
+          before_action :authenticate!
+
+          private
+
+          def authenticate!
+            auth_result = Engine.auth_strategy.authenticate(request.env)
+            unless auth_result
+              render json: {"error" => "Unauthorized"}, status: :unauthorized
+              return
+            end
+
+            request.env["braintrust.auth"] = auth_result
+            @braintrust_auth = auth_result
+          end
+
+          def parse_json_body
+            body = request.body.read
+            return nil if body.nil? || body.empty?
+            JSON.parse(body)
+          rescue JSON::ParserError
+            nil
+          end
+        end
+      end
+    end
+  end
+end