UI improvements

app/queue-server/README.md

@@ -1,100 +1,110 @@
-# Java Bluesky Interface (JBI)
+# Phoebus Queue Server Application
 
-Prototype Java client for the [Bluesky QueueServer](https://blueskyproject.io/bluesky-queueserver/) REST API.  
-It includes:
+A JavaFX-based client for the [Bluesky QueueServer](https://blueskyproject.io/bluesky-queueserver/) that provides a graphical interface for managing experiment queues, monitoring status, and editing plans at beamlines and scientific facilities.
 
-* **BlueskyHttpClient** – thread-safe singleton with rate-limiting, automatic retries and JUL logging
-* **BlueskyService**   – one blocking call per API route
-* **CLI / REPL** – *testing utilities only*
-* **JavaFX UI** – the part you actually launch in normal use
+## Features
 
----
+### Queue Management
+- **View & Edit Queue**: See all queued plans with parameters, reorder items via drag-and-drop
+- **Execute Plans**: Start, stop, pause, resume, and abort plan execution
+- **Real-time Status**: Live updates via WebSocket connections showing queue state, running plans, and RE Manager status
+- **History View**: Browse completed plans with execution results and metadata
 
-##   Prerequisites
+### Plan Editing
+- **Interactive Plan Editor**: Create and modify plans with type-safe parameter editing
+- **Python Type Support**: Full support for Python types (strings, lists, dicts, booleans, numbers)
+- **Schema Validation**: Parameters validated against plan schemas from the Queue Server
+- **Live Preview**: See plan parameters as they will be sent to the server
 
-* Java 17
-* Maven
-* Running Bluesky HTTP Server at `http://localhost:60610`  
-  *single-user API key `a` assumed below*
+### Plan Viewer
+- **Plan Details**: View plan parameters, metadata, and execution results
+- **Parameter Display**: All parameters shown with correct Python syntax and types
+- **Copy to Queue**: Duplicate plans with one click
 
-```bash
-# 1) Start RE Manager
-start-re-manager --use-ipython-kernel=ON --zmq-publish-console=ON
+### Console Monitor
+- **Live Console Output**: Real-time streaming of Queue Server console output
+- **WebSocket Support**: Efficient streaming via WebSocket connections (fallback to HTTP polling)
+- **Autoscroll**: Automatically scroll to latest output with toggle control
+
+## Quick Start
+
+### Prerequisites
 
-# 2) Start HTTP Server
-QSERVER_HTTP_SERVER_SINGLE_USER_API_KEY=a \
-    uvicorn --host localhost --port 60610 bluesky_httpserver.server:app
-````
+- **Java 17** or later
+- **Maven** (for building from source)
+- **Bluesky Queue Server** running and accessible
 
----
+### Starting a Local Queue Server
 
-##   Configure
+Use the provided Docker setup for local development:
 
 ```bash
-export BLUESKY_API_KEY=a 
+cd services/bluesky-services
+docker-compose --profile container-redis up -d
 ```
 
----
+This starts:
+- Bluesky Queue Server (RE Manager) on ports 60615/60625
+- HTTP Server REST API on port 60610
+- Redis database on port 6380
 
-##  Build & run
+For details, see [services/bluesky-services/README.md](../../services/bluesky-services/README.md)
 
-```bash
-mvn clean javafx:run
-```
+### Configuration
 
-### Verbose logging
+Set environment variables to connect to your Queue Server:
 
 ```bash
-mvn -Djava.util.logging.config.file=src/main/resources/logging.properties javafx:run
-```
+# Queue Server HTTP address (default: http://localhost:60610)
+export QSERVER_HTTP_SERVER_URI=http://localhost:60610
 
----
+# API Key authentication
+export QSERVER_HTTP_SERVER_API_KEY=a
 
-##   CLI / REPL (testing only)
+# Or use a file containing the API key
+export QSERVER_HTTP_SERVER_API_KEYFILE=~/.phoebus/qserver_api_key.txt
+```
 
-| Tool     | Start (quiet)                                                                         | Start with request tracing (`FINE`)                                                                                                                        |
-| -------- | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **CLI**  | `mvn -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineCli  -Dexec.args="STATUS" exec:java` | `mvn -Djava.util.logging.config.file=src/main/resources/logging.properties -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineCli -Dexec.args="STATUS" exec:java` |
-| **REPL** | `mvn -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineRepl exec:java`                          | `mvn -Djava.util.logging.config.file=src/main/resources/logging.properties -q -Dexec.mainClass=util.org.phoebus.applications.queueserver.RunEngineRepl -Dexec.args="STATUS" exec:java`                                                                                                                         |
+### Building & Running
 
-*CLI examples*
+#### As Part of Phoebus
 
 ```bash
-# list endpoints
-mvn -q -Dexec.mainClass=com.jbi.util.RunEngineClili -Dexec.args="list" exec:java
-
-# start the queue
-mvn -q -Dexec.mainClass=com.jbi.util.RunEngineClili -Dexec.args="QUEUE_START" exec:java
+# From phoebus root directory
+mvn clean install -DskipTests
+cd phoebus-product/target
+./phoebus
 ```
 
-`ENDPOINT [body]` accepts a JSON literal, `@file.json`, or `@-` for stdin.
+Then open **Applications → Queue Server** from the menu.
 
----
+#### Standalone Build
 
-##   How logging works
+```bash
+# From queue-server directory
+cd app/queue-server
+mvn clean install
+```
 
-* Logger name: **`com.jbi.bluesky`**
-* Levels
+## Configuration Options
 
-  * `INFO` – API errors
-  * `WARNING` – transport retries
-  * `FINE` – each HTTP call + latency
-* Enable by passing JVM flag
-  `-Djava.util.logging.config.file=src/main/resources/logging.properties`
+Configuration via **Edit → Preferences → Queue Server** or environment variables:
 
----
+| Preference        | Environment Variable              | Default                  | Description                           |
+|-------------------|-----------------------------------|--------------------------|---------------------------------------|
+| `queue_server_url`| `QSERVER_HTTP_SERVER_URI`         | `http://localhost:60610` | Queue Server HTTP address             |
+| `api_key`         | `QSERVER_HTTP_SERVER_API_KEY`     | *(none)*                 | API key for authentication            |
+| `api_key_file`    | `QSERVER_HTTP_SERVER_API_KEYFILE` | *(none)*                 | Path to file containing API key       |
+| `use_websockets`  | *(none)*                          | `true`                   | Use WebSockets for streaming data     |
+| `connectTimeout`  | *(none)*                          | `5000`                   | HTTP connection timeout (ms)          |
+| `debug`           | *(none)*                          | `false`                  | Enable HTTP request/response logging  |
 
-##   Tuning
+## Contributing
 
-```java
-// rate limit (req/sec)
-BlueskyHttpClient.initialize("http://localhost:60610",
-                             System.getenv("BLUESKY_API_KEY"),
-                             3.0);
+When making changes:
 
-// retry/back-off
-// edit src/main/java/com/jbi/util/HttpSupport.java
-MAX_RETRIES        = 5;
-INITIAL_BACKOFF_MS = 500;
-BACKOFF_MULTIPLIER = 1.5;
-```
+1. Ensure proper Python type handling in parameter editor
+2. Test with both WebSocket and HTTP fallback modes
+3. Verify API key authentication works
+4. Update documentation for new features
+5. Follow existing code style and patterns

app/queue-server/pom.xml

@@ -34,6 +34,12 @@
             <artifactId>jackson-dataformat-yaml</artifactId>
             <version>2.19.1</version>
         </dependency>
+        <!-- Jython for Python scripting -->
+        <dependency>
+            <groupId>org.python</groupId>
+            <artifactId>jython-standalone</artifactId>
+            <version>${jython.version}</version>
+        </dependency>
         <dependency>
             <groupId>org.apache.poi</groupId>
             <artifactId>poi</artifactId>

app/queue-server/src/main/java/org/phoebus/applications/queueserver/Preferences.java

@@ -15,12 +15,21 @@ public class Preferences {
     @Preference
     public static String api_key;
 
+    @Preference
+    public static String api_key_file;
+
     @Preference
     public static boolean debug;
 
     @Preference
     public static int connectTimeout;
 
+    @Preference
+    public static boolean use_websockets;
+
+    @Preference
+    public static int update_interval_ms;
+
     static
     {
         AnnotatedPreferences.initialize(Preferences.class, "/queueserver_preferences.properties");

app/queue-server/src/main/java/org/phoebus/applications/queueserver/QueueServerApp.java

@@ -1,6 +1,11 @@
 package org.phoebus.applications.queueserver;
 
+import java.io.IOException;
 import java.net.URL;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.logging.Level;
 import java.util.logging.Logger;
 
 import org.phoebus.applications.queueserver.client.RunEngineHttpClient;
@@ -16,7 +21,7 @@
 @SuppressWarnings("nls")
 public final class QueueServerApp implements AppResourceDescriptor {
 
-    public  static final Logger LOGGER = Logger.getLogger(QueueServerApp.class.getPackageName());
+    public static final Logger logger = Logger.getLogger(QueueServerApp.class.getPackageName());
 
     public  static final String NAME         = "queue-server";
     private static final String DISPLAY_NAME = "Queue Server";
@@ -30,7 +35,20 @@ public final class QueueServerApp implements AppResourceDescriptor {
 
     @Override public AppInstance create() {
 
-        RunEngineHttpClient.initialize(Preferences.queue_server_url, Preferences.api_key);
+        // Resolve server URL with default fallback
+        String serverUrl = Preferences.queue_server_url;
+        // Check if the preference wasn't expanded (still has $(VAR) syntax) or is empty
+        if (serverUrl == null || serverUrl.trim().isEmpty() || serverUrl.startsWith("$(")) {
+            serverUrl = "http://localhost:60610";
+            logger.log(Level.INFO, "Using default Queue Server URL: " + serverUrl);
+        }
+
+        // Resolve API key with priority:
+        // 1. Direct api_key preference (or QSERVER_HTTP_SERVER_API_KEY env var)
+        // 2. Read from api_key_file path (or QSERVER_HTTP_SERVER_API_KEYFILE env var)
+        String apiKey = resolveApiKey();
+
+        RunEngineHttpClient.initialize(serverUrl, apiKey);
 
         Parent root = ViewFactory.APPLICATION.get();
 
@@ -42,6 +60,45 @@ public final class QueueServerApp implements AppResourceDescriptor {
         return inst;
     }
 
+    /**
+     * Resolve the API key using the same priority as Python bluesky-widgets:
+     * 1. Check QSERVER_HTTP_SERVER_API_KEY environment variable (via api_key preference)
+     * 2. If not set, check QSERVER_HTTP_SERVER_API_KEYFILE environment variable (via api_key_file preference)
+     * 3. If keyfile path is set, read the API key from that file
+     *
+     * @return The resolved API key, or null if not configured
+     */
+    private static String resolveApiKey() {
+        // First priority: direct API key
+        String apiKey = Preferences.api_key;
+        // Check if the preference was expanded (not still $(VAR) syntax) and not empty
+        if (apiKey != null && !apiKey.trim().isEmpty() && !apiKey.startsWith("$(")) {
+            logger.log(Level.FINE, "Using API key from QSERVER_HTTP_SERVER_API_KEY");
+            return apiKey.trim();
+        }
+
+        // Second priority: read from keyfile
+        String keyFilePath = Preferences.api_key_file;
+        if (keyFilePath != null && !keyFilePath.trim().isEmpty() && !keyFilePath.startsWith("$(")) {
+            try {
+                Path path = Paths.get(keyFilePath.trim());
+                if (Files.exists(path)) {
+                    apiKey = Files.readString(path).trim();
+                    logger.log(Level.FINE, "Using API key from file: " + keyFilePath);
+                    return apiKey;
+                } else {
+                    logger.log(Level.WARNING, "API key file not found: " + keyFilePath);
+                }
+            } catch (IOException e) {
+                logger.log(Level.WARNING, "Failed to read API key from file: " + keyFilePath, e);
+            }
+        }
+
+        logger.log(Level.WARNING, "No API key configured. Set QSERVER_HTTP_SERVER_API_KEY environment variable " +
+                "or QSERVER_HTTP_SERVER_API_KEYFILE to point to a file containing the API key.");
+        return null;
+    }
+
     @Override public AppInstance create(java.net.URI resource) {
         return ApplicationService.createInstance(NAME);
     }

app/queue-server/src/main/java/org/phoebus/applications/queueserver/api/ConsoleOutputWsMessage.java

@@ -0,0 +1,19 @@
+package org.phoebus.applications.queueserver.api;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+
+/**
+ * WebSocket message from /console_output/ws endpoint.
+ * Format: {"time": timestamp, "msg": text}
+ */
+public record ConsoleOutputWsMessage(
+        @JsonProperty("time") double time,
+        @JsonProperty("msg")  String msg
+) {
+    /**
+     * Get timestamp as milliseconds since epoch.
+     */
+    public long timestampMillis() {
+        return (long) (time * 1000);
+    }
+}

app/queue-server/src/main/java/org/phoebus/applications/queueserver/api/StatusWsMessage.java

@@ -0,0 +1,30 @@
+package org.phoebus.applications.queueserver.api;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.Map;
+
+/**
+ * WebSocket message from /status/ws endpoint.
+ * Format: {"time": timestamp, "msg": {"status": {...}}}
+ */
+public record StatusWsMessage(
+        @JsonProperty("time") double time,
+        @JsonProperty("msg")  Map<String, Object> msg
+) {
+    /**
+     * Get timestamp as milliseconds since epoch.
+     */
+    public long timestampMillis() {
+        return (long) (time * 1000);
+    }
+
+    /**
+     * Get the status payload from the message.
+     * Returns null if not present.
+     */
+    @SuppressWarnings("unchecked")
+    public Map<String, Object> status() {
+        if (msg == null) return null;
+        return (Map<String, Object>) msg.get("status");
+    }
+}

app/queue-server/src/main/java/org/phoebus/applications/queueserver/api/SystemInfoWsMessage.java

@@ -0,0 +1,31 @@
+package org.phoebus.applications.queueserver.api;
+
+import com.fasterxml.jackson.annotation.JsonProperty;
+import java.util.Map;
+
+/**
+ * WebSocket message from /info/ws endpoint.
+ * Format: {"time": timestamp, "msg": {msg-class: msg-content}}
+ * Currently includes status messages and may include additional system info in the future.
+ */
+public record SystemInfoWsMessage(
+        @JsonProperty("time") double time,
+        @JsonProperty("msg")  Map<String, Object> msg
+) {
+    /**
+     * Get timestamp as milliseconds since epoch.
+     */
+    public long timestampMillis() {
+        return (long) (time * 1000);
+    }
+
+    /**
+     * Get the status payload if this is a status message.
+     * Returns null if not a status message.
+     */
+    @SuppressWarnings("unchecked")
+    public Map<String, Object> status() {
+        if (msg == null) return null;
+        return (Map<String, Object>) msg.get("status");
+    }
+}