Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Copy to .env and adjust, or export variables in your shell.
# CLI flags take precedence over environment variables.

# --- subs ---
SUBS_PORT=7777
SUBS_DATA_DIR=./data
SUBS_WALLET=my-wallet
SUBS_SPACED_RPC_URL=http://127.0.0.1:7225
# SUBS_SPACED_RPC_USER=testuser
# SUBS_SPACED_RPC_PASSWORD=secret
# SUBS_SPACED_RPC_COOKIE=/path/to/.cookie
# SUBS_PROVER_ENDPOINT=http://127.0.0.1:8888
# SUBS_REGISTRY_ENDPOINT=http://127.0.0.1:8080
# SUBS_ENV_FILE=.env
# SUBS_TEST_RIG=1
# SUBS_TEST_RIG_DIR=./testrig-data

# --- subs-prover ---
# SUBS_PROVER_SERVER=1
SUBS_PROVER_PORT=8888
# SUBS_PROVER_ENV_FILE=.env
# SUBS_PROVER_INPUT=request.json
# SUBS_PROVER_OUTPUT=receipt.bin

# --- registry-server (example) ---
REGISTRY_SERVER_PORT=8080
# REGISTRY_SERVER_ENV_FILE=.env

# --- logging (all components) ---
# RUST_LOG=subs=info,subs_prover=info,registry_server=info,tower_http=debug
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
.env
.env.*
!.env.example
.DS_Store
Cargo.lock
.vscode
Expand All @@ -11,3 +14,5 @@ target/
.idea
testrig-data
data
datamad
NOTES.md
12 changes: 10 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
[workspace]
resolver = "2"
members = ["core", "prover", "types", "subs", "examples/registry-server"]
members = [
"config-origins",
"core",
"prover",
"types",
"subs",
"examples/registry-server",
]

[workspace.dependencies]
# Internal crates
Expand Down Expand Up @@ -29,7 +36,8 @@ serde_json = "1.0"
anyhow = "1.0"
hex = "0.4"
borsh = { version = "1.5", default-features = false, features = ["derive"] }
clap = { version = "4.5", features = ["derive"] }
clap = { version = "4.5", features = ["derive", "env"] }
dotenvy = "0.15"
tokio = { version = "1" }
tracing = "0.1"
tracing-subscriber = { version = "0.3", features = ["env-filter"] }
Expand Down
95 changes: 95 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,101 @@ cargo install --path prover

For operators, use `--features cuda` on `subs-prover` for nvidia machines to enable GPU acceleration.

## Configuration

Each binary accepts the same settings via **CLI flags**, **environment variables**, or a **`.env` file** in the current working directory. Command-line flags override environment variables.

Load a custom env file path with:

- `subs`: `SUBS_ENV_FILE=/path/to/subs.env`
- `subs-prover`: `SUBS_PROVER_ENV_FILE=/path/to/prover.env`
- `registry-server`: `REGISTRY_SERVER_ENV_FILE=/path/to/registry.env`

See [.env.example](.env.example) for a full template.

### `subs`

| Variable | CLI flag | Description |
|----------|----------|-------------|
| `SUBS_PORT` | `--port` | HTTP server port (default `7777`) |
| `SUBS_DATA_DIR` | `--data-dir` | Data directory (default `./data`) |
| `SUBS_WALLET` | `--wallet` | Wallet name for signing |
| `SUBS_SPACED_RPC_URL` | `--rpc-url` | `spaced` RPC URL |
| `SUBS_SPACED_RPC_USER` | `--rpc-user` | `spaced` RPC username |
| `SUBS_SPACED_RPC_PASSWORD` | `--rpc-password` | `spaced` RPC password |
| `SUBS_SPACED_RPC_COOKIE` | `--rpc-cookie` | `spaced` RPC cookie file path |
| `SUBS_PROVER_ENDPOINT` | *(Settings UI)* | Prover URL written to `config.db` at startup |
| `SUBS_REGISTRY_ENDPOINT` | *(Settings UI)* | Registry URL written to `config.db` at startup |
| `SUBS_TEST_RIG` | `--test-rig` | Enable test rig (`1`, `true`, `yes`) |
| `SUBS_TEST_RIG_DIR` | `--test-rig-dir` | Test rig data directory |

### `subs-prover`

| Variable | CLI flag | Description |
|----------|----------|-------------|
| `SUBS_PROVER_SERVER` | `--server` | Run as HTTP server (`1`, `true`, `yes`) |
| `SUBS_PROVER_PORT` | `--server-port` | Server port (default `8888`) |
| `SUBS_PROVER_INPUT` | `-i` / `--input` | Input file (prove/compress subcommands) |
| `SUBS_PROVER_OUTPUT` | `-o` / `--output` | Output file (prove/compress subcommands) |
| `SUBS_PROVER_BENCH_EXISTING` | `--existing` | Bench: existing handle count |
| `SUBS_PROVER_BENCH_INSERT` | `--insert` | Bench: handles to insert |

### `registry-server`

| Variable | CLI flag | Description |
|----------|----------|-------------|
| `REGISTRY_SERVER_PORT` | `--port` | HTTP server port (default `8080`) |

### Examples

Using `export`:

```bash
export SUBS_SPACED_RPC_URL=http://127.0.0.1:7225
export SUBS_WALLET=my-wallet
export SUBS_DATA_DIR=./data
export SUBS_PROVER_ENDPOINT=http://127.0.0.1:8888
subs
```

Using a `.env` file:

```bash
cp .env.example .env
# edit .env, then:
subs
```

```bash
# subs-prover from .env
export SUBS_PROVER_SERVER=1
export SUBS_PROVER_PORT=8888
subs-prover
```

```bash
# registry-server
export REGISTRY_SERVER_PORT=8080
registry-server
```

Log verbosity uses the standard `RUST_LOG` variable (e.g. `RUST_LOG=subs=debug,tower_http=debug`).

On startup, each binary prints its **effective configuration** to the console with the **origin** of each value: `param` (CLI flag), `environment` (`export`), `.env` (dotenv file), or `default`. Sensitive values (passwords) are shown as `(set)` without revealing the secret. Example:

```
subs configuration:
(loaded env file: .env)
port = 7777 (.env)
data_dir = ./datamad (.env)
wallet = mad (environment)
rpc_url = http://127.0.0.1:7225 (.env)
rpc_password = (set) (.env)
server_url = http://127.0.0.1:7777 (derived from port)
```

CLI flags override environment variables; process environment overrides `.env` for the same key.

## Usage

### 1. Start the prover server
Expand Down
10 changes: 10 additions & 0 deletions config-origins/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[package]
name = "config-origins"
version = "0.1.0"
edition = "2021"
description = "Startup configuration logging with value origins for subs binaries"
publish = false

[dependencies]
clap = { workspace = true }
dotenvy = { workspace = true }
185 changes: 185 additions & 0 deletions config-origins/src/lib.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,185 @@
//! Helpers for loading `.env` files and logging effective configuration with origins.

use std::collections::{HashMap, HashSet};
use std::fmt;
use std::path::{Path, PathBuf};

use clap::parser::ValueSource;
use clap::ArgMatches;

/// Where a configuration value came from.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ConfigOrigin {
/// Command-line flag or positional argument.
Param,
/// Process environment (e.g. `export VAR=...`) before `.env` was applied.
Environment,
/// `.env` file (or file pointed to by `*_ENV_FILE`).
DotEnv,
/// Built-in default when nothing else was provided.
Default,
}

impl fmt::Display for ConfigOrigin {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
Self::Param => write!(f, "param"),
Self::Environment => write!(f, "environment"),
Self::DotEnv => write!(f, ".env"),
Self::Default => write!(f, "default"),
}
}
}

/// Result of loading a dotenv file.
#[derive(Debug, Clone)]
pub struct DotenvLoad {
/// Path loaded, if any.
pub env_file: Option<PathBuf>,
/// Variable names whose values were applied from the file (not pre-set in the process env).
pub keys_from_dotenv: HashSet<String>,
}

/// Snapshot of the process environment before loading dotenv.
type EnvSnapshot = HashMap<String, String>;

/// Load variables from a `.env` file before CLI parsing.
///
/// If `env_file_var` is set in the environment, that path is used; otherwise tries `.env`
/// in the current working directory. Existing process environment variables are not overridden.
pub fn load_dotenv(env_file_var: &str) -> DotenvLoad {
let before = snapshot_env();
let (env_file, file_keys) = resolve_env_file(env_file_var);

if let Some(ref path) = env_file {
let _ = dotenvy::from_filename(path);
} else {
let _ = dotenvy::dotenv();
}

let mut keys_from_dotenv = HashSet::new();
for key in file_keys {
if std::env::var(&key).is_ok() && !before.contains_key(&key) {
keys_from_dotenv.insert(key);
}
}

DotenvLoad {
env_file,
keys_from_dotenv,
}
}

fn snapshot_env() -> EnvSnapshot {
std::env::vars().collect()
}

fn resolve_env_file(env_file_var: &str) -> (Option<PathBuf>, HashSet<String>) {
if let Ok(path) = std::env::var(env_file_var) {
if !path.is_empty() {
let p = PathBuf::from(&path);
let keys = parse_dotenv_keys(&p).unwrap_or_default();
return (Some(p), keys);
}
}

let dot_env = PathBuf::from(".env");
if dot_env.is_file() {
let keys = parse_dotenv_keys(&dot_env).unwrap_or_default();
(Some(dot_env), keys)
} else {
(None, HashSet::new())
}
}

/// Parse variable names from a dotenv file (ignores comments and blank lines).
fn parse_dotenv_keys(path: &Path) -> std::io::Result<HashSet<String>> {
let content = std::fs::read_to_string(path)?;
let mut keys = HashSet::new();
for line in content.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
let line = line.strip_prefix("export ").unwrap_or(line).trim();
if let Some((key, _)) = line.split_once('=') {
let key = key.trim();
if !key.is_empty() {
keys.insert(key.to_string());
}
}
}
Ok(keys)
}

/// Origin for a clap argument that may also use an environment variable.
pub fn origin_from_clap(
matches: &ArgMatches,
field_id: &str,
env_var: Option<&str>,
dotenv: &DotenvLoad,
) -> ConfigOrigin {
match matches.value_source(field_id) {
Some(ValueSource::CommandLine) => ConfigOrigin::Param,
Some(ValueSource::EnvVariable) => {
env_var
.and_then(|v| origin_for_env_var(v, dotenv))
.unwrap_or(ConfigOrigin::Environment)
}
Some(ValueSource::DefaultValue) => ConfigOrigin::Default,
_ => ConfigOrigin::Default,
}
}

/// Origin for a setting that is only available via environment (not a CLI flag).
pub fn origin_for_env_var(env_var: &str, dotenv: &DotenvLoad) -> Option<ConfigOrigin> {
if std::env::var(env_var).is_err() {
return None;
}
if dotenv.keys_from_dotenv.contains(env_var) {
Some(ConfigOrigin::DotEnv)
} else {
Some(ConfigOrigin::Environment)
}
}

/// Print a startup configuration section to stdout.
pub fn log_section(component: &str, dotenv: &DotenvLoad) {
println!("{component} configuration:");
if let Some(path) = &dotenv.env_file {
println!(" (loaded env file: {})", path.display());
}
}

/// Print one configuration line.
pub fn log_entry(name: &str, value: impl fmt::Display, origin: ConfigOrigin) {
println!(" {name} = {value} ({origin})");
}

/// Print one optional configuration line.
pub fn log_entry_optional(
name: &str,
value: Option<impl fmt::Display>,
origin: Option<ConfigOrigin>,
secret: bool,
) {
match (value, origin) {
(Some(v), Some(o)) => {
if secret {
log_entry(name, "(set)", o);
} else {
log_entry(name, v, o);
}
}
_ => println!(" {name} = (not set)"),
}
}

/// Display value for sensitive settings.
pub fn display_secret(value: Option<&str>) -> String {
if value.is_some() && !value.unwrap_or("").is_empty() {
"(set)".to_string()
} else {
"(not set)".to_string()
}
}
1 change: 1 addition & 0 deletions examples/registry-server/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ name = "registry-server"
path = "src/main.rs"

[dependencies]
config-origins = { path = "../../config-origins" }
axum = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "signal"] }
tower-http = { workspace = true }
Expand Down
4 changes: 3 additions & 1 deletion examples/registry-server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,10 @@ This architecture keeps subsd private (it holds wallet keys) while the registry
# Build
cargo build --release -p registry-server

# Run
# Run (CLI or environment)
registry-server --port 8080
# REGISTRY_SERVER_PORT=8080 registry-server
# Loads .env from the current directory if present
```

Then configure subsd to use this registry:
Expand Down
Loading