feat(challenges): add term-challenge-wasm crate with Challenge trait#36
feat(challenges): add term-challenge-wasm crate with Challenge trait#36
Conversation
…implementation Add a new WASM challenge crate that implements terminal-based task evaluation using the platform-challenge-sdk-wasm SDK. The crate evaluates miners by running their agent output against a catalog of terminal tasks (file manipulation, text processing, system administration) in a sandboxed environment via host_terminal_exec. Core implementation: - evaluation.rs: Orchestrates task execution by deserializing task definitions from EvaluationInput params (or falling back to defaults), running each task via host sandbox execution, and aggregating results into an EvaluationOutput. - scoring.rs: Deterministic scoring using fixed-point i64 scaling (10,000 base). Computes pass rate across all tasks; no floating-point non-determinism sources. - tasks.rs: Defines the default task catalog with five terminal tasks spanning Easy/Medium/Hard difficulties, each with a bash test script and timeout. Provides build_test_command() to construct CommandRequest for sandbox execution. - lib.rs: no_std entry point implementing the Challenge trait for TermChallenge with name "term-challenge", version "0.1.0", and register_challenge! macro. Also updates the SDK register_challenge! macro to use a const fn new() constructor instead of Default::default(), enabling static initialization of challenge instances. The .term-challenge-ref/ reference clone is excluded via .gitignore.
|
Warning Rate limit exceeded
⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ge-wasm-crate # Conflicts: # Cargo.lock # Cargo.toml # crates/challenge-sdk-wasm/src/lib.rs
…ge-wasm-crate # Conflicts: # challenges/term-challenge-wasm/Cargo.toml # challenges/term-challenge-wasm/src/lib.rs # challenges/term-challenge-wasm/src/scoring.rs # crates/challenge-sdk-wasm/src/lib.rs
1 participant
Summary
Adds the
term-challenge-wasmcrate underchallenges/term-challenge-wasm/, implementing theChallengetrait fromplatform-challenge-sdk-wasmfor terminal-based task evaluation.Changes
term-challenge-wasm:no_std-compatible WASM challenge (cdylib) targetingwasm32-unknown-unknownlib.rs: DefinesTermChallengestruct implementingChallengetrait (name, version, evaluate, validate) and registers it viaregister_challenge!evaluation.rs: Deserializes task definitions fromEvaluationInput.paramsvia bincode, executes each task throughhost_terminal_exec, and aggregates results into anEvaluationOutputscoring.rs: Computes a deterministic score as a pass-rate scaled toi64(fixed-point withSCORE_SCALE = 10_000)tasks.rs: Defines a default task catalog (file manipulation, text processing, system admin) with difficulty levels and bash test scripts; providesbuild_test_commandto constructCommandRequestpayloadsregister_challenge!macro to use<$ty>::new()constructor instead of<$ty as Default>::default()to supportconststatic initializationchallenges/term-challenge-wasmto workspace members.gitignore: Added.term-challenge-ref/directoryNotes
platform-challenge-sdk-wasm(path),serde(no_std),bincode(no_std)