RL-Trust: Trust-Aware Resource Allocation

A DDPG-based reinforcement learning framework for allocating resources across a network of nodes, where node trust in the public entity evolves dynamically based on allocation satisfaction and neighbor influence. Two environment variants are included: a trust-aware env (TrustEnv.py) and a no-trust control (NoTrustEnv.py). Training follows a two-stage curriculum — first pre-training an allocator to minimize shortfall, then fine-tuning with full trust dynamics across diverse graph structures.

Setup

pip install torch numpy networkx

Generating Graph Datasets

python dataset_generator.py
# Writes data/graphs_train.jsonl and data/graphs_test.jsonl

Training

# Stage 1 — allocation pre-training (no trust)
python TrustEnv.py

# Stage 2 — trust-aware fine-tuning (edit checkpoint path in TrustEnv.py __main__ block first)
python TrustEnv.py

Evaluation

# RQ1 — trust-aware policies
python rq1.py \
  --stage2_ckpt checkpoints/stage2_newest_final.pt \
  --stage1_ckpt checkpoints/stage1_newest_final.pt \
  --graphs_jsonl data/graphs_test.jsonl \
  --out_dir rq1_outputs

# RQ2 — no-trust policies
python rq2.py \
  --stage2_ckpt checkpoints/stage2_no_trust.pt \
  --stage1_ckpt checkpoints/stage1_no_trust.pt \
  --graphs_jsonl data/graphs_test.jsonl \
  --out_dir rq2_outputs

Plotting

Unzip rq1_outputs and rq2_outputs and place both subfolders directly in the project root directory.

Heatmaps by family and network

To switch between RQ1 and RQ2, open each file and change value=rq1 to value=rq2 at the top.

python plot_heatmaps_per_family.py
python plot_heatmaps_per_network.py

Heatmaps by policy

python plot_heatmaps_per_policy.py --root_dir rq1_outputs --plot_dir plots/rq1/per_policy --title_prefix RQ1 --annotate --symmetric
python plot_heatmaps_per_policy.py --root_dir rq2_outputs --plot_dir plots/rq2/per_policy --title_prefix RQ2 --annotate --symmetric

Training curves (from log files)

Before running, update LOG_PATH at the top of each file to point to your .out log file.

python plot_and_eval/plotting_code_trust.py      # Stage 1 + 2 trust-aware training curves
python plot_and_eval/plotting_code_no_trust.py   # No-trust training curves

Eval metrics (from CSV)

Before running, update CSV_PATH at the top of plot_eval_results.py if needed.

python plot_and_eval/plot_eval_results.py

All plots are saved to a plots/ or logs/ directory depending on the script.