codes-scripts

Scripts for automating the building/executing processes for CODES, visualizing model graphs, and parsing/visualizing generated statistics.

Build/Exec Helper Scripts

build/build-script.sh

• Includes selection flags for building various parts of the CODES workflow (Dumpi, ROSS, and CODES) and running the CODES tests.
• User enters the path to each framework's source directory and desired build directory.

execution/workflow-combined.sh

• Via selection of a long list of available parameters, this script handles the configuration and execution of codes model-net network trace (with and without background traffic) and synthetic workload simulations as well as NeMo neuromorphic workloads.
• Currently supports generation of:
  • Fat-tree, dragonfly-custom, and slim fly network model conf files
  • Contiguous, random, clustered, and hetereogeneous allocation conf files
  • dumpi workload conf files

job_placement/nodeids-to-rankids.py

• Used within the workflow-combined.sh execution script to convert node ids generated by the clustered allocation algorithm, into rank ids expected for CODES simulations.

Simulation Post-Processing/Visualization Scripts

visualizations/cdf-model-msg-stats.py

• Parses MODEL generated output files and constructs CDF plots showing the distribution of performance across all terminals in the simulation for a given metric.
• Supported models and their filenames:
  • Dragonfly:
    • dragonfly-msg-stats
    • dragonfly-router-stats
    • dragonfly-router-traffic
  • Fat-Tree:
    • fattree-msg-stats
    • fattree-switch-stats
    • fattree-switch-traffic
  • Slim Fly:
    • slimfly-msg-stats
    • slimfly-router-stats
    • slimfly-router-traffic

visualizations/line-sampling.py

• Parses sampling output files generated at the MPI replay layer and constructs time-series line graphs where each line represents the performance over simulation time of a given MPI rank for a given metric.
• Filename used: mpi-sampling-stats

Network Layout Visualization Scripts

• Each of the network specific layout scripts labeled *-graph-layout-vis.py parses generated connection files to visualize the simulated network layout
• Connection files are generated and dumped inside the lp-io-dir if #define NETWORK_NAME_CONNECTIONS 1 is set at the top of the specific CODES network model file located in codes/src/networks/model-net/NETWORK_NAME.c
• Currently only single rail/plane configurations are supported
• Each of the network layout scripts supports the following visual graph representations (saved as pdf image):
  • Layered: Traditional multi-layered fat-tree layout
  • Force: Fruchterman Reingold force directed algorithm for spacing nodes
  • Random: Results may vary
  • Circular: All switches and terminals are arranged in a circle
  • Shell: Arranges switches and terminals in concentric circles
  • Graphviz: Formatted using graphviz utility
• Output formats (for using a separate graph visualization utility such as Gephi):
  • graphml: Contains nodes, node positions (x,y), and connections
  • gexf: Contains nodes, node coloring (green:routers, red:compute nodes), connections, and connection weights

visualizations/ftree-graph-layout-vis.py

• Fat-Tree specific graph representation (saved as pdf image):
  • Layered: Traditional multi-layered fat-tree layout

visualizations/dfly-graph-layout-vis.py

visualizations/sfly-graph-layout-vis.py

• This script supports both slim fly and fit fly network layout visualizations (
  • For fit fly, set sf_type = 1