Skip to content

Add prototype script for validating example execution#362

Closed
Tushar1733 wants to merge 5 commits intomesa:mainfrom
Tushar1733:mesa-example-validator
Closed

Add prototype script for validating example execution#362
Tushar1733 wants to merge 5 commits intomesa:mainfrom
Tushar1733:mesa-example-validator

Conversation

@Tushar1733
Copy link
Copy Markdown

@Tushar1733 Tushar1733 commented Mar 6, 2026

Summary

This PR introduces a prototype script for validating whether examples in the Mesa Examples repository execute successfully.

As Mesa evolves, some examples may break due to API changes, missing dependencies, or outdated documentation. This script demonstrates a possible approach for automatically identifying such issues.

The script:

  • discovers example directories containing app.py
  • attempts to execute each example
  • classifies results as PASS, FAIL, or TIMEOUT
  • generates a summary report and writes results to validation_report.txt

The goal of this prototype is to explore how automated validation of examples could be implemented in the future, potentially as part of a CI workflow.

Added Files

scripts/run_examples.py
scripts/README.md

Motivation

While auditing examples locally (≈20 examples), I noticed several recurring issues:

  • legacy visualization APIs
  • missing dependencies
  • outdated run commands
  • compatibility issues with newer Mesa versions

Automating example validation could help detect these issues earlier and improve the long-term maintainability of the examples repository.

Future Work

Possible next steps could include:

  • integrating validation with GitHub Actions
  • supporting examples that require solara run app.py(Done)
  • adding structured metadata for dependencies and run commands

Hii @jackiekazil @EwoutH just checking in to see if you had time to review the validator prototype. Feedbacks will help me to improve it further.

Tushar1733 and others added 5 commits March 6, 2026 15:12
Some examples are failing they can't run by "python app.py"  command.
therefore
i am adding "solara run app.py" since all the new examples are compatible with this .
@Tushar1733 Tushar1733 force-pushed the mesa-example-validator branch from 3781711 to d845055 Compare March 8, 2026 14:08
@Tushar1733
Copy link
Copy Markdown
Author

hey @EwoutH can you review it real quick ?feedback will help me improve it

@Tushar1733 Tushar1733 closed this Mar 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant