2026-04-30 Engineering Team

[joecastiglione]

Agenda

Continuing Discussion

• Poisson sampling next steps
• EET, on value of testing EET against more than SANDAG

Items with Progress

• ✅ Remove Three Zone Code and Examples ... Merged #1056 and SANDAG-Example#41 and created/merged other minor changes in #1072

• Utility not aligned #1022 - possible cause identified and fix implemented in #1027, PSRC has reviewed and found it effective.

  • This will bump our dependency on numpy up to v2. Needs additional review? Also needs tests to pass.

Items without Progress

• Park and Ride #1001 - review pending
• Speed Up Nearest Zone Calculation in Disaggregate Accessibilities #1031 - review pending
• Adding ignore list to skim load for unused skims while running with sharrow #1036 - review pending
• Implement deletion of temporary variables in summarize.py #1062 - pending docs/testing
• PopulationSim intermediate geographies camsys#1
• Model component recompilation when running MP with sharrow.
• Comprehensive testing plan.
• A bunch of other contributed or smaller PRs

[jpn--]

Meeting Overview

Agenda

• Poisson sampling pull request (from VLC) — decision on whether to merge
• Random number generator (RNG) investigation — status update and path forward
• Outstanding pull requests: three-zone code cleanup, PSRC utility alignment fix (PR #1027)

Participants

• Jeff Newman (@jpn--) — Driftless Labs
• Joe Castiglione (@joecastiglione) — Zephyr Foundation
• David Hensle (@dhensle) — RSG
• Jan Zill (@janzill) — Outer Loop
• Stefan Coe (@stefancoe) — PSRC
• Sumit Bindra — TransLink

Duration

~55 minutes

---

Executive Summary

The meeting focused on two interrelated technical decisions: whether to merge the Poisson sampling pull request from VLC, and the status of ongoing RNG optimization work being conducted by RSG and Driftless Labs. The group reached a soft consensus to defer a final decision on Poisson sampling until after a Tuesday presentation on RNG findings, while acknowledging that Poisson sampling remains the pragmatic default if RNG improvements do not materialize quickly. Several pull requests were also reviewed, including the three-zone code cleanup (now merged) and PR #1027 addressing PSRC's Sharrow utility alignment issue, which PSRC confirmed is working in production.

---

Meeting Notes

Poisson Sampling (VLC Pull Request)

Jan Zill provided an assessment that while Poisson sampling (sampling without replacement) is an interesting approach, it would require numerical investigation to validate generically, as ActivitySim is a flexible framework and the sampling specification can differ from that used in the actual choice model. Jan noted that implementing this as a true drop-in replacement for importance sampling would require more effort than expected, and that the theoretical advantages—while promising—are not yet well evidenced in practice. Jan's conclusion was that Poisson sampling is the pragmatic choice given current knowledge, but that the RNG work (see below) may change the calculus.

Sumit Bindra raised the point that the VLC Poisson sampling implementation has already been tested and run successfully in production in Australia, which distinguishes it from the RNG work, which is still experimental.

Joe Castiglione noted that of the three solution streams discussed over recent meetings—Poisson sampling, RNG optimization, and top-end/elimination-of-sampling approaches—Poisson sampling has the most evidence behind it and the lowest implementation risk. He expressed a "bird in hand" preference for merging it, while remaining supportive of letting the RNG work continue. The group agreed to hold the Poisson sampling decision for at least one more week pending the Tuesday RNG presentation, with a hard outer limit of approximately two weeks before defaulting to merging Poisson sampling.

Random Number Generator (RNG) Investigation

Jeff Newman reported that he has developed a drop-in replacement for ActivitySim's reproducible random number generation that runs approximately 100–1000x faster in isolated testing. The key insight is that the current implementation repeatedly reseeds the Mersenne Twister generator, which has a large state space (~2.5 KB per row), and advances it by drawing and discarding large numbers of random values (forwarding) to reach reproducible states. Jeff's approach instead preserves the RNG state in memory, eliminating reseeding and forwarding entirely. Modern counter-based RNGs have state spaces of ~13 bytes, making in-memory state storage practical.

David Hensle noted that the picture is more complex than simply measuring raw RNG draw speed, because the performance impact depends on how reseeding and forwarding interact with the full ActivitySim run, and different RNG designs handle offsets differently. David is preparing a presentation on these findings for Tuesday.

Jan Zill confirmed that prior profiling work showed the dominant cost in the sampling step is the forwarding of the RNG (drawing and discarding large numbers of random values), not reseeding per se. Jan's view is that Jeff's state-preservation approach directly addresses this bottleneck, and that the approach will likely translate to meaningful runtime improvements in practice.

Jeff acknowledged that he has not yet tested the new RNG on a full ActivitySim run — only on an isolated version of the random number generation component. The open question is what fraction of total model runtime is attributable to this component. David and Jeff agreed that by Tuesday they will have more information but likely not a definitive answer; the goal is to be able to show a prototype or preliminary results indicating the likely magnitude of improvement.

EET Investigation Model Runs and Second Test Model

Discussion turned to the next phase of the EET (explicit error terms) investigation, which involves re-running model scenarios to identify and diagnose unexpected differences between EET and non-EET results. David Hensle noted that the original scope was to revisit prior SANDAG model runs after incorporating recent fixes (several of which Jan Zill has prepared as pull requests), and after RNG improvements if available.

Jan raised the possibility of also using the Victoria DTP model as a second test platform. The DTP model uses a utility-based trip scheduling choice with an explicit ET implementation and is actively used in project appraisal contexts — motivating their direct interest in this work. An agreement for model sharing and running the DTP model on Outer Loop's machines is in place. Jan suggested this would be relatively straightforward, involving simple artificial scenarios (e.g., an employment change in a defined area) consistent in spirit with the SANDAG test scenarios. David agreed this would be more valuable than both teams analyzing the same dataset.

Joe Castiglione expressed some caution about expanding scope and budget, and noted the value of having the two teams bring different analytical perspectives to a common set of results. Jeff suggested deferring the second-model decision until initial work on the SANDAG model runs is underway, to avoid overcommitting budget.

PR #1027 — PSRC Sharrow Utility Alignment Fix

Jeff reported that PR #1027, which fixes the utility alignment issue experienced by PSRC when running with Sharrow enabled, has been tested by PSRC in production. Stefan Coe confirmed that the model runs successfully with Sharrow re-enabled, that Sharrow notably reduces RAM requirements (allowing runs on ~200 GB machines vs ~500 GB without), and that a test comparing Sharrow vs. non-Sharrow runs completed successfully.

In addition to the PSRC-specific fix, PR #1027 upgrades ActivitySim from multinomial v1.0 to v2.0, touching 42 files. Jeff noted that while he is ~90% confident this introduces no performance regressions, he would like confirmation before merging. A performance review — specifically comparing timing logs and memory usage with and without Sharrow, using the new code — was requested.

Stefan Coe agreed to run the PSRC model without Sharrow using the new code and report timing log results before the next meeting. David Hensle and Joe Castiglione indicated RSG and CS also have some budget available to review the PR. A target of 2–3 weeks was set for completing the review and merging.

Three-Zone Code Cleanup

Jeff confirmed that the three-zone code cleanup pull request has been merged, requiring three separate PRs to fully remove the relevant code and documentation.

---

Action Items

• David Hensle (@dhensle) — Present RNG investigation findings at Tuesday's meeting, including preliminary performance results and a characterization of the path forward.
• Jeff Newman (@jpn--) — Continue RNG testing; if possible, run the new RNG implementation on a full ActivitySim model before or shortly after Tuesday.
• Stefan Coe (@stefancoe) — Run the PSRC model without Sharrow using the PR #1027 branch and share timing logs and memory usage results before the next engineering meeting.
• David Hensle (@dhensle) / RSG — Review PR #1027 within 2–3 weeks and assess for performance regressions.
• Joe Castiglione (@joecastiglione) / CS (Andrew Kay, @andkay) — Also review PR #1027 within 1–2 weeks.
• All — Revisit the Poisson sampling merge decision following Tuesday's RNG presentation; if no clear RNG path emerges within ~2 weeks, proceed with merging the VLC Poisson sampling pull request.
• Jan Zill (@janzill) — Submit outstanding Outer Loop pull requests (noted as ready or imminent) to enable updated SANDAG model runs for the ET investigation.
• Joe Castiglione (@joecastiglione) — Review notes from the Australia/DTP meeting to confirm any outstanding follow-up items with DTP regarding model sharing and scenario testing.
• Jeff Newman (@jpn--) / Jan Zill (@janzill) — Defer decision on whether to use the DTP model as a second test platform until SANDAG model work is underway; revisit based on budget and findings.