2026-09-17 Engineering Team

[joecastiglione]

Agenda

ActivitySim

• New release:

  • v1.6.0 has been released on request
  • Not on conda-forge yet, waiting for feedstock update, blocked by sharrow not up to latest version on conda. Note this also blocks windows installer. Jan requested maintainer access on both, needs Jeff's approval.
  • Large scale runs performed with abench, Jeff's vibe-coded/agentic engineered memory and runtime profiling tool. Suggestion: Extract aggregate results, keep and compare across releases.
  • Next release at end of Phase 11

• Review outstanding:

  • #1110 Reduce peak memory: David and Jan to review. Jeff added example model tests at scale, created with abench.

• Resolve owner blockers:

  • #1112 Park-and-ride documentation: David reviewed, minor comments for Jeff to address.
  • #1107 Calibration: estimation-test failures, review readiness, user testing. Related example model PR
  • #1083 Shadow-pricing segmentation: Sijia's unit tests and SANDAG validation.
  • #1062 Summarize memory cleanup: David/RSG's test and completion status.
  • #1103 Faster RNG: Jeff to address minor comments and confirm performance/memory validation.

• Others:

  • #1086 Application and Analysis Guide: resolve review-access issues, confirm SANDAG walkthrough testing, and set final feedback deadline.
  • #1108 OMX/Zarr precedence. Jeff to comment on issue. Proposal: ZARR is a convenience cache only and is not to be modified; if both Zarr and OMX inputs exist they are assumed to contain identical content. Zarr is not an agreed industry data transfer standard and lacks widespread vendor support.
  • #1106 Automatic adaptive chunking: initial technical assessment and recommendation for Product/Community.
  • #1078 Performance-test subarea: Joe Flood's runner/data PR, warning cleanup, and agreed runtime/memory benchmarks.

PopulationSim

• #219 Stabilization: Nick addressed review comment, added test. David to give final approval before merge.
• #217 Intermediate geographies: Blocked by Stabilization PR.
• #221 Household-weight caps: Amir/Nick testing results and next fixes.

Coordination

• Modeling Mobility: proposed ActivitySim session. Note submission deadline moved to October 2.

[janzill]

Meeting Notes

Meeting Overview

Participants

• Jan Zill (@janzill) — Outer Loop
• Joe Castiglione (@joecastiglione) — Zephyr Foundation
• Shaun Tabone — Transport for NSW
• David Hensle (@dhensle) — RSG
• Sijia Wang (@i-am-sijia) — WSP
• Andrew Kay (@andkay) — Cambridge Systematics
• Joe Flood (@JoeJimFlood) — Cambridge Systematics

Duration

65 minutes

Executive Summary

ActivitySim v1.6.0 has been released on PyPI. Its conda-forge package and the Windows installer remain blocked by the outdated conda-forge sharrow package and a maintainer approval. The team supported using the new abench tooling to make large-model runtime and memory checks repeatable and to retain compact aggregate outputs for comparison across releases.

Review of the peak-memory pull request is underway. The change addresses high memory use before utility evaluation, but initial benchmark results show meaningful runtime increases in trip mode choice and disaggregate accessibility. A separate, intermittently observed memory-growth problem in sharrow-based models remains unexplained. The team also affirmed that AI-suggested low-level optimisations should not be merged when maintainers cannot adequately explain, validate and support them.

The automatic adaptive-chunking proposal was deferred for consideration with broader Phase 12 memory and performance work because current agency demand appears limited and the contribution would require substantial review. The team also agreed to turn the proposed Modeling Mobility session outline into a consortium abstract and continued discussing practical review workflows for large documentation changes and more informative GitHub issue templates.

Meeting Notes

Activity-Based Modeling Outreach

• Joe reported strong interest in activity-based modeling at the European Transport Conference, alongside continued European caution about using activity-based models for economic appraisal. Shaun described similar discussions at an Australian conference, where agencies remain interested but are still establishing how activity-based models should be applied to projects.
• Stochastic simulation, behavioural randomness and wider forecasting uncertainty are often treated as a single concern even though they are distinct. The group noted that clearer communication, concrete examples and results from repeated model realisations could help address those concerns.
• The topic could form part of the proposed Modeling Mobility session, although it was recognised as broader than the engineering team's remit.

ActivitySim v1.6.0 and Release Benchmarking

• ActivitySim v1.6.0 has been released on PyPI, the first release since November 2025.
• The conda-forge build is blocked by the sharrow feedstock update. The ActivitySim feedstock update also requires maintainer approval; Jan requested maintainer access and will follow up with Jeff. Completing these updates will also unblock the Windows installer.
• Jeff created abench to run models in controlled Linux containers and collect comparable runtime and memory profiles. Jan extended it to run the large SANDAG model on AWS and copy results to S3.
• Jan proposed retaining compact aggregate model summaries, such as choice counts by mode and segment, alongside the detailed benchmark output. Running the same inputs across releases would provide a quick statistical check for unexpected behavioural changes as well as a performance history.
• The next ActivitySim release remains targeted for the end of Phase 11.

Peak-Memory Reduction

• David and Jan had begun reviewing #1110 Reduce peak memory. The change moves chunking earlier, when chooser data are assembled and merged, to reduce the pre-simulation memory peak that explicit chunking did not address.
• The work arose from fitting a Boston-region model on a machine with approximately 64 GB of memory. Initial abench comparisons cover both MTC and SANDAG examples.
• David observed non-trivial runtime increases in trip mode choice and disaggregate accessibility in the benchmark results. He and Jan will continue investigating these results and completing their reviews.
• The gradual memory growth previously observed in some TransLink and MWCOG sharrow runs appears to be a separate problem. It does not appear in the current benchmark cases, is not consistently reproducible and could relate to cache cleanup, repeated compilation or multiprocessing. The meeting did not identify a cause.

Outstanding ActivitySim Pull Requests

• #1112 Park-and-ride documentation had received David's review and was awaiting Jeff's response to minor comments.
• #1107 Calibration remains a draft but is functionally close to completion. Its related SANDAG example-model pull request contains user-facing documentation. PSRC and MWCOG have tested the tooling successfully, and RSG is incorporating their comments and completing internal checks before marking it ready for review.
• Sijia had approval to complete the unit tests for #1083 Shadow-pricing segmentation and will resume that work after returning from leave.
• David will add the outstanding unit test for #1062 Summarise memory cleanup.
• Jan approved the main implementation of #1103 Faster random numbers, subject to a few minor comments for Jeff. The new abench workflow should make its performance and memory validation easier.

Review Standards for AI-Assisted Optimisation

• A further AI-generated review of #1103 had proposed lower-level LLVM-oriented optimisations beyond the implementation already reviewed. These might improve performance, but the maintainers could not readily assess their cross-platform behaviour, robustness or maintenance cost.
• The group supported not merging those additional optimisations at this point. Participants considered it important that human maintainers be able to explain the purpose and technology of accepted code and continue maintaining it without depending on an LLM to reconstruct its operation.
• Review depth may reasonably differ between behavioural model code and lower-level engineering infrastructure. Strong regression tests can provide confidence in plumbing-level changes, but tests do not eliminate the need for maintainers to understand the functional design and operational risks.

Application and Analysis Guide Review

• Andrew confirmed that GitHub's review interface becomes difficult for a large document receiving comments and edits over several weeks: structural changes and shifting line numbers can make unresolved comments hard to follow.
• One proposed GitHub workflow is to keep the branch under review stable while applying accepted edits on a separate branch. Each review comment could then reference the commit that addresses it, with the completed change branch merged after the review closes.
• GitHub keeps the Markdown source, comments and final publication workflow together and avoids the access restrictions some agencies face with Google Docs. A word processor, however, remains easier for many non-technical reviewers, and the group did not settle on a permanent platform.
• For the current #1086 Application and Analysis Guide, the immediate priority is to collect any outstanding Transport for NSW feedback and complete the review. A more elaborate GitHub workflow can be tested when another broad collaborative documentation review arises.

OMX and Zarr Skim Precedence

• For #1108 OMX/Zarr skim precedence, the proposed interpretation is that Zarr is a convenience cache rather than an independent source to be modified. If both OMX and Zarr inputs exist, they should be assumed to contain identical data.
• No participant objected to that approach. Jeff was expected to continue the discussion with the contributor on the issue.

Automatic Adaptive Chunking

• #1106 Automatic adaptive chunking would require further assessment of both the product need and the proposed implementation. Segment-specific chunking may be useful but could break backward compatibility, and the contribution would need substantial technical review.
• Participants did not identify an immediate need in current agency workflows. The team therefore favoured deferring the proposal until Phase 12, when it can be considered with broader memory and performance priorities, rather than spending current-phase resources on it.

Performance-Test Subarea

• Joe Flood had not received further feedback from Jeff on the runner and data prepared for #1078 Performance-test subarea. He will open a pull request so the work can be reviewed directly.
• A small automated test case would complement the large abench runs by providing routine statistical, runtime and memory comparisons without requiring a costly 512 GB cloud instance.

PopulationSim

• Nick had addressed David's final comment on #219 Dependency modernisation and cross-platform stability by moving the relevant check from packaging workflow code into a test. David will perform the final review and merge the pull request if the change is complete.
• Merging #219 will unblock #217 Intermediate geographies.
• Amir and Nick are continuing to investigate #221 Household-weight hard constraints offline. Further progress may wait until Amir returns from leave.

Modeling Mobility Submission

• The submission deadline had moved to October 2. After discussing the proposed session at two wider consortium meetings without objections to its scope, Joe recommended proceeding with the full outline developed by David and other contributors.
• Joe will convert the outline into a formal abstract, circulate it for comments and submit it as a consortium group proposal that also credits the contributors who developed the material.

Meeting-Note Closure and Issue Templates

• Joe confirmed that engineering meeting-note issues can be closed once they are an adequate record of the meeting; unresolved technical work does not require the note itself to remain open. Comments can still be added and an issue reopened if a correction is needed. Jan will review and close the accumulated engineering meeting-note issues.
• The proposed GitHub issue template is intended to make candidate Phase 12 work easier to understand and compare, especially for new consortium members. It should capture why an item matters, who needs it, possible solutions and risks rather than requiring voters to reconstruct that context from a short issue description.
• Not every issue needs the full prioritisation template. The group discussed separate forms for bugs, feature requests and user questions, with templates automatically applying labels where possible. A simple question path, whether in GitHub or redirected to Discord, would preserve external-user engagement without demanding implementation details.
• Participants were asked to review and comment on the shared template proposal. Joe plans to populate a small number of examples before the next prioritisation process rather than applying the detailed format indiscriminately to every existing issue.

Action Items

• Jan (@janzill) will follow up with Jeff (@jpn--) about maintainer access and the feedstock changes needed to publish v1.6.0 on conda-forge and unblock the Windows installer.
• David (@dhensle) and Jan (@janzill) will complete their reviews of #1110, including investigation of the observed runtime regressions.
• Sijia (@i-am-sijia) will add the outstanding unit tests for #1083.
• David (@dhensle) will add the outstanding unit test for #1062.
• Joe (@JoeJimFlood) will open a pull request for the #1078 performance-test subarea runner and data.
• David (@dhensle) will perform the final review of PopulationSim #219 and merge it if the requested change is complete.
• Joe (@joecastiglione) will draft and circulate the Modeling Mobility abstract based on the proposed group-session outline.
• Jan (@janzill) will review and close completed engineering meeting-note issues.