Describe the bug
As I develop a simple unit test for telework arrangement model specification, such as one like this, I found that certain literal forms in the Expression column of model spec CSV files are not handled consistently. Specifically, boolean literals (True/False) and numeric literals (1/0) can fail or produce inconsistent results depending on how they are parsed and evaluated.
Expression handling in model spec CSV files fails for some numeric/boolean literal forms due to two separate causes:
- CSV/pandas type inference can produce non-string Expression values.
- Expression evaluation uses two different engines with different literal rules.
To Reproduce
Use a spec with an Expression column like:
Label,Description,Expression,has_in_home_work_activity,no_in_home_work_activity
#this_works,alternative specific constant,@True,,coef_acs_no_in_home_work
#this_works,alternative specific constant,@1,,coef_acs_no_in_home_work
#this_fails,alternative specific constant,@true,,coef_acs_no_in_home_work
#this_fails,alternative specific constant,true,,coef_acs_no_in_home_work
#this_depends,alternative specific constant,1,,coef_acs_no_in_home_work
Observed behavior
@True works
@true fails
true fails
@1 works
1 and True handling is inconsistent, it fails if the Expression column is inferred by pandas as a numeric or boolean type, but if there're other string type values in the same column, it is inferred as string and work as expected. This creates a brittle situation where the success of certain literal forms depends on the presence of other values in the same column.
Expected behavior
Literal constants in Expression should be handled consistently and predictably:
Boolean literals: either consistently supported (with clear casing rules) or rejected with explicit validation errors.
Numeric literals: consistently treated as constants. Behavior should not unexpectedly depend on @ prefix unless clearly documented.
Why this matters: as users, we often define alternative constants with simple literals. Inconsistent literal handling makes configs brittle and errors hard to diagnose.
Technical context
There are two independent root causes:
- CSV read/type inference issue
- Dual evaluator literal semantics issue
@-prefixed expressions go through Python eval path (activitysim/core/simulate.py), where booleans are True/False (case-sensitive).
- non-
@ expressions go through pandas eval path via fast_eval (activitysim/core/simulate.py), where true may be treated as a name/token rather than a boolean literal.
- This creates inconsistent outcomes for syntactically similar literals.
Potential fix
- Normalize
Expression values to string before evaluation.
- Add early validation for unsupported literal forms with explicit error messages.
- Define and document one literal policy across both eval paths.
- Add unit tests for
@True, @False, @1, True, False, true, false, 1, 0.
Describe the bug
As I develop a simple unit test for telework arrangement model specification, such as one like this, I found that certain literal forms in the
Expressioncolumn of model spec CSV files are not handled consistently. Specifically,booleanliterals (True/False) andnumericliterals (1/0) can fail or produce inconsistent results depending on how they are parsed and evaluated.Expression handling in model spec CSV files fails for some
numeric/booleanliteral forms due to two separate causes:To Reproduce
Use a spec with an Expression column like:
Observed behavior
@Trueworks@truefailstruefails@1works1andTruehandling is inconsistent, it fails if the Expression column is inferred by pandas as anumericorbooleantype, but if there're otherstringtype values in the same column, it is inferred asstringand work as expected. This creates a brittle situation where the success of certain literal forms depends on the presence of other values in the same column.Expected behavior
Literal constants in Expression should be handled consistently and predictably:
Booleanliterals: either consistently supported (with clear casing rules) or rejected with explicit validation errors.Numericliterals: consistently treated as constants. Behavior should not unexpectedly depend on@prefix unless clearly documented.Why this matters: as users, we often define alternative constants with simple literals. Inconsistent literal handling makes configs brittle and errors hard to diagnose.
Technical context
There are two independent root causes:
@-prefixed expressions go through Pythonevalpath (activitysim/core/simulate.py), where booleans are True/False (case-sensitive).@expressions go through pandas eval path viafast_eval(activitysim/core/simulate.py), where true may be treated as a name/token rather than a boolean literal.Potential fix
Expressionvalues to string before evaluation.@True, @False, @1, True, False, true, false, 1, 0.