Describe the bug
column_anomalies fails with '<' not supported between 'NoneType' and 'str' on weekly buckets since 0.26.0 (fine on 0.25.1).
'<' not supported between instances of 'NoneType' and 'str'
in macro store_anomaly_test_results (macros/edr/data_monitoring/anomaly_detection/store_anomaly_test_results.sql)
called by macro test_column_anomalies (macros/edr/tests/test_column_anomalies.sql)
called by test elementary_column_<snip> (<snip>)
To Reproduce
- Have a weekly
time_bucket with a timestamp_column.
- Have a
column_anomalies test
- Data to have a row dated on the last Sunday of the most recent complete week
- Data to have enough history for that row to get an anomaly score
Example config:
config:
elementary:
timestamp_column: full_date
time_bucket:
period: week
count: 1
Expected behavior
The test should not fail with python error raised from elementary core.
Screenshots
n/a
Environment (please complete the following information):
elementary dbt package 0.26.0
edr 0.26.0
dbt-core 1.12.5, dbt-postgres
Postgres
Additional context
AI gives the following summary:
Precursor:
On Postgres, get_start_bucket_in_data calls dbt.datediff(..., 'week'), and that function starts a new week on Sunday (it compares dow, where Sunday is 0). Elementary's week buckets run Monday to Monday, so Sunday rows go into the following week.
The last Sunday before max(edr_bucket_end) maps to a bucket that doesn't exist, and the left join buckets in column_monitoring_query returns a metric row with NULL bucket_start and bucket_end. That row still gets an anomaly score.
This bug predates 0.26.0; every weekly sum has been shifted by one day.
Trigger:
0.26.0 added | sort(attribute='bucket_end') in get_anomaly_test_result_row. Sorting that NULL-bucket row against the string dates throws the error above.
(Monthly and daily buckets aren't affected.)
Describe the bug
column_anomaliesfails with'<' not supported between 'NoneType' and 'str'on weekly buckets since 0.26.0 (fine on 0.25.1).To Reproduce
time_bucketwith atimestamp_column.column_anomaliestestExample config:
Expected behavior
The test should not fail with python error raised from elementary core.
Screenshots
n/a
Environment (please complete the following information):
elementary dbt package 0.26.0
edr 0.26.0
dbt-core 1.12.5, dbt-postgres
Postgres
Additional context
AI gives the following summary:
Precursor:
On Postgres,
get_start_bucket_in_datacallsdbt.datediff(..., 'week'), and that function starts a new week on Sunday (it compares dow, where Sunday is 0). Elementary's week buckets run Monday to Monday, so Sunday rows go into the following week.The last Sunday before
max(edr_bucket_end)maps to a bucket that doesn't exist, and the left join buckets incolumn_monitoring_queryreturns a metric row with NULLbucket_startandbucket_end. That row still gets an anomaly score.This bug predates 0.26.0; every weekly sum has been shifted by one day.
Trigger:
0.26.0 added
| sort(attribute='bucket_end')inget_anomaly_test_result_row. Sorting that NULL-bucket row against the string dates throws the error above.(Monthly and daily buckets aren't affected.)