Skip to content

docs(tasks): the duplicate check's reply chain counts +2 #305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
leemour merged 1 commit into from
Oct 18, 2026
Merged

docs(tasks): the duplicate check's reply chain counts +2 #305

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions content/docs/tasks-architecture.es.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,7 +460,7 @@ Los primeros pesos se fijaron a mano y después se ajustaron con un conjunto de
| Importe: igual / distinto / falta | +6 / −4 / 0 |
| Categoría: misma / distinta / desconocida | +1 / −1 / 0 |
| Fecha límite: 2 días o menos / más / falta | +4 / −1 / 0 |
| Cadena de respuestas: sí / no | +6 / 0 |
| Cadena de respuestas: sí / no | +2 / 0 |
| Tiempo: 3 días o menos / de 4 a 14 días / más | +2 / −2 / −3 |

Ningún duplicado del conjunto tenía otra persona, otro importe u otra categoría, así que contar daría
Expand Down Expand Up @@ -490,8 +490,10 @@ duplicados entre idiomas se detectan mal (2 de 5 salieron «distinta»).
**En chats reales.** Una prueba posterior repitió 148 tareas que un agente redactó a partir de 1104
mensajes reales de 7 chats ([seguimiento con chats reales](https://github.com/WireCatLabs/cli-testing/tree/main/performance/tasks/duplicates#follow-up-real-chats-2026-10-17)).
Comparando por cuándo se escribieron las tareas, `tasks add` rechazó 74, y «probable» acertó el 27% de
las veces. Comparando por cuándo se enviaron los mensajes, rechazó 36, y «probable» acertó el 50%. Los
pesos no cambiaron: 85 pares etiquetados son muy pocos para ajustarlos.
las veces. Comparando por cuándo se enviaron los mensajes, rechazó 36, y «probable» acertó el 50%. Un
segundo conjunto llevó las etiquetas a 213 pares ([segundo conjunto etiquetado](https://github.com/WireCatLabs/cli-testing/tree/main/performance/tasks/duplicates#follow-up-a-second-labelled-set-2026-10-17)). Entre los pares mostrados
como «quizá» o «probable», los de una misma cadena de respuestas fallaron 11 veces de 13, así que la cadena de
respuestas ahora cuenta +2, no +6. Los demás pesos no cambiaron.

### 6. Decidir

Expand Down
6 changes: 4 additions & 2 deletions content/docs/tasks-architecture.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -439,7 +439,7 @@ The first weights were set by hand, then tuned on a synthetic test set:
| Amount: equal / different / missing | +6 / −4 / 0 |
| Category: same / different / unknown | +1 / −1 / 0 |
| Due date: 2 days or fewer / more / missing | +4 / −1 / 0 |
| Reply chain: yes / no | +6 / 0 |
| Reply chain: yes / no | +2 / 0 |
| Time: 3 days or fewer / 4 to 14 days / more | +2 / −2 / −3 |

No duplicate in the test set had a different person, amount or category, so counting would give
Expand Down Expand Up @@ -469,7 +469,9 @@ across languages are weak (2 of 5 scored "different").
7 chats ([real chats follow-up](https://github.com/WireCatLabs/cli-testing/tree/main/performance/tasks/duplicates#follow-up-real-chats-2026-10-17)).
Compared by when the tasks were written, `tasks add` refused 74 of them, and "likely" was right 27% of
the time. Compared by when their messages were sent, it refused 36, and "likely" was right 50% of the
time. The weights did not change: 85 labelled pairs are too few to tune them.
time. A second set brought the labels to 213 pairs ([second labelled set](https://github.com/WireCatLabs/cli-testing/tree/main/performance/tasks/duplicates#follow-up-a-second-labelled-set-2026-10-17)). Among the pairs shown as
"maybe" or "likely", the ones in one reply chain were wrong 11 times in 13, so the reply chain now counts +2, not
+6. The other weights did not change.

### 6. Decide

Expand Down
6 changes: 4 additions & 2 deletions content/docs/tasks-architecture.ru.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -453,7 +453,7 @@ ORDER BY bm25(task_stems) LIMIT 20
| Сумма: равны / разные / нет | +6 / −4 / 0 |
| Категория: та же / другая / неизвестна | +1 / −1 / 0 |
| Срок: 2 дня и меньше / больше / нет | +4 / −1 / 0 |
| Цепочка ответов: да / нет | +6 / 0 |
| Цепочка ответов: да / нет | +2 / 0 |
| Время: 3 дня и меньше / от 4 до 14 дней / больше | +2 / −2 / −3 |

Ни у одного дубликата в наборе не было другого человека, суммы или категории, поэтому подсчёт дал бы
Expand Down Expand Up @@ -484,7 +484,9 @@ ORDER BY bm25(task_stems) LIMIT 20
прогнали через проверку ([продолжение на настоящих чатах](https://github.com/WireCatLabs/cli-testing/tree/main/performance/tasks/duplicates#follow-up-real-chats-2026-10-17)).
При сравнении по времени записи задач `tasks add` отказал 74 из них, а «вероятно» было верным в 27%
случаев. При сравнении по времени отправки сообщений отказов стало 36, а «вероятно» было верным в 50%.
Веса не менялись: 85 размеченных пар — слишком мало, чтобы их настраивать.
Второй набор довёл разметку до 213 пар ([второй размеченный набор](https://github.com/WireCatLabs/cli-testing/tree/main/performance/tasks/duplicates#follow-up-a-second-labelled-set-2026-10-17)). Среди пар, показанных как «возможно» или
«вероятно», пары из одной цепочки ответов были неверны 11 раз из 13, поэтому цепочка ответов теперь даёт +2, а не
+6. Остальные веса не менялись.

### 6. Решение

Expand Down
Loading