Skip to content

Expand UtilitiesCS coverage with QuickFiler stability fixes and Codex workflow support#116

Merged
drmoisan merged 62 commits into
mainfrom
development
Apr 5, 2026
Merged

Expand UtilitiesCS coverage with QuickFiler stability fixes and Codex workflow support#116
drmoisan merged 62 commits into
mainfrom
development

Conversation

@drmoisan

@drmoisan drmoisan commented Apr 5, 2026

Copy link
Copy Markdown
Owner

Expand UtilitiesCS coverage with QuickFiler stability fixes and Codex workflow support

Summary

  • Expand UtilitiesCS coverage with a large new deterministic MSTest suite, targeted production seams, and remediation evidence, bringing aggregate UtilitiesCS line coverage to 87.4% with 61 of 63 implementation-routed files at or above 80%.
  • Fix QuickFiler/Controllers/QfcQueue.cs so RemoveItem exits cleanly when cleanup observes a pre-cancelled token, with a dedicated regression test in QuickFiler.Test/Controllers/QfcQueueTests.cs.
  • Fix QuickFiler/Controllers/KbdActions.cs so stored navigation keys use exact literal identity during registration, preventing substring collisions such as 1, 01, and 10 for the same SourceId, while preserving runtime matching behavior.
  • Make TaskMaster/AddInUtilities.cs COM entry points synchronous at the Outlook boundary.
  • Add repository-local Codex runtime assets under .agents/** and .codex/** for orchestration, commit authoring, and PR authoring workflows.
  • Reorganize feature documentation and evidence, including versioned v1/v2 coverage docs, archived completed feature folders, and refreshed audit/remediation artifacts.

Why

  • Issue #87 exists because UtilitiesCS had broad coverage gaps across shared helpers, threading, serialization, dialogs, email-intelligence modules, and Outlook-related wrappers; the feature goal was to raise every .cs file compiled by UtilitiesCS.csproj toward the repository’s >=80% coverage expectation while keeping tests deterministic and policy-compliant.
  • The current outcome is materially stronger but not absolute: phase-5 verification records 87.4% aggregate UtilitiesCS line coverage, 61 of 63 implementation-routed files at or above 80%, and two remaining below-threshold files with documented COM-constrained branches: EmailIntelligence\EmailParsingSorting\SortEmail.cs (66.7%) and EmailIntelligence\ClassifierGroups\Triage\Triage_OlLogic.cs (78.3%).
  • The QfcQueue fix was scoped to catch OperationCanceledException in RemoveItem when _token is already cancelled and return gracefully, while separately verifying that ConversationResolver.LoadConversationInfoAsync preserves the required assignment ordering before UpdateUI(...).
  • The KbdActions fix was scoped to correct a storage-identity bug: distinct stored keys for the same collection source must no longer collide, but exact duplicates must still fail and runtime keyboard-input matching semantics must remain available.
  • TaskMaster/AddInUtilities.cs was updated because COM entry points should not remain async at the Outlook boundary.
  • Codex support was added to supply a repository-local runtime for skills, orchestration, commit stewardship, and PR authoring.

What Changed

  • Core behavior / architecture

    • Expanded UtilitiesCS production seams and fixes across dialogs, email-intelligence flows, file-system wrappers, serialization, progress tracking, subject-map orchestration, and Outlook helpers.
    • Added UtilitiesCS/HelperClasses/FileSystem/PhysicalDirectoryInfoAdapter.cs and UtilitiesCS/HelperClasses/FileSystem/PhysicalFileInfoAdapter.cs to isolate file-system interactions for deterministic tests.
    • Updated UtilitiesCS/OutlookObjects/Recipient/RecipientStatic.cs alongside broader Outlook/email-intelligence changes to support safer deterministic coverage around COM-adjacent behavior.
    • Added targeted QuickFiler fixes in QuickFiler/Controllers/QfcQueue.cs and QuickFiler/Controllers/KbdActions.cs.
    • Updated TaskMaster/AddInUtilities.cs to use synchronous COM entry points.
  • Tooling / automation / CI / DevEx

    • Added repository-local skill and workflow assets under .agents/**.
    • Added Codex runtime configuration under .codex/agents/*.toml and .codex/prompts/*.md.
    • Updated .vscode/settings.json.
  • Tests

    • Added and expanded a large UtilitiesCS.Test/** suite covering dialogs, threading, serialization, email-intelligence, reusable collections, Newtonsoft helpers, helper classes, and Outlook-related code.
    • Added QuickFiler.Test/Controllers/QfcQueueTests.cs.
    • Added QuickFiler.Test/Controllers/KbdActionsTests.cs.
    • Updated QuickFiler.Test/QuickFiler.Test.csproj and UtilitiesCS.Test/UtilitiesCS.Test.csproj to compile the new test files.
  • Docs / templates / agents

    • Versioned docs/features/active/2026-03-19-utilities-coverage-part-three-87 into v1/ and v2/ documentation sets and refreshed its audit/remediation artifacts.
    • Archived completed feature folders under docs/features/archive/**.
    • Added or refreshed baseline, regression, QA-gate, audit, remediation, and coverage artifacts across the affected feature folders.
    • Added change-plan.md, coverage_output.txt, and docs/features/potential/promoted/.gitkeep.

Architecture / How It Fits Together

  • UtilitiesCS production changes are paired with mirrored UtilitiesCS.Test coverage suites and explicit project-file includes so the coverage uplift is enforced by normal solution builds and coverage-enabled MSTest runs.
  • The coverage work uses narrow seams rather than broad rewrites: file-system adapters, safer serialization/configuration hooks, progress-tracking adjustments, and orchestrator splits such as UtilitiesCS/EmailIntelligence/SubjectMap/SubjectMapSco.Orchestration.cs let deterministic tests exercise logic that previously depended on UI, COM, or host state.
  • QuickFiler regression protection is centered in controller-level tests: QfcQueueTests.cs guards pre-cancelled queue cleanup, and KbdActionsTests.cs guards exact stored-key identity and duplicate handling.
  • TaskMaster/AddInUtilities.cs remains the Outlook COM boundary, but now does so synchronously.
  • Repository automation is wired through .agents/skills/** for workflow knowledge and .codex/agents/*.toml plus .codex/prompts/*.md for runtime entry points, with docs/features/** carrying the audit and verification evidence.

Verification

Completed

  • dotnet tool run csharpier format . passed for:
    • docs/features/active/2026-03-19-utilities-coverage-part-three-87/evidence/qa-gates/phase5-csharpier.md (2026-04-03T23:57:44-04:00, EXIT_CODE: 0)
    • docs/features/active/2026-03-27-qfc-queue-remove-item-cancellation-106/evidence/qa-gates/final-qc-format.md (2026-03-27T08:56:24Z, EXIT_CODE: 0)
    • docs/features/active/2026-03-27-quickfiler-navigation-key-collision-111/evidence/qa-gates/p2-t1-format.2026-03-27T13-08.md (2026-03-27T13:08:12-04:00, EXIT_CODE: 0)
  • pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/vscode/Invoke-VSBuild.ps1 -SolutionPath TaskMaster.sln -Configuration Debug -Platform 'Any CPU' -EnableNETAnalyzers -EnforceCodeStyleInBuild passed for the same feature slices (EXIT_CODE: 0) via:
    • phase5-analyzers.md
    • final-qc-analyzer.md
    • p2-t2-analyzers.2026-03-27T13-08.md
  • pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/vscode/Invoke-VSBuild.ps1 -SolutionPath TaskMaster.sln -Configuration Debug -Platform 'Any CPU' -EnableNullable -TreatWarningsAsErrors passed for the same feature slices (EXIT_CODE: 0) via:
    • phase5-nullable.md
    • final-qc-nullable.md
    • p2-t3-nullable.2026-03-27T13-08.md
  • pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/vscode/Invoke-MSTestWithCoverage.ps1 -SearchRoot . -Configuration Debug passed for:
    • docs/features/active/2026-03-19-utilities-coverage-part-three-87/evidence/qa-gates/phase5-tests-with-coverage.md (2026-04-04T00:01:13-04:00, EXIT_CODE: 0)
    • docs/features/active/2026-03-27-qfc-queue-remove-item-cancellation-106/evidence/qa-gates/final-qc-test.md (2026-03-27T09:00:17Z, EXIT_CODE: 0)
    • docs/features/active/2026-03-27-quickfiler-navigation-key-collision-111/evidence/qa-gates/p2-t4-tests-with-coverage.2026-03-27T13-08.md (2026-03-27T13:08:12-04:00, EXIT_CODE: 0)
  • Pre-fix regression evidence was recorded for both QuickFiler bug fixes:
    • docs/features/active/2026-03-27-qfc-queue-remove-item-cancellation-106/evidence/regression-testing/expect-fail-p1t5.md (EXIT_CODE: 1)
    • docs/features/active/2026-03-27-quickfiler-navigation-key-collision-111/evidence/regression-testing/p1-t2-kbdactions-distinct-keys.2026-03-27T13-01.md (EXIT_CODE: 1)

Recommended

  • dotnet tool run csharpier format .
  • pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/vscode/Invoke-VSBuild.ps1 -SolutionPath TaskMaster.sln -Configuration Debug -Platform 'Any CPU' -EnableNETAnalyzers -EnforceCodeStyleInBuild
  • pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/vscode/Invoke-VSBuild.ps1 -SolutionPath TaskMaster.sln -Configuration Debug -Platform 'Any CPU' -EnableNullable -TreatWarningsAsErrors
  • pwsh -NoProfile -ExecutionPolicy Bypass -File scripts/vscode/Invoke-MSTestWithCoverage.ps1 -SearchRoot . -Configuration Debug
  • git diff --name-status origin/main...development

Backward Compatibility / Migration Notes

  • QuickFiler/Controllers/KbdActions.cs now distinguishes exact stored keys during registration/storage for the same SourceId; callers that relied on substring-based storage identity will now keep distinct registrations such as 1, 01, and 10.
  • QuickFiler/Controllers/QfcQueue.cs now treats pre-cancelled cleanup during RemoveItem as a graceful exit rather than propagating OperationCanceledException.
  • TaskMaster/AddInUtilities.cs changes COM-boundary behavior by making those entry points synchronous.
  • Feature-document layout changed materially:
    • docs/features/active/2026-03-19-utilities-coverage-part-three-87/* now has v1/ and v2/ subtrees.
    • Multiple completed feature folders moved from docs/features/active/** to docs/features/archive/**.
  • Deprecated sources were removed from UtilitiesCS/To Depricate/:
    • CSVDictUtilities.cs
    • FlattenArray.cs
    • StackObjectVB.cs
    • No replacement path is identified in the PR-context bundle.

Risks and Mitigations

  • Risk: Issue #87 is still not fully satisfied at the per-file threshold because SortEmail.cs and Triage_OlLogic.cs remain below 80%.
    • Mitigation: The PR includes phase-5 coverage verification, explicit follow-up documentation, and a materially improved aggregate result (87.4%).
  • Risk: The review surface is large (616 files changed, 46909 insertions), and much of it is mixed with documentation and audit churn.
    • Mitigation: Many docs changes are versioning/archive moves (R100/R098) rather than fresh runtime logic; the review guide below isolates the highest-signal paths first.
  • Risk: The KbdActions storage-identity fix could affect navigation registration semantics.
    • Mitigation: The range includes regression evidence for the failing case before the fix and final QA evidence after the fix, plus dedicated controller tests for both distinct-key and duplicate-key behavior.
  • Risk: COM/UI-adjacent code remains sensitive in UtilitiesCS and TaskMaster.
    • Mitigation: The PR adds targeted seams, deterministic tests, and a synchronous COM-boundary adjustment rather than broad architectural rewrites.
  • Risk: Rolling back the entire range at once would be expensive.
    • Mitigation: The main behavioral slices already exist as isolated merged PRs in range (#110, #112, #113, #114, #115), which provide natural rollback checkpoints.

Review Guide

  • Start with UtilitiesCS/UtilitiesCS.csproj and UtilitiesCS.Test/UtilitiesCS.Test.csproj to understand scope, compile includes, and deleted deprecated files.
  • Review the highest-risk production/test pairs in UtilitiesCS, especially:
    • UtilitiesCS/OutlookObjects/Recipient/RecipientStatic.cs + UtilitiesCS.Test/OutlookObjects/Recipient/RecipientStaticTests.cs
    • UtilitiesCS/EmailIntelligence/**/* + matching UtilitiesCS.Test/EmailIntelligence/**/*
    • UtilitiesCS/Extensions/AsyncSerialization.cs, DfDeedle.cs, IEnumerableExtensions.cs + matching tests
  • Review the new file-system and serialization seams:
    • UtilitiesCS/HelperClasses/FileSystem/PhysicalDirectoryInfoAdapter.cs
    • UtilitiesCS/HelperClasses/FileSystem/PhysicalFileInfoAdapter.cs
    • UtilitiesCS/ReusableTypeClasses/**
  • Review QuickFiler/Controllers/QfcQueue.cs with QuickFiler.Test/Controllers/QfcQueueTests.cs.
  • Review QuickFiler/Controllers/KbdActions.cs with QuickFiler.Test/Controllers/KbdActionsTests.cs.
  • Review TaskMaster/AddInUtilities.cs separately as the COM-boundary behavior change.
  • Review .agents/** and .codex/** as a separate tooling slice; they are independent of the product/runtime bug fixes.
  • Treat docs/features/** as mostly evidence, archive, audit, and versioning churn; many of those entries are mechanical moves or generated review artifacts.

Follow-ups

  • Complete or explicitly disposition the remaining below-threshold UtilitiesCS files: EmailIntelligence\EmailParsingSorting\SortEmail.cs and EmailIntelligence\ClassifierGroups\Triage\Triage_OlLogic.cs.
  • Decide whether issue #106 should be added to verified PR-closing metadata; it is referenced in the comparison range but is not listed in the verified/pending auto-close section.
  • Continue maintaining the versioned/archive feature-doc structure as open work around #87 progresses.

GitHub Auto-close

Related issues / PRs

drmoisan and others added 30 commits March 26, 2026 20:50
- repopulate collection contents in FromList and deserialization paths instead of clearing state without restoring loaded items
- route file and prompt access through injectable abstractions so backup-loading behavior can be exercised deterministically in tests
- update ScoCollection and SerializableList regression tests to assert backup-loader results and preserved target file paths
…ization

- preserve the active text-writer factory when scheduling serialize work to avoid races with later delegate replacement
- route async and fire-and-forget serialization through a shared core method that accepts the captured writer delegate
- document the synchronous and asynchronous serialization paths to clarify lock and delegate-capture behavior
- Replace direct File and MessageBox delegates with injectable file-system and prompt abstractions on SerializableList
- Snapshot the active file-system dependency before queued serialization and reuse it during fallback persistence paths
- Rewrite SerializableList tests to use Moq-backed dependency scopes instead of reflection-based field overrides
…ne evidence

- Add MSTest coverage for dialog and email-intelligence components targeted by phases P1-P14, including FolderNotFoundViewer, InputBox, MyBox, NotImplementedDialog, AutoFile, SortEmail, FilterOlFolders, FolderInfoViewer, OSBrowser, and FolderRemapController
- Register the new test files in `UtilitiesCS.Test.csproj` and add the Office interop reference needed by the new COM-facing AutoFile tests
- Capture phase-0 baseline build and coverage evidence, then update the issue #87 plan and spec to reflect completed baseline gates and in-progress coverage execution

Refs: #87
…Forms

- Wrap Application.EnableVisualStyles/SetCompatibleTextRenderingDefault
  in try-catch(InvalidOperationException) so DpiCalled is always set
  even when called after a Form has been created in the test host
- Apply CSharpier formatting to affected EmailIntelligence and Dialogs
  test files to restore toolchain-clean state
- Correct FolderRemapController_Tests reflection to target internal
  backing fields (targetModel, dragModels) via BindingFlags

Refs: #87
- Add MSTest coverage for SubjectMapEncoder rebuild, token augmentation, and encode/decode round-trip behavior
- Add SubjectMapMetrics constructor-state tests and register new EmailIntelligence test files in UtilitiesCS.Test.csproj
- Add DfDeedle array-conversion coverage and record baseline C# QA evidence while advancing plan #87 task status

Refs: #87
- Add MSTest coverage for `DfDeedle` triage normalization and date parsing edge cases
- Add WinForms STA-thread tests for `DgvForm`, `QfcTipsDetails`, `TipsController`, and `OlvExtension` behaviors
- Add `ConfigGroupBox` and `ConfigViewer` coverage, register new test files in `UtilitiesCS.Test.csproj`, and advance plan `#87` task status

`Refs: #87`
- Add EmailDataMiner_Tests: chunking, empty-source, and staging-cleanup paths (P34)
- Add EmailFiler_Tests: COM-free filing and classification scenarios
- Add IntelligenceConfig_Tests: config load, defaults, and validation paths
- Add SubjectMapSco_Tests: subject tokenization and scoring coverage
- Add ConfigController_Tests: serialization round-trips and change-detection paths
- Add AsyncMultiTasker_Tests: task scheduling, cancellation, and concurrency paths
- Add ProgressPane_Tests and ProgressViewer_Tests: state transitions and display logic
- Register all eight test classes in UtilitiesCS.Test.csproj
- Advance plan checklist to reflect delivered phases
- Add MSTest coverage for classifier-group, triage, recents-list, and OneDrive downloader behaviors, including manager reuse, build cardinality, and training callbacks
- Add helper and threading coverage for WinForms extensions, theme application, Outlook table helpers, idle timer state, and progress tracker propagation
- Mark utilities coverage plan phases 56 through 70 complete and retain the incidental ribbon XML whitespace normalization

Refs: #87
… classifiers

- Add MSTest coverage for ActionableClassifierGroup filtering and empty-data categorization paths
- Add wrapper and controller tests for ComStreamWrapper forwarding/seek behavior and StoreWrapperController population/folder selection flows
- Add ProgressTracker report, child-range propagation, and root-close coverage and mark plan phases 74 through 77 complete

Refs: #87
- Add regression coverage for Bayesian classifiers, triage training, async serialization, linked lists, file paths, timed disk writing, To-Do table handling, and UI-thread awaiter behavior
- Respect the requested train/test split, make enumerable partitioning deterministic, and guard tokenizer charset emission when code-page lookups are missing
- Resolve wrapper backing fields by naming convention before IL parsing under coverage instrumentation and mark plan phases 78 through 89 complete

Refs: #87
…th direct framework API call

- Inline `GetAvailablePhysicalMemory()` in `EmailDataMiner` using `Microsoft.VisualBasic.Devices.ComputerInfo` directly, eliminating the runtime dependency on the VBFunctions wrapper assembly
- Remove `ProjectReference` to `VBFunctions` from `UtilitiesCS.csproj` and add `Microsoft.VisualBasic` framework reference
- Update `ComputerInfo_Test.cs` to instantiate `Microsoft.VisualBasic.Devices.ComputerInfo` directly and remove the `VBFunctions` project reference from `VBFunctions.Test.csproj`
… classifiers

- Add MSTest coverage for ActionableClassifierGroup filtering and empty-data categorization paths
- Add wrapper and controller tests for ComStreamWrapper forwarding/seek behavior and StoreWrapperController population/folder selection flows
- Add ProgressTracker report, child-range propagation, and root-close coverage and mark plan phases 74 through 77 complete

Refs: #87
- Add MSTest coverage for classifier-group, triage, recents-list, and OneDrive downloader behaviors, including manager reuse, build cardinality, and training callbacks
- Add helper and threading coverage for WinForms extensions, theme application, Outlook table helpers, idle timer state, and progress tracker propagation
- Mark utilities coverage plan phases 56 through 70 complete and retain the incidental ribbon XML whitespace normalization

Refs: #87
- Add regression coverage for Bayesian classifiers, triage training, async serialization, linked lists, file paths, timed disk writing, To-Do table handling, and UI-thread awaiter behavior
- Respect the requested train/test split, make enumerable partitioning deterministic, and guard tokenizer charset emission when code-page lookups are missing
- Resolve wrapper backing fields by naming convention before IL parsing under coverage instrumentation and mark plan phases 78 through 89 complete

Refs: #87
(cherry picked from commit 364fe83)
(cherry picked from commit ebe3d29)
(cherry picked from commit 263323d)
Sync main to development
drmoisan and others added 29 commits March 27, 2026 09:12
…llation-106

Bug/qfc queue remove item cancellation 106
…ollision-111

Fix QuickFiler navigation key collisions during collection rebuild
- add deterministic MSTest coverage for dialog-driven flows in InputBox, MyBox, NotImplementedDialog, YesNoToAll, and related AutoFile and PeopleScoDictionary paths
- add focused coverage for ManagerAsyncLazy, FilterOlFolders controller/viewer, OSBrowser, SortEmail, TipsController, and AsyncSerialization edge paths
- introduce minimal dialog-invoker seams in UtilitiesCS dialog classes and capture supporting remediation evidence for utilities coverage part three

Refs: #87
- add deterministic MSTest coverage for FolderRemapController drop, sync, checked-state, and constructor paths, including a new partial test file wired into the test project
- add SubjectMapEncoder coverage for default-construction, null-guard, rebuild, augment, and string-encode branches to advance utilities coverage remediation
- update MyBox to read absolute column widths without showing the form and mark the corresponding remediation-plan tasks complete

Refs: #87
Make TaskMaster COM entry points synchronous in `AddInUtilities`
- scaffold the canonical .agents skill tree for shared Codex workflows
- add the commit-message-conventions skill, commit-steward agent, and prompt launcher
- document the migration layering and restore the Codex change plan of record
- add the shared orchestrator-workflow skill plus thin orchestrator agent and prompt launcher
- centralize canonical branch and plan-path rules in feature-promotion-lifecycle to avoid duplication
- update migration docs and change-plan guidance for the new orchestration layer
… UI helpers

- add COM-safe DfDeedle seams and extract table-to-frame conversion so Outlook table paths can be exercised without live COM dependencies
- add deterministic MSTest coverage for DfDeedle, QfcTipsDetails, ConfigGroupBox, ConfigViewer, and ProgressViewer, including headless form scenarios for handler-only paths
- register the new COM-focused DfDeedle test file and update the remediation plan to record completed coverage tasks and hardening notes

Refs: #87
- add the shared pr-authoring skill plus thin pr-author agent and generate-pr prompt for GitHub-ready PR body generation
- publish drmCopilotExtension as the canonical Codex automation surface and centralize its MCP dependency metadata in repo-automation-adapter
- update PR-context and migration guidance to prefer semantic MCP tools with explicit base-branch passing before git fallbacks
Add Codex runtime support for repository skills, orchestration, and PR authoring
- add focused MSTest coverage across dialog helpers, email-intelligence workflows, file-system wrappers, and threading utilities
- introduce targeted production seams for serialization, subject-map orchestration, and physical file-system adapters to enable deterministic tests
- capture phase 5 QA gate evidence and returned-scope decisions in the utilities coverage remediation records

Refs: #87
…ests

- replace ProgressPane-backed setup with a headless ProgressTrackerPane that uses synchronous progress reporting
- verify job-name updates through reflected pane state instead of WinForms controls on MSTest thread-pool threads
- document why the WinForms synchronization context causes these async overload tests to hang without a message pump

Refs: #87
- update SortEmail test mocks to match the boxed COM SaveAs argument and resolve the repository root from the test assembly path
- make PhysicalFileSystemAdapters tests tolerate expected timestamp-setter contention in shared VS Code and test-host environments
- exercise file adapter stream operations sequentially so assertions cover behavior without overlapping handle conflicts

Refs: #87
- inject `ShowProgressViewer` hooks in progress trackers so tests can verify initialization without displaying UI
- let `ProgressTrackerPane` ignore missing or disposed viewers while headless tests cover root and child reporting flows
- replace interactive dialog assertions with deterministic delegate checks and mark manual resource form smoke tests as ignored

Refs: #87
- add a ConfigController display seam so production code can still show the viewer while tests suppress visible windows
- switch the config controller workflow test to initialize the viewer without calling the Show() factory path
- Move the March 27 policy, code, and feature review documents into a timestamped audit folder
- Relocate the matching remediation inputs and remediation plan alongside the original audit set
- Preserve the historical review package without changing artifact contents
…-three-87-clean

Expand UtilitiesCS coverage with deterministic test seams and remediation evidence
@drmoisan
drmoisan merged commit 302dd27 into main Apr 5, 2026
2 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant