Wip/precomputed experiment segments

[danoswaltCL]

this completes the work started last release with precomputed segments

• creates precomputed-experiment-segments table
• backfills on startup if experiments don't have a row in that table (should only need to happen first time the code version runs)
• tries to mirror the precomputed-feature-flag code paths as closely as possible
• creates shared services and utilities where it's exactly the same
• clearly notes where the implementation diverge due to differences in legacy code services and the fact that experiment private lists are a bit different from feature-flag private lists (they don't need enabling events)
• global exclude lists are NOT compiled with the precomputed exclude lists, left that as is

[Copilot]

Pull request overview

This PR (WIP) introduces precomputed segment membership for experiments (mirroring feature flags) to speed up assignment-time inclusion/exclusion checks, while also removing legacy twoCharacterId fields and updating the codebase for newer TypeORM APIs and v6.6.0 versioning.

Changes:

• Add experiment_precomputed_segment storage + services, including startup backfill and assignment-time fallback when rows/tables are missing.
• Remove twoCharacterId from experiment conditions/decision points across backend + frontend models/tests and add DB migrations to drop columns.
• Update many backend queries/tests for newer TypeORM patterns (e.g., relations: { ... }, findBy(...), select: { ... }), plus frontend table trackBy and a global .break-all utility class.

Reviewed changes

Copilot reviewed 34 out of 35 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
packages/types/src/Experiment/enums.ts	Add cache prefix for experiment precomputed segments
packages/types/package.json	Bump shared types version to 6.6.0
packages/frontend/projects/upgrade/src/testing/test.mock.data.ts	Remove twoCharacterId from frontend test fixtures
packages/frontend/projects/upgrade/src/styles.scss	Add .break-all word-breaking utility
packages/frontend/projects/upgrade/src/app/features/dashboard/segments/pages/segment-root-page/segment-root-page-content/segment-root-section-card/segment-root-section-card-table/segment-root-section-card-table.component.ts	Add trackBy for segment table rows
packages/frontend/projects/upgrade/src/app/features/dashboard/segments/pages/segment-root-page/segment-root-page-content/segment-root-section-card/segment-root-section-card-table/segment-root-section-card-table.component.html	Wire up trackBy for segment table
packages/frontend/projects/upgrade/src/app/features/dashboard/feature-flags/pages/feature-flag-root-page/feature-flag-root-page-content/feature-flag-root-section-card/feature-flag-root-section-card-table/feature-flag-root-section-card-table.component.ts	Add trackBy for feature flag table rows
packages/frontend/projects/upgrade/src/app/features/dashboard/feature-flags/pages/feature-flag-root-page/feature-flag-root-page-content/feature-flag-root-section-card/feature-flag-root-section-card-table/feature-flag-root-section-card-table.component.html	Use trackBy; switch exposures field to exposureCount
packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-root-page/experiment-root-page-content/experiment-root-section-card/experiment-root-section-card-table/experiment-root-section-card-table.component.ts	Add trackBy for experiment table rows
packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-root-page/experiment-root-page-content/experiment-root-section-card/experiment-root-section-card-table/experiment-root-section-card-table.component.html	Wire up trackBy for experiment table
packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/experiment-details-page-content/experiment-payloads-section-card/experiment-payloads-table/experiment-payloads-table.component.html	Apply .break-all to decision point display
packages/frontend/projects/upgrade/src/app/features/dashboard/experiments/pages/experiment-details-page/experiment-details-page-content/experiment-decision-points-section-card/experiment-decision-points-table/experiment-decision-points-table.component.html	Apply .break-all to decision point display
packages/frontend/projects/upgrade/src/app/core/experiments/store/experiments.selector.spec.ts	Remove twoCharacterId usage in selector tests
packages/frontend/projects/upgrade/src/app/core/experiments/store/experiments.model.ts	Remove twoCharacterId from frontend experiment interfaces/DTOs
packages/frontend/projects/upgrade/src/app/core/experiments/condition-helper.service.spec.ts	Remove twoCharacterId from condition test helper
packages/frontend/package.json	Bump frontend to 6.6.0; update Angular deps
packages/backend/test/utils/database.ts	Update TypeORM Postgres options types; set migrations transaction mode
packages/backend/test/unit/services/UserService.test.ts	Update repo mocks/expectations from findByIds to findBy
packages/backend/test/unit/services/SegmentService.test.ts	Add ExperimentPrecomputedSegmentService to unit test DI setup
packages/backend/test/unit/services/ScheduledJobService.test.ts	Update relations expectations to object form
packages/backend/test/unit/services/MoocletRewardsService.test.ts	Update relations expectations to object form
packages/backend/test/unit/services/MoocletExperimentService.test.ts	Inject ExperimentPrecomputedSegmentService into MoocletExperimentService tests
packages/backend/test/unit/services/FeatureFlagService.test.ts	Update feature-flag INCLUDE_ALL/EXCLUDE_ALL tests and semantics assertions
packages/backend/test/unit/services/FeatureFlagPrecomputedSegmentService.test.ts	Adjust tests for shared base method names/log strings
packages/backend/test/unit/services/ExperimentService.test.ts	Inject experiment precomputed service; add tests for recompute on context change; add search escaping tests
packages/backend/test/unit/services/ExperimentPrecomputedSegmentService.test.ts	New unit tests for experiment precomputed segments service
packages/backend/test/unit/services/ExperimentAssignmentService.test.ts	Inject experiment precomputed service; add INCLUDE_ALL/EXCLUDE_ALL logic tests; pass logger through
packages/backend/test/unit/repositories/IndividualEnrollmentRepository.test.ts	Update select expectation to object form
packages/backend/test/unit/repositories/GroupEnrollmentRepository.test.ts	Update select expectation to object form
packages/backend/test/unit/repositories/ExperimentRepository.test.ts	Update EntityManager access from connection to dataSource
packages/backend/test/unit/repositories/ExperimentConditionRepository.test.ts	Remove tests for deleted twoCharacterId uniqueness helper
packages/backend/test/unit/repositories/DecisionPointRepository.test.ts	Remove tests for deleted twoCharacterId uniqueness helper
packages/backend/test/unit/repositories/AnalyticsRepository.test.ts	Update relations expectations to object form
packages/backend/test/unit/mockdata/raw.ts	Remove twoCharacterId from backend unit fixtures
packages/backend/test/unit/controllers/SegmentController.test.ts	Remove TypeORM useContainer wiring (TypeORM 1.0 change)
packages/backend/test/integration/PreviewExperiment/DeletePreviewAssignmentsWithExperimentUpdate.ts	Remove twoCharacterId from integration fixtures
packages/backend/test/integration/mockData/experiment/raw.ts	Remove twoCharacterId from integration fixtures
packages/backend/test/integration/mockData/experiment/index.ts	Remove twoCharacterId from integration fixtures
packages/backend/test/integration/FeatureFlags/FeatureFlagInclusionExclusion.ts	Update exposures assertion field to exposureCount
packages/backend/test/integration/ExperimentUser/NoExperimentUserOnAssignment.ts	Fix expectation to resolve to [] (not {})
packages/backend/test/integration/Experiment/update/UpdateExperiment.ts	Remove twoCharacterId from integration fixtures
packages/backend/test/integration/Experiment/onlyExperimentPoint/NoPartitionPoint.ts	Remove twoCharacterId from integration fixtures
packages/backend/test/integration/Experiment/dataLog/RepeatedMeasure.ts	Update relations to object form
packages/backend/test/integration/Experiment/dataLog/CreateLog.ts	Update relations to object form
packages/backend/test/integration/Experiment/createWithDecimal/DecimalAssigmentWeight.ts	Remove twoCharacterId from integration fixtures
packages/backend/test/integration/Experiment/conditionAndPartition/Partition.ts	Remove twoCharacterId from integration assertions
packages/backend/test/integration/Experiment/conditionAndPartition/Condition.ts	Remove twoCharacterId from integration assertions/fixtures
packages/backend/test/integration/Experiment/analytics/MonitoredPointForExport.ts	Replace getRepository usage with DataSource from DI
packages/backend/src/types/index.ts	Add shared types for segment resolution/precomputed read paths
packages/backend/src/loaders/typeormLoader.ts	Update connection options typing for TypeORM 1.0/DataSourceOptions
packages/backend/src/init/seed/backfillExperimentPrecomputedSegments.ts	Add startup backfill hook for experiment precomputed rows
packages/backend/src/database/seeds/User.ts	Migrate seeding to typeorm-extension factory manager
packages/backend/src/database/seeds/Experiment.seed.ts	Migrate seeding to typeorm-extension; fix random-weight loop
packages/backend/src/database/migrations/1783627365221-experimentPrecomputedSegment.ts	Add experiment_precomputed_segment table
packages/backend/src/database/migrations/1782416524885-remove-twoCharacterId.ts	Drop twoCharacterId columns from DB schema
packages/backend/src/database/factories/ExperimentUser.ts	Migrate factory to @faker-js/faker + typeorm-extension
packages/backend/src/database/factories/ExperimentSegment.factory.ts	Remove legacy typeorm-seeding factory
packages/backend/src/database/factories/ExperimentCondition.factory.ts	Migrate factory; remove experimentId “settings” dependency
packages/backend/src/database/factories/Experiment.factory.ts	Migrate factory; set context/backendVersion; fix faker API usage
packages/backend/src/database/factories/DecisionPoint.factory.ts	New DecisionPoint factory via typeorm-extension
packages/backend/src/app.ts	Add best-effort startup backfill for experiment precomputed table
packages/backend/src/api/services/UserService.ts	Update TypeORM query APIs; adjust get-by-email; (contains a welcome-email bug)
packages/backend/src/api/services/SegmentService.ts	Trigger recompute for both flag + experiment precomputed tables; update relations loading style
packages/backend/src/api/services/ScheduledJobService.ts	Update relations style; add not-found handling
packages/backend/src/api/services/QueryService.ts	Update relations style (nested relation graph)
packages/backend/src/api/services/PrecomputedSegmentServiceBase.ts	New shared base for flag/experiment precomputed orchestration
packages/backend/src/api/services/precomputedSegmentHelpers.ts	Shared helpers for flattening segment members + group key composition
packages/backend/src/api/services/MoocletRewardsService.ts	Update relations style (nested)
packages/backend/src/api/services/MoocletExperimentService.ts	Recompute experiment precomputed row after mooclet create/update commit
packages/backend/src/api/services/ImportExportService.ts	Update relations style for export graph
packages/backend/src/api/services/FeatureFlagService.ts	Replace removed TypeORM relation-count APIs; adjust INCLUDE_ALL semantics; add exposure-count select
packages/backend/src/api/services/FeatureFlagPrecomputedSegmentService.ts	Refactor to extend shared base + shared helpers
packages/backend/src/api/services/ExperimentUserService.ts	Move global-exclude-segment recreation after truncate to avoid module cycle
packages/backend/src/api/services/ExperimentService.ts	Hook recompute into experiment writes; remove twoCharacterId uniqueness logic; improve LIKE escaping; (contains create() await bug)
packages/backend/src/api/services/ExperimentPrecomputedSegmentService.ts	New experiment precomputed segments service extending shared base
packages/backend/src/api/services/ExperimentAssignmentService.ts	Read-path uses experiment precomputed rows with safe fallback; pass logger to exclusion logic
packages/backend/src/api/services/CheckService.ts	Update relations style
packages/backend/src/api/services/CacheService.ts	Add experiment precomputed prefix to cache bucket mapping
packages/backend/src/api/services/AnalyticsService.ts	Update relations style
packages/backend/src/api/repositories/IndividualEnrollmentRepository.ts	Update select style to object form
packages/backend/src/api/repositories/GroupEnrollmentRepository.ts	Update select style to object form
packages/backend/src/api/repositories/FeatureFlagSegmentInclusionRepository.ts	Replace onConflict(DO NOTHING) with .orIgnore()
packages/backend/src/api/repositories/FeatureFlagSegmentExclusionRepository.ts	Replace onConflict(DO NOTHING) with .orIgnore()
packages/backend/src/api/repositories/ExperimentRepository.ts	Use EntityManager dataSource; avoid seed-module import cycle
packages/backend/src/api/repositories/ExperimentPrecomputedSegmentRepository.ts	New repository for experiment precomputed segment rows
packages/backend/src/api/repositories/ExperimentConditionRepository.ts	Remove twoCharacterId uniqueness helper
packages/backend/src/api/repositories/DecisionPointRepository.ts	Remove twoCharacterId uniqueness helper
packages/backend/src/api/repositories/AnalyticsRepository.ts	Update relations style; minor formatting
packages/backend/src/api/models/FeatureFlag.ts	Add exposureCount virtual property
packages/backend/src/api/models/ExperimentPrecomputedSegment.ts	New model for experiment precomputed table
packages/backend/src/api/models/ExperimentCondition.ts	Remove twoCharacterId column/validators
packages/backend/src/api/models/DecisionPoint.ts	Remove twoCharacterId column/validators
packages/backend/src/api/DTO/ExperimentDTO.ts	Remove twoCharacterId validators from DTO
packages/backend/src/api/controllers/ExperimentController.ts	Update OpenAPI schema docs to remove twoCharacterId and extend search enums
packages/backend/package.json	Bump backend to 6.6.0; upgrade TypeORM; switch to typeorm-extension seeding
packages/backend/package-scripts.js	Remove legacy db seed/drop scripts (typeorm-seeding)
packages/backend/CLAUDE.md	Document experiment precomputed segments design and invariants
packages/backend/.env.test	Change test DB port (currently inconsistent with docker-compose)
package.json	Bump root version to 6.6.0
clientlibs/python/pyproject.toml	Bump python client version to 6.6.0
clientlibs/js/yarn.lock	Lockfile updates (dependency bumps)
clientlibs/js/packages/lite/package.json	Bump JS lite client version to 6.6.0
clientlibs/js/packages/full-node/package.json	Bump JS node client version to 6.6.0
clientlibs/js/packages/full-browser/package.json	Bump JS browser client version to 6.6.0
clientlibs/js/package.json	Bump JS client meta package version to 6.6.0
clientlibs/java/pom.xml	Bump Java client version to 6.6.0

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[danoswaltCL]

@copilot please re-review after fixes, and please focus effort on parity between experiment and feature flag implementations. The requirements in the end are the same, so these should be as DRY as they can be to avoid duplicate code, without abstracting or refactoring existing code too much. The notable differences are that experiment private lists do not use list-level 'enable' flags, so that is a simplifier, but experiments do need to factor in global-excludes, as these should remain handled separately the precomputed segments. Some code differences that perform very similar tasks are likely due to legacy codebase; it is not worth a giant refactor to get the code working exactly the same, as long as we are decently DRY and that a human reading this can follow the same general pattern for precomputed segment behavior through the two code paths. In general also: the efficiency of the read paths is more important than efficiency of the write paths.

[Copilot]

@copilot please re-review after fixes, and please focus effort on parity between experiment and feature flag implementations. The requirements in the end are the same, so these should be as DRY as the...

I re-reviewed with parity as the main lens. The experiment path now follows the same precomputed-segment pattern as feature flags via the shared base service (PrecomputedSegmentServiceBase) and shared helpers, while preserving the expected differences: experiment lists have no list-level enabled filter and global-excludes remain separate from experiment precomputed rows. I also reran targeted backend unit tests for ExperimentService + both precomputed services, and they pass.

[bcb37]

@danoswaltCL Is there any reason global excludes couldn't be folded into this too? I can't remember why we evaluate those separately, but I think we should be able to calculate them along with the others, just a matter of adding the segment of type 'global_exclude' with the same context as the experiment to the exclude lists.