-
Notifications
You must be signed in to change notification settings - Fork 167
feat: implement UNIQUE dynamic array function (HF-68) #1708
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
9eb508d
feat(unique): implement UNIQUE dynamic array function (HF-68)
marcin-kordas-hoc f73c1e1
fix(unique): enable array arithmetic for arguments (HF-68)
marcin-kordas-hoc fca2597
refactor(unique): drop unreachable vector-length guard in equalVector…
marcin-kordas-hoc 5e96194
ci: re-trigger coverage upload after dead-code removal (HF-68-unique)
marcin-kordas-hoc 4f31980
docs: move UNIQUE (HF-68) ADR to hyperformula-tests dev_docs
marcin-kordas-hoc ac05481
Merge branch 'develop' into feature/HF-68-unique
marcin-kordas-hoc 35ab505
docs(unique): drop direct Excel references from guide + JSDoc
marcin-kordas-hoc a4097a4
Merge branch 'develop' into feature/HF-68-unique
sequba File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
| /** | ||
| * @license | ||
| * Copyright (c) 2025 Handsoncode. All rights reserved. | ||
| */ | ||
|
|
||
| import {ArraySize} from '../../ArraySize' | ||
| import {CellError, ErrorType} from '../../Cell' | ||
| import {ErrorMessage} from '../../error-message' | ||
| import {ProcedureAst} from '../../parser' | ||
| import {InterpreterState} from '../InterpreterState' | ||
| import {InternalNoErrorScalarValue, InternalScalarValue, InterpreterValue} from '../InterpreterValue' | ||
| import {SimpleRangeValue} from '../../SimpleRangeValue' | ||
| import {FunctionArgumentType, FunctionPlugin, FunctionPluginTypecheck, ImplementedFunctions} from './FunctionPlugin' | ||
|
|
||
| /** | ||
| * Plugin implementing the UNIQUE spreadsheet function. | ||
| * | ||
| * UNIQUE(array, [by_col], [exactly_once]) returns the distinct rows (or columns | ||
| * when by_col is TRUE) of `array`, preserving first-occurrence order. When | ||
| * exactly_once is TRUE, only rows/columns that occur exactly once are returned. | ||
| * Equality delegates to {@link ArithmeticHelper}, so comparison honors the | ||
| * caseSensitive/accentSensitive configuration (case-insensitive by default). | ||
| */ | ||
| export class UniquePlugin extends FunctionPlugin implements FunctionPluginTypecheck<UniquePlugin> { | ||
| public static implementedFunctions: ImplementedFunctions = { | ||
| 'UNIQUE': { | ||
| method: 'unique', | ||
| sizeOfResultArrayMethod: 'uniqueArraySize', | ||
| enableArrayArithmeticForArguments: true, | ||
| parameters: [ | ||
| {argumentType: FunctionArgumentType.RANGE}, | ||
| {argumentType: FunctionArgumentType.BOOLEAN, defaultValue: false, emptyAsDefault: true}, | ||
| {argumentType: FunctionArgumentType.BOOLEAN, defaultValue: false, emptyAsDefault: true}, | ||
| ], | ||
| vectorizationForbidden: true, | ||
| }, | ||
| } | ||
|
|
||
| /** | ||
| * Corresponds to UNIQUE(array, [by_col], [exactly_once]). | ||
| * | ||
| * Errors found anywhere in the input range are propagated. An empty result | ||
| * (only reachable via exactly_once when nothing occurs exactly once) yields | ||
| * #N/A, mirroring FILTER's empty-result handling. | ||
| * | ||
| * @param {ProcedureAst} ast - the parsed function-call AST node. | ||
| * @param {InterpreterState} state - current interpreter evaluation state. | ||
| */ | ||
| public unique(ast: ProcedureAst, state: InterpreterState): InterpreterValue { | ||
| return this.runFunction(ast.args, state, this.metadata('UNIQUE'), | ||
| (range: SimpleRangeValue, byCol: boolean, exactlyOnce: boolean) => { | ||
| const data = range.data | ||
|
|
||
| const firstError = UniquePlugin.findFirstError(data) | ||
| if (firstError !== undefined) { | ||
| return firstError | ||
| } | ||
|
|
||
| // Work in "vectors": rows for the default, columns when by_col is TRUE. | ||
| const vectors: InternalScalarValue[][] = byCol | ||
| ? UniquePlugin.transpose(data) | ||
| : data.map(row => row.slice()) | ||
|
|
||
| const equalVectors = (v1: InternalScalarValue[], v2: InternalScalarValue[]): boolean => { | ||
| // v1 and v2 are always the same length here: they are rows (or columns, | ||
| // after transpose) of the same rectangular range, so no length check is | ||
| // needed before the element-wise comparison. | ||
| for (let i = 0; i < v1.length; i++) { | ||
| if (!this.arithmeticHelper.eq(v1[i] as InternalNoErrorScalarValue, v2[i] as InternalNoErrorScalarValue)) { | ||
| return false | ||
| } | ||
| } | ||
| return true | ||
| } | ||
|
|
||
| // Preserve first-occurrence order; count occurrences for exactly_once. | ||
| // Deduplication is O(n^2) in the number of vectors: equality is locale-aware | ||
| // (via arithmeticHelper.eq) and not trivially hashable, so each vector is | ||
| // compared against the distinct ones found so far. This matches Excel's | ||
| // observable behavior; for very large inputs it is the known cost. | ||
| const distinct: InternalScalarValue[][] = [] | ||
| const counts: number[] = [] | ||
| for (const vector of vectors) { | ||
| const existing = distinct.findIndex(d => equalVectors(d, vector)) | ||
| if (existing === -1) { | ||
| distinct.push(vector) | ||
| counts.push(1) | ||
| } else { | ||
| counts[existing] += 1 | ||
| } | ||
| } | ||
|
|
||
| const kept = exactlyOnce | ||
| ? distinct.filter((_, i) => counts[i] === 1) | ||
| : distinct | ||
|
|
||
| if (kept.length === 0) { | ||
| return new CellError(ErrorType.NA, ErrorMessage.EmptyRange) | ||
| } | ||
|
|
||
| const result = byCol ? UniquePlugin.transpose(kept) : kept | ||
| return SimpleRangeValue.onlyValues(result) | ||
| } | ||
| ) | ||
| } | ||
|
|
||
| /** | ||
| * Predicts the output array size for UNIQUE at parse time. | ||
| * The size is data-dependent, so we predict the input size as an upper bound | ||
| * (mirroring FILTER) and return the smaller actual result at runtime. A fresh | ||
| * ArraySize is returned so the input's `isRef` flag is not propagated (an | ||
| * ArraySize flagged as a ref is treated as scalar, which would collapse the | ||
| * spilled result into a single cell). | ||
| * | ||
| * @param {ProcedureAst} ast - the parsed function-call AST node. | ||
| * @param {InterpreterState} state - current interpreter evaluation state. | ||
| */ | ||
| public uniqueArraySize(ast: ProcedureAst, state: InterpreterState): ArraySize { | ||
| if (ast.args.length < 1 || ast.args.length > 3) { | ||
| return ArraySize.error() | ||
| } | ||
| const metadata = this.metadata('UNIQUE') | ||
| const subChecks = ast.args.map((arg) => | ||
| this.arraySizeForAst(arg, new InterpreterState(state.formulaAddress, state.arraysFlag || (metadata?.enableArrayArithmeticForArguments ?? false)))) | ||
| return new ArraySize(subChecks[0].width, subChecks[0].height) | ||
| } | ||
|
|
||
| /** Returns the first {@link CellError} found in a 2-D array, or undefined. */ | ||
| private static findFirstError(data: InternalScalarValue[][]): CellError | undefined { | ||
| for (const row of data) { | ||
| for (const cell of row) { | ||
| if (cell instanceof CellError) { | ||
| return cell | ||
| } | ||
| } | ||
| } | ||
| return undefined | ||
| } | ||
|
|
||
| /** Transposes a 2-D array (rows <-> columns). */ | ||
| private static transpose(data: InternalScalarValue[][]): InternalScalarValue[][] { | ||
| if (data.length === 0) { | ||
| return [] | ||
| } | ||
| const height = data.length | ||
| const width = data[0].length | ||
| const result: InternalScalarValue[][] = [] | ||
| for (let c = 0; c < width; c++) { | ||
| const col: InternalScalarValue[] = [] | ||
| for (let r = 0; r < height; r++) { | ||
| col.push(data[r][c]) | ||
| } | ||
| result.push(col) | ||
| } | ||
| return result | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Missing empty-range crash guard
Medium Severity
UNIQUEcan still produce a zero-width result when the input has height but no width (for example afteraddRowson an empty sheet, then a whole-row reference). That path builds[[]]and later hitsArrayValue, which rejects non-positive dimensions and throws. SiblingSORTalready guards this case before building a range value.Reviewed by Cursor Bugbot for commit a4097a4. Configure here.