Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Unreleased

### Added
- `assert_have_been_called_with_any <spy> <expected>`: passes when *any* recorded call matches, instead of only the last one (#897)
- A failed call assertion now prints the calls recorded for that spy, capped at 10 with an explicit `… and N more` (#896)
- `assert_have_been_called_with_args <spy> <arg>...`: compares the recorded arguments one by one, so `cmd "a b"` no longer matches `cmd a b` (#894)
- `BASHUNIT_COVERAGE_ENGINE=auto|xtrace|trap`: a new `xtrace` coverage engine, ~4x cheaper per captured line. Needs Bash 4.1+, so `auto` (the default) falls back to `trap` below that (#860)
Expand All @@ -12,6 +13,7 @@
- A drop-in agent skill at [bashunit.com/bashunit-skill.md](https://bashunit.com/bashunit-skill.md)

### Changed
- A failed `assert_have_been_called_with` / `_with_args` now states which call it compared (`compared 'the last of 2 calls'`) (#897)
- Deprecated forms now warn at runtime on stderr; silence with `BASHUNIT_NO_DEPRECATION_WARNINGS=true` (#866)
- `bashunit::state::print_line` / `print_tap_line` moved to `bashunit::console_results::*`; no alias kept (#868)
- `bashunit doc` and the [Assertions](https://bashunit.com/assertions) page now cover all 71 assertions, with a quick-reference table
Expand Down
3 changes: 3 additions & 0 deletions docs/ai-agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,9 @@ actually make against this API:
- Spy assertion argument order is inconsistent — check, don't guess:
`assert_have_been_called_times <count> <spy>` but
`assert_have_been_called_with <spy> <expected> [call_index]`.
- `assert_have_been_called_with <spy> <expected> [call_index]` compares a single call (the
last one by default; the failure names which). `assert_have_been_called_with_any <spy>
<expected>` matches any recorded call — prefer it unless the position is the requirement.
- A failed call assertion prints the spy's recorded calls under the diff — read that block
before guessing why it failed, it is usually enough to fix the test in one pass.
- A call assertion on a name that was never passed to `bashunit::spy` fails with
Expand Down
27 changes: 25 additions & 2 deletions docs/assertions.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ to narrow it (`bashunit doc json`).
| **JSON** | [assert_json_equals](#assert-json-equals) · [assert_json_contains](#assert-json-contains) · [assert_json_key_exists](#assert-json-key-exists) |
| **Duration** | [assert_duration](#assert-duration) · [assert_duration_less_than](#assert-duration-less-than) · [assert_duration_greater_than](#assert-duration-greater-than) |
| **Snapshots** | [assert_match_snapshot](#assert-match-snapshot) · [assert_match_snapshot_ignore_colors](#assert-match-snapshot-ignore-colors) |
| **Spies** | [assert_have_been_called](#assert-have-been-called) · [assert_not_called](#assert-not-called) · [assert_have_been_called_with](#assert-have-been-called-with) · [assert_have_been_called_with_args](#assert-have-been-called-with-args) · [assert_have_been_called_nth_with](#assert-have-been-called-nth-with) · [assert_have_been_called_times](#assert-have-been-called-times) |
| **Spies** | [assert_have_been_called](#assert-have-been-called) · [assert_not_called](#assert-not-called) · [assert_have_been_called_with](#assert-have-been-called-with) · [assert_have_been_called_with_any](#assert-have-been-called-with-any) · [assert_have_been_called_with_args](#assert-have-been-called-with-args) · [assert_have_been_called_nth_with](#assert-have-been-called-nth-with) · [assert_have_been_called_times](#assert-have-been-called-times) |
| **Manual failure** | [bashunit::fail](#bashunit-fail) |

## assert_true
Expand Down Expand Up @@ -1657,7 +1657,7 @@ function test_failure() {
## assert_have_been_called_with
> `assert_have_been_called_with "command" "expected_args" [nth]`

Reports an error if the spied `command` was not called with `expected_args`. Checks the **last** call unless a trailing all-digits `nth` selects a specific one.
Reports an error if the spied `command` was not called with `expected_args`. Checks the **last** call unless a trailing all-digits `nth` selects a specific one; the failure names the call it compared. To match any call, use [assert_have_been_called_with_any](#assert-have-been-called-with-any).

Note the argument order: the spy comes first here, but *second* in [assert_have_been_called_times](#assert-have-been-called-times).

Expand All @@ -1679,6 +1679,29 @@ function test_failure() {
```
:::

## assert_have_been_called_with_any
> `assert_have_been_called_with_any "command" "expected_args"`

Reports an error if no recorded call to the spied `command` received `expected_args`. Where [assert_have_been_called_with](#assert-have-been-called-with) compares a single call — the last one, or the one at `nth` — this one scans them all, so the assertion does not break when an unrelated call is added after it.

::: code-group
```bash [Example]
function test_success() {
bashunit::spy send_email
notify_all "a@b.c" "d@e.f"

assert_have_been_called_with_any send_email "--to a@b.c"
}

function test_failure() {
bashunit::spy send_email
notify_all "a@b.c" "d@e.f"

assert_have_been_called_with_any send_email "--to nobody@example.com"
}
```
:::

## assert_have_been_called_with_args
> `assert_have_been_called_with_args "command" "expected_arg"...`

Expand Down
9 changes: 7 additions & 2 deletions docs/public/bashunit-skill.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,8 +134,8 @@ filters it. The same list is at https://bashunit.com/assertions. **Do not invent
- JSON: `assert_json_equals`, `assert_json_contains`, `assert_json_key_exists`
- Snapshots: `assert_match_snapshot`, `assert_match_snapshot_ignore_colors`
- Spies: `assert_have_been_called`, `assert_not_called`, `assert_have_been_called_with`,
`assert_have_been_called_with_args`, `assert_have_been_called_times`,
`assert_have_been_called_nth_with`
`assert_have_been_called_with_any`, `assert_have_been_called_with_args`,
`assert_have_been_called_times`, `assert_have_been_called_nth_with`

`assert_same` compares exactly. `assert_equals` first strips ANSI colour codes, tabs and
newlines — useful for asserting on coloured CLI output, misleading everywhere else.
Expand Down Expand Up @@ -169,6 +169,10 @@ local out; out=$(failing_command) # WRONG under set -e
with `was never registered as a spy` (it used to report zero calls, so `assert_not_called`
with a typo passed while asserting nothing). Register the spy in the test that asserts on it.

**`assert_have_been_called_with` compares one call only** — the last, or the one at the
optional trailing index. Adding an unrelated later call breaks it. When the requirement is
"this happened at some point", use `assert_have_been_called_with_any`.

**`assert_have_been_called_with` joins arguments with spaces**, so it cannot see argument
boundaries: `touch "a b"` also satisfies `assert_have_been_called_with touch "a" "b"`. When
an argument may contain a space — any path — use `assert_have_been_called_with_args`, which
Expand Down Expand Up @@ -201,6 +205,7 @@ assert_have_been_called_times 2 send_email # count FIRST, then spy
assert_have_been_called_with send_email "--to a@b.c" # spy FIRST, then expected
assert_have_been_called_with send_email "--to a@b.c" 1 # ...of call #1
assert_have_been_called_nth_with 1 send_email "--to a@b.c"
assert_have_been_called_with_any send_email "--to a@b.c" # any call, not just the last
assert_have_been_called_with_args send_email "--to" "a@b.c" # boundary-exact, no index
```

Expand Down
31 changes: 31 additions & 0 deletions docs/test-doubles.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,8 @@ function test_failure() {

Reports an error if `spy` is not called with `expected`. When `call_index` is provided, the assertion checks the arguments of that specific call (starting at 1). Without `call_index` it checks the last invocation. Arguments are joined with spaces before comparison.

Only one call is compared, and the failure says which one (`compared 'the last of 2 calls'`). To match any recorded call, use [assert_have_been_called_with_any](#assert-have-been-called-with-any).

::: code-group
```bash [Example]
function test_success() {
Expand All @@ -205,6 +207,35 @@ function test_failure() {
:::


## assert_have_been_called_with_any
> `assert_have_been_called_with_any spy expected`

Reports an error if **no** recorded call of `spy` received `expected`. Use it when the requirement is "this side effect happened" rather than "this side effect happened last": `assert_have_been_called_with` only compares one call, so it breaks as soon as an unrelated call is added after it.

Arguments are joined with spaces, like `assert_have_been_called_with`.

::: code-group
```bash [Example]
function test_success() {
bashunit::spy ps

ps first
ps second

assert_have_been_called_with_any ps "first"
}

function test_failure() {
bashunit::spy ps

ps first
ps second

assert_have_been_called_with_any ps "third"
}
```
:::

## assert_have_been_called_with_args
> `assert_have_been_called_with_args spy expected...`

Expand Down
113 changes: 93 additions & 20 deletions src/test_doubles.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,45 @@ function bashunit::spy::times_to_slot() {
fi
}

_BASHUNIT_SPY_CALL_OUT=""
_BASHUNIT_SPY_CALL_TOTAL_OUT=0

# Reads the params file $1 in a single pass: the recorded line for call $2 (the
# last one when $2 is empty) lands in _BASHUNIT_SPY_CALL_OUT, the number of
# recorded calls in _BASHUNIT_SPY_CALL_TOTAL_OUT. Both are needed together,
# because the failure message names which call it compared.
# Arguments: $1 - params file, $2 - call index (optional)
function bashunit::spy::read_call_to_slots() {
local file=$1
local index=${2:-}
_BASHUNIT_SPY_CALL_OUT=""
_BASHUNIT_SPY_CALL_TOTAL_OUT=0

if [ -z "$file" ] || [ ! -f "$file" ]; then
return
fi

local current
while IFS= read -r current; do
_BASHUNIT_SPY_CALL_TOTAL_OUT=$((_BASHUNIT_SPY_CALL_TOTAL_OUT + 1))
if [ -z "$index" ] || [ "$_BASHUNIT_SPY_CALL_TOTAL_OUT" = "$index" ]; then
_BASHUNIT_SPY_CALL_OUT=$current
fi
done <"$file"
}

# Names the call a last-call/indexed assertion compared, for the failure block.
# Arguments: $1 - call index (empty for the last call), $2 - total calls
function bashunit::spy::compared_call() {
if [ -n "$1" ]; then
builtin echo "call $1 of $2"
elif [ "$2" -eq 1 ]; then
builtin echo "the only call"
else
builtin echo "the last of $2 calls"
fi
}

_BASHUNIT_SPY_CALL_LOG_MAX=10
_BASHUNIT_SPY_CALL_LOG_OUT=""

Expand Down Expand Up @@ -252,23 +291,18 @@ function assert_have_been_called_with() {
return
fi

local line=""
if [ -f "${!file_var-}" ]; then
if [ -n "$index" ]; then
line=$(sed -n "${index}p" "${!file_var}" 2>/dev/null || true)
else
line=$(tail -n 1 "${!file_var}" 2>/dev/null || true)
fi
fi

local raw
IFS=$'\x1e' read -r raw _ <<<"$line" || true
# One pass yields both the compared line and the total, which the failure
# message needs — and costs no fork, unlike the tail/sed it replaces.
bashunit::spy::read_call_to_slots "${!file_var-}" "$index"
local raw=${_BASHUNIT_SPY_CALL_OUT%%$'\x1e'*}
local total=$_BASHUNIT_SPY_CALL_TOTAL_OUT

if [ "$expected" != "$raw" ]; then
bashunit::state::add_assertions_failed
bashunit::spy::call_log_to_slot "$command"
bashunit::console_results::print_failed_test "$label" "$expected" "but got " "$raw" \
"" "" "$_BASHUNIT_SPY_CALL_LOG_OUT"
"compared" "$(bashunit::spy::compared_call "$index" "$total")" \
"$_BASHUNIT_SPY_CALL_LOG_OUT"
return
fi

Expand All @@ -293,20 +327,59 @@ function assert_have_been_called_with_args() {
return
fi

local line=""
if [ -f "${!file_var-}" ]; then
line=$(tail -n 1 "${!file_var}" 2>/dev/null || true)
fi

local actual
IFS=$'\x1e' read -r _ actual <<<"$line" || true
bashunit::spy::read_call_to_slots "${!file_var-}"
local actual=""
case "$_BASHUNIT_SPY_CALL_OUT" in
*$'\x1e'*) actual=${_BASHUNIT_SPY_CALL_OUT#*$'\x1e'} ;;
esac

if [ "$expected" != "$actual" ]; then
bashunit::state::add_assertions_failed
bashunit::spy::call_log_to_slot "$command" args
bashunit::console_results::print_failed_test "$label" \
"${expected//$'\x1f'/ }" "but got " "${actual//$'\x1f'/ }" \
"" "" "$_BASHUNIT_SPY_CALL_LOG_OUT"
"compared" "$(bashunit::spy::compared_call "" "$_BASHUNIT_SPY_CALL_TOTAL_OUT")" \
"$_BASHUNIT_SPY_CALL_LOG_OUT"
return
fi

bashunit::state::add_assertions_passed
}

function assert_have_been_called_with_any() {
local command=$1
shift
local expected="$*"

local variable
variable="$(bashunit::helper::normalize_variable_name "$command")"
local file_var="_BASHUNIT_SPY_${variable}_PARAMS_FILE"
local label
label="$(bashunit::helper::normalize_test_function_name "${FUNCNAME[1]}")"

if [ -z "${!file_var-}" ]; then
bashunit::spy::fail_unregistered "$command" "$label"
return
fi

local total=0
local found=false
local line
if [ -f "${!file_var}" ]; then
while IFS= read -r line; do
total=$((total + 1))
if [ "${line%%$'\x1e'*}" = "$expected" ]; then
found=true
break
fi
done <"${!file_var}"
fi

if [ "$found" = false ]; then
bashunit::state::add_assertions_failed
bashunit::spy::call_log_to_slot "$command"
bashunit::console_results::print_failed_test "$label" "$expected" \
"not found in any of" "${total} calls" "" "" "$_BASHUNIT_SPY_CALL_LOG_OUT"
return
fi

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -646,11 +646,18 @@ Reports an error if the spied `command` was never called. Requires `bashunit::sp
--------------
> `assert_have_been_called_with "command" "expected_args" nth`

Reports an error if the spied `command` was not called with `expected_args`. Checks the **last** call unless a trailing all-digits `nth` selects a specific one.
Reports an error if the spied `command` was not called with `expected_args`. Checks the **last** call unless a trailing all-digits `nth` selects a specific one; the failure names the call it compared. To match any call, use assert_have_been_called_with_any.

Note the argument order: the spy comes first here, but *second* in assert_have_been_called_times.


## assert_have_been_called_with_any
--------------
> `assert_have_been_called_with_any "command" "expected_args"`

Reports an error if no recorded call to the spied `command` received `expected_args`. Where assert_have_been_called_with compares a single call — the last one, or the one at `nth` — this one scans them all, so the assertion does not break when an unrelated call is added after it.


## assert_have_been_called_with_args
--------------
> `assert_have_been_called_with_args "command" "expected_arg"...`
Expand Down
52 changes: 50 additions & 2 deletions tests/functional/doubles_test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ function test_spy_call_with_args_detects_wrong_argument_boundaries() {
assert_same \
"$(bashunit::console_results::print_failed_test \
"Spy call with args detects wrong argument boundaries" \
"a b" "but got " "a\\ b" "" "" "$_BASHUNIT_SPY_CALL_LOG_OUT")" \
"a b" "but got " "a\\ b" "compared" "the only call" "$_BASHUNIT_SPY_CALL_LOG_OUT")" \
"$(assert_have_been_called_with_args spy_boundary_command "a" "b")"
}

Expand Down Expand Up @@ -183,7 +183,7 @@ function test_failed_call_assertions_dump_the_recorded_calls() {

assert_same \
"$(bashunit::console_results::print_failed_test "$label" \
"first" "but got " "second" "" "" "$log")" \
"first" "but got " "second" "compared" "the last of 2 calls" "$log")" \
"$(assert_have_been_called_with spy_with_a_call_log "first")"

assert_same \
Expand All @@ -197,6 +197,54 @@ function test_failed_call_assertions_dump_the_recorded_calls() {
"$(assert_have_been_called_nth_with 2 spy_with_a_call_log "first")"
}

function test_called_with_any_matches_a_call_in_any_position() {
bashunit::spy spy_called_three_times

spy_called_three_times first
spy_called_three_times middle one
spy_called_three_times last

assert_empty "$(assert_have_been_called_with_any spy_called_three_times "first" 2>&1)"
assert_empty "$(assert_have_been_called_with_any spy_called_three_times "middle one" 2>&1)"
assert_empty "$(assert_have_been_called_with_any spy_called_three_times "last" 2>&1)"
}

function test_called_with_any_fails_when_no_call_matches() {
bashunit::spy spy_without_a_match

spy_without_a_match first
spy_without_a_match second

bashunit::spy::call_log_to_slot spy_without_a_match

assert_same \
"$(bashunit::console_results::print_failed_test \
"Called with any fails when no call matches" \
"third" "not found in any of" "2 calls" "" "" "$_BASHUNIT_SPY_CALL_LOG_OUT")" \
"$(assert_have_been_called_with_any spy_without_a_match "third")"
}

function test_called_with_reports_which_call_it_compared() {
bashunit::spy spy_compared_call

spy_compared_call first
spy_compared_call second

bashunit::spy::call_log_to_slot spy_compared_call
local log=$_BASHUNIT_SPY_CALL_LOG_OUT
local label="Called with reports which call it compared"

assert_same \
"$(bashunit::console_results::print_failed_test "$label" \
"third" "but got " "second" "compared" "the last of 2 calls" "$log")" \
"$(assert_have_been_called_with spy_compared_call "third")"

assert_same \
"$(bashunit::console_results::print_failed_test "$label" \
"third" "but got " "first" "compared" "call 1 of 2" "$log")" \
"$(assert_have_been_called_with spy_compared_call "third" 1)"
}

function test_passing_call_assertions_print_nothing() {
bashunit::spy spy_with_a_passing_assertion

Expand Down
Loading