Fix dcc.Patch() disturbing pre-existing components (re-run callbacks + wiped persistence)

[Aaron-Wrote-This]

Fixes #3681 and #3937

Problem

When a dcc.Patch() added or rebuilt components inside a container, two separate bugs trigger,
effecting components which already existed before the Patch:

1. Initial callbacks refire for every matching MATCH/ALL output, so appending one new
   pattern matching id'd dcc.Slider reruns the initial callback for every preexisting slider
2. Persisted user edits are wiped, so edits to persistence enabled components are cleared
   because the patch added a new sibling

I tracked both items back to the way Patch resolution distinguishes components carried over from the
pre-Patch layout from freshly built ones

Fix

1. Snapshot the paths table before a Patch is resolved

2. Callbacks can suppress the initial call for carried-over components, by detecting props
   reference identity (child.props === oldPropsRef). ramda's assocPath is structurally immutable,
   so untouched nodes keep their exact props reference while rebuilt/replaced node gets a new one

3. applyPersistence now skips components that already existed pre-Patch, instead of re-running
   modProp and misreading the carried over value as a "server override" that clears the stored edit

Contributor Checklist

I've been doing a lot of work with dynamic UIs and creating extensible elements. I am loving the patch system, but I found some significant drawbacks to the both in performance and usability.

At first, I thought the fix for the initial callbacks was simple and just checking the ID more carefully would be enough, but it ended up being requiring adding an 'audit trail' for patches, which also ended up fixing the persistence issues.

Javascript isn't my 'first language', so let me know if there are any things I'm doing awkwardly, or best practices I'm missing. Happy to fix those up.
Or I understand if this is too complicated and not exactly needed. I looked for simpler options, but I couldn't find a way to differentiate patches and understand their impact in any other way, but it would be great if one existed and I just didn't see it.

• I have broken down my PR scope into the following TODO tasks
  • Record patch operations
  • Fix patch running against existing components with matching pattern matching IDs
  • Fix persisted values being wiped from existing components during a patch
• I have run the tests locally and they passed
• I have added tests, or extended existing tests, to cover any new features or bugs fixed in this PR

optionals

• I have added entry in the CHANGELOG.md
• If this PR needs a follow-up in dash docs, community thread, I have mentioned the relevant URLS as follows
  • this GitHub #PR number updates the dash docs
  • here is the show and tell thread in Plotly Dash community

[sonarqubecloud]

Quality Gate passed

Issues
6 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.9% Duplication on New Code

See analysis details on SonarQube Cloud

[T4rk1n]

Replace with PR number

[T4rk1n]

Prefer using optional chaining:

return analysis?.patchedProps[property] ? analysis : undefined;

[T4rk1n]

This code is duplicated from L981-1016, might be worth it now to refactor into a function.

[T4rk1n]

One question about how the analysis gets consumed, though:

The two consumers use different predicates:

• handleOneId (callback suppression) uses isCarriedOverByPatch, which only checks freshIds
• persistenceMods uses isUntouchedByPatch, which checks freshIds and writtenProps

So a component whose prop the patch wrote (but didn't recreate) counts as "carried over" on the callback side, and its initial calls get suppressed. I think that can strand a dependent component:

# both slider and badge already exist inside the patched container
p = Patch()
p[0]["props"]["children"][0]["props"]["value"] = 5  # writes slider.value
return p

with a callback Input(slider, "value") -> Output(badge, "children") where the badge is also inside the patched chunk:

• output side: badge isn't in freshIds, so isCarriedOverByPatch suppresses its initial call
• input side: all of the callback's outputs are inside the chunk, so the existing chunkPath filter drops it too

On dev today the over-firing bug incidentally keeps this callback alive, so the badge updates. With this PR I believe it never fires and the badge goes stale. test_cbwc014 writes deep-input.value but nothing listens to that value, so this path isn't exercised.

The write here lands on the callback's input (slider.value), while the stranded callback outputs to badge. So the fix can't live on the output side — isUntouchedByPatch is keyed on the output id (badge), which the patch neither created nor wrote, so it would still read as carried-over. The gap is really in the input-side chunkPath filter: it drops a callback whose outputs are all inside the chunk because it assumes the chunk's initial-call path will cover it, but that path is (correctly) suppressed for the carried-over output, so nothing fires. Could the filter make an exception when the triggering input prop is in writtenProps? Either way, a test with a listener on the patch-written prop (not just a write nothing consumes, like test_cbwc014) would pin the semantics down.

This is an app to repro:

"""
Repro for the freshIds/writtenProps asymmetry question on plotly/dash#3938.

Scenario:
  - `num` (an input) and `badge` both live inside `container` and exist
    before any patch
  - a callback listens to Input(num.value) and writes Output(badge.children);
    both ends are INSIDE the patched chunk
  - clicking "Patch" returns a Patch() on container.children that
      1. writes num.value          -> writtenProps case (component NOT recreated)
      2. appends a brand-new div   -> freshIds case (control: proves the
                                      suppression isn't just total)

What to look for after clicking "Patch" once:

  On dev / released dash (over-firing bug present):
      num shows the patched value AND badge updates to match
      (kept alive incidentally by the spurious initial-call refire)

  On the PR branch, IF the asymmetry is real:
      num shows the patched value but badge keeps its old text -> STALE
      - output side: badge is not in freshIds -> initial call suppressed
      - input side:  all the callback's outputs are inside the chunk
                     -> the existing chunkPath filter drops it too

  On the PR branch, IF something else covers it:
      badge updates anyway -> soften the review comment into
      "can we add a test pinning this down?"

The on-page counters make it visible without opening devtools:
  - badge callback count stuck at 1 after patching  = stale (only page load)
  - new-div MATCH count should track patches applied (control staying healthy)
"""

import threading

from dash import ALL, Dash, Input, Output, Patch, dcc, html

lock = threading.Lock()
calls = {"badge": 0, "new_div": 0}


def bump(name):
    with lock:
        calls[name] += 1


app = Dash(__name__, suppress_callback_exceptions=True)

app.layout = html.Div(
    [
        html.Button("Patch", id="patch-btn", n_clicks=0),
        html.Div(
            [
                dcc.Input(id="num", type="number", value=1),
                html.Div("badge: (initial)", id="badge"),
            ],
            id="container",
        ),
        html.Hr(),
        html.Div(id="badge-calls"),
        html.Div(id="new-div-calls"),
        html.Div("patch not applied yet", id="done"),
        dcc.Interval(id="tick", interval=500),
    ]
)


@app.callback(
    Output("container", "children"),
    Output("done", "children"),
    Input("patch-btn", "n_clicks"),
    prevent_initial_call=True,
)
def do_patch(n):
    p = Patch()
    # writtenProps: write a prop on the EXISTING input (index 0 of children)
    p[0]["props"]["value"] = 100 + n
    # freshIds: append a genuinely new component as a control
    p.append(html.Div("appended (waiting)", id={"type": "new-div", "index": n}))
    return p, f"patch #{n} applied (num should now read {100 + n})"


# The callback under test: input AND output both inside the patched chunk.
@app.callback(Output("badge", "children"), Input("num", "value"))
def update_badge(value):
    bump("badge")
    return f"badge: num value is {value}"


# Control: initial call for the appended (fresh) components should still fire.
@app.callback(
    Output({"type": "new-div", "index": ALL}, "children"),
    Input({"type": "new-div", "index": ALL}, "id"),
    prevent_initial_call=False,
)
def new_div_initial(ids):
    bump("new_div")
    return [f"appended #{i['index']} (initial call ran)" for i in ids]


@app.callback(
    Output("badge-calls", "children"),
    Output("new-div-calls", "children"),
    Input("tick", "n_intervals"),
)
def show_counts(_):
    with lock:
        return (
            f"badge callback fired: {calls['badge']} time(s) "
            "(stuck at 1 after patching = STALE; increments = updated)",
            f"new-div ALL callback fired: {calls['new_div']} time(s) "
            "(should increment with each patch)",
        )


if __name__ == "__main__":
    app.run(debug=True)