Skip to content

YeoJohnsonTransformer inverse_transform loses small nonzero values #1100

Open

Description

@ktz03

Describe the bug
YeoJohnsonTransformer.inverse_transform loses nonzero values near zero through cancellation in the inverse formulas. This reproduces on Feature-engine 1.9.4 and main d866312874a66364c6dee3c247e69476b478395c.

To Reproduce

import pandas as pd
from feature_engine.transformation import YeoJohnsonTransformer

transformer = YeoJohnsonTransformer().fit(
    pd.DataFrame({"x": [-3.0, -1.0, 0.0, 1.0, 3.0]})
)
x = pd.DataFrame({"x": [-1e-20, 0.0, 1e-20]})
transformed = transformer.transform(x)
print(transformed["x"].tolist())
print(transformer.inverse_transform(transformed)["x"].tolist())

The output is [-1e-20, 0.0, 1e-20] followed by [0.0, 0.0, 0.0]. The fitted lambda is approximately 1; no fitted attribute needs to be overridden. Normal-sized values such as [-1.0, 0.0, 1.0] round-trip correctly.

Expected behavior
The inverse should recover the nonzero values with relative accuracy, as the forward transformation does. The 1 + x and subtraction of 1 in the inverse formulas round away these values.

Desktop
Windows, Python 3.12.14, NumPy 2.2.6, pandas 2.3.3, SciPy 1.16.3.

Additional context
I am working on a small fix using log1p and expm1 for the mathematically equivalent inverse formulas, with near-zero round-trip regressions and normal-sized controls. This retains the inverse behavior introduced in #679/#696 and the index correction in #794. The open narwhals migration #965 retains the current cancellation-prone formulas, so this is a separate numerical correction.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions