Repository navigation
Add an out-of-fold target encoder (OOFMeanEncoder) #1050
Description
Is your feature request related to a problem? Please describe.
MeanEncoder replaces a category with the mean target observed for that category. When it is
estimated and applied on the same rows, every row contributes its own target to its own
encoding. With k rows per category each row is responsible for 1/k of its own encoded
value, and at k = 1 the encoding is the label itself:
| rows per category | weight of the row's own target |
|---|---|
| 1 | 100% |
| 3 | 33% |
| 10 | 10% |
| 1000 | 0.1% |
So the model scores well in training and validation collapses. MeanEncoder's documentation
warns about this, but the library offers no encoder that avoids it, and the alternatives live
in other packages.
Describe the solution you'd like
A transformer that learns its mapping out of fold, so no row is encoded using its own target.
This is the same idea as category_encoders.TargetEncoder(cv=...) and CatBoost's ordered
target statistics, neither of which integrates with the feature-engine / scikit-learn pipeline
conventions used here.
Proposed API, following the conventions of the existing encoders:
from feature_engine.encoding import OOFMeanEncoder
enc = OOFMeanEncoder(
variables=["cabin", "embarked"],
cv=5, # number of folds
smoothing=10, # additive smoothing towards the prior
unseen="encode", # encode unseen categories with the prior
shuffle=True,
random_state=0,
stratify=True,
)
X_train = enc.fit_transform(X_train, y_train) # out of fold
X_test = enc.transform(X_test) # full-data mapping
X_train_orig = enc.inverse_transform(X_train)fit_transformsplits the training data intocvfolds, learns the mapping for each fold
from the remaining folds, and encodes the held-out fold with it.fitlearns the mapping from the full training set and stores it inencoder_dict_, with
the prior iny_prior_, sotransformbehaves likeMeanEncoderat inference time.variables,missing_values,ignore_format,unseen,return_emptyandsmoothing
work as they do elsewhere inencoding.
Describe alternatives you've considered
- Tell users to wrap
MeanEncoderin aKFoldloop by hand. It works, but it is easy to get
wrong and it is the reason the issue exists. - Recommend
category_encodersor CatBoost. Both are good, but it means a second encoder
library in a project that already depends on feature-engine, andcategory_encodershas a
different API forvariables/ unseen categories / returning DataFrames.
Additional context
I have an implementation with 46 tests ready, and tests/test_encoding passes in full (389
tests) including test_check_estimator_encoders.py:
feature_engine/encoding/oof_mean_encoding.pytests/test_encoding/test_oof_mean_encoder.pydocs/api_doc/encoding/OOFMeanEncoder.rstplus the encoder table and toctree entry
Happy to open the PR, or to wait if you would rather scope it differently — for example a
cv argument on MeanEncoder instead of a separate class. I am also glad to add a
docs/whats_new entry if you tell me the format you prefer for unreleased versions.
Activity
hx-max commentedon Oct 13, 2026
Hi @solegalli, I've opened PR #1102 with a complete implementation of the API sketched in this
issue.
What's in it:
feature-engine conventions (variables, missing_values, ignore_format, unseen, return_empty,
smoothing, plus cv / shuffle / random_state / stratify).
all three unseen modes, stratify degradation on continuous targets, smoothing (int, float and
'auto'), missing_values, ignore_format, return_empty, inverse_transform and feature-name
handling.
the project's own check_feature_engine_estimator.
The branch is rebased on current main (d866312). Locally: tests/test_encoding passes in full
(392 tests), flake8 is clean, the sphinx -W docs build has no warnings, and mypy reports no
issues in the files this PR touches -- the one remaining mypy error is a pre-existing one in
feature_engine/datetime/datetime_subtraction.py that reproduces on unmodified main.
I see PR #1097 also references this issue. Happy for you to pick whichever implementation fits
the roadmap, and glad to share the test suite if that helps consolidate.