Add grouped lag, rolling and expanding features

docs/user_guide/timeseries/forecasting/ExpandingWindowFeatures.rst

@@ -471,4 +471,30 @@ forecast time series using traditional machine learning models, like linear regr
 or gradient boosting machines.
 
 By purchasing them you are supporting `Sole <https://linkedin.com/in/soledad-galli>`_,
-the main developer of feature-engine.
+the main developer of feature-engine.
+
+Independent series in one dataframe
+-----------------------------------
+
+Use ``groupby`` to identify independent series by one column or a combination of
+columns. ``sort_by`` orders observations using a column, overriding ``sort_index``::
+
+    transformer = ExpandingWindowFeatures(
+        variables=["sales"], groupby=["store", "product"], sort_by="date",
+        min_periods=1
+    )
+    X_transformed = transformer.fit_transform(X)
+
+No lag, rolling aggregate, or expanding aggregate crosses a group boundary.
+Grouping and sorting columns are retained and excluded from automatic feature
+selection. The dataframe must retain a unique, non-missing row index; dates may
+repeat across groups when stored in the ``sort_by`` column. Within each group,
+``sort_by`` values must be unique and non-missing. Group labels must be complete.
+Use a datetime sorting column or index for time-based windows and frequency shifts.
+
+The output preserves row labels and follows the requested sorting order. With
+``sort_by=None`` and ``sort_index=False``, calculations follow input order within
+each group. ``drop_na`` removes rows with incomplete generated features, and
+``transform_x_y`` aligns the target with the retained rows. The transformer uses
+only the rows passed to ``transform``; it does not carry training history into a
+later batch. With ``groupby=None``, existing behaviour is unchanged.

docs/user_guide/timeseries/forecasting/LagFeatures.rst

@@ -674,4 +674,30 @@ forecast time series using traditional machine learning models, like linear regr
 or gradient boosting machines.
 
 By purchasing them you are supporting `Sole <https://linkedin.com/in/soledad-galli>`_,
-the main developer of feature-engine.
+the main developer of feature-engine.
+
+Independent series in one dataframe
+-----------------------------------
+
+Use ``groupby`` to identify independent series by one column or a combination of
+columns. ``sort_by`` orders observations using a column, overriding ``sort_index``::
+
+    transformer = LagFeatures(
+        variables=["sales"], groupby=["store", "product"], sort_by="date",
+        periods=[1, 2]
+    )
+    X_transformed = transformer.fit_transform(X)
+
+No lag, rolling aggregate, or expanding aggregate crosses a group boundary.
+Grouping and sorting columns are retained and excluded from automatic feature
+selection. The dataframe must retain a unique, non-missing row index; dates may
+repeat across groups when stored in the ``sort_by`` column. Within each group,
+``sort_by`` values must be unique and non-missing. Group labels must be complete.
+Use a datetime sorting column or index for time-based windows and frequency shifts.
+
+The output preserves row labels and follows the requested sorting order. With
+``sort_by=None`` and ``sort_index=False``, calculations follow input order within
+each group. ``drop_na`` removes rows with incomplete generated features, and
+``transform_x_y`` aligns the target with the retained rows. The transformer uses
+only the rows passed to ``transform``; it does not carry training history into a
+later batch. With ``groupby=None``, existing behaviour is unchanged.

docs/user_guide/timeseries/forecasting/WindowFeatures.rst

@@ -657,4 +657,30 @@ forecast time series using traditional machine learning models, like linear regr
 or gradient boosting machines.
 
 By purchasing them you are supporting `Sole <https://linkedin.com/in/soledad-galli>`_,
-the main developer of feature-engine.
+the main developer of feature-engine.
+
+Independent series in one dataframe
+-----------------------------------
+
+Use ``groupby`` to identify independent series by one column or a combination of
+columns. ``sort_by`` orders observations using a column, overriding ``sort_index``::
+
+    transformer = WindowFeatures(
+        variables=["sales"], groupby=["store", "product"], sort_by="date",
+        window=3, min_periods=1
+    )
+    X_transformed = transformer.fit_transform(X)
+
+No lag, rolling aggregate, or expanding aggregate crosses a group boundary.
+Grouping and sorting columns are retained and excluded from automatic feature
+selection. The dataframe must retain a unique, non-missing row index; dates may
+repeat across groups when stored in the ``sort_by`` column. Within each group,
+``sort_by`` values must be unique and non-missing. Group labels must be complete.
+Use a datetime sorting column or index for time-based windows and frequency shifts.
+
+The output preserves row labels and follows the requested sorting order. With
+``sort_by=None`` and ``sort_index=False``, calculations follow input order within
+each group. ``drop_na`` removes rows with incomplete generated features, and
+``transform_x_y`` aligns the target with the retained rows. The transformer uses
+only the rows passed to ``transform``; it does not carry training history into a
+later batch. With ``groupby=None``, existing behaviour is unchanged.

feature_engine/timeseries/forecasting/base_forecast_transformers.py

@@ -77,6 +77,8 @@ def __init__(
         missing_values: str = "raise",
         drop_original: bool = False,
         drop_na: bool = False,
+        groupby=None,
+        sort_by=None,
     ) -> None:
 
         if missing_values not in ["raise", "ignore"]:
@@ -104,6 +106,12 @@ def __init__(
         self.missing_values = missing_values
         self.drop_original = drop_original
         self.drop_na = drop_na
+        self.groupby = _check_variables_input_value(groupby)
+        if sort_by is not None and not isinstance(sort_by, (str, int)):
+            raise ValueError("sort_by must be a column name or None.")
+        if sort_by is not None and groupby is None:
+            raise ValueError("sort_by requires groupby.")
+        self.sort_by = sort_by
 
     def _check_index(self, X: pd.DataFrame):
         """
@@ -178,6 +186,8 @@ def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None):
         # Otherwise, when we merge the new features we will duplicate rows.
         self._check_index(X)
 
+        self._check_grouping(X)
+
         # find or check for numerical variables
         if self.variables is None:
             self.variables_ = find_numerical_variables(
@@ -186,6 +196,18 @@ def fit(self, X: pd.DataFrame, y: Optional[pd.Series] = None):
         else:
             self.variables_ = check_numerical_variables(X, self.variables)
 
+        if self.groupby is not None:
+            excluded = self._group_columns() + (
+                [] if self.sort_by is None else [self.sort_by]
+            )
+            if self.variables is not None and set(self.variables_) & set(excluded):
+                raise ValueError("Grouping and sorting columns cannot be variables.")
+            self.variables_ = [v for v in self.variables_ if v not in excluded]
+            if not self.variables_ and not self.return_empty:
+                raise ValueError(
+                    "No numerical variables remain after excluding group columns."
+                )
+
         # check if dataset contains na
         if self.missing_values == "raise":
             self._check_na_and_inf(X)
@@ -221,6 +243,7 @@ def _check_transform_input_and_state(self, X: pd.DataFrame) -> pd.DataFrame:
         # Dataframes must have unique values in the index and no missing data.
         # Otherwise, when we merge the created features we will duplicate rows.
         self._check_index(X)
+        self._check_grouping(X)
 
         # check if dataset contains na
         if self.missing_values == "raise":
@@ -229,11 +252,91 @@ def _check_transform_input_and_state(self, X: pd.DataFrame) -> pd.DataFrame:
         # reorder variables to match train set
         X = X[self.feature_names_in_]
 
-        if self.sort_index is True:
+        if self.sort_by is not None:
+            X = X.sort_values(self.sort_by, kind="stable")
+        elif self.sort_index is True:
             X.sort_index(inplace=True)
 
         return X
 
+    def _group_columns(self):
+        return self.groupby if isinstance(self.groupby, list) else [self.groupby]
+
+    def _check_grouping(self, X):
+        if self.groupby is None:
+            return
+        columns = self._group_columns()
+        required = columns + ([] if self.sort_by is None else [self.sort_by])
+        if any(c not in X.columns for c in required):
+            raise ValueError("Grouping and sorting columns must be present in X.")
+        if X[required].isna().any().any():
+            raise ValueError(
+                "Grouping and sorting columns cannot contain missing values."
+            )
+        if self.sort_by is not None and X.duplicated(required).any():
+            raise ValueError("sort_by must be unique within each group.")
+
+    def _transform_grouped(self, X, kind):
+        """Compute features within groups, then align on group and time together."""
+        if not self.variables_:
+            return X
+        data = X[self.variables_].copy()
+        if self.sort_by is not None:
+            data.index = pd.Index(X[self.sort_by])
+        keys = [X[c].to_numpy() for c in self._group_columns()]
+        target_index = pd.MultiIndex.from_arrays(keys + [data.index])
+        grouped = data.groupby(keys, sort=False, observed=True)
+        levels = list(range(len(keys)))
+        parts = []
+        if kind == "lag":
+            shifts = self.freq if self.freq is not None else self.periods
+            shifts = shifts if isinstance(shifts, list) else [shifts]
+            for shift in shifts:
+                if self.freq is None:
+                    part = grouped.shift(shift, fill_value=self.fill_value)
+                    part.index = X.index
+                else:
+                    part = data.copy()
+                    part.index = pd.MultiIndex.from_arrays(
+                        keys + [data.index.shift(1, freq=shift)]
+                    )
+                    part = part.reindex(target_index)
+                    part.index = X.index
+                    if self.fill_value is not None:
+                        part = part.fillna(self.fill_value)
+                parts.append(part)
+        else:
+            windows = self.window if kind == "window" else [None]
+            windows = windows if isinstance(windows, list) else [windows]
+            for window in windows:
+                if kind == "window":
+                    part = grouped.rolling(
+                        window=window, min_periods=self.min_periods
+                    ).agg(self.functions)
+                else:
+                    part = grouped.expanding(min_periods=self.min_periods).agg(
+                        self.functions
+                    )
+                if self.freq is None:
+                    part = part.groupby(level=levels, sort=False).shift(self.periods)
+                else:
+                    arrays = [part.index.get_level_values(i) for i in levels]
+                    time = part.index.get_level_values(-1)
+                    part.index = pd.MultiIndex.from_arrays(
+                        arrays + [time.shift(self.periods, freq=self.freq)]
+                    )
+                part = part.reindex(target_index)
+                part.index = X.index
+                parts.append(part)
+        features = pd.concat(parts, axis=1)
+        features.columns = self._get_new_features_name()
+        result = pd.concat([X, features], axis=1)
+        if self.drop_original:
+            result = result.drop(columns=self.variables_)
+        if self.drop_na:
+            result = result.dropna(subset=features.columns)
+        return result
+
     def _more_tags(self):
         tags_dict = _return_tags()
         tags_dict["allow_nan"] = True

feature_engine/timeseries/forecasting/expanding_window_features.py

@@ -101,6 +101,16 @@ class ExpandingWindowFeatures(BaseForecastTransformer):
         Whether the NAN introduced in the created features should be removed.
 
 
+    groupby: str, int or list, default=None
+        Columns whose combined values identify each independent time series.
+        Grouping columns are excluded from automatic variable selection.
+
+    sort_by: str or int, default=None
+        Column used to order observations before grouped transformations. Overrides
+        ``sort_index``. Required to be non-missing, and unique within each group.
+        Use a datetime column for time-based windows or frequency shifts. Requires
+        ``groupby``. The output follows this sorted order and preserves row indexes.
+
     Attributes
     ----------
     variables_:
@@ -164,6 +174,8 @@ def __init__(
         missing_values: str = "raise",
         drop_original: bool = False,
         drop_na: bool = False,
+        groupby=None,
+        sort_by=None,
     ) -> None:
 
         if not isinstance(functions, (str, list)) or not all(
@@ -182,7 +194,13 @@ def __init__(
             )
 
         super().__init__(
-            variables, return_empty, missing_values, drop_original, drop_na
+            variables,
+            return_empty,
+            missing_values,
+            drop_original,
+            drop_na,
+            groupby=groupby,
+            sort_by=sort_by,
         )
 
         self.min_periods = min_periods
@@ -208,6 +226,9 @@ def transform(self, X: pd.DataFrame) -> pd.DataFrame:
         # Common dataframe checks and setting up.
         X = self._check_transform_input_and_state(X)
 
+        if self.groupby is not None:
+            return self._transform_grouped(X, "expanding")
+
         tmp = (
             X[self.variables_]
             .expanding(min_periods=self.min_periods)

feature_engine/timeseries/forecasting/lag_features.py

@@ -88,6 +88,16 @@ class LagFeatures(BaseForecastTransformer):
     drop_na: bool, default=False.
         Whether the NAN introduced in the lag features should be removed.
 
+    groupby: str, int or list, default=None
+        Columns whose combined values identify each independent time series.
+        Grouping columns are excluded from automatic variable selection.
+
+    sort_by: str or int, default=None
+        Column used to order observations before grouped transformations. Overrides
+        ``sort_index``. Required to be non-missing, and unique within each group.
+        Use a datetime column for time-based windows or frequency shifts. Requires
+        ``groupby``. The output follows this sorted order and preserves row indexes.
+
     Attributes
     ----------
     variables_:
@@ -147,6 +157,8 @@ def __init__(
         missing_values: str = "raise",
         drop_original: bool = False,
         drop_na: bool = False,
+        groupby=None,
+        sort_by=None,
     ) -> None:
 
         if not (
@@ -172,7 +184,13 @@ def __init__(
             )
 
         super().__init__(
-            variables, return_empty, missing_values, drop_original, drop_na
+            variables,
+            return_empty,
+            missing_values,
+            drop_original,
+            drop_na,
+            groupby=groupby,
+            sort_by=sort_by,
         )
 
         self.periods = periods
@@ -197,6 +215,9 @@ def transform(self, X: pd.DataFrame) -> pd.DataFrame:
         # Common dataframe checks and setting up.
         X = self._check_transform_input_and_state(X)
 
+        if self.groupby is not None:
+            return self._transform_grouped(X, "lag")
+
         # if freq is not None, it overrides periods.
         if self.freq is not None: