reshape

reshape#

Reshaping helpers for long-format sensor data: per-id resampling, IQR outlier removal, and long-to-wide pivoting.

API Reference#

Reshaping helpers for long-format sensor data.

Long format here means one row per (datapoint_id, timestamp) with a value column — the typical shape of data pulled from a time-series database before pivoting to a wide, plot-ready frame.

pyedautils.data_prep.reshape.aggregate_long(df: DataFrame, freq: str | None, func: str = 'mean', id_col: str = 'datapoint_id', time_col: str = 'timestamp', value_col: str = 'value') → DataFrame[source]#

Resample long-format data per id.

Parameters:

  • df – Long-format DataFrame with id, timestamp and value columns.

  • freq – A pandas offset alias (e.g. "1h", "1D", "1W", "1ME"). None returns df unchanged (raw resolution).

  • func – Aggregation function name ("mean", "min", "max", "median", "sum", …). Default "mean".

  • id_col – Id column name. Default "datapoint_id".

  • time_col – Timestamp column name. Default "timestamp".

  • value_col – Value column name. Default "value".

Returns:

Same long format, resampled per id.

Return type:

pd.DataFrame

pyedautils.data_prep.reshape.remove_outliers_iqr(df: DataFrame, multiplier: float = 3.0, id_col: str = 'datapoint_id', value_col: str = 'value') → DataFrame[source]#

Drop outliers per id using the IQR fence.

Values outside [Q1 - multiplier·IQR, Q3 + multiplier·IQR] are removed, computed independently per id.

Parameters:

  • df – Long-format DataFrame.

  • multiplier – IQR multiplier. Default 3.0.

  • id_col – Id column name. Default "datapoint_id".

  • value_col – Value column name. Default "value".

Returns:

Filtered DataFrame (same columns).

Return type:

pd.DataFrame

pyedautils.data_prep.reshape.to_wide(df_long: DataFrame, id_to_label: Dict[str, str] | None = None, id_col: str = 'datapoint_id', time_col: str = 'timestamp', value_col: str = 'value') → DataFrame[source]#

Pivot long-format data to wide (timestamp index, one column per id).

Parameters:

  • df_long – Long-format DataFrame.

  • id_to_label – Optional mapping from id (as str) to a column label. Ids not present in the mapping are dropped. If None, the ids themselves are used as column labels.

  • id_col – Id column name. Default "datapoint_id".

  • time_col – Timestamp column name. Default "timestamp".

  • value_col – Value column name. Default "value".

Returns:

Wide DataFrame indexed by timestamp. Duplicate (timestamp, id) pairs keep the first value.

Return type:

pd.DataFrame