Correlation & Scatter

Goal

Explore linear relationships between variables using a correlation matrix heatmap and scatter plots. Correlation analysis reveals which sensors move together, which are independent, and whether relationships are positive or negative.

Data Basis

Dataset: flatTempHum.csv — Hourly indoor temperature and humidity measurements from four residential flats (A–D).
Columns used: All temperature and humidity columns
Time range: October 2018 – September 2020

Data Preparation

import pandas as pd

url = "https://raw.githubusercontent.com/retomarek/edap/main/edap/sampleData/flatTempHum.csv"
df = pd.read_csv(url, sep=";")
df["time"] = pd.to_datetime(df["time"])
df = df.set_index("time", drop=True)
df.head()

	FlatA_Hum	FlatA_Temp	FlatB_Hum	FlatB_Temp	FlatC_Hum	FlatC_Temp	FlatD_Hum	FlatD_Temp
time
2018-10-03 00:00:00	53.0	24.43	38.8	22.40	44.0	24.5	49.0	24.43
2018-10-03 01:00:00	53.0	24.40	38.8	22.40	44.0	24.5	49.0	24.40
2018-10-03 02:00:00	53.0	24.40	39.3	22.40	44.7	24.5	48.3	24.38
2018-10-03 03:00:00	53.0	24.40	40.3	22.40	45.0	24.5	48.0	24.33
2018-10-03 04:00:00	53.3	24.40	41.0	22.37	45.2	24.5	47.7	24.30

Correlation Matrix

The correlation matrix shows the Pearson correlation coefficient between every pair of variables. Values range from −1 (perfect negative correlation) to +1 (perfect positive correlation).

from pyedautils.plots import plot_correlation

fig = plot_correlation(df, title="Correlation Matrix — All Sensors")
fig.show()

Interpreting the correlation matrix

• Dark red (+1) — variables move together (e.g. temperatures in adjacent flats)

• Dark blue (−1) — variables move in opposite directions (e.g. outdoor temperature ↑, heating demand ↓)

• White (0) — no linear relationship

• Temperature–humidity pairs are positive in these flats: warming the same air lowers its relative humidity, but over the year indoor humidity follows the humid summer air, and this seasonal effect wins. The Mollier h-x Diagram separates both effects with the absolute humidity.

Scatter Plot

A scatter plot reveals the shape and density of the relationship between two variables. Low opacity makes dense regions visible.

from pyedautils.plots import plot_scatter

fig = plot_scatter(
    df,
    x_column="FlatA_Temp",
    y_column="FlatA_Hum",
    title="Flat A: Temperature vs. Humidity",
    xlab="Temperature [°C]",
    ylab="Humidity [%]",
)
fig.show()

fig = plot_scatter(
    df,
    x_column="FlatA_Temp",
    y_column="FlatB_Temp",
    title="Flat A vs. Flat B — Temperature",
    xlab="Flat A [°C]",
    ylab="Flat B [°C]",
)
fig.show()

Customization

• Adjust opacity (0.0–1.0) to control how dense regions appear. Lower values are better for large datasets.

• The color_scale parameter in plot_correlation() accepts any Plotly colorscale (e.g. "Viridis", "Plasma", "RdBu_r").

• Use text_format=".3f" for more decimal places in the correlation matrix.