Window Functions in Pandas (Rolling, Expanding, EWMA, Shift) The Media Gen

Window functions are powerful when working with time series or ordered data. They let you look at a value in the context of nearby values – for example, a 3-day moving average of stock prices.

Unlike groupby, window functions don’t reduce the number of rows. They add new columns with calculations based on a moving or growing window.

1. Sample Time Series Data

We’ll work with a small stock price dataset:

import pandas as pd

# Time series data
dates = pd.date_range('2024-01-01', periods=10, freq='D')
stock_data = {
    'Date': dates,
    'Price':  [100, 102, 98, 105, 107, 103, 108, 110, 106, 112],
    'Volume': [1000, 1100, 900, 1200, 1300, 1000, 1400, 1500, 1100, 1600]
}

df_stock = pd.DataFrame(stock_data)
df_stock.set_index('Date', inplace=True)

st.subheader("Stock Data:")
st.write(df_stock)

Output:

We’ll add new columns step by step as we explore different window functions.

2. Rolling Window – Moving Average with `rolling().mean()`

What is rolling?

A rolling window looks at a fixed number of rows at a time, moving one step forward each time. For example, a 3-day moving average uses the current day and the previous 2 days.

Example: 3-Day Moving Average of Price

# Rolling mean (Moving Average)
df_stock['MA_3'] = df_stock['Price'].rolling(window=3).mean()

st.subheader("With 3-day Moving Average:")
st.write(df_stock[['Price', 'MA_3']])

Explanation

.rolling(window=3) creates a rolling object that considers 3 consecutive rows at a time.
.mean() calculates the average inside each window.
The first 2 rows don’t have enough data for a 3-row window, so they become NaN.

Output:

This is your classic moving average indicator, often used in finance and analytics.

3. Rolling Window with Multiple Statistics

You’re not limited to just the mean. The same rolling window can produce min, max, standard deviation, and more.

Example: Rolling Min, Max, and Std Dev

df_stock['Rolling_Min'] = df_stock['Price'].rolling(window=3).min()
df_stock['Rolling_Max'] = df_stock['Price'].rolling(window=3).max()
df_stock['Rolling_Std'] = df_stock['Price'].rolling(window=3).std()

st.subheader("Multiple rolling statistics:")
st.write(df_stock[['Price', 'MA_3', 'Rolling_Min', 'Rolling_Max', 'Rolling_Std']])

Explanation

.min() → smallest price in the 3-day window
.max() → largest price in the 3-day window
.std() → standard deviation (how much the values vary inside the window)

Output (rounded for readability):

This kind of rolling summary is useful for volatility analysis, local ranges, and detecting spikes.

4. Expanding Window – Cumulative Statistics with `expanding()`

What is expanding?

An expanding window starts at the first row and keeps growing. Each step uses all data from the beginning up to the current row.

Example: Cumulative Max and Cumulative Mean

df_stock['Cumulative_Max'] = df_stock['Price'].expanding().max()
df_stock['Cumulative_Mean'] = df_stock['Price'].expanding().mean()

st.subheader("Expanding window:")
st.write(df_stock[['Price', 'Cumulative_Max', 'Cumulative_Mean']])

Explanation

.expanding().max() → the highest price seen so far at each date.
.expanding().mean() → the average price from the start up to that date.

Output:

These are great for running totals, running averages, and “highest so far” style metrics.

5. Exponential Weighted Moving Average (EWMA) with `ewm()`

What is EWMA?

A Exponential Weighted Moving Average (EWMA) is like a moving average, but it gives more weight to recent values and less to older values, instead of treating them equally.

Example: EWMA with `span=3`

df_stock['EWMA'] = df_stock['Price'].ewm(span=3).mean()

st.subheader("Exponential Weighted Moving Average:")
st.write(df_stock[['Price', 'MA_3', 'EWMA']])

Explanation

.ewm(span=3) defines how fast the weights decay. Smaller span → more weight on recent data.
.mean() then computes the EWMA.

Output:

Notice how EWMA moves smoother than the raw price, but reacts faster than a simple moving average.

6. Shift Operations – Lag, Lead, and Percentage Change

What is `shift()`?

shift() moves the data up or down by a number of periods:

shift(1) → previous row (lag).
shift(-1) → next row (lead).

This is essential for comparing the current value to the previous or next value.

Example: Previous Price, Next Price, and Price Change

df_stock['Previous_Price'] = df_stock['Price'].shift(1)
df_stock['Next_Price'] = df_stock['Price'].shift(-1)
df_stock['Price_Change'] = df_stock['Price'] - df_stock['Previous_Price']
df_stock['Price_Change_Pct'] = df_stock['Price'].pct_change() * 100

st.subheader("Shift operations:")
st.write(df_stock[['Price', 'Previous_Price', 'Next_Price',
                'Price_Change', 'Price_Change_Pct']])

Explanation

Previous_Price → yesterday’s price in today’s row.
Next_Price → tomorrow’s price in today’s row.
Price_Change → difference from the previous day.
pct_change() → percentage change compared to the previous row.

Output:

Complete Code:

import pandas as pd
import streamlit as st

# Time series data
dates = pd.date_range('2024-01-01', periods=10, freq='D')
stock_data = {
    'Date': dates,
    'Price': [100, 102, 98, 105, 107, 103, 108, 110, 106, 112],
    'Volume': [1000, 1100, 900, 1200, 1300, 1000, 1400, 1500, 1100, 1600]
}

df_stock = pd.DataFrame(stock_data)
df_stock.set_index('Date', inplace=True)
st.subheader("Stock Data:")
st.write(df_stock)

# Rolling mean (Moving Average)
df_stock['MA_3'] = df_stock['Price'].rolling(window=3).mean()
st.subheader("With 3-day Moving Average:")
st.write(df_stock)

# Rolling with multiple statistics
df_stock['Rolling_Min'] = df_stock['Price'].rolling(window=3).min()
df_stock['Rolling_Max'] = df_stock['Price'].rolling(window=3).max()
df_stock['Rolling_Std'] = df_stock['Price'].rolling(window=3).std()
st.subheader("Multiple rolling statistics:")
st.write(df_stock)

# Expanding window (cumulative)
df_stock['Cumulative_Max'] = df_stock['Price'].expanding().max()
df_stock['Cumulative_Mean'] = df_stock['Price'].expanding().mean()
st.subheader("Expanding window:")
st.write(df_stock[['Price', 'Cumulative_Max', 'Cumulative_Mean']])

# Exponential Weighted Moving Average
df_stock['EWMA'] = df_stock['Price'].ewm(span=3).mean()
st.subheader("Exponential Weighted Moving Average:")
st.write(df_stock[['Price', 'MA_3', 'EWMA']])

# Shift - lag and lead values
df_stock['Previous_Price'] = df_stock['Price'].shift(1)
df_stock['Next_Price'] = df_stock['Price'].shift(-1)
df_stock['Price_Change'] = df_stock['Price'] - df_stock['Previous_Price']
df_stock['Price_Change_Pct'] = df_stock['Price'].pct_change() * 100
st.subheader("Shift operations:")
st.write(df_stock[['Price', 'Previous_Price', 'Price_Change', 'Price_Change_Pct']])

Summary

In this window functions chapter, you learned how to:

Use rolling() for moving averages and rolling statistics.
Use expanding() for cumulative metrics.
Use ewm() for exponential weighted moving averages.
Use shift() and pct_change() for lagged comparisons and returns.

You can now reuse this pattern for stock prices, sensor readings, website traffic, or any time-based data.

1. Sample Time Series Data

2. Rolling Window – Moving Average with rolling().mean()

What is rolling?

Example: 3-Day Moving Average of Price

Explanation

Output:

3. Rolling Window with Multiple Statistics

Example: Rolling Min, Max, and Std Dev

Explanation

Output (rounded for readability):

4. Expanding Window – Cumulative Statistics with expanding()

What is expanding?

Example: Cumulative Max and Cumulative Mean

Explanation

Output:

5. Exponential Weighted Moving Average (EWMA) with ewm()

What is EWMA?

Example: EWMA with span=3

Explanation

Output:

6. Shift Operations – Lag, Lead, and Percentage Change

What is shift()?

Example: Previous Price, Next Price, and Price Change

Explanation

Output:

Summary

2. Rolling Window – Moving Average with `rolling().mean()`

4. Expanding Window – Cumulative Statistics with `expanding()`

5. Exponential Weighted Moving Average (EWMA) with `ewm()`

Example: EWMA with `span=3`

What is `shift()`?