Backtesting Your Portfolio Strategy with Python: Quantitative Finance Series

Keywords: Backtesting, Portfolio, Python, Strategy

In our last article, we used Modern Portfolio Theory (MPT) to find an “optimal” portfolio. This is a fantastic theoretical result, but it comes with a big question: would this strategy have actually made money in the past? And how does it stack up against a simpler approach?

Welcome to the third installment of our hands-on quantitative finance series. Today, we bridge the gap between theory and reality by building a backtesting engine. Backtesting is the process of simulating an investment strategy using historical data to see how it would have performed. It is one of the most critical steps in the life of a quant. A strategy must be rigorously backtested before deploying with real capital.

We will build a system to test our max Sharpe Ratio portfolio strategy. We will rebalance it periodically. Then, we will compare its performance against a simple benchmark. This guide will provide you with a robust framework for testing your own investment ideas.

Why Backtest?

An optimal portfolio calculated today is based on historical data up to this point. But markets change. A portfolio that was optimal for the last five years might not be optimal for the next five.

A back-test helps us assess the robustness of a strategy. Instead of calculating the optimal weights once, we’ll go back in time and pretend we are at the start of each year. At that point, we’ll use only the data available up to that time to calculate the optimal portfolio. We then “hold” that portfolio for the next year and record its performance. We repeat this process year after year.

This “walk-forward” approach gives us a realistic picture of how the strategy would have performed. It helps answer key questions:

Performance: Did the strategy generate positive returns?
Outperformance (Alpha): Did it beat a simple benchmark, like an equal-weight portfolio or an index like the S&P 500?
Risk: How volatile was the strategy? What was its worst loss (maximum drawdown)?
Consistency: Did it perform well across different market conditions (e.g., bull markets, bear markets)?

A good backtest also requires us to define key performance metrics:

Cumulative Return: The total return of the strategy over the entire back-test period.
Annualized Return (CAGR): The geometric average amount of money earned by an investment each year over a given time period.
Annualized Volatility: The standard deviation of the returns, a measure of risk.
Sharpe Ratio: The risk-adjusted return, just like in our previous article, but this time calculated on the backtested performance.
Maximum Drawdown: The largest peak-to-trough decline in the portfolio’s value. This is a crucial indicator of downside risk.

Let’s Get Coding: Building the Backtester

We’ll build our backtester to test the max Sharpe Ratio strategy on the same set of stocks, rebalancing annually.

Prerequisites

The required libraries are the same as before:

pip install numpy pandas pandas-datareader matplotlib scipy

pip install numpy pandas pandas-datareader matplotlib scipy

Step 1: Fetching a Longer History of Data

For a meaningful backtest, we need a longer time series of data. Let’s go back to 2010.

import pandas as pd
import numpy as np
import yfinance as yf
import matplotlib.pyplot as plt
from datetime import datetime
from scipy.optimize import minimize

# Use the same tickers as the previous article
tickers = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'JPM', 'V', 'PG', 'JNJ']

# Fetch a longer history of data for the backtest
start_date = datetime(2010, 1, 1)
end_date = datetime(2024, 12, 31)

# Fetch the 'Adj Close' prices using yfinance
adj_close_df = pd.DataFrame()
for ticker in tickers:
    # Use yf.download to fetch data
    data = yf.download(ticker, start=start_date, end=end_date)
    adj_close_df[ticker] = data['Close']

# Calculate daily log returns
log_returns = np.log(adj_close_df / adj_close_df.shift(1)).dropna()

print("Data shape:", log_returns.shape)
print(log_returns.head())

import pandas as pd
import numpy as np
import yfinance as yf
import matplotlib.pyplot as plt
from datetime import datetime
from scipy.optimize import minimize

# Use the same tickers as the previous article
tickers = ['AAPL', 'MSFT', 'GOOGL', 'AMZN', 'JPM', 'V', 'PG', 'JNJ']

# Fetch a longer history of data for the backtest
start_date = datetime(2010, 1, 1)
end_date = datetime(2024, 12, 31)

# Fetch the 'Adj Close' prices using yfinance
adj_close_df = pd.DataFrame()
for ticker in tickers:
    # Use yf.download to fetch data
    data = yf.download(ticker, start=start_date, end=end_date)
    adj_close_df[ticker] = data['Close']

# Calculate daily log returns
log_returns = np.log(adj_close_df / adj_close_df.shift(1)).dropna()

print("Data shape:", log_returns.shape)
print(log_returns.head())

Step 2: The Optimization Function

We’ll need our optimization function from the previous article. Let’s package it neatly. This function will take a slice of historical returns and find the optimal weights for the max Sharpe Ratio portfolio.

# We'll assume a constant risk-free rate for simplicity
risk_free_rate = 0.02

def get_optimal_weights(log_returns_slice):
    """
    Finds the optimal portfolio weights for the max Sharpe ratio.
    """
    mean_returns = log_returns_slice.mean()
    cov_matrix = log_returns_slice.cov()
    num_assets = len(tickers)

    def get_portfolio_stats(weights):
        returns = np.sum(mean_returns * weights) * 252
        volatility = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))) * np.sqrt(252)
        return np.array([returns, volatility, (returns - risk_free_rate) / volatility])

    def minimize_negative_sharpe(weights):
        return -get_portfolio_stats(weights)[2]

    constraints = ({'type': 'eq', 'fun': lambda weights: np.sum(weights) - 1})
    bounds = tuple((0, 1) for _ in range(num_assets))
    initial_weights = np.array([1./num_assets] * num_assets)

    result = minimize(minimize_negative_sharpe, initial_weights, method='SLSQP', bounds=bounds, constraints=constraints)
    return result.x

# We'll assume a constant risk-free rate for simplicity
risk_free_rate = 0.02

def get_optimal_weights(log_returns_slice):
    """
    Finds the optimal portfolio weights for the max Sharpe ratio.
    """
    mean_returns = log_returns_slice.mean()
    cov_matrix = log_returns_slice.cov()
    num_assets = len(tickers)

    def get_portfolio_stats(weights):
        returns = np.sum(mean_returns * weights) * 252
        volatility = np.sqrt(np.dot(weights.T, np.dot(cov_matrix, weights))) * np.sqrt(252)
        return np.array([returns, volatility, (returns - risk_free_rate) / volatility])

    def minimize_negative_sharpe(weights):
        return -get_portfolio_stats(weights)[2]

    constraints = ({'type': 'eq', 'fun': lambda weights: np.sum(weights) - 1})
    bounds = tuple((0, 1) for _ in range(num_assets))
    initial_weights = np.array([1./num_assets] * num_assets)

    result = minimize(minimize_negative_sharpe, initial_weights, method='SLSQP', bounds=bounds, constraints=constraints)
    return result.x

Step 3: Setting Up and Running the Backtest Loop

This is the core of our backtesting engine. We’ll define a rebalancing frequency (annual) and loop through time.

# Define rebalancing frequency (annually)
rebalance_dates = log_returns.resample('YE').first().index

# We need at least one year of data to start, so we begin from the second year
backtest_start_date = rebalance_dates[1]

# Store portfolio returns
portfolio_returns = []

# Loop through rebalancing dates
for i in range(len(rebalance_dates) - 1):
    start_period = rebalance_dates[i]
    end_period = rebalance_dates[i+1]

    # 1. Get historical data slice for optimization
    # Ensure we don't include the end_period in the optimization data
    #optimization_data = log_returns.loc[:start_period]

    # Define the start of the lookback period
    lookback_start = start_period - pd.DateOffset(years=3)
    optimization_data = log_returns.loc[lookback_start:start_period]

    # 2. Calculate optimal weights for this period
    # Using a try-except block in case optimization fails for a period
    try:
        optimal_weights = get_optimal_weights(optimization_data)
    except:
        # If optimization fails, fall back to equal weights
        optimal_weights = np.array([1./len(tickers)] * len(tickers))

    # 3. Get the returns for the holding period (the next year)
    # Ensure we include the end_period in the holding period data
    holding_period_returns = log_returns.loc[start_period:end_period]

    # 4. Calculate the portfolio's returns for this period
    period_portfolio_return = np.dot(holding_period_returns, optimal_weights)

    # 5. Append the returns to our list as a pandas Series
    # Convert the numpy array to a pandas Series with the correct index
    portfolio_returns.append(pd.Series(period_portfolio_return, index=holding_period_returns.index)) # Convert to Series

# Concatenate all period returns into a single Series
backtest_returns = pd.concat(portfolio_returns)
backtest_returns.name = "MPT Strategy"

# Define rebalancing frequency (annually)
rebalance_dates = log_returns.resample('YE').first().index

# We need at least one year of data to start, so we begin from the second year
backtest_start_date = rebalance_dates[1]

# Store portfolio returns
portfolio_returns = []

# Loop through rebalancing dates
for i in range(len(rebalance_dates) - 1):
    start_period = rebalance_dates[i]
    end_period = rebalance_dates[i+1]

    # 1. Get historical data slice for optimization
    # Ensure we don't include the end_period in the optimization data
    #optimization_data = log_returns.loc[:start_period]

    # Define the start of the lookback period
    lookback_start = start_period - pd.DateOffset(years=3)
    optimization_data = log_returns.loc[lookback_start:start_period]

    # 2. Calculate optimal weights for this period
    # Using a try-except block in case optimization fails for a period
    try:
        optimal_weights = get_optimal_weights(optimization_data)
    except:
        # If optimization fails, fall back to equal weights
        optimal_weights = np.array([1./len(tickers)] * len(tickers))

    # 3. Get the returns for the holding period (the next year)
    # Ensure we include the end_period in the holding period data
    holding_period_returns = log_returns.loc[start_period:end_period]

    # 4. Calculate the portfolio's returns for this period
    period_portfolio_return = np.dot(holding_period_returns, optimal_weights)

    # 5. Append the returns to our list as a pandas Series
    # Convert the numpy array to a pandas Series with the correct index
    portfolio_returns.append(pd.Series(period_portfolio_return, index=holding_period_returns.index)) # Convert to Series

# Concatenate all period returns into a single Series
backtest_returns = pd.concat(portfolio_returns)
backtest_returns.name = "MPT Strategy"

Let’s break down the loop:

For each year, we select all historical data up to the start of that year.
We feed this data into our get_optimal_weights function to decide our asset allocation for the upcoming year.
We then select the actual returns that occurred during that year.
We apply our calculated weights to these actual returns to see how our portfolio performed.
We save these returns and move to the next year.

Step 4: Creating a Benchmark

A strategy is only good if it’s better than a simple alternative. Let’s create an equal-weight (1/n) portfolio as our benchmark.

# Create benchmark returns (equal weight)
num_tickers = len(tickers)
equal_weights = np.array([1/num_tickers] * num_tickers)
benchmark_returns = log_returns.dot(equal_weights)
benchmark_returns = benchmark_returns.loc[backtest_returns.index] # Align dates
benchmark_returns.name = "Equal Weight Benchmark"

# Create benchmark returns (equal weight)
num_tickers = len(tickers)
equal_weights = np.array([1/num_tickers] * num_tickers)
benchmark_returns = log_returns.dot(equal_weights)
benchmark_returns = benchmark_returns.loc[backtest_returns.index] # Align dates
benchmark_returns.name = "Equal Weight Benchmark"

Step 5: Analyzing and Visualizing the Performance

Now we can compare our MPT strategy against the benchmark. We’ll calculate cumulative returns and plot them.

# Calculate cumulative returns
cumulative_mpt_returns = (1 + backtest_returns).cumprod()
cumulative_benchmark_returns = (1 + benchmark_returns).cumprod()

# Plot the results
plt.figure(figsize=(14, 8))
cumulative_mpt_returns.plot(label=backtest_returns.name, legend=True)
cumulative_benchmark_returns.plot(label=benchmark_returns.name, legend=True)
plt.title('Backtest: MPT Strategy vs. Equal Weight Benchmark')
plt.xlabel('Date')
plt.ylabel('Cumulative Growth of $1')
plt.grid(True, linestyle='--', alpha=0.6)
plt.show()

# Calculate cumulative returns
cumulative_mpt_returns = (1 + backtest_returns).cumprod()
cumulative_benchmark_returns = (1 + benchmark_returns).cumprod()

# Plot the results
plt.figure(figsize=(14, 8))
cumulative_mpt_returns.plot(label=backtest_returns.name, legend=True)
cumulative_benchmark_returns.plot(label=benchmark_returns.name, legend=True)
plt.title('Backtest: MPT Strategy vs. Equal Weight Benchmark')
plt.xlabel('Date')
plt.ylabel('Cumulative Growth of $1')
plt.grid(True, linestyle='--', alpha=0.6)
plt.show()

This plot is the moment of truth. It shows how $1 invested in each strategy would have grown over the back-test period.

Step 6: Calculating Performance Metrics

Visuals are great, but we need hard numbers to properly evaluate the strategies. Let’s create a function to calculate our key metrics.

def calculate_performance_metrics(returns):
    """Calculates key performance metrics for a series of returns."""
    total_return = (1 + returns).prod() - 1
    num_years = len(returns) / 252
    annualized_return = (1 + total_return) ** (1/num_years) - 1
    annualized_volatility = returns.std() * np.sqrt(252)
    sharpe_ratio = (annualized_return - risk_free_rate) / annualized_volatility

    # Calculate Maximum Drawdown
    cumulative_returns = (1 + returns).cumprod()
    peak = cumulative_returns.expanding(min_periods=1).max()
    drawdown = (cumulative_returns/peak) - 1
    max_drawdown = drawdown.min()

    metrics = {
        'Cumulative Return': f"{total_return:.2%}",
        'Annualized Return (CAGR)': f"{annualized_return:.2%}",
        'Annualized Volatility': f"{annualized_volatility:.2%}",
        'Sharpe Ratio': f"{sharpe_ratio:.2f}",
        'Maximum Drawdown': f"{max_drawdown:.2%}"
    }
    return pd.Series(metrics)

# Calculate and display metrics for both strategies
mpt_metrics = calculate_performance_metrics(backtest_returns)
benchmark_metrics = calculate_performance_metrics(benchmark_returns)

results_df = pd.DataFrame({
    'MPT Strategy': mpt_metrics,
    'Equal Weight Benchmark': benchmark_metrics
})

print("\n--- Backtest Performance Metrics ---")
print(results_df)

def calculate_performance_metrics(returns):
    """Calculates key performance metrics for a series of returns."""
    total_return = (1 + returns).prod() - 1
    num_years = len(returns) / 252
    annualized_return = (1 + total_return) ** (1/num_years) - 1
    annualized_volatility = returns.std() * np.sqrt(252)
    sharpe_ratio = (annualized_return - risk_free_rate) / annualized_volatility

    # Calculate Maximum Drawdown
    cumulative_returns = (1 + returns).cumprod()
    peak = cumulative_returns.expanding(min_periods=1).max()
    drawdown = (cumulative_returns/peak) - 1
    max_drawdown = drawdown.min()

    metrics = {
        'Cumulative Return': f"{total_return:.2%}",
        'Annualized Return (CAGR)': f"{annualized_return:.2%}",
        'Annualized Volatility': f"{annualized_volatility:.2%}",
        'Sharpe Ratio': f"{sharpe_ratio:.2f}",
        'Maximum Drawdown': f"{max_drawdown:.2%}"
    }
    return pd.Series(metrics)

# Calculate and display metrics for both strategies
mpt_metrics = calculate_performance_metrics(backtest_returns)
benchmark_metrics = calculate_performance_metrics(benchmark_returns)

results_df = pd.DataFrame({
    'MPT Strategy': mpt_metrics,
    'Equal Weight Benchmark': benchmark_metrics
})

print("\n--- Backtest Performance Metrics ---")
print(results_df)

--- Backtest Performance Metrics ---
                         MPT Strategy Equal Weight Benchmark
Cumulative Return             812.02%                834.58%
Annualized Return (CAGR)       16.01%                 16.20%
Annualized Volatility          22.61%                 18.03%
Sharpe Ratio                     0.62                   0.79
Maximum Drawdown              -39.06%                -31.66%

--- Backtest Performance Metrics ---
                         MPT Strategy Equal Weight Benchmark
Cumulative Return             812.02%                834.58%
Annualized Return (CAGR)       16.01%                 16.20%
Annualized Volatility          22.61%                 18.03%
Sharpe Ratio                     0.62                   0.79
Maximum Drawdown              -39.06%                -31.66%

This final table gives us a clear, quantitative comparison of the two strategies across several dimensions of risk and return.

Conclusion: The Verdict

You have now built a complete backtesting engine. You’ve gone beyond simple optimization. You have rigorously tested a dynamic strategy against historical data. You compared it to a sensible benchmark using industry-standard metrics.

What do the results tell us? Equal weighted portfolio is performing better then periodically rebalanced MPT portfolio. The MPT strategy’s underperformance here stems from its reliance on historical data to predict future returns. This approach to predicting risks can sometimes be misleading. And, of course, it is not easy to beat the market but we can try.

There are several changes, that can be implemented to improve the MPT strategy’s performance.

1. Adjust the Rebalancing Frequency

Our current strategy rebalances annually ('YE'). This means the “optimal” weights are calculated only once a year. Market conditions can change much faster than that.

The Change: Try rebalancing more frequently, such as quarterly or semi-annually.
How to Implement: Change the resampling frequency.

rebalance_dates = log_returns.resample('QE').first().index # 'QE' for Quarter-End

rebalance_dates = log_returns.resample('QE').first().index # 'QE' for Quarter-End

More frequent rebalancing allows the portfolio to adapt more quickly to changing market trends and volatility patterns. However, be aware that in the real world, this would also increase transaction costs.

2. Refine the Lookback Window for Optimization

The current script uses an expanding window for optimization. This means that for each rebalancing, it uses all historical data from the very beginning (2010). The market dynamics of 2010 might not be relevant for making decisions in 2020.

The Change: Switch to a rolling lookback window, for example, using only the last 3 or 5 years of data to calculate the optimal weights.
How to Implement: Modify the line that defines optimization_data.

# Define the start of the lookback period
lookback_start = start_period - pd.DateOffset(years=3) 
optimization_data = log_returns.loc[lookback_start:start_period]

# Define the start of the lookback period
lookback_start = start_period - pd.DateOffset(years=3) 
optimization_data = log_returns.loc[lookback_start:start_period]

A rolling window makes the optimization more sensitive to recent market behavior. This approach may be a better predictor of the immediate future than very old data.

3. Modify the Optimization Constraints

The current optimization sets the weight for each asset to be between 0% and 100%. This can sometimes lead to highly concentrated portfolios if one or two assets have had exceptionally strong historical performance.

The Change: Add an upper limit for the weight of any single asset to ensure better diversification. For example, you could cap each asset at 30% of the portfolio.
How to Implement: In the get_optimal_weights function, change the bounds variable.

bounds = tuple((0, 0.30) for _ in range(num_assets))

bounds = tuple((0, 0.30) for _ in range(num_assets))

This forces the portfolio to be more diversified and less reliant on the historical performance of a few star stocks, reducing concentration risk.

4. Use a Dynamic Risk-Free Rate

The Sharpe ratio calculation, which is at the heart of the optimization, uses a hardcoded risk_free_rate of 2%. The actual risk-free rate fluctuates over time.

The Change: Fetch historical risk-free rate data (e.g., the 3-Month Treasury Bill) and use the rate corresponding to the rebalancing period.
How to Implement:
1. Fetch the data for the T-bill rate (ticker ^IRX on Yahoo Finance).
2. Pass this rate into your get_optimal_weights function and use it in your Sharpe Ratio calculation.

# Add this after fetching stock data
rf_data = yf.download('^IRX', start=start_date, end=end_date)
risk_free_rates = (rf_data['Close'] / 100).resample('YE').last() # Get year-end rates

# Modify the function call in the loop
# ...
current_risk_free_rate = risk_free_rates.loc[start_period]
optimal_weights = get_optimal_weights(optimization_data, current_risk_free_rate)
#...

# Add this after fetching stock data
rf_data = yf.download('^IRX', start=start_date, end=end_date)
risk_free_rates = (rf_data['Close'] / 100).resample('YE').last() # Get year-end rates

# Modify the function call in the loop
# ...
current_risk_free_rate = risk_free_rates.loc[start_period]
optimal_weights = get_optimal_weights(optimization_data, current_risk_free_rate)
#...

You would also need to adjust the get_optimal_weights and calculate_performance_metrics functions to accept this dynamic rate.

This provides a more accurate, time-varying measure for the Sharpe ratio, leading to potentially better and more realistic portfolio allocations.

By experimenting with these parameters, rebalancing frequency, lookback window, and constraints, you can significantly alter the behavior and performance of your MPT strategy. You can also test its robustness under different assumptions.

This framework is your starting point. You can now experiment with different rebalancing frequencies, different sets of assets, or even different optimization goals (e.g., minimizing volatility instead of maximizing Sharpe).

You’ve completed a major part of the quant workflow: from idea generation (MPT) to implementation (optimization) and finally to validation (backtesting). This is a powerful skill set for anyone serious about data-driven finance.

Happy backtesting!

SimplifiedZone

Backtesting Your Portfolio Strategy with Python: Quantitative Finance Series

Why Backtest?

Let’s Get Coding: Building the Backtester

Prerequisites

Step 1: Fetching a Longer History of Data

Step 2: The Optimization Function

Step 3: Setting Up and Running the Backtest Loop

Step 4: Creating a Benchmark

Step 5: Analyzing and Visualizing the Performance

Step 6: Calculating Performance Metrics

Conclusion: The Verdict

1. Adjust the Rebalancing Frequency

2. Refine the Lookback Window for Optimization

3. Modify the Optimization Constraints

4. Use a Dynamic Risk-Free Rate

Like this:

Leave a ReplyCancel reply

Backtesting Your Portfolio Strategy with Python: Quantitative Finance Series

Why Backtest?

Let’s Get Coding: Building the Backtester

Prerequisites

Step 1: Fetching a Longer History of Data

Step 2: The Optimization Function

Step 3: Setting Up and Running the Backtest Loop

Step 4: Creating a Benchmark

Step 5: Analyzing and Visualizing the Performance

Step 6: Calculating Performance Metrics

Conclusion: The Verdict

1. Adjust the Rebalancing Frequency

2. Refine the Lookback Window for Optimization

3. Modify the Optimization Constraints

4. Use a Dynamic Risk-Free Rate

Share this:

Like this:

Leave a ReplyCancel reply

Discover more from SimplifiedZone

Discover more from SimplifiedZone