Forecasting Stock Index With Deep Learning¶
- In this project I will be using multivariate time series forecasting using an LSTM model architecture
- The goal is predict the next 5 days "SPY" index prices using the last 30 days prices as well as other features including support and resistance bands and momentum
In [25]:
# Import necessary libraries
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
import requests
import numpy as np
from keras.utils.vis_utils import plot_model
import datetime
%load_ext tensorboard
In [2]:
# Call AlphaVantage api to fetch "SPY" index data
api_key = 'OAK4S5QP4CT156KH'
ticker = "SPY"
url = f'https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&symbol={ticker}&apikey={api_key}'
response = requests.get(url=url, params={'outputsize':'full'})
json_data = response.json()['Time Series (Daily)']
In [3]:
# Read json data into pandas dataframe
df = pd.DataFrame(json_data).transpose()
df.columns = [c.split('. ')[1] for c in df.columns]. # Clean up column names
# convert data to floats and reverse order: older to newer ticker prices
df = df.apply(pd.to_numeric, errors='coerce')
df = df.reindex(index=df.index[::-1])
# Add support and resistance indicators (+/- 2 std devs) from a 30 day moving average
df['ma_close'] = df['close'].rolling(window=5).mean()
ma = df['close'].rolling(window=30).mean()
stdev = df['close'].rolling(window=30).std()
df['upper_boll'] = ma + 2*stdev
df['lower_boll'] = ma - 2*stdev
# Add 5 day percent change
df['5_day_pct_change'] = df['close'].pct_change(5)
# Add future 5 day returns
df['future_5_return'] = df['5_day_pct_change'].shift(-5)
# Add 5 day volume change
df['5_day_vol_change'] = df['volume'].pct_change(5)
# Add 1 and 5 day momentum
close = df['close']
close_past_5 = df['close'].shift(5)
close_past_1 = df['close'].shift(1)
df['momentum_1'] = close - close_past_1
df['momentum_5'] = close - close_past_5
# drop null rows
df.dropna(axis=0,inplace=True)
df.head()
Out[3]:
In [4]:
# Quick plot to visualize support and resistance bands
plt.figure(figsize=(12,8))
plt.plot(df['close'][-500:], color='g')
plt.plot(df['ma_close'][-500:], color='black')
plt.plot(ma[-500:], color='b')
plt.plot(df['upper_boll'][-500:], color='r')
plt.plot(df['lower_boll'][-500:], color='r')
plt.show()
Features included in the model (8 total features)¶
- 5 day moving average of closing prices (What we are trying to forcast)
- Volume
- Upper Bolling Band (30 day moving average + 2 std)
- Lower Bolling Band (30 day moving average - 2 std)
- 5 day percent change in prices
- 5 day percent change in volume
- 1 day momentum
- 5 day momentum
In [5]:
# extract main features to train on
features_considered = ['ma_close','volume','upper_boll','lower_boll','5_day_pct_change',\
'5_day_vol_change', 'momentum_1','momentum_5']
features = df[features_considered]
# create train test split vectors
TRAIN_SPLIT = 4000
dataset = features.values
# Normalize features in dataset based on training set mean and std
data_mean = dataset[:TRAIN_SPLIT].mean(axis=0)
data_std = dataset[:TRAIN_SPLIT].std(axis=0)
dataset = (dataset-data_mean)/data_std
# Rows: days
# Columns: features (ex. ma, momentum, upper_bol etc...)
print(dataset)
print(f'Data Dimensions: {dataset.shape}')
Transforming Dataset Into Multivariate Format¶
- We are trying to transform our data into an array of shape (# days, 30, 8)
- 30 represents how many time steps in the past we want to look in order to make our predictions
- 8 represents how many features we want to analyze for each time step
credit: https://www.tensorflow.org/tutorials/structured_data/time_series
In [6]:
def multivariate_data(dataset, target, start_index, end_index, history_size,
target_size, step, single_step=False):
data = []
labels = []
start_index = start_index + history_size
if end_index is None:
end_index = len(dataset) - target_size
for i in range(start_index, end_index):
indices = range(i-history_size, i, step)
data.append(dataset[indices])
if single_step:
labels.append(target[i+target_size])
else:
labels.append(target[i:i+target_size])
return np.array(data), np.array(labels)
In [7]:
# How many time steps in the past we want to look at to make our prediction
past_history = 30
# How many time steps into the future we want to forcast
future_target = 5
# When set to 1, the function will slide the 30 day window by 1 time step each iteration
STEP = 1
x_train_multi, y_train_multi = multivariate_data(dataset, dataset[:, 0], 0,
TRAIN_SPLIT, past_history,
future_target, STEP)
x_val_multi, y_val_multi = multivariate_data(dataset, dataset[:, 0],
TRAIN_SPLIT, None, past_history,
future_target, STEP)
In [8]:
x_train_multi.shape
Out[8]:
In [9]:
BUFFER_SIZE = 3900
BATCH_SIZE = 360
# Taking x_train and y_train numpy arrays and converting them to a tensorflow dataset
train_data_multi = tf.data.Dataset.from_tensor_slices((x_train_multi, y_train_multi))
# Shuffling and batching data
train_data_multi = train_data_multi.cache().shuffle(BUFFER_SIZE).batch(BATCH_SIZE).repeat()
val_data_multi = tf.data.Dataset.from_tensor_slices((x_val_multi, y_val_multi))
val_data_multi = val_data_multi.batch(BATCH_SIZE).repeat()
In [13]:
# Defining plotting functions
def create_time_steps(length):
return list(range(-length, 0))
#Plots actual vs. predicted values
def multi_step_plot(history, true_future, prediction):
plt.figure(figsize=(12, 6))
num_in = create_time_steps(len(history))
num_out = len(true_future)
plt.plot(num_in, np.array(history[:, 0]), label='History')
plt.plot(np.arange(num_out)/STEP, np.array(true_future), 'bo',
label='True Future')
if prediction.any():
plt.plot(np.arange(num_out)/STEP, np.array(prediction), 'ro',
label='Predicted Future')
plt.legend(loc='upper left')
plt.show()
# Plots training and validation loss
def plot_train_history(history, title):
loss = history.history['mae']
val_loss = history.history['val_mae']
epochs = range(len(loss))
plt.figure()
plt.plot(epochs, loss, 'b', label='Training loss')
plt.plot(epochs, val_loss, 'r', label='Validation loss')
plt.title(title)
plt.legend()
plt.show()
In [11]:
for x, y in train_data_multi.take(1):
multi_step_plot(x[0], y[0], np.array([0]))
Summary of Model Architecture:¶
I am using 2 LSTM layers with 512 hidden units followed by a Dropout layer, which reduces overfitting in the model. The final dense layer outputs 5 values, which are the predictions for the next five days prices. I use an Adam optimizer and a "mean squared error" loss function. The clipvalue = 4 ensures that the model doesn't experience exploding gradients, which is a common problem with deep recurrent network architectures.
To learn more about LSTM's see https://en.wikipedia.org/wiki/Long_short-term_memory
In [21]:
model = tf.keras.models.Sequential([
tf.keras.layers.LSTM(512, return_sequences=True, input_shape=x_train_multi.shape[-2:],activation='relu'),
tf.keras.layers.LSTM(512, activation='relu'),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Dense(5,activation='linear'),
])
model.compile(optimizer=tf.keras.optimizers.Adam(clipvalue=4.0), loss='mse', metrics=['mae'])
I define 2 callbacks here. The first will stop model training if the mean average error on the validation set falls below 0.06. The second callback connects with tensorboard so you can visualize your model's training later on.
In [22]:
class myCallback(tf.keras.callbacks.Callback):
def on_epoch_end(self, epoch, logs={}):
if logs.get('val_mae') < 0.06:
print('Done Training')
self.model.stop_training = True
callbacks = myCallback()
log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir)
In [23]:
history = model.fit(train_data_multi, epochs=100,
steps_per_epoch=11,
validation_data=val_data_multi,
validation_steps=2,
verbose=1, callbacks=[callbacks, tensorboard_callback])
Uncommenting the next line will allow you to view the results in tensorboard
In [27]:
#%tensorboard --logdir logs/fit
In [28]:
plot_train_history(history, 'Multi-Step Training and validation loss')
Examing the results!¶
As you can see below the model had varied success. In some cases the model accurately predicts the trend of the next 5 days. Note that the y-axis is on its normalized scale. It is important to realize that predicting stock prices is a very, very difficult tasks. There is a lot of information such as news headlines that could help the model's performance
In [ ]:
for x, y in val_data_multi.take(5):
multi_step_plot(x[0], y[0], model.predict(x)[0])
Final Conclusions:¶
Overall, this is a great example of how deep learning can solve difficult tasks like forcasting stock prices. Perhaps with future tweaks, someone can get the model to perform even better!