LSTMs For Time Series Forcasting

Prashant Sharma
Prashant Sharma

Data Scientist , Machine Learning and Deep Learning Engineer

We as human beings are always forcasting about the future based upon the knowledge we have in our hands, be it your prediction for the time taken to reach a coffee shop to meet a friend, or to predict the winner of the IPL. As an evolved species we like to know what’s ahead in future. May be thats why clairvoyants are so popular.

Let me Introduce to you the Clairvoyant for our Data. The one of the most popular neural network archietecture, Long Short-Term Memory or LSTM. LSTMs are part of a greater family of Recurrent Neural Networks. Unlike regular neural network archietectures RNNs unpack in time, means they have additional Time dimension.

RNN unfolding in time
LSTM archietecture

A common LSTM unit is composed of a cell, an input gate, an output gate and a forget gate. The cell remembers values over arbitrary time intervals and the three gates regulate the flow of information into and out of the cell.

Now lets try some clairvoyance for the Time-series data and what is better than knowing about the stock price trends of Apple Technology company. We will grab the stocks price data of Apple’s share from 2011 to 2017.

We can clearly see a trend in the data. This data will be a good input for our LSTM. But before feeding this to our LSTM neural net we need to split our dataset into training and testing set. Test set will help us evaluate our model once we train our deep LSTM model.

#split_data
split = 0.2
split_index = int((1 - split) * len(new_df))

X_train = np.array(new_df[:split_index])
date_train = df.iloc[:split_index,0]
X_test = np.array(new_df[split_index:])
date_test = df.iloc[split_index:,0]


print(f"shape of training dataset: {X_train.shape}")

print(f"shape of validation_dataset: {X_test.shape}")

We have one more problem to solve before feeding, we need to create a window for the data as Neural networks expect features and labels in a supervised learning environment. So lets just do that along with scaling.

#scaling
scaler= StandardScaler()
S_train = scaler.fit_transform(X_train)
S_test = scaler.transform(X_test)

def windowing(series , window_size ,  batch_size , shuffle_buffer):
    ds = tf.data.Dataset.from_tensor_slices(series)
    ds = ds.window(window_size + 1 , shift = 1 , drop_remainder = True)
    ds = ds.flat_map(lambda w : w.batch(window_size + 1))
    ds = ds.shuffle(shuffle_buffer)
    ds = ds.map(lambda w: (w[:-1] , w[-1:,1]))
    return ds.batch(batch_size)

window_size = 20
batch_size = 32
shuffle_buffer= 1000
dataset = windowing(S_train,window_size ,batch_size, shuffle_buffer)

we got ourself a perfect dataset, our window function will keep moving along the dataset with the step size of 1 and storing first 19 records as features and 20th record(high price) as label and will pack them in shuffled batches. This step is key to forcasting.

Now time to create our first LSTM model. we will be using TensorFlow 2.0 framework.

model = tf.keras.models.Sequential()
model.add(tf.keras.layers.LSTM(32, input_shape=[None,5], return_sequences = True))
model.add(tf.keras.layers.LSTM(16))
model.add(tf.keras.layers.Dense(1))
model.add(tf.keras.layers.Activation('linear'))
model.compile(loss = "mse" , optimizer= tf.keras.optimizers.Adam(learning_rate= 0.001),
            metrics = ["mae"])
model.summary()

Now the moment of truth, will it train……

history = model.fit(dataset , epochs=100 , verbose = 0 )

So our loss is consistently decreasing which is a sign that our model is learning the parameters from our dataset.

lets see how it did on our training dataset.

That is a perfect fit. Model is predicting well on the training set.

Now most crucial part of any machine learning algorithm is Evaluation. we have to evaluate the performance of our model on an unseen data. The neural net has’nt seen this that and will try to predict the future values.

forecast = []
actual = []
for time in range(len(S_test) - window_size):
    y = model.predict(S_test[time:time + window_size][np.newaxis])
    forecast.append(y[0])
i = 0   
for value in S_test:
    if i <= window_size:
        i += 1
        continue
    else:
        actual.append(value[1])



plt.figure(figsize=(10, 6))

plt.plot(date_test[20:],forecast, label = "Forecast")
plt.plot(date_test[21:],actual, alpha = 0.4 , label="Actual")
plt.legend()
plt.title("Test data")

Conclusion: So the model we have created is working well in predicting the future values. LSTMs are good in learning the context within data thats why they are so good in handling time series problems.

Caution: Do not indulge in trading based on this model.

1 thought on “LSTMs For Time Series Forcasting”

Leave a Comment

Your email address will not be published. Required fields are marked *