Back to Home

1) EDA and Forecasting on weekly petrol and diesel data in the UK

Jupyternotebook links:

• Exploratory Data Analysis (EDA) on weekly petrol and diesel data

• LSTM forecasting without Trend removal

• LSTM forecasting with Trend removal

Details about the data:

• This data is from June 2003 till May 2020. When the data was extracted, the latest data was from May 2020.
• The data was for a weekly road fuel prices on the cost for the unleaded petrol (ULSP) and unleaded diesel (ULSD).
• Univariate forecasting method was used
• 7-day windowed data size was chosen due to the data being weekly. Also, weekly data would seem appropriate to choose to forecast future weeks.
• A bidirectional LSTM was used for this model. L2 and dropout regularization were used to reduce overfitting.

  Forecasting without trend removal

• For this section, I looked took the petrol pump price data over time and used bi-directional LSTM model to fit and predict future values of weekly petrol data in the UK.

• In this section, I did not implement de-trending to view how the results would fare compared to detrending the data. A univariate approach was used for this, as when additional features such as petrol_duty_rate, petrol_vat_percent_rate, and Year were added, it did not improve the model. However, this model can be adapted to a multivariate time series, by feeding it x columns data and changing the num_x to numbers of features.

• Furthermore, 90% (841 values) of the data was used for training and 5% (37 values) was used for testing.

• In the end, predicted 7 weeks of data for the petrol prices into the future with the model.

Training and testing loss:

• Results on the training and validation are good as validation loss is lower compared to the training loss. This helps as the model will be able to predict future values better.

• Results on the training set indicate that the predicted values are similar to the actual values.

• Although, not perfect. The results from the predicted values from the test/validation data show that it can predict a similar trend compared to the actual data of the petrol price.

Train MSE: 5.102

Test MSE: 10.504

Predicting the future:

7 weeks into the future shows there is going to be a downwards trend in the petrol price in the following weeks.

Forecasting with trend removal

• For this section, I looked took the petrol pump price data over time and used bi-directional LSTM model to fit and predict future values of weekly petrol data in the UK.

• I also applied detrending before fitting the model as in papers it was a recommended step for pre-processing. Also, due to detrending, this will make the data stationary which will allow the model to train well..

• A univariate approach was used for this, as when additional features such as petrol_duty_rate, petrol_vat_percent_rate, and Year were added, it did not improve the model. However, this model can be adapted to a multivariate time series, by feeding it x columns data and changing the num_x to numbers of features.

• The cross-validation for this data set was changed a little bit. 90% (841 values) of the data was used for training and 8% (71 values) was used for testing. Lastly, 2% of the data (11 values) were used to test unknown future values.

• obs(t) - obs(t-1) to detrend the data

Training and testing loss:

Predicted and Actual values on the Training set:

Predicted and Actual values on the Test set:

Train MSE: 0.679

Test MSE: 0.571

• From the training and testing loss graph and with the MSE, it shows the model is doing well in the test data set. There is no overfitting compared to the model without conducting the detrending. With the trend values added back to the data, it significantly improved MSE and its prediction on the test set.

Future predictions: