Machine Learning for Time Series Forecasting

Time series forecasting using machine learning enables businesses to use ML-driven digital solutions. They help analyze observed data to predict future values for specific time periods.

What is time series forecasting? 

Time series forecasting is considered one of the most applied data science techniques that are used in different industries such as finance, supply chain management, production, and inventory planning. 

Stock price forecasting, weather forecasting, business planning, and resource allocation are only a few of the many possible applications for time series forecasting.

The time series forecast models based on machine learning found wide implementation in time series projects required by various businesses for facilitating the predictive distribution of time and resources.

In this post, we want to share our experience while working on deep learning for forecasting projects.

First, we'll talk about time series data and its role in forecasting techniques. Later we'll discuss various classical and machine learning methods for analysis and forecasting so that you can gain a deeper understanding of using machine learning in business. We'll walk through the main steps taken while implementing time series machine learning forecast projects and analyze the main challenges that may arise during the project.

Time series forecasting in machine learning

Before anything else, it is important to review first what time series is, as well as time series analysis and forecasting.

Time Series pertains to the sequence of observations collected in constant time intervals, be it daily, monthly, quarterly or yearly. Time Series Analysis involves developing models used to describe the observed time series and understand the "why" behind its dataset. This involves creating assumptions and interpretations about a given data. Time Series Forecasting makes use of the best-fitting model essential to predicting future observations based on the complex processing of current and previous data.

Machine learning forecasting proved to be the most effective in capturing the patterns in the sequence of both structured and unstructured data and its further time series analysis forecasting.

Speaking of applying a suitable model for deep learning for time series forecasting, it is important to understand the components of the time series data:

✔Trends (to describe increasing or decreasing behavior of the time series frequently presented in linear modes).

✔Seasonality (to highlight the repeating pattern of cycles of behavior over time) 

✔Irregularity/Noise (to regard the non-systematic aspect of time series deviating from the common model values)

✔Cyclicity (to identify the repetitive changes in the time series and define their placement in the cycle). 

Modeling time series

Among the numerous methods aimed at achieving precision and minimizing errors and losses within forecasting, there are several classical and modern machine learning methods that prove their accuracy and computational relevance. 

There are various types of time series analysis. The most widely applied models for deep learning for time series forecasting projects by categories are as follows: 

Classical Methods

Tried-and-tested classical methods are the most widely applied to develop time series forecasts. These models can deliver accurate time series predictions using spreadsheets and formulas applied to data sets.

However, classical methods have some drawbacks that make them insufficient for building accurate forecasts in rapidly-changing environments.

Naïve model

In most cases, naïve models are applied as a random walk (with the last observed value used as a unit for the next period forecast) and a seasonal random walk (with a value from the same period of the last observed time span used as a unit of the forecast).

Exponential Smoothing Model 

The method uses the foundation of machine learning time series classification. Forecasts are made on the basis of captured weighted averages and according to weights decreasing as the observer tracks back in time. 

The method implies a smoothing constant that helps make previous data less essential and the newest data more crucial to build time-series predictions. The constant adjusts when building forecasts to make more recent data more or less valuable compared to historical data. The model can not help detect trends and seasonality. However, a number of extensions of the simple exponential smoothing (SES) have been introduced to include the trend/damped trend and seasonality.

ARIMA/SARIMA

ARIMA stands for the combination of Autoregressive (AR) and Moving Average (MA) approaches within building a composite model of the time series. ARIMA models include parameters to account for season and trend (for instance, dummy variables for weekdays and they're distinguishing). In addition, they allow for the inclusion of autoregressive and moving average terms to handle the autocorrelation embedded in the data.

SARIMA stands for Seasonal Autoregressive Integrated Moving Average: it widens the application of the ARIMA by including a linear combination of past seasonal values and/or forecast errors. 

Linear Regression method 

Linear regression is the simple statistical technique commonly used for predictive modeling. Breaking it down to basics, it comes to providing an equation of independent variables on which our target variable is built upon. The defined relationship between variables helps predict future values.

Machine Learning Methods

ML-driven methods use machine learning to analyze large amounts of data using trained models.

Multi-Layer Perceptron (MLP)

As an applied machine learning approach, the time series model forecasting implies the triple structure of the initial layer of the network, which takes in an input, a hidden layer of nodes, and an output layer used to make a prediction.

MLP is a feedforward neural network (Image Source)

Recurrent Neural Network (RNN)

RNNs are basically neural networks with memory that can be used for predicting time-dependent targets. Recurrent neural networks can memorize the previously captured state of the input to make a decision for the future time step. Recently, lots of variations have been introduced to adapt Recurrent Networks to a variety of domains.

Long Short-Term Memory (LSTM)

LSTM cells (special RNN cells) were developed to find the solution to the issue with gradients by presenting several gates to help the model make a decision on what information to mark as significant and what information to ignore. GRU is another type of gated recurrent network.

Image Source

Besides methods mentioned above, Convolutional Neural Network models, or CNNs for short, as well as decision tree-based models like Random Forest and Gradient Boosting variations (LightGBM, CatBoost, etc.) can be applied to time series forecasting.

Evaluating Model Accuracy 

Speaking of time series methods, we should mention that visually identifying which machine learning model has the best accuracy is not always producing reliable results. 

Among the time series prediction models, the method of calculating the MAPE (Mean Absolute Percent Error) shows the best results as a quicker way to compare the overall forecast accuracy of a proposed model.  

The measures represent the percentage of average absolute error that occurred. The overall idea of calculations for the evaluation of model accuracy comes to the following proportion: the lower the MAPE, the better the forecast accuracy.

Time series forecasting process

To avoid any detrimental consequences and ensure the project's success in terms of designing the predictive time model, deep learning for time series forecasting is being implemented by taking the following steps.

1. Project Goal Definition

It is the first step of the time series machine learning tutorial. Prior to discussing the project in detail, make sure that you understand the subjective. It means understanding the specifics of the business domain of forecast operation, including terms and key definitions, as well as common business models pertaining to the particular domain. Thus, this stage implies defining the project specifics through extensive research within the area of knowledge.  

2. Data Gathering and Exploration

Defining the basics leads to a clear view of the scope of data you need to collect to facilitate the further discovery of data insights. With implementing techniques of building the plot graphs and visualization charts, the domain knowledge reception reaches a level required for strategic data exploration and estimating hinges and trends for further evaluating the variations volume. It also helps specify the forecasting task and conduct preliminary exploratory analysis successfully. 

3. Data Preparation

At this stage, the development team performs cleaning data for relevant insights and further subtracting the variables of importance. The data preparation process for feature engineering is being launched. The key component of feature engineering is targeting the areas of knowledge of the domain that is crucial for designing the new features in the existing dataset. 

4. Applying the Time Series Forecasting Method

On the basis of preliminary data preparation and exploratory analysis of a range of time series forecasting conducted at the previous stage, the team works with several models and chooses one based on the criteria of relevance and projected accuracy of the forecast. Fitting the model for project performance ensures the proper model development and consideration of variables essential within the forecasting process.  

5. Evaluation/Validation and Performance Comparison

This step covers the optimization of the forecasting model parameters and achievements of high performance. By applying a cross-validation tuning method implying the data split, data scientists train forecasting models with different sets of hyper-parameters. Completing this stage requires applying performance score estimation and diverse test datasets evaluation. It is important to take an out-of-sample approach to receive adequate performance evaluation when processing a particular type of data.   

6. Deployment

This stage includes the forecasting model integration into production. At this particular stage, we highly recommend setting a pipeline to aggregate new data to use for the next AI features. It helps in data preparation work when performing your future projects.   

The forecast development requires agility in approach as it is an iterative process.

The iterative loops incorporate a series of exploration and visualization to get data. Once visualization has been performed, it might be required to take a step back to gather additional data. The models are revised and updated as new data and new insights are made available.

Thus, this stage shifts focus to the ongoing development and refinement of one or more models until a relevant level of performance is achieved. 

APPLICATIONS OF ML-DRIVEN FORECASTING

Time series ML analysis is used for forecasting in many different industries.

ML-driven data series analysis can help predict the following:

• Demand and sales. Machine learning can help analyze historical data to predict customer demand or sales. Some methods also analyze seasonality or other repetitive events to develop accurate forecasts.

• Stock prices. Traders widely use time series prediction machine learning to forecast the prices for stocks, bonds, securities, or other assets in the long- and short-term perspective.

• Harvest volume. Farmers can use data sets collected from different sources and apply machine learning to turn them into insights. With the help of this technology, they can forecast the harvest value to plan the number of resources required for crop collecting.

• Web traffic. Online businesses can analyze all the data sets they collect to define patterns. Using the collected insights, they can adjust allocated resources to increase efficiency and reduce downtime.

• Climate and weather. Tons of different weather data types are collected all around the world constantly. ML can help analyze the collected information to predict weather and climate changes.

• Diseases. Machine learning can help doctors spot diseases or drastic health changes that need an immediate response.

• Academics. Fast-paced analytics boosts the number of new scientific discoveries. Time series forecasting models can help scientists receive valuable insights faster than using conventional analytics.

• Demographic changes. Most classical formulas imply greek letters because they were initially developed in Ancient Greece to analyze demographic changes. These days, machine learning helps get more accurate forecasts on demographic changes.

The technology can help turn any data into valuable insights and accurate predictions if appropriately processed.

COMPANIES THAT USE ML-ENABLED FORECASTING

ML-driven solutions are widely used by top companies in many different industries. 

Let's learn more about three successful technology adopters and explore machine learning demand forecasting examples.

BlackRock

BlackRock is considered the most successful company in the investment, advisory, and risk assessment industries. The company managed to achieve a significant competitive advantage by becoming an early adopter of machine learning and artificial intelligence. 

For instance, Aladdin Risk, one of the company's products, uses machine learning to analyze time series to deliver sophisticated risk analytics.

Amazon

Amazon is the leading company in the eCommerce industry. The company constantly evolves and implements new technologies to keep the leading positions.

It collects a lot of data about its customers. The company uses ML models to analyze all the data to get detailed insights, build accurate time series predictions, spot unusual activity, etc. With the help of machine learning, the company can analyze time-series data to offer the best purchasing suggestions to its customers and minimize the fraud rate.

Walmart

One of the most popular retail companies in the US uses machine learning to predict sales and optimize its supply chain. The company applies forecasting algorithms to turn raw data collected by its stores into useful insights. The use of ML-driven forecasts helps the company satisfy the ever-changing customer demand and optimize its operations.

Time Series Forecasting Project Challenges 

We would like to share the experience we acquired in performing forecasting projects and pinpoint the challenges that the development team might face. 

Lack of Data

The bigger the datasets are, the more training data the system can access, which leads to higher accuracy of predictions. However, there are limitations to using machine learning associated with a lack of historical or seasonality data for a target variable. Consequently, a lack of data might result in an overall decrease in forecasting precision. 

Lack of Domain Knowledge

Apparently, without adequate domain knowledge, the stage of feature engineering itself as the key component of ML implementation falls under high risk. In general, domain knowledge can help improve the quality of models in any project. To prevent the problem arising from the lack of domain knowledge, the expertise of business niche specialists is required.  

In addition to the problems already mentioned, while working on the stock price forecasting project, our major concerns were related to the heteroscedasticity and chaoticness of stock prices.

Poor Data Quality

The quality of data directly affects the accuracy of results. Data cleaning is a heavy-lifting task that should be performed to achieve the best results. 

The main problems that should be tackled when preparing data for further analysis are:

• Missed values
• Improperly formatted data
• Inconsistent values
• Unreliable data sources
• Irrelevant data attributes

TOOLS FOR ML-DRIVEN FORECASTING

Software engineers use a large number of different tools to develop well-tailored ML-driven solutions. The most popular programming language used for creating machine learning time series predictions is Python due to its many advantages.

The most popular services that offer access to versatile tools for developing ML-enabled forecasts are:

Google Cloud

Google Cloud offers access to a broad range of products that help develop AI- and ML-enabled solutions. The most popular are:

• TensorFlow
• BigQuery
• VertexAI

Software engineers widely use these tools to train machine learning models for getting data-driven forecasts. BigQuery and VertexAI can use Google's cloud computing infrastructure to analyze large datasets.

Azure Machine Learning

Azure Machine Learning is a service developed by Microsoft. It offers access to many tools for the end-to-end development of machine learning models. Also, users can train and deploy their models using the solution. 

Using the Azure Time Series Insights tool, developers can visualize all the data in live time for further analysis.

AWS Machine Learning

AWS Machine Learning is a forward-looking service from Amazon. It offers access to a cloud-based infrastructure and versatile tools that help analyze datasets. Amazon Forecast is a popular tool that helps analyze time series data to develop predictive models. The solution helps inspect time series data used for machine learning and pick the best models.

Final Thought

The complexity of time series forecasting project implementation demands the highest quality of development, which our team experts can provide.

With the knowledge the company accumulated in similar project performance, we surely meet your project requirements in regards to thoroughly considering domain specifics and business goals of the particular time series prediction example.