Statistical and Machine Learning forecasting methods: Concerns and ways forward

Summary

Machine Learning (ML) methods have been proposed in the academic literature as alternatives to statistical ones for time series forecasting. Yet, scant evidence is available about their relative performance in terms of accuracy and computational requirements. The purpose of this paper is to evaluate such performance across multiple forecasting horizons using a large subset of 1045 monthly time series used in the M3 Competition. After comparing the post-sample accuracy of popular ML methods with that of eight traditional statistical ones, we found that the former are dominated across both accuracy measures used and for all forecasting horizons examined. Moreover, we observed that their computational requirements are considerably greater than those of statistical methods. The paper discusses the results, explains why the accuracy of ML models is below that of statistical ones and proposes some possible ways forward. The empirical results found in our research stress the need for objective and unbiased ways to test the performance of forecasting methods that can be achieved through sizable and open competitions allowing meaningful comparisons and definite conclusions.

Key Points

• 1Machine Learning methods are outperformed by traditional statistical methods in accuracy for time series forecasting
• 2Their computational requirements are significantly higher than those of statistical approaches
• 3The paper proposes objective benchmarks for evaluating forecasting methods
• 4Findings suggest that current Machine Learning models require substantial refinement to improve their forecasting accuracy
• 5The study utilizes a comprehensive dataset from the M3 Competition to ensure robust comparisons.

Related articles

• Uncertainty and Error Analysis Review: Significant Figures and Scientific Notation

  This document provides an overview of uncertainty and error analysis in measurements, focusing on significant figures, scientific notation, accuracy vs. precision, types of errors, and sample problems related to these concepts.

• REVIEWER IN MATHEMATICS 9 FIRST TERM

  This document serves as a comprehensive review of key mathematical concepts related to relations, functions, and linear equations, structured for educational purposes. It presents definitions, rules, and methods of representation to aid understanding and application.

• The Measurement Calculus

  This document introduces the Measurement Calculus, a mathematical framework for measurement-based quantum computation. It develops syntax, operational semantics, and a calculus for reasoning about quantum computation patterns that are foundational in quantum programming languages.

• An Introduction to Statistical Learning with Applications in Python

  This book serves as a comprehensive introduction to statistical learning techniques, demonstrating applications in Python. It addresses key concepts and methods in the field while providing practical examples through lab exercises to facilitate hands-on learning.

• Quantitative Methods with Simulations and Modeling

  This module introduces students to quantitative methods and basic statistics essential for decision-making in various fields, particularly Information Technology. It covers descriptive statistics, probability, visualizations, and practical applications through hands-on activities.