Stochastic modeling: a thorough guide to evaluate, pre-process, model and compare time series with m

Download Full Version ebookmass - Visit ebookmass.com

Stochastic Modeling: A Thorough Guide to Evaluate, Pre-Process, Model and Compare Time Series with MATLAB Software First Edition Hossein Bonakdari https://ebookmass.com/product/stochastic-modeling-athorough-guide-to-evaluate-pre-process-model-and-comparetime-series-with-matlab-software-first-edition-hosseinbonakdari/

OR CLICK HERE DOWLOAD NOW

Discover More Ebook - Explore Now at ebookmass.com

Instant digital products (PDF, ePub, MOBI) ready for you Download now and discover formats that fit your needs...

Applied Time Series Analysis. A Practical Guide to Modeling and Forecasting Terence C. Mills https://ebookmass.com/product/applied-time-series-analysis-apractical-guide-to-modeling-and-forecasting-terence-c-mills/ ebookmass.com

Optimizations and Programming: Linear, Non-linear, Dynamic, Stochastic and Applications with Matlab Abdelkhalak El Hami https://ebookmass.com/product/optimizations-and-programming-linearnon-linear-dynamic-stochastic-and-applications-with-matlababdelkhalak-el-hami/ ebookmass.com

Deterministic and Stochastic Modeling in Computational Electromagnetics Dragan Poljak https://ebookmass.com/product/deterministic-and-stochastic-modelingin-computational-electromagnetics-dragan-poljak/ ebookmass.com

Corn Book for Young Folk Charles Burgess Williams

https://ebookmass.com/product/corn-book-for-young-folk-charlesburgess-williams/ ebookmass.com

Enjoy our Universe: you have no other choice First Edition De Rújula https://ebookmass.com/product/enjoy-our-universe-you-have-no-otherchoice-first-edition-de-rujula/ ebookmass.com

Six Steps to Managing Alzheimer's Disease and Dementia: A Guide for Families Budson Md https://ebookmass.com/product/six-steps-to-managing-alzheimersdisease-and-dementia-a-guide-for-families-budson-md/ ebookmass.com

Random in Death 1st Edition J. D. Robb

https://ebookmass.com/product/random-in-death-1st-edition-j-d-robb/

ebookmass.com

Antelope Hill Writing Competition 2022: Small Victories Antelope Hill Publishing https://ebookmass.com/product/antelope-hill-writingcompetition-2022-small-victories-antelope-hill-publishing/ ebookmass.com

AUDITING and ASSURANCE SERVICES 9th Edition Timothy J. Louwers https://ebookmass.com/product/auditing-and-assurance-services-9thedition-timothy-j-louwers/ ebookmass.com

Strategy: An Introduction to Game Theory (Third Edition) 3rd Edition – Ebook PDF Version https://ebookmass.com/product/strategy-an-introduction-to-game-theorythird-edition-3rd-edition-ebook-pdf-version/ ebookmass.com

STOCHASTIC MODELING

STOCHASTIC MODELING A Thorough Guide to Evaluate, Pre-Process, Model and Compare Time Series with MATLAB Software

HOSSEIN BONAKDARI Department of Soils and Agri-Food Engineering, Laval University, Québec, Canada

MOHAMMAD ZEYNODDIN Department of Soils and Agri-Food Engineering, Laval University, Québec, Canada

Elsevier Radarweg 29, PO Box 211, 1000 AE Amsterdam, Netherlands The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom 50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States Copyright © 2022 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

ISBN: 978-0-323-91748-3 For Information on all Elsevier publications visit our website at https://www.elsevier.com/books-and-journals Publisher: Candice Janco Editorial Project Manager: Aera Gariguez Production Project Manager: Paul Prasad Chandramohan Cover Designer: Victoria Pearson Esser Typeset by Aptara, New Delhi, India

Dedication

Deepest thanks to our amazing families for their endless support.

v

CONTENTS

Preface

ix

Acknowledgements Abbreviations

xi xiii

1. Introduction

1

1.1 Time series 1.2 Stochastic and stochastic with exogenous variables 1.3 Data preprocessing References

2. Preparation & stationarizing

1 5 7 9

13

2.1 Missing data 2.2 Detecting outliers 2.3 Time series structure and attributes 2.4 Stationarity 2.5 Deterministic terms detection tests 2.6 Stationarizing methods 2.7 Exercise References

13 23 36 40 48 67 82 84

3. Distribution evaluation and normalizing

87

3.1 Distribution visualization 3.2 Normal distribution definition 3.3 Skewness 3.4 Kurtosis 3.5 Common tests and transforms 3.6 Data distribution tests 3.7 Normalization transforms 3.8 Exercise References

4. Stochastic modeling 4.1 Modeling methods overview 4.2 Deterministic models 4.3 Probabilistic statistical models 4.4 Stochastic concepts

87 87 89 89 91 91 104 129 129

133 133 133 133 134 vii

viii

Contents

4.5 Differencing operators in stochastic models 4.6 Stochastic models equations 4.7 Identify appropriate models and parameters’ orders 4.8 Estimation of stochastic models’ parameters 4.9 Univariate stochastic modeling 4.10 Stochastic models with exogenous inputs 4.11 Fitting stochastic and stochasticX models by econometric modeler app 4.12 Invertibility constraint for MA models 4.13 Chapter summary 4.14 Exercise References

135 136 137 140 140 156 167 182 183 183 184

5. Goodness-of-fit & precision criteria

187

5.1 Model adequacy 5.2 Model parsimony 5.3 Conventional performance measure 5.4 Cross-validation in time series 5.5 Exercise References

187 198 202 248 260 260

6. Forecasting time series by deep learning and hybrid methods 6.1 Deep learning introduction 6.2 Hybrid modeling 6.3 Exercise References Appendix Index

265 265 288 316 316 321 353

Preface

Considering the fact that the development of all high-quality ongoing measurement technologies leads to the generation of datasets overtime or time series, data analysis, and predictive models have become necessary. Efficient and reliable means of time series modeling is crucial in building a comprehensive methodology for data analysis and predictive models. However, accurate analysis and modeling of time series data are a challenging task due to the randomness inherent in the deterministic terms that produce an intricate nonlinear data pattern over time, which can be addressed properly by focusing on improving the accuracy of the identification and capturing of both the direct and indirect factors affecting the deterministic terms in the predictive model using advancements in data analysis/processing. This book is written with the aim of providing a new overview of the principles necessary to develop accurate and reliable techniques based on stochastic methods for time series modeling, interpretation, and prediction.

The topics covered in the book are designed to address all necessary steps for time series modeling based on several user-friendly developed codes in MATLAB.This book bridges over the existing gap between theory and application of stochastic methods to enhance the knowledge and skill of readers to evaluate, preprocess, model, predict, and compare time series with user-friendly codes in MATLAB. The book consists of six chapters. Chapter 1 includes the introduction and basic concepts, while the rest covers the steps in assessing, preprocessing, and modeling a time series and evaluation of the models for future predictions. At the end of each chapter, some exercises are provided for interested readers.

The FIRST chapter contains the introduction and a brief description of time series concept and methods of utilizing these forms of data in analyzing, modeling, and interpreting phenomena. The chapter also includes reasons for applying tests and preprocessing methods, as well as descriptions of stochastic model concepts.

The SECOND chapter focuses on the preparation and stationary concepts, tests, methods of obtaining the stationarity, and how to code them in MATLAB. The collected data may be incomplete and have missing data points or have components that make it unsuitable for interpretation and modeling. Therefore, in this section, methods of preparation of time series are presented.

The THIRD chapter describes the normal distribution and reasons for requiring normal data. Following the descriptions, the tests to evaluate normality of time series are presented. Finally, several transforms for normalizing of non-normal time series are provided and an instance is given for each one.

ix

x

Preface

The FOURTH chapter is dedicated to stochastic modeling. In this chapter, the concepts of stochastic models, different types of this subcategory of statistical models, and properties of each are defined. After presenting theoretical descriptions and the pros and cons of each model, the chapter explains how to obtain the order of parameters for each model and the technique of modeling in MATLAB software.

The FIFTH chapter relates the methods of evaluation of the produced models. The surveying of the model’s functionality and precision is a prerequisite of each modeling procedure. In this section of the book, the tests and indices required to obtain the most accurate model with the simplest structure are presented. The criteria that assess the adequacy and parsimony of the model are introduced. Finally, the concepts of crossvalidation and how to evaluate the time series-based models by cross-validation are presented.

The SIXTH chapter is dedicated to deep learning modeling and integration of stochastic models with artificial intelligence models. The hybridization of the linear, nonlinear methods is a subject that has attracted scholars’ attention widely. This method utilizes both linear and nonlinear method characteristics and therefore, most of the time produces better results than using individual models. Since the area of artificial models is vast, covering all of them is not possible, and is not within the scope of this book.

Therefore, only the most recent and successful Long-Short-Term Memory model, along with its hybridization with stochastic models, is introduced and described in this book.

In the appendix, there will be a brief review of MATLAB software and a basic tutorial, needed for coding. Further, the earliest version of the MATLAB that contains the utilized functions are highlighted as “Introduced in MATLAB XXX” for each new function.

Therefore, in that version and higher versions, the mentioned functions exist. However, for this book, MATLAB R2020b version is used. Also, to find the MATLAB version and installed ToolBoxes on your PC and required ToolBoxes for this book’s tutorials, related commands, and methods of achieving this information are presented at the end of the appendix. We hope that this book will be used as a knowledge-based decision support tool in time series modeling taking into account social, economic, environmental, and ethical governance issues in related fields in applied sciences. To conclude, we believe that this book provides essential knowledge for reliable time series modeling, which enhances the existing body of knowledge on this important topic in applied sciences. Comments and suggestions from the readers will be genuinely appreciated.

Please refer the below companion site link to view the MATLAB codes and Video tutorials. https://www.elsevier.com/books-and-journals/book-companion/9780323917483

Acknowledgments

The authors acknowledge the financial support provided by the Natural Science and Engineering Research Council of Canada (NSERC) Discovery Grant (#RGPIN-202004583). We would also like to acknowledge with thanks to the Department of Soils and Agri-Food Engineering, Université Laval for all effective assistance and logistic support throughout this work. We particularly thank Prof. Denis Roy, Dean of the Faculty of Agriculture and Food and Prof. Damien de Halleux, Director of the Department of Soils and Agri-Food Engineering, Université Laval for their support and encouragement during the preparation of this book.

Hossein Bonakdari, PhD, P.Eng. Department of Soils and Agri-Food Engineering, Laval University, Québec, Canada Ranked in the 2% list of the world’s top scientists (h-index=44) Editorial board, Informatics, Journal Editorial board, Natural Resources Informatics Section Board Member, Sustainability Journal ORCID ID: https://orcid.org/0000-0001-6169-3654 Researcher ID: http://www.researcherid.com/rid/B-9305-2018 http://www.researchgate.net/profile/Hossein_Bonakdari https://scholar.google.ca/citations?user=bBWIpAUAAAAJ&hl=en Mohammad Zeynoddin Department of Soils and Agri-Food Engineering, Laval University, Québec, Canada RG: https://www.researchgate.net/profile/Mohammad-Zeynoddin Google scholar: https://scholar.google.com/citations?user=e9Jx7UMAAAAJ Linkedin: https://www.linkedin.com/in/mohammad-zeynoddin-0bb322112/?original Subdomain=ir

xi

Abbreviations

ACF AD ADF AI ANN AR ARIMA ARIMAX ARMA ARMAX ARX Cfw Chw Cmdw CPU CV Diff DL ESDG F GPU Iω J KS LOO-CV LPO-CV LS LSTM MA MAD ML MW P PACF Pchip PCHS PP PSD RNN ROCV S SARIMA

Auto-Correlation Function Anderson–Darling Augmented Dickey–Fuller Test Artificial Intelligence Artificial Neural Network Auto-Regressive Auto-Regressive Integrated Moving Average ARIMA Models with Extra Inputs (Exogenous Series) Auto-Regressive Moving Average ARMA Models with Extra Inputs (Exogenous Series) AR Models with Extra Inputs (Exogenous Series) Current Folder Window Command History Window Command Window Central Processing Unit Cross-Validation Nonseasonal Differencing Deep Learning Generalized Extreme Studentized Deviate Test Fisher Graphics Processing Unit Periodogram Jump Kolmogrov–Smirnov Leave-One-Out Cross-Validation Leave-P-Out Cross-Validation Lillifors Long-Short Term Memory Moving Average Scaled Median Absolute Deviation Machine Learning Mann–Whitney Period Partial Auto-Correlation Function Shape-Preserving Piecewise Cubic Spline Interpolation Peace Wise Cubic Hermite Spline Phillips–Perron Test Power Spectral Density Recurrent Neural Network Rolling Origin Cross-Validation Stochastic Terms Seasonal ARIMA xiii

xiv

Abbreviations

SARIMAX Sdiff Sf SGDM Sk-Fold SMK SMK SStd Std Stochasticx SW T Wsw

SARIMA Models With Extra Inputs (Exogenous Series) Seasonal Differencing Spectral Analysis Stochastic Gradient Descent With Momentum Stratified K-Fold Cross-Validation Mann–Kendal Seasonal Mann–Kendal Seasonal Standardization Nonseasonal Standardization Stochastic Models With Extra Inputs (Exogenous Series) Shapiro–Wilk Trend Workspace Window

CHAPTER 1

Introduction 1.1 Time series Many phenomena naturally have variation with time; these phenomena encompass a wide variety of fields of study. One such example is water quality data from a stream.

The water quality parameters (alkalinity, turbidity, organic content, etc.) may change throughout the year and interannually. The approach used to study these phenomena of interest is to collect data regarding the parameters that have affected them in the past and those that will influence them in the future. Following this approach, time series may be considered the best tools for analyzing the collected past and present data to be able to make future decisions (Akhbari et al., 2019). In fact, time series are very important in the development of planning and management policies. With the help of time series, you can see the trend of changes in phenomena from the past to the present along with the differences between the observed and expected values due to fluctuations in the phenomena. A clear picture of the behavior of phenomena of interest can be developed through time series modeling. Periodic fluctuations and seasonal changes can be observed in phenomena, allowing their behavior to be understood and related to other influencing phenomena. If we consider the example of water quality data once more, well-developed time series models may allow one to understand how seasonal differences influence water quality indicators, such as turbidity. Not only time series allow us to explain the influence of seasonal changes on water quality indicators, but they also allow us to understand interannual changes for the same season. This information can allow municipal officials to develop watershed management policies or allow for the optimization of drinking water production for example. The management and planning decisions that are made are based on the prediction of future conditions from past and present time series data. Forecasts are constantly needed, and over time, the effects of these predictions on actual performance are measured. From the constant measurement of performance, the predictions are regularly updated, and the decisions corrected. This cycle continues in an iterative fashion in order to achieve the desired conclusion (Azari, Soori, and Bonakdari, 2017; Langridge, Gharabaghi, McBean, Bonakdari, and Walton, 2020).

Over the recent years, vast improvements in technology have made recording data for time series modeling more practical and reliable. As a result, time series modeling is one of the most practical tools for investigating a variety phenomenon in science, engineering, and economics (Azimi, Bonakdari, Ebtehaj, Gharabaghi, and Khoshbin, 2018; Stochastic Modeling: A Thorough Guide to Evaluate, Pre-Process, Model and Compare Time Series with MATLAB Software.

DOI: https://doi.org/10.1016/B978-0-323-91748-3.00001-X

c 2022 Elsevier Inc. Copyright All rights reserved.

1

2

Stochastic modeling

Figure 1.1 Schematic of environmental epidemiology.

Binesh and Bonakdari, 2014; Bonakdari, Ebtehaj, Gharabaghi, Vafaeifard, and Akhbari, 2019; Hui Pu, Bonakdari, Lassabatère, Joannis, and Larrarte, 2010). The concept of time series modeling allows researchers to assess the outcomes of a variety of phenomena at any time with minimum costs and efforts. Using forecasted data, they can plan possible solutions to problems, make decisions, and implement control measures.

1.1.1 Time series in environmental epidemiology One of field that has widely benefited from time series concept is environmental epidemiology (Bhaskaran, Gasparrini, Hajat, Smeeth, and Armstrong, 2013; Bonakdari, Pelletier, and Martel-Pelletier, 2020a, 2020b; Corcuera Hotz and Hajat, 2020; Tejasvini, Amith, and Shilpa, 2020). Environmental epidemiology (Fig. 1.1) allows researchers to forecast the outcomes of any phenomenon like health field or assess the impact of environmental exposures such as weather, air pollutants, and other contributing factors impacting health condition. Using study data to forecast future events enables decision makers to plan solutions and implement control measures in a way that is much simpler and much more cost-effective when compared to other methods, such as randomized control tests (Bonakdari et al., 2021; Bonakdari, Pelletier, and Martel-Pelletier, 2020c; Bonakdari, Tardif, Abram, Pelletier, and Martel-Pelletier, 2020).

1.1.2 Engineering and sequential data Another domain of application of time series is in engineering. One example of an emerging application of time series modeling is in water management. As communities

Introduction

Figure 1.2 Cycle of data curing, analyzing, and decision making.

continue to expand and grow, so do their water requirements, whether it be for energy production, drinking water supply, or agricultural practice (Fig. 1.2). Hydrological time series data are critical in order to allow policy makers to make effective decisions regarding water management and resource sustainability (Kazemian-Kale-Kale et al., 2020; Lotfi et al., 2019; Lotfi et al., 2020; Zaji, Bonakdari, and Gharabaghi, 2019). With the help of these time series, costs associated with the implementation of water management policies (including operation and maintenance of water utilities) can be greatly reduced, and the management of water resources efficiently conducted (Soltani et al., 2021; Zinatizadeh, Pirsaheb, Bonakdari, and Younesi, 2010). The example of water resources is one of several applications of time series in engineering, which is more exorbitant than other

3

4

Stochastic modeling

Figure 1.3 Preprocessing, analyzing, and modeling data in economy.

examples due to the global water crisis. Time series are also used in other engineering sections, for instance, estimating and forecasting the amount of energy consumed by industries and home consumers, surveying natural cycles in these uses and managing energy consumption, modeling the amount of traffic in a corridor and forecast future needs, study the production and harvest of agricultural products and compare with the needs of society and many other examples.

1.1.3 Historical data for forecasting future economy Further application of time series modeling can be found in the study of economics (Fig. 1.3). With this tool, producers, sellers, and industry owners can observe and interpret markets in order to identify supply and demand requirements (Larsson and Nossman, 2011; Qiu, Ren, Suganthan, and Amaratunga, 2017). Moreover, by using the appropriate methods, they can forecast future demands in order to prepare themselves for potential downturns or periods of sustained economic growth. From the above discussion, we can see that time series play a significant role in almost all scientific and management fields. Therefore, an intimate knowledge of how to implement time series modeling

Introduction

Figure 1.4 Abstract model categorization.

and the required pre- and post-processing steps is vitally important (Ebtehaj, Zeynoddin, and Bonakdari, 2020; Moeeni, Bonakdari, and Fatemi, 2017; Zeynoddin, Ebtehaj, and Bonakdari, 2020).

1.2 Stochastic and stochastic with exogenous variables 1.2.1 Stochastic models The ability to model, implement those models, and analyze model outcomes are fundamental skills that are required of many real-world applications. These applications span a diverse range of sectors from medical to civil and military (Moazamnia and Bonakdari, 2014; Mojtahedi, Ebtehaj, Hasanipanah, Bonakdari, and Amnieh, 2019; Momplot et al., 2012). In practice, in order to achieve an effective plan, appropriate modeling should be done with intelligible analysis and review, at the lowest cost.

Different approaches are needed to model phenomena and predict parameters.

One of the approaches used to model phenomena and analyze time series is to use models, which, based on mathematical concepts and relationships describe systems and make it possible to predict future parameters and conditions. These models can be classified into three categories: (1) statistical models; (2) artificial intelligence (AI) models; (3) and integrated models (Fig. 1.4). Integrated models are formed by combination

5

6

Stochastic modeling

of statistical and AI models (Azimi, Bonakdari, and Ebtehaj, 2017; Bonakdari, 2011; Moeeni, Bonakdari, and Ebtehaj, 2017b; Moeeni, Bonakdari, Fatemi, and Zaji, 2017; Wang, Hu, Ma, and Zhang, 2015). Each category of models may be further sub-divided into several categories, each of which may be employed in the resolution of different types of problems. In this text, stochastic modeling, a sub-class of statistical methods, is presented. These models are lauded amongst industry members and the academic community alike as they have an easily comprehensible structure, can be readily applied to a variety of problems, and have high precision in short-term forecasts (Zeynoddin, Bonakdari, Ebtehaj, Azari, and Gharabaghi, 2020).

1.2.2 Stochastic model structure Stochastic models are tools for estimating the probabilistic distributions of results using random variables by one or more inputs. These random variables are based on the fluctuations, observed in historical data for a given period using data standardization techniques. These models can be used to predict time series, such as the amount of precipitation, number of patients, customers, or any other parameter at a given time in the future. In these models, it is assumed that the data are stationary and normal (Brockwell and Davis, 2016; Zeynoddin and Bonakdari, 2019). Therefore, the datasets used for modeling must be prepared and preprocessed before modeling.

1.2.3 Model classifications The first statistical stochastic model to be introduced was the auto-regressive (AR) model, which was able to establish a correlation between current and previous values in the series.

This model quickly gained popularity due to the simplicity of its structure and is still used in many annual or seasonal modeling applications. The AR model works well for modeling phenomena whose parameters are relatively stable and exhibit relatively small changes without dramatic fluctuation. If the series changes exhibits dramatic fluctuations under certain conditions, such as an outbreak of a disease, a sudden growth in market share, or flood conditions in a river, the AR model will not perform well. In order to address this deficiency of the AR model, a new model was developed through the addition of moving average algorithm creating the auto-regressive moving average (ARMA) model. If there are significant seasonal fluctuations, seasonal ARMA models can be created.

Further investigation revealed that the series, which were not stationary on average, can become stationary by considering the differential change. Therefore, by integrating the difference operator in the ARMA model, a new model called the auto-regressive integrated moving average (ARIMA) was developed. Compared to previous models, this model requires fewer parameters, which leads to reduced modeling costs and the creation of a parsimonious model (Moeeni, Bonakdari, and Ebtehaj, 2017a). If seasonal parameters

Introduction

Figure 1.5 Stochastic modeling methods.

are used in this model, seasonal ARIMA or SARIMA or also known as multiplicative ARIMA is produced (Zeynoddin et al., 2019). Stochastic models using extra inputs (StochasticX: ARX, ARMAX, ARIMAX, and SARIMAX) are an additional form of model that utilizes exogenous inputs in order to forecast a specific variable (Fig. 1.5).

In this type of stochastic model, several time series which have mutual impacts on each other, such as precipitation, air temperature, and moisture are used as inputs to escalate the accuracy of the models (Box, Jenkins, Reinsel, and Ljung, 2015; Moeeni and Bonakdari, 2018).

1.3 Data preprocessing 1.3.1 Definition of preprocessing In the analysis of time series, there are two main goals. The first goal is to identify the pattern of the phenomenon presented by a sequence of observations, and the second goal is to predict future values. To achieve these two goals, it is necessary to understand the structure and pattern of changes in the observed time series and to be able to interpret them.Once these structures and the patterns of change have been identified,the necessary

7

8

Stochastic modeling

Figure 1.6 Stationarizing time series.

data can be interpreted and analyzed. After analyzing the data, it is possible to predict future phenomena by extracting the identified patterns.

1.3.2 Relationship between forecasting and time series structure When studying most natural phenomena, however, complex time series structures are formed that make it difficult to analyze, interpret, model, and predict variables (Fig. 1.6).

These complex structures can occur in series with ascending and descending trends, seasonal alternations, drastic jumps that emerge as a result of sudden changes in phenomena, and the synergy of these factors make analysis even more sophisticated (Ebtehaj, Bonakdari, Zeynoddin, Gharabaghi, and Azari, 2020). Broadly, the purpose of using time series is to facilitate the interpretation of events and predict future conditions with optimal accuracy. However, many models used in predicting time series have been formed based on simplistic initial assumptions which are required to make modeling and prediction of time series possible. An example of one such simplistic assumption is the development of statistical models based on normal data distribution and stationarity.

1.3.3 Distribution and its impact on time series forecasting As a result of the inherent structural complexity involved in most natural phenomena, methods are needed that provide the possibility to eliminate or reduce the model complexity of time series, as well as to make them easier to analyze and interpret. Furthermore, these methods should aim to increase the model precision while maintaining the validity of the produced models. To accomplish this goal, preprocessing is required, and is considered a nonseparable component to any modeling exercise. These preprocesses can include estimating missing data; identifying and deleting outliers; normalizing data

Introduction

Figure 1.7 Normalization of non-normal time series.

by logarithmic, BoxCox, John-Draper, Manly, Johnson, and Yeo-Johnson transforms and many other conversions (Fig. 1.7) (Zeynoddin and Bonakdari, 2019; Zeynoddin et al., 2018); stationarizing by eliminating periodic patterns in series using spectral analysis; jump elimination; trend analysis; and data standardization (Moeeni and Bonakdari, 2017; Moeeni, Bonakdari, and Fatemi, 2017). Each one of the above methods is used in special circumstances and according to the underlying nature of the existing structures in the series being studied.

References Akhbari, A., Ibrahim, S., Zinatizadeh, A.A., Bonakdari, H., Ebtehaj, I., Khozani, Z.S., Gharabaghi, B., 2019.

Evolutionary prediction of biohydrogen production by dark fermentation. CLEAN-Soil, Air, Water, 47 (1), 1700494.

Azari, A., Soori, S., Bonakdari, H., 2017. The application of imperialist competitive algorithm in determining the optimal parameters of empirical area reduction method to predict the sedimentation process in Dez Dam. Geography and Sustainability of Environment 7 (3), 1–14.

Azimi, H., Bonakdari, H., Ebtehaj, I., 2017. Sensitivity analysis of the factors affecting the discharge capacity of side weirs in trapezoidal channels using extreme learning machines. Flow Measurement and Instrumentation, 54, 216–223. https://doi.org/10.1016/j.flowmeasinst.2017.02.005.

Azimi, H., Bonakdari, H., Ebtehaj, I., Gharabaghi, B., Khoshbin, F., 2018. Evolutionary design of generalized group method of data handling-type neural network for estimating the hydraulic jump roller length.

Acta Mechanica, 229 (3), 1197–1214. https://doi.org/10.1007/s00707-017-2043-9.

Bhaskaran, K., Gasparrini, A., Hajat, S., Smeeth, L., Armstrong, B., 2013. Time series regression studies in environmental epidemiology. International Journal of Epidemiology, 42 (4), 1187–1195.

Binesh, N., Bonakdari, H., 2014. Investigating different models for estimation of longitudinal velocity distribution in rectangular open channels. Applied Mathematics in Engineering, Management and Technology 1, 19–27.

9

10

Stochastic modeling

Bonakdari, H., 2011. Entropy and its application in computation of velocity distribution in sewers. In: World Environmental and Water Resources Congress 2011: Bearing Knowledge for Sustainability.

Bonakdari, H., Ebtehaj, I., Gharabaghi, B., Vafaeifard, M., Akhbari, A., 2019. Calculating the energy consumption of electrocoagulation using a generalized structure group method of data handling integrated with a genetic algorithm and singular value decomposition. Clean Technologies and Environmental Policy, 21 (2), 379–393.

Bonakdari, H., Jamshidi, A., Pelletier, J.-P., Abram, F., Tardif, G., Martel-Pelletier, J., 2021. A warning machine learning algorithm for early knee osteoarthritis structural progressor patient screening. Therapeutic Advances in Musculoskeletal Disease, 13. 1759720X21993254.

Bonakdari, H., Pelletier, J.-P., Martel-Pelletier, J., 2020a. A continuous data driven translational model to evaluate effectiveness of population-level health interventions: Case study, smoking ban in public places on hospital admissions for acute coronary events. Journal of Translational Medicine, 18 (1), 1–21.

Bonakdari, H., Pelletier, J.-P., Martel-Pelletier, J., 2020b. A reliable time-series method for predicting arthritic disease outcomes: New step from regression toward a nonlinear artificial intelligence method. Computer Methods and Programs in Biomedicine, 189, 105315.

Bonakdari, H., Pelletier, J.-P., Martel-Pelletier, J., 2020c. Viewpoint on time series and interrupted time series optimum modeling for predicting arthritic disease outcomes. Current Rheumatology Reports, 22, 1–11.

Bonakdari, H., Tardif, G., Abram, F., Pelletier, J.-P., Martel-Pelletier, J., 2020. Serum adipokines/related inflammatory factors and ratios as predictors of infrapatellar fat pad volume in osteoarthritis: Applying comprehensive machine learning approaches. Scientific Reports, 10 (1), 1–12.

Box, G.E.P., Jenkins, G.M., Reinsel, G.C., Ljung, G.M., 2015. Time series analysis: Forecasting and control (Wiley Series in Probability and Statistics), 5th ed. Wiley, Hoboken, NJ. http://gbv.eblib.com/patron/ FullRecord.aspx?p=2064681.

Brockwell, P.J., Davis, R.A., 2016. Introduction to time series and forecasting. © Springer International Publishing Switzerland.

Corcuera Hotz, I., Hajat, S., 2020. The effects of temperature on accident and emergency department attendances in London: A time-series regression analysis. International Journal of Environmental Research and Public Health, 17 (6), 1957.

Ebtehaj, I., Bonakdari, H., Zeynoddin, M., Gharabaghi, B., Azari, A., 2020. Evaluation of preprocessing techniques for improving the accuracy of stochastic rainfall forecast models. International Journal of Environmental Science and Technology, 17 (1), 505–524.

Ebtehaj, I., Zeynoddin, M., Bonakdari, H., 2020. Discussion of “Comparative assessment of time series and artificial intelligence models to estimate monthly streamflow: A local and external data analysis approach”. Journal of Hydrology, 583, 124614.

Hui Pu,J.,Bonakdari,H.,Lassabatère,L.,Joannis,C.,Larrarte,F.,2010.Profil de vitesses turbulent:Une nouvelle loi pour les canaux étroits. La Houille Blanche 96 (3), 65–70.

Kazemian-Kale-Kale, A., Gholami, A., Rezaie-Balf, M., Mosavi, A., Sattar, A. A., Azimi, A. H., Gharabaghi, B., Bonakdari, H., 2021. Uncertainty assessment of entropy-based circular channel shear stress prediction models using a novel method, Geosciences (Switzerland) 11, 308.

Langridge, M., Gharabaghi, B., McBean, E., Bonakdari, H., Walton, R., 2020. Understanding the dynamic nature of time-to-peak in UK streams. Journal of Hydrology, 583, 124630.

Larsson, K., Nossman, M., 2011. Jumps and stochastic volatility in oil prices: time series evidence. Energy Economics 33 (3), 504–514.

Lotfi, K., Bonakdari, H., Ebtehaj, I., Delatolla, R., Zinatizadeh, A.A., Gharabaghi, B., 2020. A novel stochastic wastewater quality modeling based on fuzzy techniques. Journal of Environmental Health Science and Engineering 18 (2), 1099–1120.

Lotfi, K., Bonakdari, H., Ebtehaj, I., Mjalli, F.S., Zeynoddin, M., Delatolla, R., Gharabaghi, B., 2019. Predicting wastewater treatment plant quality parameters using a novel hybrid linear-nonlinear methodology.

Journal of Environmental Management, 240, 463–474.

Moazamnia, M., Bonakdari, H., 2014. Velocity distribution and estimation of discharge in sewers by Shannon entropy concept. Journal of Water and Wastewater; Ab Va Fazilab (in Persian) 25 (2), 26–35.

Moeeni, H., Bonakdari, H., 2017. Forecasting monthly inflow with extreme seasonal variation using the hybrid SARIMA-ANN model. Stochastic Environmental Research and Risk Assessment 31 (8), 1997– 2010. https://doi.org/10.1007/s00477-016-1273-z.

Introduction

Moeeni, H., Bonakdari, H., 2018. Impact of normalization and input on ARMAX-ANN model performance in suspended sediment load prediction. Water Resources Management, 32 (3), 845–863.

Moeeni, H., Bonakdari, H., Ebtehaj, I., 2017a. Integrated SARIMA with neuro-fuzzy systems and neural networks for monthly inflow prediction. Water Resour. Manage. 31 (7), 2141–2156. https://doi.org/ 10.1007/s11269-017-1632-7.

Moeeni, H., Bonakdari, H., Ebtehaj, I., 2017b. Monthly reservoir inflow forecasting using a new hybrid SARIMA genetic programming approach. Journal of Earth System Science, 126 (2), 2. https://doi.org/ 10.1007/s12040-017-0798-y.

Moeeni, H., Bonakdari, H., Fatemi, S.E., 2017. Stochastic model stationarization by eliminating the periodic term and its effect on time series prediction. Journal of Hydrology, 547, 348–364. https://doi.org/ 10.1016/j.jhydrol.2017.02.012.

Moeeni, H., Bonakdari, H., Fatemi, S.E., Zaji, A.H., 2017. Assessment of stochastic models and a hybrid artificial neural network-genetic algorithm method in forecasting monthly reservoir inflow. INAE Letters 2 (1), 13–23. https://doi.org/10.1007/s41403-017-0017-9.

Mojtahedi, S.F.F., Ebtehaj, I., Hasanipanah, M., Bonakdari, H., Amnieh, H.B., 2019. Proposing a novel hybrid intelligent model for the simulation of particle size distribution resulting from blasting. Engineering with Computers 35 (1), 47–56.

Momplot,A.,Bonakdari,H.,Mignot,E.,Lipeme Kouyi,G.,Rivière,N.,Bertrand-Krajewski,J.-L.,2012.Effects of computational meshes on hydrodynamics of an open channel junction flow using CFD technique.

In: Proceedings of the 9th International Conference on Urban Drainage Modelling. Belgrade.

Qiu, X., Ren, Y., Suganthan, P.N., Amaratunga, G.A.J., 2017. Empirical mode decomposition based ensemble deep learning for load demand time series forecasting. Applied Soft Computing 54, 246–255.

Soltani, K., Amiri, A., Zeynoddin, M., Ebtehaj, I., Gharabaghi, B., Bonakdari, H., 2021. Forecasting monthly fluctuations of lake surface areas using remote sensing techniques and novel machine learning methods.

Theoretical and Applied Climatology 143 (1), 713–735.

Tejasvini K.N., Amith G.R., Akhtharunnisa, Shilpa H., 2020. Air pollution forecasting using multiple time series approach. In: Mandal J., Mukhopadhyay S. (Eds.) Proceedings of the Global AI Congress 2019.

Advances in Intelligent Systems and Computing, vol 1112. Springer, Singapore.

Wang,J.,Hu,J.,Ma,K.,Zhang,Y.,2015.A self-adaptive hybrid approach for wind speed forecasting.Renewable Energy 78, 374–385. https://doi.org/10.1016/j.renene.2014.12.074.

Zaji, A.H., Bonakdari, H., Gharabaghi, B., 2019. Advancing freshwater lake level forecast using Kingâs castle optimization with training sample adaption and adaptive neuro-fuzzy inference system. Water Resources Management 33 (12), 4215–4230.

Zeynoddin, M., Bonakdari, H., 2019. Investigating methods in data preparation for stochastic rainfall modeling: A case study for Kermanshah synoptic station rainfall data, Iran. Journal of Applied Research in Water and Wastewater 6 (1), 32–38.

Zeynoddin, M., Bonakdari, H., Azari, A., Ebtehaj, I., Gharabaghi, B., Madavar, H.R., 2018. Novel hybrid linear stochastic with non-linear extreme learning machine methods for forecasting monthly rainfall a tropical climate. Journal of Environmental Management 222, 190–206.

Zeynoddin, M., Bonakdari, H., Ebtehaj, I., Azari, A., Gharabaghi, B., 2020. A generalized linear stochastic model for lake level prediction. Science of the Total Environment, 723, 138015.

Zeynoddin, M., Bonakdari, H., Ebtehaj, I., Esmaeilbeiki, F., Gharabaghi, B., Haghi, D.Z., 2019. A reliable linear stochastic daily soil temperature forecast model. Soil and Tillage Research, 189, 73–87.

Zeynoddin, M., Ebtehaj, I., Bonakdari, H., 2020. Development of a linear based stochastic model for daily soil temperature prediction: One step forward to sustainable agriculture. Computers and Electronics in Agriculture 176, 105636. https://doi.org/10.1016/j.compag.2020.105636.

Zinatizadeh, A.A.L., Pirsaheb, M., Bonakdari, H., Younesi, H., 2010. Response surface analysis of effects of hydraulic retention time and influent feed concentration on performance of an UASFF bioreactor. Waste Management. (Oxford) 30 (10), 1798–1807.

11