Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees

International Journal of Sediment Research xxx (xxxx) xxx

Contents lists available at ScienceDirect

International Journal of Sediment Research journal homepage: www.elsevier.com/locate/ijsrc

Original Research

Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S. Sadra Shadkani a, Akram Abbaspour a, Saeed Samadianfard a, Sajjad Hashemi a, Amirhosein Mosavi b, c, *, Shahab S. Band d, e, ** a

Department of Water Engineering, Faculty of Agriculture, University of Tabriz, Tabriz, Iran Environmental Quality, Atmospheric Science and Climate Change Research Group, Ton Duc Thang University, Ho Chi Minh City, Viet Nam Faculty of Environment and Labour Safety, Ton Duc Thang University, Ho Chi Minh City, Viet Nam d Institute of Research and Development, Duy Tan University, Da Nang, 550000, Viet Nam e Future Technology Research Center, College of Future, National Yunlin University of Science and Technology, 123 University Road, Section 3, Douliou, Yunlin, 64002, Taiwan, China b c

a r t i c l e i n f o

a b s t r a c t

Article history: Received 16 June 2020 Received in revised form 30 September 2020 Accepted 5 October 2020 Available online xxx

Monitoring sediment transport is essential for managing and maintaining rivers. Estimation of the sediment load in rivers is fundamental for the study of sediment movement, erosion, and flood control.

In the current study, three machine learning modelsdmulti-layer perceptron (MLP), multi-layer perceptron-stochastic gradient descent (MLP-SGD), and gradient boosted tree (GBT)dwere utilized to estimate the suspended sediment load (SSL) at the St. Louis (SL) and Chester (CH) stations on the Mississippi River, U.S. Four evaluation criteria including the Correlation Coefficient (CC), Nash Sutcliffe Efficiency (NSE), Scatter Index (SI), and Willmott's Index (WI) were utilized to evaluate the performance of the used models. A sensitivity analysis of the models to the input variables revealed that the current day discharge variable had the most effect on the SSL at both stations, but in the absence of current-day discharge data (Qt), a combination of input parameters including SSLt 3 ; SSL t 2 ; SSLt 1 ; Qt 3 ; Qt 2 ; Qt 1 can be used to estimate the SSL. The comparative outcomes indicated the high accuracy of MLP-SGD-5 model with a CC of 0.983, SI of 0.254, WI of 0.991, and NSE of 0.967 at station CH and the MLP-SGD-6 model with a CC of 0.933, SI of 0.576, WI of 0.961, and NSE of 0.867, respectively, at station SL. The results of MLP models were improved by SGD optimization. Therefore, the MLP-SGD method is recommended as the most accurate model for SSL estimation. © 2020 International Research and Training Centre on Erosion and Sedimentation/the World Association for Sedimentation and Erosion Research. Published by Elsevier B.V. All rights reserved.

Keywords: Suspended sediment transport Ensemble learning Multilayer perceptron Stochastic gradient descent Machine learning

1. Introduction Sediment transport in rivers plays a key role in many water engineering features such as reservoir volume, water quality and

* Corresponding author. Environmental Quality, Atmospheric Science and Climate Change Research Group, Ton Duc Thang University, Ho Chi Minh City, Viet Nam. ** Corresponding author. Institute of Research and Development, Duy Tan University, Da Nang, 550000, Viet Nam. E-mail addresses: sshadkani@gmail.com (S. Shadkani), a-abbaspour@tabrizu.ac. ir (A. Abbaspour), s.samadian@tabrizu.ac.ir (S. Samadianfard), hashemisajjad2009@ gmail.com (S. Hashemi), amirhosein.mosavi@tdtu.edu.vn (A. Mosavi), shamshirbandshahaboddin@duytan.edu.vn (S.S. Band).

health, river topography, the aquatic environment, navigability, and several other hydrological characteristics. So, as one of the crucial factors in global water recourses and environmental management, river suspended sediment estimation, and forecasting with high accuracy has great importance. Suspended sediment load (SSL) forecasting is an intricate and non-linear phenomenon, which includes the interaction of several physical and hydrological parameters that change in space and time, and, thus, is not a simple task.

Based on the foregoing features, as one of the most significant subjects in river basin research, establishing a relation between sediment load and discharge has drawn the attention of many investigators all over the world (Salih et al., 2020).

https://doi.org/10.1016/j.ijsrc.2020.10.001 1001-6279/© 2020 International Research and Training Centre on Erosion and Sedimentation/the World Association for Sedimentation and Erosion Research. Published by Elsevier B.V. All rights reserved.

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

2

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

In recent years, due to the progression of soft computing and data-driven techniques, hydraulic engineers have tended to forecast SSL with heuristic regression and artificial intelligence methods. Traditional methods (multi-linear regressions, autoregressive models, and sediment rating curves (SRC)) have some problems because of their restricted capability to consider nonlinearity and non-stationarity in environmental and hydrological data which have been solved in the afore mentioned novel computing techniques (Jain, 2001). Such machine learning methods have been demonstrated to be strong alternatives for common traditional methods. Several types of research have shown the superiority of the new techniques (e.g., Cigizoglu, 2004; Kisi et al., 2009; Rajaee et al., 2009; Cobaner et al., 2009; Melesse et al., 2011; Pham et al., 2017; Taşar et al., 2017).

Horvat and Horvat (2020) modeled sediment transport and bed evolution under one dimensional unsteady flow. The model was evaluated by confirming it for the Danube, Tisa, and Sava rivers for a five-year time period, and according to the obtained results, the developed model had the best validity. Bhattacharya et al. (2005) utilized an artificial neural network to develop a process for building a model based on data to predict sediment transport amounts. They considered field information and published flume data from many of investigators to generate the artificial neural network (ANN) technique. Finally, they concluded that the estimation accuracy of the developed technique was higher than recognized sediment transport methods like that of Engelund and Hansen (1967). Building a model based on the ANN method for sediment yield and runoff simulation on weekly and diurnal scales for the Nagwan watershed of India, Raghuwanshi et al. (2006) used a five-year data set for training and applied a two-year dataset for testing the model. They also used the same data sets to predict runoff and sediment yield by linear regression-based methods. In comparison with linear regression techniques, they found that the ANN models had more accurate results in forecasting both sediment yield and runoff on weekly and daily modeling scales. In another study, utilizing data from the Longchuanjiang River of China, Zhu et al. (2007) modeled the suspended sediment flux on a monthly scale using an ANN model. In this method, average rainfall, flow discharge, rainfall intensity, and temperature parameters were applied to simulate the suspended sediment flux. It was concluded that the ANN model simulated monthly suspended sediment flux with high accuracy regarding proper variables and their time series for the suspended sediment flux. Applying two ANN models, radial basis function and feed-forward back propagation models, Alp and Cigizoglu (2007) built the connection among total daily SSL and hydro-meteorological variables (rainfall and flow). The afore

mentioned models were trained using rainfall and SSL information for the Juniata catchment in the U.S. Comparison of the results of conventional multilinear regression with ANN methods illustrated the superiority of ANN models according to the chosen performance indices.

Kisi et al. (2012) implemented support vector machine (SVM), ANN, adaptive Neuro-fuzzy inference system (ANFIS), and genetic programming (GP) methods to simulate daily SSL at the Pineville and Barbourville stations on the Cumberland River in the U.S. They found the GP model to be the best method with high accuracy in comparison with the other three methods for daily SSL forecasting.

Lafdani et al. (2013) estimated daily SSL for the Doiraj River in Iran using ANN and SVM techniques. They utilized 11 years of data of streamflow and rainfall and concluded that ANN models had better performance than SVM models. It was also found that the best model of ANN model with a correlation coefficient (CC) of 0.99 and root mean square error (RMSE) of 0.34 had lower errors in comparison with other studied models. Nourani and Andalib (2015) estimated the SSL for the Mississippi River on monthly and daily scales. They contrasted the performance of an ANN and a Waveletbased Least Square Support Vector Machine (WLSSVM) technique in their research. Their results revealed that the ANN model predicted the monthly SSL more accurately in comparison with the WLSSVM model, but the WLSSVM model presented yielded smaller errors than the ANN on a daily scale.

In the past decade, researchers have estimated SSL effectively by hybrid methods. Rajaee (2011) used a wavelet artificial neural network (WANN) technique to simulate SSL at the diurnal scale for the Yadkin River in the U.S. and compared the results with multilinear regression (MLR), ANN, and SRC methods. The comparison of the accuracy of the estimation methods demonstrated that the WANN was the most accurate method for SSL prediction. Moreover, Olyaie et al. (2015) examined the technique of Rajaee (2011) and compared it with three methods (ANN, ANFIS, and SRC). They also found the same results that the WANN model has the best performance among the methods. Implementing classical and novel feedforward neural network (FNN) methods, i.e., Hybrid FNN, double parallel FNN, multilayer FNN, and Hybrid double FNN (HDFNN) techniques, Chen and Chau (2016) estimated SSL at a daily scale.

The results of the statistical parameters used in the study found that the HDFNN model performs better than the other three methods.

Heuristic regression approaches, such as model trees and support vector regression (SVR) methods, have been applied for SSL modeling in recent years. Reddy and Ghimire (2009) predicted daily SSL using Tree Model (M5), Gene Expression Programming

Fig. 1. Graphical design of GBT model.

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

3

Fig. 2. Location of studied hydrological stations in the U.S.

Fig. 3. Distribution of SSL and discharge of the Mississippi River at station SL.

(GEP), MLR, and SRC approaches and found that the M5 model yielded better simulations with low errors compared with three other methods. Similarly, utilizing M5 model and comparing with reduced-error pruning tree (REPTree), radial basis function (RBF), fuzzy logic (FL), and ANN methods, Kumar et al. (2012) modeled the suspended sediment concentration (SSC) at Kasol, India. The M5 model was selected as the most exact model in their study. Also, Goyal (2014) obtained similar outcomes in which the M5 model and wavelet regression techniques estimated sediment yields with higher accuracy than ANN. Using ANN, Least-Squares Support Vector Regression (LSSVR), M5 model, classification and regression trees, MLR, and radial basis functions ANN approaches, Kaveh et al.

(2017) trained an ANFIS model using the Levenberg-Marquardt (LM) learning algorithm and simulated the time series of SSC of the Schuylkill River in the U.S. with SSC and river discharge data.

The results indicated that the models trained with LevenbergMarquardt (LM) learning algorithm had better efficiency. Kumar et al. (2016) simulated daily suspended sediment and illustrated that ANN and LSSVR approaches had lesser inaccuracy than other methods. Kargar et al. (2019) utilized neuro-fuzzy (NF) and GEP methods for modeling sediment transport in open channels, and it was concluded from this study that NF model performs better than GEP model for modeling sediment transport in open channels.

Finally, Bharti et al. (2017) investigated five machine-learning

Fig. 4. Distribution of SSL and discharge of the Mississippi River at station CH.

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

4

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

Fig. 5. Scatter plots of observed and predicted SSL values using the high performing MLP-SGD models at station SL.

methods (ANN scaled conjugate gradient, Artificial Neural Network Levenberg-Marquardt (ANNLM), REPTree, M5 model, and LSSVR) to forecast sediment load and runoff using hydro-meteorological variables. The ANNLM technique was chosen as the most accurate model among the studied methods.

In the current study, three techniques, namely gradient boosted tree (GBT), multilayer perceptron (MLP), and multilayer perceptron-stochastic gradient descend (MLP-SGD) models, have been implemented in SSL prediction. In this regard, time series of SSL and discharge on the Mississippi River at St. Louis (SL) and Chester (CH) in the U.S. were used, and the results were compared to select the best model with low errors for the study sites. Moreover, with the aim to obtain the best estimation accuracy for the SSL, various scenarios for the modeling input-output architecture have been considered. The remarkable feature of the research is that to the best of the author's knowledge, neither of GBT and hybrid MLP-SGD

models have been utilized in hydrological research. So, the aim of this study is to examine the GBT and hybrid MLP-SGD models for SSL estimation for the afore mentioned stations and demonstrate the high effectiveness of GBT and MLP-SGD models in the field of sediment transport modeling, which has not been studied in previous research.

2. Materials and methods 2.1. Gradient boosted trees (GBT) Decision trees can build the data target values by applying the data features, and each part in the tree denotes a section of the data features. Each leaf node signifies an estimation of the data target value of the present data feature. Moreover, the other benefits of decision trees are as follows: 1) envisioning of the decision tree is

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

5

Fig. 5. (continued).

easy; 2) the arrangement of the decision tree is obvious, and it is simple to determine the effects of various features on the outcome; and 3) the decision tree is computationally capable. Consequently, numerous researchers have used decision trees for various subjects all over the world. Decision trees have been implemented in epidemiology (Ivancevic et al., 2016); power systems (Liu et al., 2014), transportation (Fernandez et al., 2015), sediment hydrology (Ebtehaj et al., 2018; Kumar et al., 2012) and stock indices trading (Zhou et al., 2019). Furthermore, Lopez-Suarez et al. (2018) utilized decision trees in agriculture.

The boosting algorithm is based on the opinion that when deciding an intricate function, the consequences of merging decisions of several specialists will be superior to the decision of a particular specialist. Due to the afore mentioned features, the accuracy of a weak classification algorithm can be improved by the boosting algorithm (Friedman, 2001), such as gradient boosted. In gradient boosted methods, a group of several simple models are the basis of the training forecasting model and each simple model is trained to compensate for the inaccuracy of the preceding simple model. A GBT model, as depicted in Fig. 1, has been extensively utilized by data investigators to obtain improved outcomes in many machine learning tasks (Chen & Guestrin, 2016). Gradient boosted decision trees, which account for interactions, are more amenable to apply non-linear and crossing changes on the input features, unlike linear models like logistic regression. Especially, due to hierarchical configuration, they are scalable, strong to consider outliers, and have the capability to simulate non-linear decision borders naturally. This type of model is appropriate for the management of statistical features and certain features with tens of categories.

Assume that there are N training samples fðxi :yi ÞgN i¼1 , where xi is a sample and yi represents the label of sample xi . Let FðxÞ be a linear combination of single decision trees, and Lðy; FðxÞÞ be a loss function. For any sample xi , Fðxi Þ is the classification (the ith decision tree) of xi , and Lðyi ; Fðxi ÞÞ is the loss between Fðxi Þ and yi .

The aim of GBT is to learn an ideal model FðxÞ such that n P Lðyi ; Fðxi ÞÞ is diminished for a specified loss function Lðy; FðxÞÞ. i¼1

To do that, the GBT algorithm first develops an original decision tree F0 ðxÞ, then iteratively builds m new trees. In each iteration, a

new tree hðxÞ is joined to decrease the residuals, which are attained by the given loss function Lðy; F ðxÞÞ. Therefore, the ideal model F *ðxÞ of GBT can be calculated as follows:

F * ðxÞ ¼ F0 ðxÞ þ v*

m X

rt *ht ðxÞ

(1)

t¼1

where m represents the iteration number; 0 < v < 1 denotes the reduction parameter that regulates the learning rate of the GBT; ht ðxÞ represents the tree trained in the tth iteration, and rt is the weight of ht ðxÞ.

2.2. Multi-layer perceptron (MLP) The ANN is a soft computational technique and robust data simulation mechanism which is able to capture and demonstrate intricate input/output interactions and has been extensively applied in the modeling of hydrological processes. ANNs are collected nodes (neurons) that are organized in sets (layers), and linked through weights. Any simple neural network arrangement involves three layers: the input layer (information), the hidden layers and the output layer (yield). The modeling process of the method is that the input information is presented to the input layer; this information is lead through and functioned on by the neural network yielding the output which is attained at the output layer. Other neurons send many inputs to each neuron using weighted networks. These weighted inputs are further added up and build the argument for a transmission function, for instance, a linear, logistic, or hyperbolic tangent function that consecutively constructs the last output of the neuron (Talebizadeh et al., 2009).

As previously mentioned, ANN is efficiently utilized for the simulation of complicated hydrological processes since it does not need comprehensive statistics concerning the physical procedure prevailing systems.

There are many varieties in ANNs and training algorithms that have been implemented in several studies. The usual ANN method in hydrological research is the MLP. The MLP method is a transformative and multifunctional kind of ANN comprised of an input layer, hidden layers, and a single output layer (Rai et al., 2005). In general, ANNs are divided into two categories based on the

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

6

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

Table 1 Statistical parameters for training, testing, and all data sets at station SL. Statistical parameters

Mean Minimum Maximum Standard deviation Coefficient of variation Skewness

Training set

Test set

All data

Q (cms)

SSL (tons per day)

Q (cms)

SSL (tons per day)

Q (cms)

SSL (tons per day)

6618.169 1625.387 21,605.754 4109.501 0.621 1.008

217,691 4230 2,750,000 302,648 1.391 2.710

6870.328 1874.575 23,389.715 3761.947 0.548 1.282

246,539.450 4470 3,600,000 389,920 1.581 3.367

6693.887 1625.387 23,389.715 4009.600 0.599 1.071

226,353.905 4230 3,600,000 331,500 1.460 3.117

Table 2 Statistical analysis for training, testing, and all data sets at station CH. Statistical parameters

Mean Minimum Maximum Standard deviation Coefficient of variation Skewness

Training set

Test set SSL (tons per day)

Q (cms)

SSL (tons per day)

Q (cms)

SSL (tons per day)

6896.883 1730.159 21,860.606 4247.861 0.616 0.961

247,502.874 5870 2,480,000 326,970 1.320 2.338

6870.328 1874.575 23,389.715 3761.940 0.548 1.282

234,212.811 4590 2,000,000 325,600 1.391 2.336

6983.354 1730.159 26,617.836 4181.073 0.599 1.070

243,512.151 4590 2,480,000 326,590 1.340 2.336

Table 3 The autocorrelations between SSL and discharge for all data at station SL. Parameters

Qt

Qt 1

Qt 2

Qt 3

SSLt 1

SSLt 2

SSLt 3

SSLt

Qt Qt 1 Qt 2 Qt 3 SSLt 1 SSLt 2 SSLt 3 SSLt

1

0.998 1

0.976 0.996 1

0.944 0.974 0.993 1

0.747 0.754 0.731 0.698 1

0.723 0.748 0.753 0.733 0.922 1

0.691 0.726 0.741 0.757 0.810 0.921 1

0.757 0.736 0.699 0.641 0.923 0.811 0.687 1

direction of data transmission and processing: feed-forward and backward networks. MLPs are part of a feed-forward network.

Studies have shown that MLP networks, with a hidden layer are widely used, also MLP networks have the ability to and high accuracy in estimating nonlinear functions (Hornik et al., 1989; Schalkoff, 1997). The neural network used in the current study is the multilayer perceptron network and its training method is back propagation. The MLP method can be expressed as:

Y ¼F

m X

Wkj :F

j¼1

n X

All data

Q (cms)

Wji Xi þ Bj þ Bk

(2)

i¼1

The weights between hidden layers and output layers are represented by Wkj; the weights between hidden layers and output layers are represented by Wji; the input variables are shown as Xi; m is the hidden layer neuron number; n is the input layer neurons number, Bj is the bias amount of the neuron in the hidden layer; Bk is the bias amount of the neurons in the output layer; F is the

transmission function; and Y is the output function. In many studies, the ANNs use a sigmoid function for function activation. In the current research, a sigmoid (Sig) function was used. The Sig transmission function for different Z values is as follows:

FðZÞ ¼

1 1 þ expð ZÞ

2.3. Multilayer perceptron-stochastic gradient descent (MLP-SGD) Stochastic gradient descent (SGD) is an iterative technique for optimizing an objective function with appropriate softness properties. Because the method substitutes the actual gradient (calculated from the whole of the data) by an estimation of the gradient (calculated from a randomly selected subset of the data), it can be considered as a stochastic approximation of gradient descent optimization. Especially for big data applications, this approach decreases the computational weight, attaining faster iterations for a slightly slower convergence rate (Taddy, 2019).

As previously mentioned, SGD utilizes a randomly-selected subsection of the training instances to estimate the gradient of the target function given by Eq. (4). The number of training instances utilized for this estimation is named the batch size. The SGD training simply converts into gradient descent when the batch size is N. As a small batch size is implemented, the parameters are updated more repeatedly than for the gradient descent, and the convergence is analyzed. When the size of the batch is 1, the maximum repetition of updates is done, leading to a straightforward perceptron-like algorithm (Tsuruoka et al., 2009).

aw ¼

N X

Kðj:wÞ C

i¼1

Table 4 The autocorrelations between SSL and discharge for all data at station CH. Parameters

Qt

Qt 1

Qt 2

Qt 3

SSLt 1

SSLt 2

SSLt 3

SSLt

Qt Qt 1 Qt 2 Qt 3 SSLt 1 SSLt 2 SSLt 3 SSLt

1

0.998 1

0.969 0.994 1

0.956 0.984 0.991 1

0.784 0.781 0.760 0.713 1

0.776 0.782 0.785 0.763 0.950 1

0.754 0.777 0.781 0.789 0.863 0.953 1

0.789 0.761 0.711 0.666 0.953 0.860 0.750 1

(3)

X j ui j

(4)

i

where N is the batch size, C is the meta-parameter, ui is weight of the feature i. Separately from applying a single training sample to estimate the gradient, the optimization process is the same as for the simple gradient descent, so the updated weights of the features at training sample j expressed as:

ukþ1 ¼ uk þ hk

v C X Kðj:uÞ j ui j vu N i

(5)

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx Table 5 Data combinations used to predict SSL. Combination number

Parameters

1 2 3 4 5 6

SSLt 1 ; Qt 1 SSLt 2 ; SSLt 1 ; Qt 2 ; Qt 1 SSLt 3 ; SSLt 2 ; SSLt 1 ; Qt 3 ; Qt 2 ; Qt 1 SSLt 1 ; Qt ; Qt 1 Qt 2 ; Qt 1 ; SSLt 2 ; SSLt 1 ; Qt SSLt 3 ; SSLt 2 ; SSLt 1 ; Qt ; Qt 3 ; Qt 2 ; Qt 1

In Eqs. (4) and (5), K is the iteration numerator and hk is the learning rate, which reduces as the iteration progress. MLP-SGD is a MLP neural network based method that is optimized using SGD. The network includes many hidden layers comprising neurons with tanh (Hyperbolic tangent function) or in the other words a scaled sigmoid, rectifier (where x is the input value, select the maximum of (0, x)), max out (Select the maximum input vector coordinates), and ExpRectifier (Exponential Rectifier function) activation operations. Advanced facilities including epochs, rho, L1 or L2 adjustment, momentum training, adaptive learning rate, and rate annealing enable high prediction accuracy.

The size of the weight updates are defined by the user specified learning rate when the adaptive learning rate is inactivated and is a function of the difference between the estimated value and the objective value. That variance commonly named delta is only presented at the output layer. Backpropagation is applied to improve the accuracy of the output at each hidden layer. By authorizing previous iterations to effect the present update, momentum adjusts backpropagation. Utilizing the momentum parameter helps to avoid local minima and the associated fluctuations. The momentum is ramped up slowly since excessive momentum can result in fluctuations. The rate annealing, momentum training, dropout rate annealing, and momentum training parameters activate if the adaptive learning rate is disabled.

Longitude: 89 490 48.7600 W) were acquired from U.S. Geological Survey (USGS). Figs. 3 and 4 show the daily variations of SSL and Q data for the studied time period of 2004e2017 at both stations SL and CH. The statistical parameters including mean, minimum, maximum, standard deviation, coefficient of variation and skewness for training, testing, and all data sets at SL and CH stations were determined and listed in Tables 1 and 2, respectively. The mean is the sum of the observed values divided to the number of values, the standard deviation is a measure of the values of variation or dispersion of a set of data, a low standard deviation shows that the data tend to be close to the average of the set of data, while a high standard deviation shows that the amounts of data are spread out over a wider range, and the coefficient of variation indicates the extent of variability of data in relation to the mean of the data. As can be seen from these tables, skewness coefficients are in the appropriate range because as the skewness coefficient decreases, the negative effects on modeling are reduced. The most appropriate value for this parameter is zero (Altun et al., 2007). The statistical parameters are defined as follows:

P Mean ¼

Oi nd

The Mississippi River is the fourth-longest river and fifteenthlargest river in terms of discharge in the world and it also is the second longest river on the North American continent. The Mississippi River drains from parts of 32 U.S. states and two Canadian provinces. The locations of the studied stations SL and CH are shown in Fig. 2. Daily SSL and Q data from the Mississippi River at St. Louis, MO (SL, Station ID: 07,010,000, Latitude: 38 37044.400 N and Longitude: 90 100 47.200 W) and the Mississippi River at Chester, IL (CH, Station ID: 07,020,500, Latitude: 37 540 02.6700 N and Table 6 Performance of the models for SSL estimation at station SL. Models

MLP

GBT

MLP-SGD

Error parameters

(6)

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi P ðOi MeanÞ2 Standard deviation ¼ nd 1

Coefficient of variation ¼

3 Mean Skewness ¼

2.4. Data and study area

7

(7)

Standard deviation Mean nd þ1 2

(8)

(9)

Standard deviation

where Oi is observed data for the i th measurement and nd is the number of data.

For accurate SSL prediction, it is necessary to select the appropriate input parameters for the implemented models. To achieve this aim, the autocorrelation between SSL and discharge were obtained and are listed in Tables 3 and 4, for stations SL and CH, respectively. It can be seen from Tables 3 and 4 that SSLt has a higher correlation with Qt, Qt 1, SSLt 1, SSLt 2 (t denotes the daily time step).

2.5. Evaluation criteria

Combination number 1

2

3

4

5

6

CC SI WI NSE

0.904 0.685 0.943 0.812

0.913 0.651 0.953 0.830

0.928 0.591 0.960 0.860

0.922 0.617 0.958 0.847

0.926 0.618 0.953 0.847

0.929 0.593 0.958 0.859

CC SI WI NSE

0.890 0.723 0.936 0.791

0.901 0.694 0.944 0.807

0.913 0.655 0.947 0.828

0.908 0.676 0.943 0.817

0.919 0.642 0.949 0.835

0.922 0.633 0.950 0.840

CC SI WI NSE

0.910 0.663 0.947 0.824

0.919 0.642 0.949 0.835

0.929 0.586 0.961 0.862

0.927 0.602 0.956 0.855

0.932 0.580 0.960 0.865

0.933 0.576 0.961 0.867

Several evaluation parameters were utilized to evaluate the efficiency of the implemented methods and their accuracy. In the current study, Correlation Coefficient (CC), Scatter Index (SI), Nash Sutcliffe Efficiency (NSE), and Willmott's Index (WI) were utilized.

As the CC, NSE, and WI parameters get closer to one and SI get closer to zero, the accuracy of prediction increases. These criteria are described as follows (Samadianfard et al., 2020): nd P

CC ¼

i¼1 nd P i¼1

O2i

1 nd

Oi Pi n1d nd P i¼1

Oi

nd P

!i¼1

nd P

Oi

Pi

i¼1 nd P i¼1

Pi2 n1d

nd P

!

(10)

Pi

i¼1

Bold represents the best results.

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

8

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

Table 7 Performance of the models for SSL estimation at station CH. Models

MLP

GBT

MLP-SGD

Error parameters

sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi nd P 1 ðPi Oi Þ2 nd

Combination number 1

2

3

4

5

6

CC SI WI NSE

0.955 0.414 0.976 0.911

0.977 0.296 0.988 0.955

0.979 0.343 0.985 0.939

0.975 0.326 0.986 0.945

0.982 0.261 0.991 0.965

0.980 0.260 0.991 0.965

CC SI WI NSE

0.942 0.467 0.970 0.887

0.971 0.330 0.985 0.944

0.971 0.333 0.958 0.943

0.974 0.316 0.986 0.948

0.978 0.288 0.989 0.957

0.977 0.293 0.988 0.955

CC SI WI NSE

0.956 0.413 0.977 0.911

0.978 0.294 0.988 0.955

0.979 0.285 0.989 0.958

0.979 0.284 0.989 0.958

0.983 0.254 0.992 0.967

0.981 0.259 0.991 0.965

Bold represents the best results.

SI ¼

i¼1

nd P

NS ¼ 1

(11)

O

i¼1 nd P

ðPi Oi Þ2 (12) ðOi OÞ2

i¼1

2 6 6 WI ¼ 1 6 n 4 Pd

nd P

ðOi Pi Þ2

3

7 7 7 5 ðjPi Oi j þ jOi Oi jÞ2 i¼1

(13)

i¼1

where Pi and Oi are estimated values and observed data for time i, respectively, and O is the average observed data.

Fig. 6. Scatter plots of observed and predicted SSL values using the high performing MLP-SGD models at station CH.

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

9

Fig. 6. (continued).

Also, Taylor's diagram (TD) provides a graphical representation of the predicted and actual data. In the current study, Taylor's diagram was utilized for an accurate investigation of the models.

Some speciďŹ cations of the predicted values and observed data can be merged into Taylor's diagram (Taylor, 2001). For example, the SD, CC, and RMSE between the estimated and actual data can be shown at the same time in this diagram. The azimuth angle shows correlation coefďŹ cient amount and the radial distance from the observed data point and the radial distance from the (0, 0) point, signify the root mean square error and standard deviation amounts, respectively.

3. Results and discussions According to the autocorrelation values of SSL and Q for the different daily time steps (Tables 3 and 4), six different input combinations, including SSL and Q in current and previous daily values, were considered for estimating SSL by the studied GBT, MLP-SGD, and MLP models (Table 5). It should be noted that SSL has the units of tons per day and Q has the units of cubic meters per second (cms) and also the index t represents the current day and the index t n denotes n days ago. Moreover, available data, in the time period of 2004e2017, were divided into two parts, 70 percent of the data (3513 days) considered as the training dataset and 30

Fig. 7. Comparative graphs of observed and predicted SSL using the high performance MLP-SGD models at station SL.

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

10

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

Fig. 8. Comparative graphs of observed and predicted SSL using the high performance MLP-SGD models at station CH.

percent of data (1506 days) considered as the testing dataset were used for SSL modeling.

For the MLP model, two hidden layers were utilized in the current study, although previous investigations have revealed that a single hidden layer is adequate for an ANN to predict any complicated nonlinear function (Cybenco, 1989). For the MLP-SGD model, after the activation operations test, it was found that the max out function has the best performance in modeling SSL. Also, the adaptive learning rate was activated due to better efficiency and was considered equal to 0.005 for SSL prediction. Moreover, the values of epochs, rho, L1, and L2 were considered 10, 0.99, 0.00001 and 0, respectively, in the current study. Then, various input combinations were investigated and the corresponding results of the evaluated GBT, MLP, and MLP-SGD models at the test stage for both

stations SL, and CH are listed in Tables 6 and 7, respectively. As can be seen from the results presented in Tables 6 and 7, all three methods of MLP, MLP-SGD, and GBT indicate acceptable results, but the most accurate estimations are related to combination 6 at station SL and combination 5 at station CH. These mentioned the input Combinations (5 and 6) utilize most of the input variables, which have significant effects on the accurate estimation of SSL. Furthermore, Combination 3 can be useful for predicting SSL in times of data scarcity and lack of current day data at station CH. At the station SL, MLP-SGD-6 provided the most accurate estimations of SSL with CC of 0.933, SI of 0.576, WI of 0.961, and NSE of 0.867. After that, MLP-SGD-5 with CC of 0.932, SI of 0.580, WI of 0.960, and NSE of 0.865 and MLP-SGD-3 with CC of 0.929, SI of 0.586, WI of 0.961, and NSE of 0.862, maybe beneficial for

Fig. 9. Taylor diagrams of predicted SSL amounts using the most accurate models for stations SL and CH.

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx Table 8 Influence of eliminating input variables on the MLP-SGD-6 model efficiency for SSL prediction at station SL. Model

1 2 3 4 5 6 7 8

MLP-SGD-6

Statistical parameter

Input parameters

SI

All Remove Remove Remove Remove Remove Remove Remove

0.576 0.957 0.781 0.764 0.838 1.025 0.777 0.862

Qt Qt 1 Qt 2 Qt 3 SSLt 1 SSLt 2 SSLt 3

Table 9 Influence of eliminating input variables on the MLP-SGD-5 model efficiency for SSL prediction at station CH. Model

1 2 3 4 5 6

MLP-SGD-5

Statistical parameter

Input parameters

SI

All Remove Remove Remove Remove Remove

0.254 0.691 0.428 0.431 0.775 0.454

Qt Qt 1 Qt 2 SSLt 1 SSLt 2

estimating SSL. Additionally, MLP-SGD-5 with CC of 0.983, SI of 0.254, WI of 0.991, and NSE of 0.967 is the most accurate model at station CH. The MLP-SGD model improved all error parameters of MLP model in all scenarios for instance: the MLP-SGD-6, as the top model in station SL, had acceptable performance to improve CC 0.431%, SI 2.686%, WI 0.09% and NSE 0.931%.

To better understand the relevant results of the estimated data, distribution graphs derived from the afore mentioned best models relative to the observed data and the regression equations between observed and estimated values of SSL are shown in Figs. 5 and 6 for stations SL and CH, respectively. The solid black line in these graphs shows the bisector line (y ¼ x) that as the trendline (black dotted line) gets closer to the bisector line, the performance of the models increases. Also, the hollow points closer to the bisector line illustrate higher precision. For a visual comparison of the results of the computational results of SSL obtained from the different models compared to the available observational data, a time series diagram of the estimated SSL values with respect to observed data are shown in Figs. 7 and 8 for stations SL and CH, respectively.

Finally, an individual evaluation of observed and predicted SSL values yielded by the most accurate models was done for stations SL and CH. Taylor diagrams (TDs) are shown for both studied stations in Fig. 9. As shown in Fig. 9, MLP-SGD-5 and MLP-SGD-6 predicted SSL values accurately at stations CH and SL, respectively.

4. Sensitivity analysis To check the effect of input parameters on SSL prediction, the SI evaluation parameter was utilized for various input variables. To achieve this purpose, the MLP-SGD-6 and MLP-SGD-5 were selected for sensitivity analysis as the most efficient models at stations SL and CH, respectively. Tables 8 and 9 show the effect of removing each input parameter on the accuracy of the MLP-SGD-6 and MLP-SGD-5 models. As can be seen from Tables 8 and 9, the accuracies of the MLP-SGD-6 and MLP-SGD-5 models were reduced if any of the parameters including in these combinations were removed. Also, the SSLt 1 and Qt have the most influence in

11

improving the estimation efficiency and eliminating these parameters caused sharp increases in the SI error values.

5. Conclusions SSL modeling in rivers is very important because of its impact on the management and operation of hydraulic structures, calculating the filled volume of dams, as well as causing changes in river morphology. So, in the current study, the SSL values were estimated using GBT, MLP-SGD, and MLP models for the stations SL and CH on the Mississippi River. For this purpose, data from the time period of 2004e2017, were divided into two parts (70% for training and 30% for testing) and the performance of MLP, MLP-SGD and GBT models in estimating the SSL were evaluated. The results of the models were compared with each other. The obtained results indicated that the MLP-SGD-6 with input parameters of SSLt 3 ; SSLt 2 ; SSLt 1 ; Qt ; Qt 1 ; Qt 2 ; and Qt 3 with a CC of 0.933, SI of 0.576, WI of 0.961 and, NSE of 0.867, had the highest capability for SSL estimation at station SL. The MLP-SGD-5 with input parameters of SSLt 2 ; SSLt 1 ; Qt ; Qt 2 ; and Qt 1 with a CC of 0.983, SI of 0.254, WI of 0.991, and NSE of 0.967 provided the most accurate estimations of SSL compared to the other models at station CH. The overall results indicated that the amount of current-day discharge (Qt) has the most impact on the accuracy of SSL estimation at both stations, also SGD also improved the accuracy of all results of the MLP model in all combinations, and increased the efficiency of this method for predicting SSL, so MLP-SGD model is recommended for SSL estimation due to its high accuracy. Future work can be done on evaluating the accuracies of the studied GBT and MLP-SGD models for estimating SSL in different rivers and climates. Moreover, implementing other data driven techniques for SSL estimation in stations SL and CH can be useful.

Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments The support of the Alexander von Humboldt Foundation is acknowledged.

References Alp, M., & Cigizoglu, H. K. (2007). Suspended sediment load simulation by two artificial neural network methods using hydro meteorological data. Environmental Modelling & Software, 22(1), 2e13, https://doi.org/10.1016/ j.envsoft.2005.09.009 Altun, H., Bilgil, A., & Fidan, B. C. (2007). Treatment of multi-dimensional data to enhance neural network estimators in regression problems. Expert Systems with Applications, 32(2), 599e605, https://doi.org/10.1016/j.eswa.2006.01.054 Bharti, B., Pandey, A., Tripathi, S. K., & Kumar, D. (2017). Modelling of runoff and sediment yield using ANN, LS-SVR, REPTree and M5 models. Hydrology Research, 48(6), 1489e1507, https://doi.org/10.2166/nh.2017.153 Bhattacharya, B., Price, R. K., & Solomatine, D. P. (2005). Data-driven modelling in the context of sediment transport. Physics and Chemistry of the Earth, 30, 297e302, https://doi.org/10.1016/j.pce.2004.12.001 Chen, X. Y., & Chau, K. W. (2016). A hybrid double feedforward neural network for suspended sediment load estimation. Water Resources Management, 30, 2179e2194, https://doi.org/10.1007/s11269-016-1281-2 Chen, T., & Guestrin, C. (2016 August). Xgboost: A scalable tree boosting system. In Proceedings 22nd ACM SIGKDD international conference on knowledge discovery and data mining (pp. 229e236), https://doi.org/10.1145/2939672.2939785. San Francisco.

Cigizoglu, H. K. (2004). Estimation and forecasting of daily suspended sediment data by multi-layer perceptrons. Advances in Water Resources, 27(2), 185e195, https://doi.org/10.1016/j.advwatres.2003.10.003

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001

12

S. Shadkani et al. / International Journal of Sediment Research xxx (xxxx) xxx

Cobaner, M., Unal, B., & Kisi, O. (2009). Suspended sediment concentration estimation by an adaptive neuro-fuzzy and neural network approaches using hydro-meteorological data. Journal of Hydrology, 367(1e2), 52e61, https:// doi.org/10.1016/j.jhydrol.2008.12.024 Cybenco, G. (1989). Approximation by superpositions of a sigmoidal function.

Mathematics of Control, Signals and Systems, 2, 303e314, https://doi.org/10.1007/ BF02551274 Ebtehaj, I., Bonakdari, H., & Zaji, A. H. (2018). A new hybrid decision tree method based on two artificial neural networks for predicting sediment transport in clean pipes. Alexandria Engineering Journal, 57(3), 1783e1795, https://doi.org/ 10.1016/j.aej.2017.05.021 Engelund, F., & Hansen, E. (1967). A monograph on sediment transport in alluvial streams. Copenhagen, Denmark: Teknisk Forlag.

Fernandez, C., Izquierdo, R., Llorca, D. F., & Sotelo, M. A. (2015). A comparative analysis of decision trees based classifiers for road detection in urban environments. In 2015 IEEE 18th international conference on intelligent transportation systems (pp. 719e724), https://doi.org/10.1109/ITSC.2015.122. Las Palmas.

Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29(5), 1189e1232, https://doi.org/10.1214/aos/ 1013203451 Goyal, M. K. (2014). Modeling of sediment yield prediction using M5 model tree algorithm and wavelet regression. Water Resources Management, 28(7), 1991e2003, https://doi.org/10.1007/s11269-014-0590-6 Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359e366, https://doi.org/ 10.1016/0893-6080(89)90020-8 Horvat, M., & Horvat, Z. (2020). Long term sediment transport simulation of the Danube, Sava, and Tisa rivers. International Journal of Sediment Research, 35(5), 550e561, https://doi.org/10.1016/j.ijsrc.2020.03.014 Ivancevic, V., Igic, N., Terzic, B., Knezevic, M., & Lukovic, I. (2016). Decision trees as readable models for early childhood caries. In Intelligent decision technologies 2016, https://doi.org/10.1007/978-3-319-39627-9_39 Jain, S. K. (2001). Development of integrated sediment rating curves using ANNs. Journal of Hydraulic Engineering, 127(1), 30e37, https://doi.org/10.1061/(ASCE) 0733-9429, 2001)127:1(30.

Kargar, K., Safari, M. J. S., Mohammadi, M. A., & Samadianfard, S. (2019). Sediment transport modeling in open channels using neuro-fuzzy and gene expression programming techniques. Water Science and Technology, 79(12), 2318e2327, https://doi.org/10.2166/wst.2019.229 Kaveh, K., Bui, M. D., & Rutschmann, P. (2017). A comparative study of three different learning algorithms applied to ANFIS for predicting daily suspended sediment concentration. International Journal of Sediment Research, 32(3), 340e350, https://doi.org/10.1016/j.ijsrc.2017.03.007 Kisi, O., Dailr, A. H., Cimen, M., & Shiri, J. (2012). Suspended sediment modeling using genetic programming and soft computing techniques. Journal of Hydrology, 450e451, 48e58, https://doi.org/10.1016/j.jhydrol.2012.05.031 Kisi, O., Haktanir, T., Ardiclioglu, M., Ozturk, O., Yalcin, E., & Uludag, S. (2009).

Adaptive neuro-fuzzy computing technique for suspended sediment estimation. Advances in Engineering Software, 40(6), 438e444, https://doi.org/10.1016/ j.advengsoft.2008.06.004 Kumar, A. R. S., Ojha, C. S. P., Goyal, M. K., Singh, R. D., & Swamee, P. K. (2012).

Modeling of suspended sediment concentration at Kasol in India using ANN, fuzzy logic, and decision tree algorithms. Journal of Hydrologic Engineering, 17(3), 394e404, https://doi.org/10.1061/(ASCE)HE.19435584.0000445 Kumar, D., Pandey, A., Sharma, N., & Flügel, W. A. (2016). Daily suspended sediment simulation using machine learning approach. Catena, 138, 77e90, https:// doi.org/10.1016/j.catena.2015.11.013 Lafdani, E. K., Nia, A. M., & Ahmadi, A. (2013). Daily suspended sediment load prediction using artificial neural networks and support vector machines. Journal of Hydrology, 478, 50e62, https://doi.org/10.1016/j.jhydrol.2012.11.048 Liu, C., Sun, K., Rather, Z. H., Chen, Z., Bak, C. L., Thogersen, P., & Lund, P. (2014).

A systematic approach for dynamic security assessment and the corresponding preventive control scheme based on decision trees. IEEE Transactions on Power Systems, 29(2), 717e730, https://doi.org/10.1109/TPWRS.2013.2283064 Lopez-Suarez, M., Armengol, E., Calsamiglia, S., & Castillejos, L. (2018). Using decision trees to extract patterns for dairy culling management. In IFIP international conference on artificial intelligence applications and innovations (pp. 231e239),

https://doi.org/10.1007/978-3-319-92007-8_20. May 25-27, 2018, Rhodes, Greece.

Melesse, A. M., Ahmad, S., McClain, M. E., Wang, X., & Lim, Y. H. (2011). Suspended sediment load prediction of river systems: An artificial neural network approach. Agricultural Water Management, 98(5), 855e866, https://doi.org/ 10.1016/j.agwat.2010.12.012 Nourani, V., & Andalib, G. (2015). Daily and monthly suspended sediment load predictions using wavelet based artificial intelligence approaches. Journal of Mountain Science, 2, 85e100, https://doi.org/10.1007/s11629-014-3121-2 Olyaie, E., Banejad, H., Chau, K. W., & Melesse, A. M. (2015). A comparison of various artificial intelligence approaches performance for estimating suspended sediment load of river systems: A case study in United States. Environmental Monitoring and Assessment, 187, 189, https://doi.org/10.1007/s10661-015-4381-1 Pham, B. T., Shirzadi, A., Bui, D. T., Prakash, I., & Dholakia, M. B. (2017). A hybrid machine learning ensemble approach based on a radial basis function neural network and rotation forest for landslide susceptibility modeling: A case study in the himalayan area, India. International Journal of Sediment Research, 33(2), 157e170, https://doi.org/10.1016/j.ijsrc.2017.09.008 Raghuwanshi, N., Singh, R., & Reddy, L. (2006). Runoff and sediment yield modeling using artificial neural networks: Upper Siwane River, India. Journal of Hydrologic Engineering, 11(1), 71e79, https://doi.org/10.1016/j.ijsrc.2017.09.008/. ASCE) 1084-0699(2006)11:1(71.

Rai, P., Majumdar, G. C., DasGupta, S., & De, S. (2005). Prediction of the viscosity of clarified fruit juice using artificial neural network: A combined effect of concentration and temperature. Journal of Food Engineering, 68(4), 527e533, https://doi.org/10.1016/j.jfoodeng.2004.07.003 Rajaee, T. (2011). Wavelet and ANN combination model for prediction of daily suspended sediment load in rivers. The Science of the Total Environment, 409(15), 2917e2928, https://doi.org/10.1016/j.scitotenv.2010.11.028 Rajaee, T., Mirbagheri, S. A., Zounemat-Kermani, M., & Nourani, V. (2009). Daily suspended sediment concentration simulation using ANN and neuro-fuzzy models. The Science of the Total Environment, 407(17), 4916e4927, https:// doi.org/10.1016/j.scitotenv.2009.05.016 Reddy, M. J., & Ghimire, B. N. (2009). Use of model tree and gene expression programming to predict the suspended sediment load in rivers. Journal of Intelligent Systems, 18(3), 211e228, https://doi.org/10.1515/JISYS.2009.18.3.211 Salih, S. Q., Sharafati, A., Khosravi, K., Faris, F., Kisi, O., Tao, H., Ali, M., & Yaseen, Z. M. (2020). river suspended sediment load prediction based on river discharge information: Application of newly developed data mining models. Hydrological Sciences Journal, 65(4), 624e637, https://doi.org/10.1080/ 02626667.2019.1703186 Samadianfard, S., Hashemi, S., Kargar, K., Izadyar, M., Mostafaeipour, A., Mosavi, A., Nabipour, N., & Shamshirband, S. (2020). Wind speed prediction using a hybrid model of the multi-layer perceptron and whale optimization algorithm. Energy Report, 6, 1147e1159, https://doi.org/10.1016/j.egyr.2020.05.001 Schalkoff, R. J. (1997). Artificial neural networks. New York: McGraw-Hill.

Taddy, M. (2019). Business data science: Combining machine learning and economics to optimize, automate, and accelerate business decisions. New York: McGraw-Hill.

Talebizadeh, M., Morid, S., Ayyoubzadeh, S. A., & Ghasemzadeh, M. (2009). Uncertainty analysis in sediment load modeling using ANN and SWAT model. Water Resources Management, 24, 1747e1761, https://doi.org/10.1007/s11269-009-9522-2 Tasar, B., Kaya, Y. Z., Varcin, H., Unes, F., & Demirci, M. (2017). Forecasting of suspended sediment in rivers using artificial neural networks approach. International Journal of Advanced Engineering Research and science, 4, 79e84, https:// doi.org/10.22161/ijaers.4.12.14 Taylor, K. E. (2001). Summarizing multiple aspects of model performance in a single diagram. Journal of Geophysical Research Atmospheres, 106, 7183e7192, https:// doi.org/10.1029/2000JD900719 Tsuruoka, Y., Tsujii, J., & Ananiadou, S. (2009). Stochastic gradient descent training for L1-regularized log-linear models with cumulative penalty. In 47th annual meeting of the ACL and the 4th IJCNLP of the AFNLP (pp. 477e485), https:// doi.org/10.3115/1687878.1687946. Suntec, Singapore.

Zhou, F., Zhang, Q., Sornette, D., & Jiang, L. (2019). Cascading logistic regression onto gradient boosted decision trees for forecasting and trading stock indices.

Applied Soft Computing, 84, 105747, https://doi.org/10.1016/j.asoc.2019.105747 Zhu, Y. M., Lu, X. X., & Zhou, Y. (2007). Suspended sediment flux modeling with artificial neural network: An example of the Longchuanjiang River in the upper yangtze catchment, China. Geomorphology, 84(1e2), 111e125, https://doi.org/ 10.1016/j.geomorph.2006.07.010

Please cite this article as: Shadkani, S et al., Comparative study of multilayer perceptron-stochastic gradient descent and gradient boosted trees for predicting daily suspended sediment load: The case study of the Mississippi River, U.S., International Journal of Sediment Research, https:// doi.org/10.1016/j.ijsrc.2020.10.001