Conway, Jack_Senior Thesis 2026

Using Machine Learning to Measure Clutchness in the NBA Jack Conway Senior Thesis | 2026

Using Machine Learning to Measure Clutchness in the NBA

Jack Conway Boston University Academy Senior Thesis Paper Alexander Chernikov March 20, 2026

Conway 1 Abstract This paper investigates whether machine learning models can provide meaningful insights into the concept of “clutchness” for National Basketball Association (NBA) players. Shots late in close NBA games are widely considered higher pressure and more difficult. While fans and analysts frequently debate which players are the most “clutch” in these late game situations, the idea of “clutchness” has been notoriously difficult to define and measure, especially with traditional statistical methods. This paper aims to better determine “clutchness” by using machine learning models.

The main dataset used has raw shot-by-shot data for every shot attempt in the NBA over the last 19 seasons. For each shot the dataset specifies who was shooting, where and when the shot was taken, the type of shot (driving dunk, step back jump shot, etc.), the defense, and more. We also merged this data set with another that contained player statistics from previous seasons. We applied multiple machine learning models and techniques to train a predictive model with the ability to predict shot outcomes under these various conditions. This was the main focus of the project as it required extensive model tuning through trial and error.

Finally, to evaluate clutchness, this predictive model was applied while withholding information about the time and score of the game, effectively isolating player performance from contextual pressure. By comparing the model’s expected outcomes to the actual results of the high-pressure shots, we can quantify how individual players perform relative to statistical expectations in late-game situations. This approach allows for a data-driven assessment of “clutchness,” highlighting which players consistently exceed or fall short of predicted performance when the stakes are highest.

This research contributes to the current body of knowledge by applying machine learning to the often difficult-to-measure

Conway 2 notion of “clutchness.” There has been little publicly available research on this topic, especially research utilizing newer, evolving machine learning models.

Introduction and Background This project intends to use machine learning to determine which NBA players are the most clutch. Simply measuring a player’s field goal percentage in clutch moments is outdated and unable to represent which players are truly the “clutchest.” By instead training a general model to predict individual shot outcomes, we can determine which players elevate their performance under pressure.

A linear regression model is considered one of the foundational algorithms in machine learning (Brown, 2021).

Therefore, we used this as the starting point for our NBA shot-making model. The fundamental idea behind a linear regression is that it predicts a continuous outcome by looking at the relationship between features and the target variables through a weighted sum. In our case for all of the models, the features consist of all of the data about where the shot was taken, by whom, what type of shot, etc., and the target variable is whether or not the shot was made. In its simplest form, a linear regression assumes the output, Y, can be written as 𝑌 = 𝑏0 + 𝑏1𝑥1 + 𝑏2𝑥2 +... + 𝑏𝑛𝑥𝑛 where each xi represents a feature and each b represents the weight applied to that feature in predicting Y. The model is able to “learn” the weights by finding the values which minimize the error using gradient descent, which is similar to backpropagation for neural networks (Kirsanov, 2024).

At an abstract level, this is a procedure that helps the weights in the model be adjusted in a way that its output converges towards the actual output. The first step in gradient descent is a forward pass

Conway 3 where the data passes through the current model to get an output.

This output is then compared with the actual expected value using a loss function that penalizes incorrect predictions. Once the loss is calculated, this process is performed on the model in reverse to calculate how each weight contributed to this loss. This is achieved using the chain rule, as each weight contributes to the calculation of the loss, and each of these weights has a corresponding partial derivative. Gradient descent is especially valuable because even a model with a large number of features can still be trained using this technique, as each column simply adds a dimension mathematically. Once the gradients are obtained, an optimization technique adjusts the weights iteratively. This process repeats until the minimization of the loss encounters diminishing returns, thus producing a trained model. ​ One important parameter for gradient descent is the learning rate. The learning rate determines the magnitude of the changes to the weights for each time through the cycle. A learning rate that is too small will make the model very slow, as it will take a lot of changes to get to the minimum error. On the other hand, a large learning rate won’t actually find the weights for minimum error. Therefore, an effective learning rate must strike a balance of being large enough to ensure efficient convergence, while also being small enough to avoid overshooting the minimum.

One of the best aspects of a linear regression model is its interpretability. Since every feature has a coefficient, one can read off the coefficients to determine which factors play the most important role in predicting whether a shot is going to be made.

This level of transparency, which is rare with many other models, is useful for sports analytics. The ability to connect the model to real basketball concepts makes a linear regression model a good starting point to find a simple and explainable benchmark. A linear regression is also computationally efficient, as it trains quickly and

Conway 4 does not require much tuning. Even though shot making isn’t exactly a continuous variable because it’s binary (you either make or miss), the linear regression framework gives a rough first approximation of how different features influence shot quality.

However, this does not align with the true nature of this problem.

The model might produce predictions anywhere along the real number line from 0 to 1, even though the true outcome must be one of these extremes. Therefore, one must create thresholds for the probabilities to map them to these binary outcomes. Here, a probability less than 0.5 maps to a miss while at least 0.5 would be a make. This mapping can cause the model to be somewhat less accurate.

Furthermore, linear regressions have obvious limitations when working with complex basketball data. In an actual NBA game, interactions between features are definitely not linear. For instance, the impact of a shot’s distance on make probability depends on whether a shooter is contested. Specifically, a step-back contested three is a better shot for Jayson Tatum than for Ben Simmons. Some players are more comfortable playing in certain arenas compared to others. Linear regression models cannot capture these more complicated patterns because a linear regression tends to “flatten” these relationships into straight lines, struggling to fit complex data well.

Hence, the assumption of linearity often leads to underfitting. Underfitting happens when a model is too simple to understand the patterns within the training data. This is mostly the case when a model doesn’t have enough parameters or a model assumes too much, just like how linear relationships cannot represent these advanced interactions in an NBA game. One important way to test for and prevent underfitting is using sklearn’s train_test_split() method. This method splits the data set into training and testing subsets. This way, the model is tested on new

Conway 5 data that it hasn’t seen before, allowing for one to evaluate the separate training and testing accuracies to determine if the model is underfitting. Having both low training and low testing accuracies is typically a sign that a model is underfitting.

On the other hand, overfitting occurs when a model fits the training data too closely. Overfitting tends to focus on finding specific details in the data instead of looking for broad patterns characteristic of all data. One can tell that a model is overfitting when the model performs well on the training data but struggles on the testing data. To avoid this, one must create a balance between the complexity of the model and making sure it can generalize to new data.

The initial linear regression was clearly underfitting. For a linear regression, one good way to analyze accuracy is using the R2 value, which represents the percentage of variation in the dependent variable that can be explained by the independent variable. In essence, this corresponds to how well the linear regression fits the data. The linear regression using this dataset produced an R2 value of 0.104. For a model like the one used for this research with such a large and noisy dataset, this isn’t as bad as it may seem. An R2 between 0.2-0.3 is quite good, and anything over 0.3 is excellent for high-variance data like this. In addition, there are definitely ways to improve this score, such as reducing multicollinearity, but, regardless, this value shows that linear regressions will not provide the level of detail needed. Even so, as mentioned, the interpretability of the model is certainly a strength.

For example, it was interesting to see that the most negative coefficient was if a shot was a jump shot at -.655, suggesting that other shots like layups and dunks are much more effective. Other factors in the top 5 most negative included hook shots, tip shots, and fadeaway jump shots, which are all considered relatively difficult. Meanwhile, some of the most positive coefficients were

Conway 6 being a slam-dunk shot or being in the left or right corner. This also makes sense, as good shooters often prefer three-point shots from the corner, especially since they are usually more open due to a defense helping in the paint.

After establishing a baseline with a linear regression model, we looked at more complex nonlinear models. We began with a Random Forest classifier, but to understand that, decision trees must be covered, which are a more straightforward algorithm. A decision tree takes data and will predict outcomes by creating branches based on feature values. It is almost like a flowchart: if the shot distance is farther than twenty feet go left, otherwise go right. Then if you went left, if the shooter is a corner three specialist, go left again, and so on. The tree splits the dataset in a way that attempts to group the data in partitions that are as homogeneous as possible, which is called reducing the entropy.

For binary outcomes, such as making or missing a shot, the tree will try to continue partitioning the dataset until each leaf node has only makes or misses.

Decision trees are intuitive because they mimic the actual decision making process that a human may engage in. However, a decision tree doesn’t generalize well to new data and it is very susceptible to overfitting. This happens because it often memorizes the training data too closely, creating branches that reflect “noise” rather than real patterns. For example, just because at some point of the tree every person whose name starts with an “A” made their shot, that doesn’t mean that this pattern would continue with new data it wasn’t trained on.

While similar, a Random Forest model is able to prevent some of these issues. Random Forest is an example of an ensemble learning technique, where the predictive capability is developed through the combination of many decision trees, since a single tree causes overfitting (Random Forest Classifier, 2020). These many

Conway 7 decision trees are formed using random samples of the overall data, both in terms of rows as well as columns. Then, the prediction is made for each tree, allowing the final prediction to be made through voting.

One of the reasons why Random Forest is so useful is its ability to introduce diversity among its trees. By using “bootstrap samples” of the data, which are random samples drawn with replacement from the original dataset, the model is designed to ensure that each individual tree is not highly correlated with every other tree in the forest. Any single tree might be a weak or overfit predictor, but the whole ensemble of trees as a whole will be able to pick up on meaningful patterns in the data (Breiman, 2001).

However, one of the main disadvantages of Random Forest models is that they aren’t interpretable, as hundreds of trees are generated and no single consensus tree is created. This means that when a Random Forest model has low accuracy, it can be hard to understand why. One can attempt to combat this by looking at the built-in feature importances, which highlight which variables the model is relying on most heavily.

Conway 8 Figure 1: Random Forest model classification report Random Forest was relatively accurate for this research, as it was able to combine numerical and categorical data well. One of the best Random Forest models had an accuracy of 0.645, as seen in the classification report above.

The classification report is a good way to evaluate models beyond just a raw accuracy score, which can be misleading, especially if the data is unbalanced. The values in the precision column are the proportion of predicted positives that were correct to how many were actually positive. In this case, of course, positive means a made shot, corresponding to the row marked as “1” in the table. Recall, on the other hand, is the number of predicted made shots that were indeed made divided by the total number of shots that were actually made. This answers “Of all the actual positive cases, how many did the model detect?”

To put this another way, precision is calculated as True Positives / (True Positives + False Positives) whereas Recall is True Positives / (True Positives + False Negatives). Using the confusion matrix at the bottom of the classification report, these can easily be calculated by hand as well.

Conway 9 Figure 2: Explanation of confusion matrix The next column in the classification report is F1-score, which combines the precision and recall using harmonic mean. The harmonic mean punishes extreme imbalances between these two quantities, making the F1-score a good way to measure performance even when the classes are unevenly distributed. In practice, this can be the most useful single metric for comparing classification models. Even so, the raw accuracy score is usually used to evaluate the quality of a model, which is simply the number of correct predictions over the total number of cases.

Finally, support is simply the number of actual true values for each class in the testing dataset. It does not measure performance directly, yet it provides helpful context when interpreting other metrics. For example, having strong precision or recall in a row with low support is not that important, since it represents success on a small portion of the data, whereas having low precision or recall on high-support classes can have a much larger impact on overall performance.

While this was a relatively strong accuracy, this particular model took 21 minutes to run, even with parallel processing which will be discussed later. One way to speed this up is by only using the most important features in the dataset. For example, the graph below shows the accuracy of one Random Forest model using a certain number of features on a logarithmic scale. Having more features certainly helps at first, but it gets to the point where one must decide whether it is worth it to drastically decrease a model’s efficiency just for small improvements in accuracy.

Conway 10

Figure 3: Accuracy vs. Number of Features Furthermore, cross-validation helps to assess and improve the accuracy of models like Random Forest, as it is an important method for controlling model complexity. Rather than a single train-test-split, as discussed above, cross-validation repeatedly splits up the dataset into folds of training and validation subsets.

The system is trained on some of these folds and then evaluated on others, improving itself each time, which boosts performance for data it hasn’t seen before.

In general, cross-validation is used to tune the model and effectively “prune” it by choice of hyperparameters, which are model settings that can be adjusted to optimize performance such as the number of trees and tree depth. Even as individual trees within a Random Forest often grow deep, the overall depth, minimum samples per leaf, and the number of features considered at each split can be controlled systematically through cross-validation. This allows the model to determine when increased complexity stops helping and is maybe even hurting

Conway 11 performance. For example, while increasing the number of trees tends to reduce variance, the returns generally diminish at some point. Cross-validation allows the user to track validation performance as a function of forest size to determine where additional trees start to become unwarranted.

Generating all of these trees takes a long time, and some models will take hours or days to run, especially when they involve cross validation and advanced models. Thus, parallel processing becomes very important. In the majority of models, learning algorithms repeat the same tasks again and again, whether this is an evaluation of the gradient, scanning the features, or adjusting models. Through the execution of the same tasks on multiple processors or cores, parallel processing enables the speedy training of models on bigger datasets, like the one being used for this research, that would otherwise have taken too long to process. For example, the computer used for this research has 10 cores, but a program would typically only use one core to run. However, when running models that would normally take a long time, the computer can be programmed to divide the “work” for the program among the 9 other cores, which vastly speeds up the processing.

In tree-based models like Random Forest, each core can work in parallel to make many trees that eventually come together for the final model. Such parallel computing enables the model to process data more efficiently than other algorithms with ordered data that cannot be analyzed in parallel.

Parallel processing is also a key concept in the development of other models and neural networks, especially during gradient descent. Although the forward and backward propagation techniques have to be done sequentially, the calculations performed in each layer, including the multiplication of large matrices, can be handled and sped up using parallel processing. This is how a complex model is able to process a

Conway 12 massive amount of data all at once, reducing what would have been a slow process.

The next model that was tried was XGBoost, which stands for Extreme Gradient Boosting, a machine learning algorithm that is based on “boosting” a decision tree. XGBoost forms many trees just like Random Forest, but every new tree is used to correct the faults made by a combination of preceding trees. For this reason, it creates “smarter” trees that ultimately lead to higher model accuracy (Kavlakoglu & Russi, 2024).

Behind the XGBoost algorithm is the concept of gradient boosting, in which additional regression trees are created to optimize a loss function through a gradient-based approach, just like with a linear regression. For each “boosting cycle,” the function calculates the gradients of the loss function with respect to the predictions made so far. The gradients obtained are then used to decide how to split the data in the next regression tree. Thus, every tree makes a small contribution to the entire model. This process of correction is similar to how natural learning processes occur, as corrections are gradually implemented instead of having all corrections implemented in a single cycle. Further research is currently underway to investigate how traditional, biological learning processes can be used for machine learning.

Furthermore, XGBoost has its own regularization method that reduces the complexity of the trees, thus reducing the risk of overfitting. To do this, the depth, weights, and learning rate are all restricted. This means that the testing accuracy maintains accuracy closer to the training accuracy. Overall, the model is fast, efficient, accurate, and scalable, especially given that the opportunities XGBoost allows for parallel processing. XGBoost also handles missing values automatically, making it well suited for large datasets (XGBoost Documentation, 2022). Finally, this model

utilized sklearn’s “GridSearchCV” in hyperparameters, further increasing its accuracy.

Conway 13 order to tune

Figure 4: Classification Report for XGBoost Therefore, it is not surprising that XGBoost had a solid accuracy of 0.657. It was also very fast, as it only took 6 seconds to run. This makes it ideal for evaluating clutchness, as the model must be used many times repeatedly. We ultimately ended up using this as the model to evaluate clutchness.

Results and Discussion The most basic approach to evaluating clutchness is using a player’s raw field goal percentage in these big moments. However, by using machine learning, we can assess clutchness based on expected shot difficulty. This is especially important because “clutch” shots are not randomly distributed, as they tend to be from farther distances and with greater defensive pressure, both of which force difficult shots. It’s rare to see a game winning shot be a wide open layup, as it’s more likely to be a step-back, closely

Conway 14 guarded deep three. Simply stated, it is difficult to make accurate assessments of shooters based on raw percentages when shots become more difficult.

To train this model to evaluate clutchness, we must train it without any information that might relate to clutchness, such as the time remaining or score differential. This ensures that the model is predicting whether or not a shot will be made independently of the importance of the moment. Then, taking a subset of the dataset that consists only of clutch shots, which we define as attempts in the last 5 minutes of the fourth quarter in a 5 point game or less, this model predicts how many of these shots each player will make by summing the probability of each individual shot being made. This is calculated using Scikit-learn’s predict_proba() function. By comparing this predicted total with how many shots the player actually made, one can see which players consistently exceed or fall short of statistical expectations in these high-pressure moments. Two basic metrics are used for this analysis. The first is “clutch over expectation,” which is simply the difference between the number of shots made and the sum of probabilities of expected shots made (expected values for each shot). This of course measures whether, and to what extent, a player converted more or fewer shots than predicted given shot difficulty. The second metric is clutch field-goal percentage above expectation, which normalizes this difference by the number of clutch attempts, which allows for fairer comparison across players with different volumes of clutch shots. Even so, this metric is not clearly superior, because having the courage to take, and make, a higher volume of big shots is what makes some players great.

Conway 15

Figure 5: The top 10 players from the 2024-2025 NBA season by the number of clutch shots they made over expectation To make the list, one must have attempted at least 25 of these shots over the course of the season, which applied to 50 NBA players. Interestingly, 30 of these 50 players made fewer shots than expected, which supports the hypothesis that these shots under high-pressure are more difficult. Darius Garland was at the top of this list, converting on 12.6% more shots than expected in the clutch. He did have fewer attempts than many other players at the top of the list, though, likely because he was the second scoring option on a Cleveland Cavaliers team led by Donovan Mitchell.

Each year, the NBA awards the “Clutch Player of the Year Award” to “the player who best comes through for his teammates in the clutch.” Last year, this award went to Jalen Brunson, with Anthony Edwards and Nikola Jokic as the two other finalists.

Brunson was 5th on this list with 3.02 makes over expectation, while Anthony Edwards was 3rd with 3.24 and Nikola Jokic was 14th with 0.58. Jokic being lower on the list may be because this model only accounts for shooting, as NBA voters likely factored in his excellent passing and rebounding ability. It’s compelling that all three of these players tied for second in the league with 29 made shots in clutch moments (behind Demar Derrozan who shot a relatively poor 31/71), indicating that voters may simply be looking at the number of times each succeeded instead of more in depth analysis about how players raise their performance in big

Conway 16 moments. Voters may simply see the game winning shots that go viral online instead of looking at a player’s whole body of work.

Perhaps with the rise of machine learning, voters will look more at analytics like these to truly see who is the most clutch.

Figure 6: The 10 worst players in clutch moments last season In last place is Deven Vassell, who shot an abysmal 13.7% below expected in these high-pressure moments. At position 47 on the list is Celtics star Jayson Tatum, who made 3.93 fewer shots than expected in the clutch. Meanwhile, his teammate Jaylen Brown was much better, in 22nd place. This trend was also true in the 2023-2024 season, as Tatum was in second to last place, shooting an incredible 15.5% below expected in big moments.

Meanwhile, Brown was 16th out of 53 players converting 3.22% more shots than expected. This is certainly in line with many Boston sports fans’ opinions that Tatum shouldn’t always be the one taking shots in the big moments.

We can also take a broader view and look at many years of data. For example, below is the top 5 and bottom 5 most clutch players in the last 20 seasons, though the dataset was filtered to only include players who have scored 10,000 points over that span to prevent the dataset from being too large. It was interesting to see that players like Buddy Hield and Russell Westbrook who are well known for shining in big moments were atop this list, while players like Kawhi Leonard and Kevin Durant, whose clutch abilities are sometimes questioned even though they are both great individual

Conway 17 players, fell to the bottom. Other household names that may have been lower than expected included Stephen Curry in 20th, LeBron James in 22nd, and Kyrie Irving in 30th.

Figure 7: Top 5 most clutch players in the last 20 seasons

Figure 8: Bottom 5 least clutch players in the last 20 seasons ​ Perhaps even more meaningful was the fact that only 9 of these 38 players shot better than expected in these close, late-game situations, with an average of 2.89% below their expected field goal percentage. This suggests that even these great players tend to perform below expectation when faced with the pressure and intensity that comes with these moments.

Conclusion In conclusion, this project shows that machine learning can provide meaningful insight about the NBA. Since we found that XGBoost was the best performing model by balancing high accuracy and efficiency, it seems that future NBA machine learning research should use this or other advanced classification models.

More research could also be done using some of the newer, more advanced models like transformers. Since transformers are time-based, this could be especially helpful to look at how momentum affects a game. For example, many debate whether

Conway 18 shooters get “hot,” meaning that after they make a couple of shots they are more likely to make the next one because they will be more confident. This could be investigated with a transformer model, or even just using a regression by adding a column that indicates whether or not a player made their previous shot. If the coefficient column was positive, that would indicate that being “hot” does increase your chance of making a shot.

Finally, there are some limitations of this research. For one, it’s difficult to translate all the aspects of a basketball shot into a dataset. Thus, it’s impossible for the model to determine the exact difficulty of a shot. Furthermore, for each season, there’s a relatively small sample size of “clutch” shots. While there’s over 200,000 total shots in a season, no individual player had over 80 clutch shots during that time. For this reason, there are probably examples where just because a player was at the top or bottom of the list for a season, it may be due to luck and random chance rather than how “clutch” they actually are.

Furthermore, our definition of clutch moments, shots taken in the last 5 minutes of a game where the score is separated by 5 points or fewer, could be debated. Game winning shots or shots in the last minute of a close game certainly are more important and maybe deserve more weight. For example, Jayson Tatum is known for making a number of game winners, but this model ranks him as less clutch because of his overall performance in the last 5 minutes of these games. Last, the greatest issue with this research is the fact that it only includes regular season data. The most “clutch” shots are the ones that come in the playoffs, when a team’s whole season is on the line. Therefore, looking at how players perform in the playoffs can be a true measure of clutchness.

Conway 19 References Artem Kirsanov. (2024, March 31). The Most Important Algorithm in Machine Learning. YouTube. https://www.youtube.com/watch?v=SmZmBKc7Lrs. Bhavya. (2022). NBA Shots Dataset (2000 - Present). Kaggle.com. https://www.kaggle.com/datasets/techbaron13/nba-shots-da taset-2001-present.

Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.

Brown, S. (2021, April 21). Machine Learning, Explained. MIT Sloan. https://mitsloan.mit.edu/ideas-made-to-matter/machine-lear ning-explained.

DomSamangy. (2024). NBA regular season shot location data from the 2003-04 season to 2024-25. GitHub. https://github.com/DomSamangy/NBA_Shots_04_25.

DomSamangy. (2025). NBA player play type data from 2012-2025.

GitHub. https://github.com/DomSamangy/NBA_Play_Types_12_2.

GeeksforGeeks. (2020, September 4). Random Forest Classifier using Scikitlearn. GeeksforGeeks. https://www.geeksforgeeks.org/dsa/random-forest-classifier -using-scikit-learn/.

Kavlakoglu, E., & Russi, E. (2024, May 9). XGBoost. Ibm.com. https://www.ibm.com/think/topics/xgboost.

Szymon Jóźwiak. (2024). NBA Traditional Boxscores 1997-2024.

Kaggle.com. https://www.kaggle.com/datasets/szymonjwiak/nba-traditio nal. xgboost developers. (2022). XGBoost Documentation — xgboost 1.5.1 documentation. Xgboost.readthedocs.io. https://xgboost.readthedocs.io/en/stable/.