1
Increased Golf Hole Size Effect
IEE 572: Design of Engineering Experiments
Dr. Douglas Montgomery
Team Members:
Hank Dettlaff (1:30)
Ankitha Chandran (1:30)
Sahar Ehsani Valencia (4:30)
Table of Contents
Statement of Problem…...... 3
Selection of Factors…………………….....3
Selection of Response Variable………..…4
Choice of Experimental Design……....….4
Conducting the Experiment……………...5
Data Analysis……………………………...5
Conclusions and Recommendations……15
Appendix A: Run Order………………...18
Appendix B: Model ANOVA Output…..25
Statement of Problem:
The golf industry has been facing a decline in the numbers of new golfers both coming to and continuing to enjoy the game. This trend has been attributed to a number of factors including the large time requirement to play a complete round and a large learning curve to become proficient at the game. One recently proposed idea that has been backed by golf legend Jack Nicklaus is to increase the size of the cup. The intent of this is to make it easier for players of all abilities to make putts. Subsequently, the reduction in total putts during a round would help to improve pace of play and reduce the overall time requirement for a round. In this experiment, we wish to test the effect of the larger hole on increasing the conversion rate of putts for players of varying ability.
Selection of Factors:
The primary design factor of this experiment is the hole size. The current regulation size of 4.25 inches has been the official number since the Royal and Ancient Golf Club adopted it from the first known golf hole cutter used at the Musselburgh Links in Scotland. As the motivation is to increase the putt conversion rate, this size will be used as the low level of this factor. The number for the increased hole size that has been predominately discussed is 8 inches. For this reason, this will be used as the high level of the cup size factor.
Future experiments could deal more directly with the optimization of this factor. For the selection of the other design factors (Table 1), it is desired to provide a representation of putts that are commonly encountered on the course. In a round, a golfer will encounter putts of varying distance and amount of break (curvature of the ball path). For the low level, a distance representing a common second putt length that is expected to have a high but not 100% conversion rate for an experienced player but may still be troublesome for a beginner is desired. A distance of 3 feet is selected for this. For the high level, 10 feet is selected as it represents a longer putt following a greenside chip or a relatively short putt following an approach. Longer putts would be expected to have a very low conversion rate even for experienced players so would likely not yield as meaningful results without extensive numbers of putts attempted. For putt curvature, a putt on as flat as surface as possible will be selected so that the putting line crosses the center of the cup. For a high level, a breaking putt will be selected and attempts will be made to make the angle of the starting target line from the hole line similar for putts of varying distance. In order to minimize potential variations, a mat will be used to represent the larger hole such that exact same putt is encountered.
Factor / Type / Low Level / High LevelHole Size / Numerical / 4.25” / 8”
Putt Distance / Numerical / 3’ / 10’
Putt Curvature / Categorical / Flat/Straight / Breaking
Player Experience / Categorical / Experienced / Beginner
Table 1: List of factors and corresponding levels
Asides from the design factors, it is possible to encounter varying quality of greens (speed, smoothness, grass type), varying weather conditions, better understanding of the putting line, changes in mentality and confidence, and potential wearing in of a track to the hole after many putts. The green quality and weather conditions will be considered as a held-constant factor as we will carry out the experiment in a small time period all on the same greens. A green representative of average quality will be selected to typify what most people will encounter. A day with minimal wind and no rain will be selected to minimize weather factors. The other experiment related factors will be considered allowed-to-vary factors but their effect on results will be minimized by randomizing all putts rather than carrying out a set of putts at each factor combination at once.
Selection of Response Variable:
As the goal of this experiment is to provide data relevant to determining if a reduction in strokes is obtained using the larger hole, a figure of putt conversion rate will be used as the response variable. This conversion rate will calculated as the percentage of putts made out of a set number of attempts for each factor level.
Choice of Experimental Design:
The experiment will be conducted using 24 factorial design totaling 16 factor combinations. The randomized run order is generated using a 20 replicate 24 factorial design in the JMP resulting in 320 total runs (Appendix A). The experiment will be carried out in a single session on a day with stable weather conditions and low wind speed. As a result, blocks will not be required and variability due to nuisance factors is minimized. The data will then be analyzed using a single replicate 24 factorial table in Design Expert.
Conduction of the Experiment:
Factor 1 / Factor 2 / Factor 3 / Factor 4 / Response 1Std / Run / A:Hole Size / B:Distance / C:Curvature / D:Player / R1
in / ft
1 / 14 / 4.25 / 3 / Straight / Beginner / 0.85
3 / 6 / 4.25 / 10 / Straight / Beginner / 0.3
5 / 4 / 4.25 / 3 / Breaking / Beginner / 0.7
7 / 9 / 4.25 / 10 / Breaking / Beginner / 0.15
2 / 10 / 8 / 3 / Straight / Beginner / 1
4 / 1 / 8 / 10 / Straight / Beginner / 0.35
6 / 16 / 8 / 3 / Breaking / Beginner / 0.9
8 / 15 / 8 / 10 / Breaking / Beginner / 0.35
9 / 8 / 4.25 / 3 / Straight / Experienced / 1
11 / 13 / 4.25 / 10 / Straight / Experienced / 0.5
13 / 3 / 4.25 / 3 / Breaking / Experienced / 1
15 / 7 / 4.25 / 10 / Breaking / Experienced / 0.35
10 / 11 / 8 / 3 / Straight / Experienced / 1
12 / 5 / 8 / 10 / Straight / Experienced / 1
14 / 2 / 8 / 3 / Breaking / Experienced / 1
16 / 12 / 8 / 10 / Breaking / Experienced / 0.65
Table 2: Randomized run order created using Design Expert.
The experiment was conducted and the percentage values were entered into factorial table generated by Design expert for each factor level combination (Table 2). The experiment was carried over a 3 hour span on 11/30/2011 from 10:50 to 13:50 at the ASU Karsten Golf Course.The temperature ranged from 66˚F to 74˚F, the wind speed was 4.6 miles per hour, with a humidity of 22 percent and it was mostly clear. The course superintendent reported green speeds of 9.5 feet using a stimpmeter. There was no dew present on the greens at any time during the experiment that would affect this. The players took frequent water breaks to make sure they were not exhausted and their performance would not be affected.
Data Analysis:
There are a total of four factors for this experiment at two levels each. The effects list considering all the interaction terms was carried out and the result is as follows:
Term / Effect / SumSqr / % ContribtnRequire / Intercept
Model / A-Hole Size / 0.175 / 0.1225 / 8.30861
Model / B-Distance / -0.475 / 0.9025 / 61.2124
Error / C-Curvature / -0.1125 / 0.050625 / 3.43366
Model / D-Player / 0.2375 / 0.225625 / 15.3031
Model / AB / 0.0875 / 0.030625 / 2.07715
Error / AC / 0 / 0 / 0
Model / AD / 0.025 / 0.0025 / 0.169563
Error / BC / -0.05 / 0.01 / 0.678253
Model / BD / 0.1 / 0.04 / 2.71301
Error / CD / -0.0125 / 0.000625 / 0.0423908
Error / ABC / -0.0125 / 0.000625 / 0.0423908
Model / ABD / 0.1125 / 0.050625 / 3.43366
Error / ACD / -0.05 / 0.01 / 0.678253
Error / BCD / -0.075 / 0.0225 / 1.52607
Error / ABCD / -0.0375 / 0.005625 / 0.381518
Lenth's ME / 0.240992
Lenth's SME / 0.489249
Table 3: Effects list with sum of squares and percent contribution to variability
The half normal probability plot for the same is shown below (Figure 1) and it can be seen that only factors A (hole size), B (distance), D (player) and ABD have significant effects on the response variable as they are outliers on the half normal probability plot. The ANOVA is then carried out taking these significant effects into consideration along with effects BD, AB and AD to maintain hierarchical order. The results of the refined model ANOVA are shown below in Table 4.
Figure 1:Half Normal Probabilty Plot
ANOVA for selected factorial modelAnalysis of variance table [Partial sum of squares - Type III]
Sum of / Mean / F / p-value
Source / Squares / df / Square / Value / Prob > F
Model / 1.374375 / 7 / 0.196339 / 15.70714 / 0.0004 / Significant
A-Hole Size / 0.1225 / 1 / 0.1225 / 9.8 / 0.014
B-Distance / 0.9025 / 1 / 0.9025 / 72.2 / < 0.0001
D-Player / 0.225625 / 1 / 0.225625 / 18.05 / 0.0028
AB / 0.030625 / 1 / 0.030625 / 2.45 / 0.1562
AD / 0.0025 / 1 / 0.0025 / 0.2 / 0.6666
BD / 0.04 / 1 / 0.04 / 3.2 / 0.1114
ABD / 0.050625 / 1 / 0.050625 / 4.05 / 0.079
Residual / 0.1 / 8 / 0.0125
Cor Total / 1.474375 / 15
Std. Dev. / 0.1118034 / R-Squared / 0.932175
Mean / 0.69375 / Adj R-Squared / 0.872827
C.V. % / 16.1158052 / Pred R-Squared / 0.728699
PRESS / 0.4 / Adeq Precision / 9.803061
Table 4: Design-Expert ANOVA output for designated model
It can be seen from the analysis that factors A (hole size), B (distance), and D (player) are significant. The value of R2 is 0.9321. This shows that the percentage of the variability explained by the model is 93.21% and thus, the design is a good fit to the model. A small value of PRESS (0.4) and a high value of predicted R2 (0.7286) indicate that the model can satisfactorily predict the variability in new data. Adequate Precision gives the signal to noise ratio. This has a value of 9.803061 which indicates that there is adequate signal as usually a ratio of greater than four is desirable.
The normality assumption is checked using the normal plot of residuals (Figure 2). Using the “fat pencil” test, we find that all points fall reasonably close to the line. This indicates the normality assumption has been satisfied.
Figure 2: Normal Probabilty Plot of residuals
The residual vs. predicted plot (Figure 3) is shown below. It can be seen that the points do not take on any distinctive shape. The points at a prediction level of 0.82 (standard order runs #12 and #16) have noticeably higher residuals and correspond to either straight or breaking putts for the experienced player using the 8 inch cup from 10 feet. It is believed that these are attributable to the tendency of the beginner to hit the ball very firm therefore minimizing variation that would be encountered by a more typical putting style. This may have lessened the effect of factor C (curvature) but is not such a strong issue as to fully invalidate conclusions drawn from the data. This is discussed further in the conclusion and recommendations.
Figure 3:Plot of Residuals vs Predicted
Although the untransformed data did not show any major issues, the arcsine square root transform is investigated since this is often useful for response variables that are in percentage form. The resulting plot of residuals is very similar (Figure 4) however the figures of fit quality are reduced. Therefore, further analysis is carried out without the transformation.
Figure 3:Plot of Residuals vs Predicted with Arcsine Square Root Transformation
The next plot is that of Predicted vs. Actual and the points lie along a straight line indicating that the fit between the predicted model and actual data is reasonably good.
Figure 4:Plot of Predicted vs Actual
At first glance, the residuals vs. individual factors plots indicate that there may be an issue with non-constant variance. Upon further inspection, the points leading to this conclusion are #12 and #16 of the standard run order. Again, we conclude that this will not strongly detract from the validity of our results.
Figure 5: Plot of Residuals vs Hole Size
Figure 5:Plot of Residuals vs Distance
Figure 7:Plot of Residuals vs Player
Figure 8: Hole Size – Distance (AB) interaction plots for beginner (left) and experienced player (right). Black lines are at a 3 foot distance and red lines are from 10 feet.An inspection of the interaction plots helps visualize the results of the ANOVA and draw conclusions relevant to the effect of hole size on putting conversion rate. The AB (hole size and distance) interaction is shown in Figure 8 with the player experience at low and high levels. This interaction has a p-value of 0.15 which is moderately low but not considered significant. At the average of the two plots shown, the interaction lines are slightly off parallel which reflect this. The very noticeable effect on these interactions caused by factor D (player) is in agreement with the low p-value of 0.079 for this interaction. For the beginner, we note that the larger hole size results in a similar improvement for both levels of distance. The confidence bounds of the prediction may even include no improvement at both distances for this player. This is suggestive that a larger hole size may increase the strength of the data in future experiments. For the experienced player, all putts were made at a range of 3 feet therefore no improvement is noted. In future experiments, a longer low distance level might provide more relevant information on hole size effect. At the higher distance, considerable improvement is noted for the experienced player.
Figure 9: Hole Size – Player Level (AD) interaction plots for 3 foot distance (left) and 10 foot distance (right). Red line is the for the beginner and the green line is for experienced player.Inspecting the AD (hole size and player) interaction for high and low levels of distance (Figure 9) again displays the strength of the ABD interaction as the B (distance) factor greatly affects this interaction. With a p-value of 0.667 this interaction is unlikely to have been responsible for the variance in the data. This is reflected as these interaction lines are nearly parallel for at the average distance level. As was previously mentioned, all putts were made at the shorter distance for the experienced player. In this case, this is suggestive that a smaller hole size might provide better data however since the goal is to increase putting conversion rate increasing the low level distance would be preferable. At this range, it is clear that a greater advantage is encountered by the beginner. At the 10 foot distance, the advantage is reversed as the larger hole contributes to a clear improvement for the experienced player but a small predicted improvement for the beginner. The confidence bounds of the predicted interaction for the beginner include no improvement at their boundary. Again, this suggests that the larger cup size may need to be even larger to increase the strength of this prediction.
Figure 10: Distance – Player (BD) interaction for 4.25 inch cup (left) and 8 inch cup (right). Beginner characteristic is in red and experienced is in green.The BD (distance and player) interaction plots (Figure 10) also help to visualize the effect of hole size. For the standard cup size (4.25 inches), distance has a similar effect for both player levels on the conversion percentage. However, the larger cup lessens the effect of distance for the experienced player but remains approximately the same for the beginner. A small but uniform improvement over the distance range is noted for this player.
We also generated the time series make or miss for the beginner (Figure 11) and experienced player (Figure 12) to inspect for any notable changes in conversion rate through out the experiment. For the beginner the density of misses is lower towards the end than the beginning but this difference is at a low enough level that the experimental randomization should minimize any effect. The plot for the experienced player shows no strong trends as is expected. It can be seen from these graphs that the randomization of the experiment was effective.
Figure 11: Time Series of Make/Miss for beginner
Figure 12:Time Series of Make/Miss for Experienced Player
Conclusion and Recommendations:
In this experiment, factors A (hole size), B (distance), and D (player) were found to be significant at a level of 0.05 using an ANOVA model fitting these factors, their three factor interaction, and corresponding hierarchical combinations. Their three factor interaction ABD may also be important as reflected by a p-value of 0.079 in the ANOVA and by inspecting the interaction plots.
Plots of residuals versus predicted values and factors primarily support that assumptions of constant variance are met with the exception of factor combinations #12 and #16 which were either straight or breaking putts using the large hole from 10 feet for the experienced player. During the experiment, the beginner, whom had never played before, had a tendency to hit the ball very firm therefore reducing the influence of break in a putt a potentially drastic amount. As a result, the D factor effect (curvature) was minimized in the overall experiment and its likely true effect was observed as high residual points for the experienced player.
An additional source of unaccounted error relating to this is the use of a mat in place of the hole. Since it would be difficult to judge if a putt going over the hole would go in or bounce out due to excess speed, some putts may have been counted as a make that may have came out. Since less excessive speed is required to pop out of the smaller hole, it is likely that conversion percentage involving this level was inflated, especially in the case of the beginner. For future experimentation, it is recommended to make sure the beginner understands the importance of putting such that misses are still near the hole prior to the experiment. Overall, the following conclusions drawn from the data can be considered informative but require careful consideration of these issues.
The goal of the experiment, to better understand the effect of a larger hole on putting and overall scoring, is now more directly discussed. Overall, the predicted conversion rate showed at least some degree of increase using the larger hole except for in cases where there was no room for improvement. Although there were some cases where no improvement was within the prediction confidence interval for the beginner, it is likely that the conclusion of improvement would be stronger given a better understanding of the need to keep misses near the hole as mentioned above. This conclusion supports the desired goals of the larger cups to reduce total putts and therefore improve pace of play as well as to make the game more fun for new players by enabling them to feel the success of making more difficult putts.
It is also desired to know how the larger hole size might effect different levels of players differently. Since the make percentage was maxed out for the experienced player at the shorter distances a comparative advantage using the smaller hole is noted for the beginner. However, as can be noted in the BD (distance and player) interaction plot, there is a positive relationship with distance and the comparative advantage for the experienced player. This is probably not a desirable trait however it may be outweighed by the fact that the beginner is predicted to experience a fairly uniform increase in conversion rate over the range of distances. For future experiments, this data suggests increasing both the short and long putt distance to avoid saturation in the case of the short putt for the experienced player and also as a tool to increase prediction power in the case of the long putt since a higher proportion was made at this distance then expected.