Simple Sensitivity Analysis with R
A sensitivity analysis is a technique used to determine how different values of an independent variable impact a particular dependent variable under a given set of assumptions. This technique is used within specific boundaries that depend on one or more input variables, such as the effect that changes in interest rates have on bond prices.
As soon as you build a predictive model, you might be interested in which predictor variable is the most important or the most influential. This is not an easy task to do and there are multiple ways to address it, with multiple assumptions.
In the video below, I am using simulated squat jump data, namely max Power, body weight, FV slope and push off distance to predict jump height. Once I plug things into a model, I might become interested into how changing one variable affects the outcome (in this case jump height). In the video, I am explaining the use of beta coefficients (i.e. scaling the predictors), log-log analysis and very simple sensitivity analysis that produces very intuitive visualization, which is quite handy in understanding variable importance.
Unfortunately, with this we still can’t make ‘ought’ from ‘is’ (see Hume), but it is indeed a step in the right direction. This is because we are using observational data to make inferences on how changing one variable (and controlling for others) will affect theoutcome variable. For example, we might have a bunch of individuals performing FV profiling and we might use sensitivity analysis to get a glimpse of what parameter improvement might give us the biggest improvement in the vertical jump of a particular individual (explained in the video). In a way, we are making interventional claims from observational data (“We can’t derive ‘ought’ from ‘is’”) and we need to watch for it. But it is a good starting point to consider assuming the model.
Below the video, you can download a free R code with the functions for sensitivity analysis. Also, please let me know if you have any further questions or critiques. Enjoy the video and I am certain that you will pick up this very simple sensitivity analysis that can be quite intuitive to coaches and other consumers of your analysis.
EDIT: In the video, I have mentioned that the function might have an error with the use of polynomial fit. It doesn’t. I just compared two different models and got different effects of max power on jump height, which is a nice example of predictions and sensitivity being conditional on the model and assumptions. Multivariate modeling is not easy, but it is bloody fun (at least for me).
Additional Information
If you are interested in these topics, I suggest checking “Interpretable Machine Learning” by Christoph Molnar.
You can also check this great video on LIME (good sources as well: Visualizing ML Models using LIME, “Why Should I Trust You?”.
Responses