Comparing Linear Regression to Shrinkage Regression Algorithms (RR, Lasso, El Net) Using PTSD Patients’ Data

Document Type : Research Paper

Author

Department of Psychology, Faculty of Humanities, Tarbiat Modares University, Tehran, Iran

Abstract

The purpose of this research was to introduce the alternative model of regression algorithms and having it compared to linear regression. To do this, we need to use modern algorithms such as Ridge, Lasso, and Elastic net regression in which precision is maximized by regularizing the cost function. In this paper theoretical basis and practical implications have been explained.. The target population was patients diagnosed with Post Traumatic Stress Disorder (PTSD) in 2020 for the comparison. 97 PTSD patients (73 females and 24 males) in Tehran were measured in 8 variables related with the intensity of the trauma re-experience. The linear regression, Ridge, Lasso, and Elastic Regressions were used with R software. The results indicated that compared to linear regression, Elastic. LASSO and Ridge explained more variances and had more R square and less MSE respectively. When the main assumptions of Linear regression are not met, using shrinkage regressions seems to be reasonable and accurate.

Keywords

References

فراهانی، ح.، و عریضی، ح. (1387). روش‌های پیشرفته پژوهش در علوم انسانی. اصفهان: جهاد دانشگاهی اصفهان.
Cohen, J., Cohen, P., West, S., G., Aiken, L., S. (2003). Applied multiple regression and correlation analysis for the behavioral sciences. Third Edition. New York: Routledge.
Ernst, A. F., & Albers, C. J. (2017). Regression assumptions in clinical psychology research practice—a systematic review of common misconceptions. PeerJ. 16(5), e3323.
Fox, J. (2016). Applied regression analysis and generalized linear models (3rd Ed.). Thousand Oaks, CA: Sage publications.
Hair, J. F., Babin, B. J., Anderson, R. E., & Black, W. C. (2018). Multivariate Data Analysis. 8th edition: USA.
Hastie, T., Tibshirani, R., & Wainwright, M. (2015). Statistical learning with sparsity: The Lasso and generalizations. Chapman Hall, London.
Howell, D. (2013). Statistical Methods for Psychology. USA: Wadsworth.
Liu, H., & Zhang, J. (2009). Estimation consistency of the group Lasso and its applications. J Mach Learn Res Workshop Conf Proc. 5, 376–83.
Maronna, R. A. (2011). Robust ridge regression for high-dimensional data. Technometrics. 53(1), 44-53.
Montgomery, D. C., Peck, E. A., &Vining, G. G. (2012). Introduction to linear regression analysis. USA: Wiley & Sons.
Park, H., & Konishi, S. (2015). Robust logistic regression modelling via the elastic net-type regularization and tuning parameter selection. Journal of Statistical Computation and Simulation. 86(7), 1-12.
Saleh, A. K. M. E., Arashi, M., & Kibria, B. M. G. (2019). Theory of Ridge Regression Estimation with Applications. Wiley, Hoboken, NJ, USA.
Samkar, H., & Alpu, O. (2010). Ridge regression based on some robust estimators. Journal of Modern Applied Statistical Methods. 9(2). 495-501.
Stein, C. M. (1981). Estimation of the mean of a multivariate normal distribution. Annals of Statistics.9(6), 1135-1151.
Wilcox, R. R. (2019). Multicolinearity and ridge regression: results on type I errors, power and heteroscedasticity. Journal of Applied Statistics. 46(5), 946-957.
Williams, M. N., Grajales, C., & Kurkiewicz, D. (2013). Assumptions of multiple regression: correcting two misconceptions. Practical Assessment. Research & Evaluation. 18(11), 1–14.
Zou, H., & Hastie, T. (2005). Regularization and Variable Selection via the Elastic Net. Journal of the Royal Statistical Society (B). 67(2), 301-320.
Journal of Applied Psychological Research
Volume 11, Issue 3
November 2020
Pages 193-206

Files

History

  • Receive Date: 06 July 2020
  • Accept Date: 29 August 2020
  • First Publish Date: 17 November 2020
  • Publish Date: 21 November 2020

Share

How to cite

Statistics