Machine Learning for Underwriting Life and Health Insurance
Underwriting is a risk assessment process that classifies insurance applications into different categories. Traditional underwriting is costly, time-consuming and perceived as a barrier for the underserved population. Despite many attempts in automating life and health insurance underwriting, the dominant approach is a mix between rule-based engines and traditional underwriting. Applications would be first assessed by rule-based engines; only a third of all applications are processed by these engines. The remaining applications that cannot be assessed by the rule-based engines are reviewed by underwriters or even referred to the reinsurers. The rule-based engines have no predictive ability of applications that do not fit into the existing rules encoded into the engines. Therefore, the natural-step improvement to the current approach is predictive machine learning models.
This research aims to construct predictive machine learning models to predict underwriting decisions for life and health insurance applications, using reinsurer data that are predominantly applications with complex medical conditions and large sum insured. The models are designed to provide an end-to-end solution, so machine learning techniques such as natural language processing and clustering analysis are used to process real-world data; in particular, free-text descriptions of impairments and occupations, which the traditional statistical models cannot process. Text mining tools such as word clouds are used as part of preliminary analyses and give some insights into medical underwriting. Various feature selection methods such as mutual information and recursive feature elimination are used to improve prediction accuracies. Lastly, machine learning algorithms such as XGB, Random Forest and bagging are used to predict the underwriting decisions.
The accuracies of various machine learning models are then compared, and the extreme gradient boost algorithm with a combination of feature selection methods performs the best, with 94% accuracy on the training set and 71% accuracy on the testing set. This result is a significant improvement from the rule-based engines that can process only a third of the applications. The feature ranking function of the extreme gradient boost algorithm gives additional underwriting insights such as the features that are the most important for the model to determine underwriting decisions. The remarkable increase in prediction accuracies, as well as additional functionalities such as feature ranking, indicate that predictive machine learning models could be used to improve the current underwriting process significantly.