Two hundred years ago a captain may have had only a sounding line and his experience to navigate through uncharted waters. Today, a captain has access to many other data sources and tools to aid in his navigation, including; paper charts, online charts, engineering surveys, a depth sounder, radar, and GPS. These new tools don’t make the old tools obsolete, but any mariner would be safer and more accurate in their piloting by employing all the tools at their disposal.
In the same vein, actuaries who solely use traditional reserving techniques, such as triangle based methods, aren’t capitalizing on new technologies. Actuaries should start adopting other techniques such as Machine Learning (ML). ML is a field of predictive analytics that focuses on ways to automatically learn from the data and improve with experience. It does so by uncovering insights in the data without being told exactly where to look.
ML is the GPS for actuaries. As GPS improved navigation, ML has the potential to greatly enhance our reserves. It is important to note though that ML is not just about running algorithms; it is a process. At a high level this process includes defining the problem, gathering data and engineering features from the data, and building and evaluating the models. As in the actuarial control cycle, it is important to continually monitor results.
Through our research, we have found significant improvements in the prediction of reserves by employing this ML process. Overall we have found a reduction in the standard and worst case errors by 10 percent. To assist actuaries in testing the value of ML for themselves, this paper will provide an outline of the ML process.
Similar to the Actuarial Control Cycle, the first step is to define the problem. In our context, we are interested in efficiently calculating the unpaid claims liability. We want to calculate this quantity in an accurate manner that minimizes the potential variance in the error of our estimate.
Actuaries often use various triangle-based methods such as the Development and the Paid Per Member Per Month (Pd PMPM) to set reserves. These methods in principle attempt to perform pattern recognition on limited information contained within the triangles. Although, these methods continue to serve actuaries well, information is being left out that could enhance the overall reserve estimate. To make up for the lack of information used to estimate the reserves, an actuary relies heavily on his/her judgment. Although judgment is invaluable, biases and other elements can come into play leading to large variances, and the need for higher reserve margins.
As described in our prior article (Cap, Coulter, & McCoy, 2015), the range of reserve estimate error present in company statements pulled from the Washington State Office of the Insurance Commissioners website, was -10 percent to 40 percent. This represents a wide range of error, and has significant implications. These can include an impact to the insurers rating status, future pricing and forecasting decisions, calculation of Premium Deficiency Reserves, or even unnecessary payout of performance bonuses.
Gathering data is something that actuaries are already good at. Leveraging off their expertise along with other subject matter experts will be helpful in identifying all available sources for use. There is often a saying with ML that more data often beats a sophisticated algorithm.
Once the data has been gathered it will need to be engineered to improve the predictive power of the model. This is referred to as feature engineering, and can include the transformation, creation, conversion, or other edits/additions to the data that will benefit the process. As an example, suppose we were estimating the cost of a house with only two fields, the length and the width of the house. We could help improve the model by feature engineering a new feature called square footage, where we would multiply the length and width.
The gathering and engineering of the data can be a difficult stage to get through, and without the right people on the team, it could lead to a wasted effort. Having domain knowledge on the team enables a more thoughtful consideration of what sources and features are important. In our research we have found many features that have predictive power for reserve setting. The following is a sample list of features that could provide value:
- Number of workdays
- Check runs in a month
- Large claims
- Inpatient admits/days
- Membership mix by product
- Change in duration
- Cost sharing components
- Place of service
Once the data has been prepared, the user will apply various ML models to the dataset. In general, there are two types of data, the first of which is called the training set, and the second the testing set.
The training set is the data that is used to train and cross-validate the model and comprises historical data (in the case of reserving, historical completed data). The testing data on the other hand includes only the data from which you wish to derive predictions (for example, the current month’s reserve data).
To evaluate the model, a portion of the training set is withheld in order to cross validate the results. The models that are identified to perform well on the withhold set are then applied to the testing data to create the predictions.
There are many different machine learning models, each of which has its own strengths and weaknesses. Thus there is no one model that works best on all problems.
For our research we used supervised learning techniques classified as regression. We ran various ML models and determined which ones were the most appropriate for the problem based on cross validation techniques. We then used an ensemble method to blend the various model outputs for an overall prediction. An example of this type of technique can be found in our prior article (Cap, Coulter, & McCoy, 2015).
These results were then compared against typical triangle-based methods, where we tested the percent range of UCL error over 24 different reserving months. Overall we found that ML added significant value in reserve setting, and we highly encourage reserving teams to explore this process for themselves.
Predictive analytics are not new to actuaries. Methods like these are fairly common on the casualty side and have recently become more popular within healthcare for predicting fraud, readmission, and other aspects. However, those within healthcare are often being led by Data Science teams, who continue to fill a larger analytics role within the health space. It is only a matter of time before these techniques become standard to reserving. The question is who will fill this role, will Actuaries stay at the helm, or will we transfer some of our functions to Data Science teams.
We hope that the process outlined above will provide some guidance, and at least prepare the actuary for their first step in this space.