Why would you need to prove the consistency and accuracy of a decisioning system? I can think of several parties for whom it is critical to delve deeper.
First, of course, are lenders. Lenders must be able to measure their ability to price risk. Second, the Financial Services Authority needs to be shown that auto-decisioning treats customers fairly. Third, funding providers – be they warehouse banks, portfolio purchasers or securitisation investors – must know the loans they are funding are created with due credit control. So it is critical that auto-decisioning performance is transparent.
But if you need to measure the performance of an auto-decisioning system, how do you do it? Human underwriters’ decisions are based on simple factors. A debt to net income calculation is about the limit of complexity, so if a human decision looks odd you can question it and probably understand the answer.
Machines can only do what they’re told. But if they’re told what to do by a small group of intelligent, computer-savvy credit specialists, even if you know enough to ask the right questions the answers you get are likely to be complicated.
If an affordability model and a credit score assimilates dozens of data items per case and applies algorithms to generate seemingly arbitrary scores, how can you be sure a decision to lend is good?
Performance is the obvious measure of a successful credit policy but this only applies over the longer term. A year or more of data is required for meaningful conclusions to be drawn. On an everyday basis, the only way for lenders to police their decisioning systems is with robust audit and quality control processes.
Checking that the functioning of black box systems is compliant has not been advertised as a concern for the FSA yet but I am quite sure this will appear on its agenda as the popularity of auto-decisioning grows.
Making funders comfortable with auto-decisioning is an interesting area. If funders approve product criteria and general lending policy all should be well if the decisioning system writes business to specification. But I expect to see more interest being taken in the workings of decisioning systems.
Traditionally, the decision process has been carefully guarded by lenders, and rightly so. But lenders must bend a little and allow more scrutiny of their processes to boost volume and liquidity across the industry.