In search of the honest credit repair shop

FTC made the headlines recently complaining about inaccurate consumer credit reports. The Wall Street Journal (link) has a typical report on this research. Here's their summary:

In the FTC study, 262 of the 1,001 people who reviewed their credit reports spotted at least one potential "material" mistake, such as a credit-card account that wasn't theirs or a late payment that they didn't believe was late.

That sounds like a worrisome 25% error rate.

***

In Chapter 2 of Numbers Rule Your World, I explain why focusing on reporting inaccuracies is counterproductive, particularly and ironically for those who have good credit. The reporters clearly haven't read the 370-page report the FTC has put on line.

If they did, they may have noticed the chart shown on the right (in the main document). The chart shows the proportion of consumers in their sample who have all the disputes denied (green), who have all the disputes deniedaccepted (blue), and those who have some of their disputes accepted (yellow).

One feature of this chart that can easily get lost is the uniformly short columns corresponding to those with high credit scores and the much taller columns for those who bad credit scores.

Those with low scores are much more likely to find mistakes in their reports than those with higher scores.

If the reporters flipped to Appendix D, they would have seen the following table:

Forty-five percent of those with credit scores below 590 claimed at least one potential mistake while only five percent of those with 790 or higher credit scores did.

What kinds of mistakes are being reported? Refer back to the WSJ quote above. A late payment that may not have happened will be disputed - what about a late payment that the credit bureau didn't know about? Would the consumer volunteer that information? Similarly, if the account not belonging to the consumer has great credit, thus pulling up one's credit score, would the consumer ask the bureau to take it out?

If the FTC encourages consumers to dispute credit reports, the end result is that the credit scores become less accurate, not more accurate. The dispute process introduces a bias so that the reports tend to contain mistakes that cause overestimation but not those that cause underestimation.

Worse still, this policy penalizes the good people, as it allows less creditworthy consumers to unfairly inflate their scores. In general, it hurts the predictive power of credit-scoring algorithms.

***

It's ok for the credit reports to have errors so long as they are unpredictable. This means that positive errors are as likely as negative errors.The following table is overwhelming proof that the credit dispute process introduces bias:

Pretty much all disputes are resolved to the favor of the consumer. I don't think the credit reports only contain errors that penalizes (low-scoring) consumers.

***

Final question: we know there are credit repair agencies who can help you correct your credit scores. Have you seen any of these agencies advertise as the honest agency who will help lower your credit score to make it more accurate?

 

Comments

[Kevin Henry]

You claim that resolving mistakes introduces bias; but this requires an assumption about the underlying distribution of errors. Yes, if positive and negative errors are equally common then only correcting negative errors will introduce bias. But if negative errors are more common than positive errors, correcting negative errors will tend to reduce bias.

So I don't think you can support your conclusion without giving evidence about that underlying distribution.

[Kaiser]

Kevin: When there is a mix of positive and negative errors, they tend to cancel each other out. Correcting only the negative errors will expose the positive errors, and thus increase bias on average.

In my book - as other credit scoring books - I look at the types of algorithms used to make such predictions. With an understanding of how it works, I think it's even more difficult to believe that these algorithms only make errors in one direction.

[Ken]

A point as well is that what the credit agencies are doing is similar to what happens when analysing large administrative datasets when exact identifying information is removed, and we are left with things like date of birth and city. It then becomes a case of deciding how close do the identifiers have to be to considered the same subject. Choose wrongly and the analysis becomes pointless either because we miss a lot of matches or we have lots of spurious matches.

[Jack]

If credit scores become less able to distinguish good and bad customers then that is a good thing. Credit scoring amplifies existing inequalities by tending to reward people who are already well off whilst penalising those who are not.

[Kaiser]

Jack: The evidence is strong that credit scoring technology has vastly expanded the number of Americans who are able to get credit. Credit is not a right, it's a responsibility. Expanded credit has absolutely increased the average quality of life in America.

That credit scoring is not perfect is not a reason to reject the technology. One must compare it against the alternative. Before credit scoring was human scoring. Does human scoring not "reward people who are already well off"? I cover all this and more in Chapter 2.

[Ari]

'It's ok for the credit reports to have errors so long as they are unpredictable.'

This is true in the aggregate; it sucks for the hapless individual with negative errors; its a boon for the person with positive errors.