Scammers are very good at impersonation. So when the phone rings and the person calling claims to be a debt collector, you need to be suspicious—even if they have a lot of personal information about you. It could be a con artist running the "phantom debt collector" scam.
These telephone swindlers often pretend to be with a law firm, government agency or police department. Victims say the callers can be mighty aggressive.
"They might threaten garnishment of your wages or seizure of your assets, all the way up to arrest and jail time if the consumer does not pay on this debt right away," said John Breyault, who runs the National Consumers League's Fraud.org website.
These phone bandits commonly target people who've taken out—or simply applied for—an online payday loan. They sound credible because they have all the personal information needed to apply for the loan. They probably know your bank and they might have all or part of your Social Security number.
"The fact that they have this incredible amount of personal information is part of the reason why people pay them," said Elizabeth Scott, an attorney with the Federal Trade Commission. "Victims are convinced that only someone who they legitimately owe money to would have this information."
Thousands of people have complained about phantom debt collectors. The FTC has already filed four cases involving fraudulent collection of online payday loans. The total loss from just these cases is estimated to be close to $20 million.
Maria, a victim in one of these cases, gave a sworn statement to the FTC that she and her husband received numerous harassing calls at home and work from a Mr. Hunt. She said he screamed at them and made numerous threats about a payday loan debt they did not owe.
Read MoreBig data knows you're pregnant (and that's not all)
"He claimed that his company would see me in court and that I should hire a good attorney; and that the police would arrive at my house within a half hour if I did not pay the debt," she said in court papers. "Both my husband and I were very scared."
Maria and her husband had never taken out a payday loan. She had been used as a reference by someone who applied for an online cash-advance loan.
These phantom debt collectors often say you owe thousands, but they're willing to settle for a couple of hundred dollars. They typically want payment by wire transfer or Green Dot MoneyPak.
"That's a red flag," Breyault said. "This is not how a legitimate lender accepts payment for your loan."
Protect yourself
It's not easy to tell if a site offering payday loans is legitimate or run by crooks who want to snag the personal information you provide to use for fraudulent purposes. That's why fraud fighters urge you not to apply for a payday loan online.
Here's what to do if a phantom debt collector calls:
- Tell the caller you will not do anything until you get a written "validation notice" that spells out what you owe and to whom, and provides information about the collector. This notice is required by law. Anyone who says they can't send it is a fraudster.
- Never send money to an unknown caller who demands payment of a loan—or provide any additional information about yourself—even if you owe the money. If you're not sure if you have an unpaid debt, talk to your original lender. Find out if there's a balance due and/or if they sent you to collection.
- Don't let them scare you into doing something foolish. You cannot be arrested for being delinquent on paying back a loan. You can be sued, but the police are not coming to get you.
"It's hard when you're being threatened or harassed, but slow it down," Scott advised. "Hang up the phone and take the time to see if you do actually owe someone money and if the person calling really represents the company to whom you owe the money. You may owe money, but it may not be to the people who are calling."
Read MoreWhy some of us are more vulnerable to online fraud
If you get a call from a phantom debt collector, file a complaint with your state attorney general's office and the Federal Trade Commission. This is how prosecutors build a case and go after these crooks. The FTC has information about phantom debts and fake collection notices on its website.
—By CNBC contributor Herb Weisbaum. Follow him on Facebook and Twitter @TheConsumerman or visit The ConsumerMan website.