Key Takeaways

OpenAI has zeroed in on romance scams as a dominant misuse of its AI models, including ChatGPT. 

The AI research company detailed a specific incident in a report titled “Disrupting Malicious Uses of our Models.” Within the last year, OpenAI discovered that a Cambodia-based “network of accounts” used AI to create a fake dating agency called Klub Romantis.

When investigating the scam, OpenAI found that the scammers were combining human prompts with AI automation, a move it called “unusual,” most likely because scammers typically want the process to be more automated, not less. 

Blending AI and human intelligence suggests that some scammers are willing to sacrifice speed for more human-sounding and believable communication with targets. 

Klub Romantis may have been uncovered as a scam, but OpenAI posits that hundreds of men may still be suffering from the financial and emotional fallout. 

The Ping, the Zing, and the Sting

OpenAI broke the scamming process down into three steps: “The ping, the zing, and the sting.” 

The “ping” is when a scammer initially reaches out to the masses — a cold call, if you will. In Klub Romantis’ case, men who searched for terms like “golf” and “yachts” and “fine dining” were more likely to come across paid ads for the fake dating agency — that’s the ping. 

Clicking on the ads was only the beginning for these men. The scam deepened when they spoke to what turned out to be an AI-generated receptionist, whose warm, flirty tone endeared them to Klub Romantis. 

This describes “the zing,” or when the scammer attempts to pull at the target’s heartstrings, generate a spark, or otherwise motivate them to engage.

These increasingly-sexual receptionist personas, and later the authoritative “mentor” personas, were the keys to the scam, as they developed trust with the targets while convincing them to pay more and more money. 

After sending the “kill” sum  to the scammer — the biggest payment, AKA the sting — the victim would be blocked from contacting Klub Romantis. Consider them pinged, zinged, and stung. 

Texts Are Passé, but Social Media Ads Are the Future 

Klub Romantis also stands out for graduating from the tried-and-true SMS and email outreach formats to paid social media ads. This made it easier for Klub Romantis to target men of a specific age and lifestyle; in this case, middle-aged men with a taste for the finer things in life. 

With its targeted ads and hybrid human/AI tone, OpenAI described Klub Romantis as a scam network of the modern age: “This underscores the importance of studying the nature of threat actors and the ways in which they behave, as well as the content they generate.” 

Most of us already keep an eye out for texts, DMs, or dating app messages from potentially AI-generated sources. But OpenAI pointed out that social media ads are a breeding ground for threat actors. It’s a brazen move, but one that can pay off for the scammers, if OpenAI’s warning is any indication. 

Is ChatGPT a Scammer’s Perfect Target? 

Open AI has not been able to verify exactly how much money was lost in the Klub Romantis scam, or just how many people were duped. All we’re left with is an ominous question mark about the role AI models will play in future romance scams.

When I read OpenAI’s report about these recent AI scams, I got the impression that OpenAI is clinging to authority with increasingly slippery palms. Is its control waning as scammers become more deceptive and more sophisticated? 

It’s possible that ChatGPT and other LLMs are exactly the types of targets scammers look for. They’re wild, new, and desperate for engagement — at least, that’s how scammers may perceive them to be. 

But when OpenAI “disrupts malicious uses” of its AI models, it’s sending a strong message to scammers: We’re not going to allow ChatGPT to become a scammer’s playground… for as long as we possibly can, that is.