Key Takeaways

At a restaurant in Lisbon, a group of people arrived on the same night for what they thought would be a first date with a Tinder match. They ended up being part of an experiment.

The social experiment, which was conducted by tech startup Humanity Protocol, involved using AI automation to convince people — and platforms — that they were flirting with a real person, when all the while they were interacting with AI. 

Humanity Protocol’s AI successfully created fake profiles and sneaked around Tinder’s ID verification, an impressive feat considering Tinder’s high-profile Face Check tool. Of the 296 Tinder users who interacted with Humanity Protocol’s four AI-generated accounts, 40 agreed to meet up in person for a real date. 

The experiment put AI’s constantly evolving manipulation prowess into sharp relief.

The experiment’s goal wasn’t to humiliate online daters or even to call out Tinder’s safety measures, but to put AI’s constantly-evolving manipulation prowess into sharp relief. 

The experiment is a wake-up call to platforms on the fence about investing in AI. AI hackers are getting more realistic and more high-tech every day. For platforms, the challenge isn’t just accepting AI as the future of app development, but as the future of trust and safety, too. 

“I think the winning approach will be risk-based and privacy-first,” Terence Kwok, founder of Humanity Protocol, told DatingNews. This means platforms can best balance users’ need for privacy and safety by implementing strong safeguards throughout the matchmaking process, and while using as few humanity-proving techniques as possible. 

Attackers Often Strike After Onboarding 

Humanity Protocol’s experiment really took off during the matchmaking stage, when it used AI automation to carry on 100 conversations at the same time. “If a small team can do this as an experiment, imagine what coordinated bad actors could achieve with malicious intent,” Kwok said in the press release detailing the experiment. 

After all, many platforms have a similar approach to safety: The bulk of AI verification happens during onboarding, and then it’s up to the user to report suspicious behavior when they see it.

But Humanity Protocol found that attackers see onboarding as the main hurdle between them and potential victims. Once they bypass this hurdle — which many inevitably do — it’s much easier for them to make connections with innocent users. 

Attackers see onboarding as the main hurdle between them and potential victims.

“Our experiment showed you can still manufacture a believable ‘person’ end-to-end with widely available generative tools, then keep that profile convincingly active at scale,” Kwok said. 

Putting strong AI tools like liveness verification during onboarding is vital, but it’s not the only step platforms must take to protect users from AI attackers. Users are even more vulnerable during the messaging stage because their guards are down. They think, “What are the chances of someone sneaking through liveness verification?” 

Well, the chances are higher than we’d like to admit. Kwok explained that once AI attackers bypass ID verification, they typically “move on to the part that really creates trust, like consistent messaging, attention, and responsiveness” — all human qualities that AI is becoming better and better at replicating. 

A Balancing Act Between AI and Safety

If it wasn’t already clear, one-time liveness verification or optional ID checks can’t be a platform’s sole defense against AI scammers. 

The real challenge is for platforms to find a way for users to repeatedly prove their humanity without it becoming a hassle, an inconvenience, or worse, an intrusion into their privacy or data. 

“What makes this problem so persistent is that it’s technical, behavioral, and regulatory all at once, and the three keep pulling in different directions,” Kwok told us. It’s hard enough to strike a balance between AI innovation and safety; now platforms have to convince users to prove themselves as humans? 

The inconvenience that additional safety measures may cause sounds like a small price to pay for security, but people ultimately go to dating sites for quick connections, not to prove their humanity (“You can literally watch the conversion drop as you add screens,” Kwok told us). 

“What makes this problem so persistent is that it’s technical, behavioral, and regulatory all at once.”

Biometric data is the most personal info a platform can have about users (it doesn’t get more personal than a map of your face, after all). But it’s still not always as strong a tool as document-based verification, where a user provides a photo of a government document, such as their driver’s license, to confirm that they’re a real person.  

But as Kwok told DatingNews, this, too, comes with trust downsides, as there are many people who aren’t comfortable uploading a photo of their most personal ID documents to a dating site: 

“Dating apps live on a tension: you need enough certainty to catch impersonators and scammers, but push verification too far and you kill the spontaneity people came for,” Kwok explained. 

To Catch Attackers, Platforms Must Follow Behavioral Clues

The solution? Humanity Protocol suggests marketing repeated verification as a journey — a way users can feel safe over an extended period of time, and not just at the beginning of the onboarding process. 

Kwok praised Tinder’s Face Check as a step in the right direction. The user will first encounter the tool during the onboarding process, but it stores an encrypted map of the user’s face to ensure their face repeatedly matches up with their profile over time. 

“That kind of step-up control is directionally right, as long as it’s paired with strong privacy guardrails,” he said. If platforms want to zero in on bad actors or bots, they’ll have to look beyond ID verification. 

After all, AI attackers are always going to try to access your platform. On the bright side, they’ll also always leave a digital trail. Kwok recommends paying attention to how each account actually behaves. 

AI attackers will almost always leave a digital trail.

Does one person create new accounts at a suspiciously fast rate? Does an account sign in on one too many different devices? Is a single account carrying on an inhuman number of conversations at once? 

When uninvestigated, this behavior may not look too out of the ordinary. But Humanity Protocol is clear: Even a whiff of nonhuman behavior can be a sign that AI attackers have infiltrated your human-first platform. 

Or it could be a sign that a social experiment has unfolded on your dating app — and revealed your security vulnerabilities as a result. It sounds far-fetched, but it’s happened once before.