Could You Be the Next Victim? Harvard Highlights the Rise of Convincing AI-Generated Phishing Emails

“I’d never fall for suspicious spam emails written in awkward Japanese.” If that’s what you’re thinking, you may already be the perfect target for the latest generation of scams.

News headlines are filled every day with reports of the astonishing advances in artificial intelligence (AI). Its capabilities have become so powerful that concerns about malicious use continue to grow. In fact, efforts are underway both in Japan and overseas for governments and major global corporations to strengthen cooperation in preparation for cyberattacks that exploit cutting-edge AI.

For example, highly advanced AI systems such as Claude Mythos, developed by the American startup Anthropic, are said to be capable of instantly identifying vulnerabilities in IT systems, including computer operating systems (OS). While AI’s convenience offers tremendous benefits to society, the risk of misuse is always present.

Against this backdrop, one particularly thought-provoking report serves as a warning for our everyday lives. Titled “How to Avoid Being Deceived by AI Scam Emails,” the report raises a stark alarm: “A Harvard experiment shows that we have entered an era in which one in every two people can be deceived.”

According to the report’s author, Yu Kashimura, Chief Researcher at the Daiichi Life Research Institute, suspicious emails with subject lines such as “Congratulations! You’ve won!” are a thing of the past. Today, he argues, legitimate emails and scam emails have become so sophisticated that they are virtually indistinguishable.

A shocking reality: half of people are deceived by AI scams

Let’s first examine the findings of an experiment conducted by a research team at Harvard University to assess the dangers of scam emails generated with AI. In the study, 101 participants were randomly divided into four groups, each receiving a different type of email. Researchers then measured the percentage of people who clicked on a malicious link attached to the message.

The four groups received: (1) conventional spam emails, (2) emails created by experts (professional humans), (3) emails generated entirely by AI, and (4) emails generated by AI and then refined by humans.

The results were striking. While only 12 percent of participants who received the conventional spam emails clicked the dangerous link, the figure rose to 54 percent for both the expert-created and fully AI-generated emails. For the AI-generated emails that were edited by humans, the rate climbed even higher, reaching 56 percent. In other words, AI scam emails have already become as sophisticated as those crafted by professional fraudsters, and we have entered an era in which one in every two people may be deceived.

An even more alarming fact emerges when examining the study more closely. The participants were university affiliates from a variety of specialized fields, including the natural sciences. Compared with the general public, these individuals are typically regarded as possessing high levels of information literacy. Moreover, these findings are already two years old. Since then, AI technology has advanced dramatically, including the emergence of systems such as Claude Mythos introduced earlier.

Sophisticated tactics that exploit weaknesses through social media

Regarding the evolution of scam emails, Kashimura explains:

“In the past, identical emails were sent to large numbers of people in the hope that a few recipients might be deceived. Recently, however, the trend has shifted toward personalized emails targeting specific individuals.” (Kashimura, hereafter)

What exactly does this mean?

“AI gathers information, organizes it, and makes highly sophisticated inferences.”

As Kashimura describes, today’s AI can automatically search the internet using a target’s name as a starting point. It systematically collects publicly available information, including social media posts on platforms such as Facebook, self-introductions on company websites, and past news articles.

It then analyzes that information to determine what kind of work the person does, what interests they have, and what activities they engage in. Ultimately, the AI compiles a detailed profile that identifies the individual’s vulnerabilities and the factors that make them more likely to trust certain messages.

The Harvard research team also interviewed participants to understand why they had believed the AI-generated scam emails were genuine. Responses included comments such as, “The content was relevant to me,” and, “It felt like a special email addressed specifically to me.” These findings revealed just how sophisticated such messages have become, to the point where almost anyone could find them convincing.

For example, after referencing the name of a project the target recently worked on or mentioning their area of expertise, an email might arrive saying:

“We would greatly appreciate your cooperation regarding this matter.”

Faced with wording like this, it becomes extremely difficult to remain suspicious.

Furthermore, sophisticated AI scam emails are now being sent from overseas in fluent Japanese. In the past, awkward Japanese phrasing or the inclusion of Simplified Chinese characters often made it easy to recognize a scam. Those days are over.

Must-read! Five ways to defend yourself against AI scams

According to Kashimura, most of these scam emails involve money-related matters, such as investment solicitations.

“They exploit human desires and psychology.”

With this observation, Kashimura outlines “five countermeasures” to protect yourself from scam emails.

Five ways to protect yourself

１＿Be suspicious even of emails from people you know: Even if the sender claims to be a parent, sibling, or spouse, you should be cautious if they suddenly say things like, “I urgently need a large sum of money.”

２＿Do not click links or buttons: There is a high possibility that they are traps designed to redirect you to fake websites that closely resemble legitimate ones.

３＿Use official channels to verify information: Do not use the contact details provided in the email. Always confirm through official routes, such as the organization’s official website.

４＿Review your social media privacy settings: Limit the visibility of information that can identify you personally, such as your employer, job title, recent travel destinations, and hobbies, as a form of self-protection.

５＿Enable multi-factor authentication: Rather than relying solely on passwords, strengthen your security by using multiple methods of identity verification, such as authentication codes and biometric authentication.

Even voices can be faked? The terrifying evolution of AI

Recent advances in AI have made it possible to accurately imitate human voices and even carry out fluent conversations.

“Voice technology is what’s truly frightening now.”

As Kashimura explains, there is a growing risk of receiving fake phone calls from people impersonating relatives or acquaintances using cloned voices, making it easy for victims to believe they are speaking to someone they know.

Finally, Kashimura offered the following warning regarding advanced technologies such as AI and the current state of Japanese society:

“Many Japanese people have low levels of IT literacy. We should begin properly educating children about AI from the elementary and junior high school stages.”