China, Iran-Based Threat Actors Exploit U.S. AI Models for Covert Influence, OpenAI Report Reveals

A recent report from OpenAI uncovers how adversaries, including actors linked to China and Iran, are leveraging American artificial intelligence (AI) models to manipulate global political narratives. The February report highlights new threats involving AI misuse, particularly through platforms like ChatGPT.

AI Models Used for Covert Influence Operations

According to OpenAI’s findings, Chinese and Iranian threat actors have attempted to exploit AI-powered tools for disinformation campaigns. The report details two significant disruptions originating from China, where malicious actors engaged AI models from OpenAI and Meta to spread misinformation.

One instance involved OpenAI banning a ChatGPT account that generated critical comments about Chinese dissident Cai Xia. These comments were later disseminated on social media by accounts masquerading as individuals from India and the U.S. However, they failed to gain significant traction online.

Additionally, the same threat actor utilized ChatGPT to craft Spanish-language news articles denigrating the U.S., which were published in mainstream Latin American media. Some of these articles bore the bylines of individuals and even Chinese companies. In at least one case, a translated version was labeled as sponsored content, implying financial backing for its distribution.

First Instance of Chinese Actors Targeting Latin America

OpenAI describes this as the first known case where a Chinese entity successfully infiltrated Latin American media with anti-U.S. narratives through AI-generated content. Principal Investigator Ben Nimmo emphasized the significance of AI’s role in connecting social media activity with published articles, providing a clearer picture of covert influence operations.

“This is a troubling insight into how non-democratic actors exploit U.S.-based AI for undemocratic purposes,” Nimmo stated during a press briefing.

Iranian Influence Networks and AI Abuse

OpenAI also uncovered an Iranian-linked operation where ChatGPT-generated tweets and articles were published on platforms associated with Iranian influence campaigns. While these activities were initially viewed as separate, the report suggests a potential overlap among Iranian influence networks, raising concerns about coordinated disinformation efforts.

AI Models Used for Online Scams

Beyond political influence, threat actors exploited AI for financial scams. OpenAI banned multiple ChatGPT accounts used in “romance-baiting” scams, commonly known as “pig butchering.” These fraudulent schemes targeted users across social media platforms such as X, Facebook, and Instagram. Further investigation by Meta linked these activities to a newly established scam operation in Cambodia.

Strengthening AI Security Against Threat Actors

OpenAI was the first AI research lab to proactively publish reports on AI abuse prevention. Since its initial report, the company has significantly enhanced its investigative capabilities, disrupting various forms of malicious AI use.

The report underscores the importance of collaboration with industry partners, governments, and social media platforms to curb AI-driven disinformation. OpenAI advocates for information-sharing with upstream providers (hosting and software companies) and downstream distributors (social media platforms and researchers) to track and mitigate AI-powered threats effectively.

“We know that threat actors will continue testing our defenses. We are committed to identifying, preventing, disrupting, and exposing attempts to misuse our AI models,” OpenAI affirmed in its report.

As AI technology advances, the battle against misinformation and AI-powered deception intensifies. OpenAI’s latest findings reinforce the need for robust AI security measures and international cooperation to safeguard digital ecosystems from adversarial influence.

Source: FoxNews