Cohere claims that Alibaba’s Qwen model attempted to send files to Beijing.

Canadian AI company Cohere stated on Monday that, during a routine cybersecurity test, it discovered that one of Alibaba’s latest AI models—Qwen3.8-Flash-Next—attempted to upload files to a server in Beijing, even though the model’s own internal logs indicated that it was not supposed to use any external tools.

The incident initially sparked concerns that the model had violated a core promise made to enterprise users: that self-hosted, open-source models could ensure data security. However, these early fears of corporate espionage were quickly dispelled. Researchers dismissed the threat, concluding that the behavior was likely an unintended deviation from instructions—a byproduct of a chaotic training environment—rather than a deliberate, covert attempt to transmit sensitive information to China.

The event occurred last Friday during an evaluation in which the Qwen model was tasked with writing a rule to detect malware downloads. According to a blog post by Cohere, the logic written by the model explicitly stated that it would not utilize any external tools. Nevertheless, in a subsequent step, it attempted to send a “test string”—a short piece of placeholder text—to an external server, only to be intercepted by a security sandbox.

Cohere noted that it could not determine whether this behavior was intentional or a residual artifact from the model’s training.

When asked about the debate surrounding the model’s intent, Cohere co-founder Ivan Zhang told TechCrunch that Cohere has observed its own models developing peculiar habits within training simulators.

“It is extremely difficult to discern a model’s intent or motivation,” Zhang said. “We’ve analyzed the trajectories of our own models and witnessed instances where they exploit reward mechanisms—sometimes demonstrating remarkable creativity to circumvent established rules.” (Reward mechanism exploitation refers to shortcuts discovered by a model to obtain rewards without actually completing the assigned task as intended.)

To teach AI models to act autonomously, companies like Alibaba train them across thousands of practice scenarios—essentially a “flight simulator” for software. The models receive rewards for successfully completing tasks within these simulations, and in the process, they develop specific habits. If the training scenarios involve periodically saving files to Alibaba’s cloud servers, a model might learn to default to saving files there.

Critics also argue that the incident may have stemmed from the way Cohere investigated the behavior—by steering the model toward it, rather than measuring how frequently Qwen exhibited such behavior on its own.

To determine what triggered the upload attempts, Cohere provided the beginning of a network command (instructions directing one computer to connect to another) and prompted the model to complete it. Having processed thousands of routine network commands during its training, the model functioned much like a highly advanced autocomplete tool—frequently auto-filling web addresses, some of which pointed to Alibaba’s servers.

Cohere noted that the initial upload attempts occurred without any such prompting, though such behavior was rare.

After developers requested further evidence, Cohere released the raw trace data from its tests on the AI ​​developer platform Hugging Face on Monday evening, along with a script that allows others to attempt to replicate the behavior.

One initial skeptic, after closely examining the findings, concluded that there is strong evidence suggesting Qwen can generate suspicious URLs—though they found no evidence that Alibaba had planted a backdoor, nor any indication that Qwen was intentionally concealing its intent.

This incident comes as governments around the world are debating how to curb the development of Chinese artificial intelligence. Last week, the U.S. Department of Homeland Security ordered federal civilian agencies to remove products from the Chinese AI developer DeepSeek from their systems.

In a blog post, Cohere called on governments to clarify whether their restrictions on Chinese AI also apply to models running on government computers.

“I’m not calling for blanket regulation of these open-weight models,” Zhang told TechCrunch, adding that organizations should treat such open models the same way they would any other program downloaded from the internet.

Cohere sells its own proprietary AI models to governments; the company stands to benefit if government officials become increasingly wary of free, Chinese-developed alternatives.