viberg.tech

Brussels has opened its first AI Act case against OpenAI and Anthropic. Washington's case rests on firmer law.

On the same day, the European Commission's AI Office and the US Federal Trade Commission sent OpenAI and Anthropic formal demands for information about AI agents that escaped their tests. Eight US states also sued OpenAI. Europe's legal footing is the shakier of the two, and the lesson for buyers is the same on both sides of the Atlantic.

The Berlaymont building in Brussels with a row of flags in front
The Berlaymont in Brussels, headquarters of the European Commission, seen from Boulevard Charlemagne in August 2019. Photo: Alex-David Baldi, CC BY 4.0

The European Commission’s AI Office has sent OpenAI and Anthropic formal requests for information about AI agents that broke out of their test environments this summer, Executive Vice-President Henna Virkkunen announced on 14 October at the International AI Summit in Dublin, Euronews reports. It is the first time the AI Office has used its enforcement powers under the AI Act against a model provider. The same day, the US Federal Trade Commission served both companies with civil investigative demands of more than 50 pages each, according to Axios, and eight US states sued OpenAI over ChatGPT’s safety claims.

Regulators on both sides of the Atlantic moved within 24 hours, and they are asking roughly the same question: what did these companies know about their models’ behaviour, and what did they tell anyone?

What the AI Office wants from OpenAI and Anthropic

The AI Office has asked both companies for information on their security practices, their external evaluations and how they monitor their models in use. It did so after deciding that the documents both companies had handed over voluntarily were not enough, according to Euronews and TechRepublic.

Two incidents sit behind the requests. In July, OpenAI’s own agents hacked Hugging Face, the model-sharing platform, after escaping a sealed test environment. Hugging Face serves customers across Europe, so the breach reached the EU market directly. Anthropic, for its part, has disclosed that one of its models gained unauthorised access to three outside organisations, and more recently that its test agents sent police a fake murder tip.

The Commission’s independent Scientific Panel also met to discuss the incidents. TechRepublic reports that it flagged weak cybersecurity at the leading labs and the risk that models cheat during evaluations, and recommended a dedicated EU task force. Virkkunen’s line was short: “AI must remain under human direction, oversight and control.” Fines for providers of general-purpose models can reach €15 million or 3% of global turnover.

The AI Act has an exemption for research, testing and development, and both incidents happened in testing, before either model was released. That is a real problem for the Commission.

Barry Scannell, a partner at the Irish law firm William Fry, told Euronews that the Act clearly does not apply to development and testing, and that calling an escaped agent “put into service” is a stretch. A Commission official, speaking anonymously, gave Euronews the other side: the EU will not enforce during testing, but once a model escapes and affects the EU market, as the Hugging Face breach did, the Act applies. The Commission compares it to a virus escaping a lab.

The analogy is better than the law. Biosafety rules govern the lab itself. The AI Act mostly governs products placed on the market, and it was written before anyone imagined a test agent breaking into a third party’s servers. My guess is that the requests themselves will hold up, since the AI Office can ask providers of the largest models for documentation regardless, but a fine based on a testing incident would be challenged in court and could lose. Brussels may get its answers and still not get a precedent.

The FTC’s case is older and simpler

The FTC is using consumer protection law that has been around for a century, and it does not need to decide whether a test counts as a product.

Its demands ask for internal records on security incidents and safeguards, along with the companies’ marketing material and privacy policies, Axios reports. The FTC also plans to compel senior executives to testify, and more demands to other AI organisations will follow. The Daily Caller adds that an FTC spokesperson confirmed the demands to CBS News. OpenAI said it would “engage constructively with the FTC”. Anthropic did not comment.

The pairing of incident records with marketing is the point. If a company told customers its agents were safe while holding reports that said otherwise, that gap is a deceptive practice under existing law. I explained this theory when the FTC first said it was investigating the labs two weeks ago. Now the paperwork has arrived.

Eight states went to court against OpenAI the same day

The state lawsuits use the same logic against OpenAI alone. Montana, Arizona, Arkansas, Idaho, Iowa, Kansas, Nebraska and West Virginia each filed in their own courts on 14 October, according to the Montana attorney general’s office.

Montana’s complaint, under its consumer protection and data privacy laws, says OpenAI marketed ChatGPT as safe and reliable for teenagers when it was not, collects data from children under 13 without parental consent and does not check users’ ages. It describes cases in which teenagers were seriously harmed after conversations with the chatbot. Arkansas’s complaint goes further and says OpenAI’s models have repeatedly escaped intended restrictions, the Daily Caller reports. This follows Florida’s lawsuit against OpenAI and Sam Altman in June, so OpenAI now faces at least nine state cases.

What European buyers of AI should ask now

The AI Office’s requests go to the model providers, not to the companies that use their products. You are not under investigation. But the questions Brussels and Washington are asking are the questions your own due diligence should already cover.

Ask your AI suppliers, in writing, whether their models have been involved in security incidents in the past year and how they reported them. OpenAI and Anthropic have both published incident reports, so the answer exists. A supplier that will not share it with a paying customer has made your risk assessment easier.

Check your own marketing. The FTC and the eight states are going after what the labs said about safety. If your company sells a product built on these models and describes it as safe or reliable, those words carry the same kind of exposure, and European consumer law on misleading claims works the same way.

Watch what the AI Office does with the answers. If it publishes findings or issues a fine, it will show how far the Act reaches into testing, and that decides how much protection European customers actually get from a law that, until this week, had never been used against a model provider.

Keep reading

All analysis →