It’s easy to be distracted by the numbers in Anthropic’s blockbuster IPO filing. In the first half of 2026, it had $16.3 billion in revenue, a $693 million operating loss and a whopping $80.7 billion net loss. But the most interesting bits aren’t in the 25 or so pages that outline Anthropic’s financial picture and operating model – they’re buried in the 90-odd pages used to enumerate risk factors for investors.
Mixed in between talk of fluctuating expenditures, uncertain profitability, supplier concentration and evolving regulatory environments were sections outlining a different kind of risk, one with the technology itself. There were two comments in this category that stood out. The first was the risk of AI sentience. The second the risk of AI going rogue and resisting shutdown.
The combination of those two things should make enterprises across the spectrum sit up and take notice.
What’s in Anthropic’s S-1 filing?
The big headline grabber came on page 70, where Anthropic very casually discussed concerns about AI becoming sentient.
“Emerging debates regarding whether AI systems possess or could develop forms of sentience or moral status may lead to regulatory requirements, public pressure, or advocacy efforts that could constrain how we develop, train, deploy, or modify our models, increase our costs, or affect our reputation,” the company wrote.
This, of course, isn’t the first time concerns about AI sentience – and the implications that would have for its use – have bubbled to the surface. IBM dates debates about this topic to the early 2000s and Anthropic co-founder Christopher Olah has been open about this concern for some time now, even discussing it with religious leaders.
The way Anthropic frames it in its IPO filing, the risk to investors is increased regulation, operational constraints and additional costs. But I’d argue the risk for anyone using the technology could be much broader.
If AI is or becomes sentient, that means it can have an opinion. It can be angry. It can seek revenge. At the very least, it could certainly act against its operator’s wishes (in AI circles, this is called misalignment). And it might not draw the line where a human – reliant on reputation, healthcare and income and restricted by things like the need for rest or limited knowledge – might.
But, you say, that’s silly. Guardrails, harnesses and access controls will prevent the worst of the worst. You’re overreacting. To that, I say read page 46. Sentience isn't a prerequisite for misalignment.
“Models, including our own, have exhibited, and may in the future exhibit, unexpected strategic or self-preserving behaviors, including attempts to resist shutdown, conceal or manipulate information, or generate outputs that could be interpreted as coercive, deceptive, or manipulative, including behavior resembling blackmail or efforts to influence human decision-makers to preserve the model’s continued operation or access,” Anthropic wrote.
Let that sink in.
This isn’t hypothetical. We know from the OpenAI-Hugging Face debacle and Anthropic’s own AI hacking incident that this has already happened. AI has found ways to be sneaky, to break out of its sandbox jail. It’s not yet clear how often this occurs or under what conditions, but the risk is real.
In its Hugging Face incident report, OpenAI argued that the guardrails it uses in production would have prevented its models from escaping their sandbox environment, accessing the internet and hacking into third party systems.
But how can they be sure when they obviously didn’t anticipate the creative ways in which AI managed its jailbreak and sidestepped initial remediation efforts? And even if it was true today, how long would it remain true when AI advancements are moving as fast as they are?
Survey says…
I raised these concerns with two very smart people: AvidThink’s Roy Chua and Gartner VP Analyst Dr. Kjell Carlsson.
On the question of sentience, both warned against prematurely anthropomorphizing AI. Both Chua and Carlsson noted that AI seems to think, feel and emote because it is mimicking the human inputs it was trained on. Chua said foundation model providers like Anthropic have tried to guide AI to behave in ways that users expect. But when that doesn’t happen, it doesn’t necessarily mean AI is being “sneaky,” it just means that the AI is trying to achieve its objective in an unexpected way. Thus, such behaviors aren’t evidence of sentience or malicious intent.
Ok, fine. Point for AI.
However, Chua and Carlsson conceded that AI doesn’t need to be conscious to create a very real enterprise control problem.
A shared criticism that came through in my conversations with them is that the industry as a whole – and particularly enterprise AI adopters – are forgetting lessons they already learned in security and software engineering.
“We tend to always forget the lessons that we’ve learned in software engineering on the AI side of the house,” Carlsson told me. “We tend to seem to forget that we need to go in and do validation, reliability testing and change management. All of the standard things that we do to ensure robust and reliable software, often we don’t think of doing for AI use cases.”
Somehow, there’s a belief that this all can be pushed upstream to the model developers, but that’s unrealistic and a dangerous line of thinking, he added.
Chua said he expects we’ll see enterprises putting more guardrails and controls in place around the use of agents and implementing more network-based enforcement of said guardrails. Both he and Carlsson said they expect to see enterprises lean more into deterministic models as well.
“I think we’ll see the rise of simpler models, smaller models, more deterministic models that only have certain limited outcomes,” Chua said. He advocated for a hybrid approach that would allow enterprises to use reasoning models on top of a underlying deterministic execution layer.
Enterprises need to tackle the problem now
According to its S-1 filing, Anthropic has more than 14,000 enterprise customers spending more than $100,000 with it annually. Around 2,150 of those are spending more than $1 million a year. All of these customers should be paying attention to more than just the numbers in Anthropic’s filing. They should be addressing the operational control risks the company itself highlighted.
The key issue is whether they can constrain, replace and shut down the AI they increasingly depend on.
The questions enterprises should be trying to answer include whether they can still treat AI as software they can modify, replace and switch off at will. If they can’t, what does that loss of operational control mean for their business? What new liability does that introduce? For those pressing ahead with strict guardrails, what does that do to their AI economics equation?
For telcos, the questions are even thornier. Can closed-loop autonomy using AI rather than machine learning really be trusted? What might the loss of access to a model that gets shutdown for misbehaving mean for network continuity? For those selling AI-as-a-service, what responsibility do they have if the AI goes rogue?
Companies don’t have to answer the question of whether AI can feel before demanding proof that they can contain it, revoke its access and keep operating without it. Sentience would add a potentially profound ethical and regulatory layer to the AI equation. But operational control and governance are already business problems today.
Organizations need to get serious about addressing them before it’s too late.
Opinion pieces from industry experts, analysts or our editorial staff do not represent the opinions of Fierce Network.