The AI Challenges Businesses Are Actually Focused On Right Now

September 18, 2026 · Episode Links & Takeaways

Author

AI Daily Brief
September 18, 2026

HEADLINES

Anthropic puts numbers on the pace of AI development

Two weeks into the safety discourse dominating the conversation, the thing almost everyone is asking for is specificity, and Anthropic has answered with a proposed three-axis measurement of how fast advanced AI is actually developing: AI's ability to build the next version of itself, the company's ability to oversee and intervene in agent actions, and the scale of resources going into model development. The headline figure from its new R&D Automation Index is that Claude now "leads" 26% of Anthropic's R&D work and "collaborates" on more than 90% — up from 1% led in March, 12% in May after Mythos, and doubling since. But the definition does a lot of work: AI-led still means a human sets the high-level goal, not the AI, and Anthropic is careful to say Claude "is not operating fully autonomously for any measured subset of AI R&D work." On oversight, roughly 30,000 agents are doing research and engineering work at any given moment, with claimed 100% coverage, a 0.002% real-time escalation rate, and around 50 of 100,000 weekly transcripts escalated to human review. The goal here isn't perfect measurement — it's a proposal for metrics any lab could report under a regulatory regime, and the rare move that most sides of the debate agree is better existing than not.

Recursive self-improvement isn't just a US lab story

Any slowdown discourse that doesn't include China is basically no discourse at all, and Z.ai supplied the reminder this week with a blog post titled "Toward Recursive Self-Improvement: How GLM Built Its Own Inference Infrastructure." The framing is striking: the team describes watching GLM complete an infrastructure task that would previously have taken experienced engineers weeks, and writing that "our successors are the AI systems we are creating ourselves." GLM-5.3 helped build the infrastructure running GLM-5.3-Flash, tripling inference throughput in under two weeks.

Bridgewater's Greg Jensen wants compute regulated like systemically important banks

Bridgewater CIO Greg Jensen told The Information that the major AI risk worth regulating is concentration of power, not the models themselves. He's uneasy about open source — Bridgewater runs its own reinforcement learning on powerful open models and acknowledges there's no way to track who else is doing what — but says that risk pales next to the frontier labs controlling 35% to 50% of the world's compute within two years. His proposal: designate anyone holding more than roughly 5% of US or global compute as a systemically important institution, with regulation and ownership caps to match. Applying G-SIB-style regulation to compute raises a week's worth of hard questions, but what's valuable in the framing is that it moves past simplistic binaries and asks where the actual locus of power sits — in the models, or in the power to run them. Those have very different intervention points.

The DOJ may carve out antitrust space for safety coordination

One of the practical sticking points in the slowdown debate is whether labs coordinating on safety implicates antitrust law — the theory being that a coordinated reduction in research spending raises profitability at the consumer's expense, loosely analogous to Apple and Samsung agreeing to stop working on new phones. Associate US Attorney General Stanley Woodward says the administration is weighing an update to interagency guidance that would extend the existing cybersecurity carve-out to AI safety. Curiously, despite multiple AI executives publicly calling for exactly this, Woodward says nobody has contacted his office about it. Even absent new guidance, he sees no problem: "It doesn't occur to me that coordinating on cybersecurity or security is anti-competitive." Europe's antitrust chief Teresa Ribera agreed at the same event, calling it a classic game theory problem where shared risk makes cooperation everyone's interest.

Andrew Ng calls extinction risk science fiction

Coursera co-founder Andrew Ng used a Bloomberg TV interview to push back hard on the tenor of the last two weeks, arguing that doomers have run this playbook repeatedly over the past decade to generate publicity and shape regulation. "I was quite dismayed over the past two weeks," he said. "This wave of PR has kicked up again, for probably similar purposes." He acknowledges real risks, largely around cybersecurity, but frames the dangers as "practical engineering problems" and calls the extinction framing "much more science fiction than science. It's very damaging."

Mistral hacked again, IP for sale on the dark web

Speaking of real, here-and-now cybersecurity problems, Mistral has been hacked for a second time, with hackers offering full source code, internal development files, web app code and additional proprietary information. In May, roughly five gigabytes of internal source code and 450 private repos were exfiltrated and put up for sale. An X user called Beni says they contacted the seller and confirmed the dump included model weights, post-training pipelines and dataset construction; the asking price was $25,000 before the post was deleted. Mistral later said it found no evidence of unauthorized access after a thorough investigation — suggesting this may be the May material coming back around.

Gemini 3.8 Live and the case for a voice front door

One release that went under the radar this week was Gemini 3.8 Live Extended Thinking, a live speech model that processes continuous conversation rather than turn-based exchanges. It topped the Artificial Analysis speech-to-speech index, beating GPT-Live-1 Astra and Grok Voice Think Fast 2.0, and adds near real-time visual input, automatic detection across 97 languages, and tool-call handoff so it can complete tasks in the background. Google developer relations lead Tim Messerschmidt demoed it powering a Reachy Mini robot through a conversation weaving between English and German. Greg Isenberg took the implication further, arguing 90%-plus of vertical SaaS will need a voice front door: the contractor says what went wrong on the job site, and by the time he's back in the truck the quote is sent, inventory is checked, the CRM is updated and the customer has been texted.

MAIN STORY

The AI Challenges Businesses Are Actually Focused On Right Now

The AI safety debate has completely broken containment over the last two weeks, with polling showing fast-shifting public attitudes and political resonance exploding. Enterprises, meanwhile, are stuck out here trying to work out whether any of it has real implications for them or whether they just need to keep on keeping on. The short answer is that the safety discourse barely touches the enterprise short term — but it does sharpen trend lines that were already forming, particularly around security spend and the case for owning your own models.

ACTUALLY CHALLENGES IN THE ENTERPRISE

The WSJ Technology Council Summit
Few fear extinction, half want a slowdown anyway
When the Wall Street Journal asked for a show of hands on Monday, only a few attendees said they were worried AI might kill us all — yet around half favored slowing frontier research and prioritizing better guardrails. The panel takeaway was that a slowdown carries few implications for how enterprises use AI right now, with FedEx Dataworks president Talwar saying "it's in our hands, and it's up to us to apply AI for good." The discussion centered on prudent governance around normal business risks rather than the existential risks dominating the media. This comports with the thesis that a slower pace of new development might actually increase what companies spend on AI, since relentless change disincentivizes comprehensive transformation programs that risk being obsolete before they finish.

Larry Fink
The data center backlash worries him more than x-risk
At the Canada Investment Summit, the BlackRock CEO was far more concerned about construction delays than existential risk, warning that the backlash could make AI the "domain of large firms." His framing: "The faster we can build out more capacity, the more we can democratize and make it available for everyone."

The startup response
Private markets are funding the answer to the concern
Founders are moving toward the governance and monitoring tools businesses will need as agents get more powerful, with one venture investor telling The Information they're now focused on startups building model security and infrastructure tech. Private markets responding to concern by funding companies that address the specifics of that concern is exactly what you want to see.

Microsoft's code of conduct
Don't surrender your proprietary data to the model providers
Microsoft used the moment to route the safety discussion back toward the AI sovereignty argument it has been making for months, dropping a 15,000-word code of conduct with the single overriding objective that humans retain meaningful control over AI. Most of the discourse fixed on its outright rejection of AI consciousness and its prohibition on designing AI to even imitate consciousness. But Satya Nadella's post pulled out the business implication: firms must retain full control over their unique and tacit knowledge, building their own continuous learning loops "without becoming dependent on any one model provider." Power concentration in the labs is partly a function of what enterprises hand over, and the way to deal with it is not to surrender proprietary data in the first place.

Ramp's spend data
Down 10% from July, and probably not for the obvious reason
Ramp's latest AI index found AI spend declined among the top 1% of businesses spending on AI, falling to $7,200 per employee per month in August, down 10% from a July peak of $8,000. The usual caveats apply — Ramp's audience is highly concentrated among tech-forward early adopters and its own products are about cost efficiency — and summer seasonality looks underweighted in the analysis. Ramp's read is that the most sophisticated users are getting adept at complex architectures that don't lean on the most expensive models at all times, with price cuts and a growing share of spend shifting to standard and light models. To Ramp's credit, its more recent reporting also found Fable 5.1 climbing quickly to 22.5% of enterprise spend after the data retention requirements were removed — which was always the obvious explanation for the earlier adoption numbers that generated so many frantic headlines.

Aaron Levie's field report
Cyber, agent identity, process re-engineering, evals, legacy systems
Box's CEO spent a week with technology leaders across banking, media, information services, insurance and consulting, and published what he heard. On security: "Everyone is nervous about the growing rate of vulnerabilities coming at them from AI and the implications of the OpenAI Hugging Face incident. The conversation is not as existential as it is in Silicon Valley, but still highly concerned and pragmatic." On agent identity, the tension is that enterprises want to issue identities to every agent and control what they do, but sometimes the agent needs to act exactly as the user. Worth noting that a lot of the security concern isn't about malicious actors at all — now that agents are in the hands of non-engineers, companies are finding the agents powerful enough to escape containment even when nobody is trying to do anything problematic.

Security software is where the budget is going
One category enterprises are increasing spend on
The concern shows up in the numbers. Ramp's data has AI security software as a growth category, with three trending vendors making software specifically designed to monitor agents in production in the wake of the Hugging Face hack. Those specific vendors might not have stopped that attack, but the category is clearly a focus for business buyers and startup builders alike.

Model battles and architecture churn
"We tried X and it didn't work, so have gone with Y"
Most companies are deploying multiple frontier models because standardizing is too hard and preferences differ across teams and use cases, though dollars stay concentrated in a few vendors. Open weights remain in infancy at scale, often for lack of domestic frontier open source options — plenty of appetite, few places to go. Paired with that is a ruthless adjusting of architectures: companies swapping systems out multiple times in the past year or two, because innovation moves fast enough that nobody waits around for a vendor to get it right.

Astra for Law and Cooley's Go Public
The labs want everything consolidated inside their environment
OpenAI and Anthropic are clearly trying to keep enterprises in their own environments, with OpenAI in particular pushing model prices down as far as they can go while building vertical solutions. Alongside the Astra for Law launch, OpenAI and law firm Cooley co-launched Go Public, designed to draft the S1 filings companies submit to the SEC before an IPO — a workflow Cooley has run thousands of times and has now codified into an agentic system.

Latham & Watkins buys its own servers
The perfect counter-example, from the same industry
As if to offer a clean comparison, the US's second largest law firm was reported to be buying Nvidia servers to stand up in-house systems as an alternative to OpenAI and Anthropic models. The rationale: "Sometimes we may have information that is so sensitive, client information that we really wanna protect, we don't wanna put it on any cloud vendor." One way for enterprises to stay out of the safety fray entirely is to own their own models — a point Mistral CEO Arthur Mensch made rather more bluntly.

Jaya Gupta on open weights as a SKU
"Pace the frontier may be the greatest invitation software incumbents have ever gotten"
Foundation Capital's Jaya Gupta argues that while the labs debate how fast intelligence should advance, software companies should be racing to commoditize the intelligence that already exists. Pharma and banks are picking up open weight models partly for margins and partly because a revocable lab API is a dependency they don't want. Her prescription: every major software company should become a model factory for its own vertical — own the evals, post-train open weights on the workloads it uniquely sees, serve those models to customers, and use production feedback to improve them.

The bottom line
The hardest path just got more defensible
The response from businesses to the safety discussion is largely business as usual, because the pace and challenges of enterprise adoption were always distinct and always weighed down by institutional inertia. But the conversation reinforces trend lines that were already forming: more time and resources on cyber and security, and more reasons to explore open weights and owned model alternatives to getting into bed with the big vendors — not least the increasing likelihood of regulatory disruption. The changing safety discourse doesn't much affect the enterprise short term, but it does mean companies willing to try the hardest things, like investing in their own owned architectures, have more potential to differentiate from their peers than they did before.