The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one.
The bottleneck is compute and data, not the model. That's why they could only gate it for a bit. The ITAR thing proves it: no nationality controls in place, so the only option was killing the whole thing. Not exactly what an all-powerful gatekeeper does.
Regulatory capture is the OpenAI and Anthropic end goal, for certain.
But I also think they exist in a sort of un-designed corporate narcissism, which is a common trait in bubble economies — I am not judging them particularly severely.
Netscape under Clark and Andreessen and Sun under McNealy both fell into corporate narcissism: the belief that only they really mattered, that they were chosen, and that the world needed to rearrange itself to just let them shine. They arguably let themselves get played by Oracle (a corporate psychopath) and others as a result.
OpenAI's position is profoundly corporate-narcissistic: all we need is all the money in the economy and not to have to do anything upsetting like think about turning a profit for the next four years. Like rich kids. It would be nice if you believed we were so important that we should get an enormous stipend for just being us.
Anthropic's position is: we think we're so unique and ominous that government needs to make us both essential and terrifying. We have to exist otherwise worse people will.
HN is the builder side of the conversation, and in my experience, few safety people congregate here.
The safety side of tech is a PTSD inducing shit show. Governments are more than happy to champion age verification laws, because parents, around the world, are clamoring for anything to pump the breaks on the social media experiment.
Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.
> Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.
I don't think anyone in tech is really truly engaging with how quickly the shine has come off the tech industry. Except maybe Apple, who even so still have some work to do.
Technology and science is the intersection that is supposed to make our lives better, easier, more prosperous. The last decade or two what marvelous technology has came from silicon valley that hasn't served primarily the billionaire class and made life worse for the common people.
The yoke of silicon valley is feeling heavy. People might just throw it off.
As someone on the "safety side of tech", social media is being exploited to increase surveillance and government control precisely because its actual social influence is heavily on the wane, and capital is happy to sacrifice what's left to increase the profits of the expanding public/private tech surveillance industry (with "protect the children" controls on social media like age verification being the usual backdoor route it always is).
Society may be growing tired of Tech, but governments aren't, and in fact they're heavily expanding their back channel reliance on not-traditionally-military Tech as an extension of their Defense spending.
Cyber security has the maturity that trust and safety hopes to achieve at some point.
Social media was being exploited from inception. Palantir had sales documents for sock puppet management software back in the PHP era.
I don’t disagree that Government is interested in tech, but I will push back on the dismissal of child safety that is inherent in your comment, intended or not.
For all that some people in the firm may have tried to do the right thing, Social media firms have created bad outcomes for children, and executives were briefed on the harms they were going to cause.
This is the dismissal that concerns me, because it ends up miscalculating the level of anger and unhappiness amongst the voting populace, and therefore the political will to pass regulation to reign tech in.
The political will is already captured and redirected.
There are numerous bills to limit AI access for consumers, to combat deepfakes hurting children. There are no bills introduced or passed to prevent AI being used to target dronestrikes that kill children abroad, or surveil children domestically.
What the public wants doesn't actually matter right now, only what the government will allow to let pass, which in this case is additional internet surveillance.
Under a future, better government this may change, but (sadly) nothing is going to sink tech's dominance right now.
The anger and unhappiness against tech is good, and hopefully someday they'll burn down all the data centers and I'll never have to hear the words Cloud Computing again, but (to paraphrase a famous Eve Online interaction) it's not going to be today, and it's not going to be us.
> Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.
s/Tech/Tech Companies
Tech did it to themselves. People like and want technology. What they don't like and don't want more of is enshittified, user hostile technology. The answer is out there, but our collective school systems failed to teach computing irt free software/open source and instead schools themselves all bought in on enshittified, proprietary tech, or even just dumped trying to teach computing at all outside of "how to login to google classroom and google docs"
I grew up lucky, in that my dad was a dev, my first PC as a kid ran red hat, my high school had an intro to programming class (in BAISC lol). It shaped how I approached computing growing up, and my values. It makes me look at the things we have now and think "No, you're just repackaging community free software and selling it back to me, I'll pass on that."
That experience isn't available to anyone born after that specific era, instead their tech experience is shaped by walled gardens, vendor lock-in, and straight up hostile and manipulative software, so its no wonder they are tired of it. They don't even know a different world (of software) exists.
Spot on. There's a certain level of drinking the kool-aid or getting high on their own supply. Anthropic is a lot worse than OpenAI but OpenAI had to go through rounds of shedding.
To be maximally fair to them, I think it is difficult to be one of the key businesses in a market bubble and not fall victim to this kind of thinking, especially when the continued inflation of the bubble depends on you — lots of people lose their shirts if you don't push hard to be "special".
But as you say, there is a measure of getting high on one's own supply now.
And there's the curious solipsistic energy of Sam Altman whimsically musing in public that it turns out his product is too expensive for people and they complain when you make the price realistic (when it possibly needs to be more expensive for OpenAI to survive).
They seem to believe that the ordinary rules either will not or somehow must not apply to them; it's increasingly bizarre to watch.
Maybe the people around pets.com were this bizarre; we didn't have so much livestreamed interview content to show us.
A\ and OpenAI each have their own unique kind of nonsense. I think OpenAI has just been less successful with persuading the rest of the world that they should have all the money in the world.
Anthropic has been surprisingly successful at convincing them that they should control frontier models because they're so dangerous that... only Anthropic can be trusted with them.
(If they're really so dangerous, the right way to deal with them is through a democratic process and taking them out of the hands of a for-profit private entity.)
what Dario wants is to retain any influence whatsover on how the research progresses before the inevitable nationalization of the frontier. he gets to keep the N-2 tech and maybe influence the N-1 tech, but the only influence on the frontier he has is today; whatever he imprints in the pipeline the government takes over.
IOW I don't think he thinks in the same categories as most folks here.
Best-possible-model (N) - Two Generations (2), same with N-1, N is the SOTA in this example. I'm not sure that actually clarifies what the comment is trying to say other than they think the models will be nationalized (can't even imagine what that would look like).
> ...the research progresses before the inevitable nationalization of the frontier.
Hacker News has been telling me America beats China at "innovation" because of the "freedoms" - especially frew enterprise. I wonder how a nationalized frontier lab would perform.... Andhow the non-citizen researchers would feel about working for the US government that doesn't trust them to use frontier models.
But he already got it, no? Claude Fable can only be made available to US citizens, which implies that every user who wants to use Claude Fable must provide proof of citizenship in some way, basically KYC.
> The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one.
But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems like some do, most don't, Anthropic and Claude remains very popular among the people who use LLMs, there is no denying that.
I have a question that perhaps you or someone else here has an answer for: I enjoy using Opus via Google Antigravity (usually agy) for perhaps 90 minutes a week. For Google’s subsidized $20/month plan they seem to give out a reasonably generous amount of Claude tokens. How does this compare with Anthropic’s $20/month plan using Claude Code?
BTW, I also use DeepSeek v4 Flash very frequently: fast and so cheap it is almost free.
Anthropic's plans are based on user experience of usage, not raw token counts, so you get to run through so many conversation turns, etc. within a 5 hour usage window. (Cursor, OpenCode Go, and others are similar.)
Cursor's $20 a month plan provides a reasonable amount of Opus tokens as well.
It’s really hard to translate minutes to tokens, it depends on how you’re using it.
The best answer would be to pull session stats from your harness and compare that against the limits. I think Anthropic publishes the limits of each plan.
If you’re using a pretty stock harness and not doing crazy multi-agent stuff with it, you’re probably fine.
My girlfriend built a whole (but simple) React app with it and only hit the limits of the $20 plan once. In fairness, she was trying to get it to clean up a bunch of 800ish line React files at once with a vague “make it look nice” prompt that she ran a few times. I think it was just churning for like half an hour straight before she burned all her credits.
It’s probably enough if you’re not on a fully agentic development strategy, it’s plenty to have it write tests and do comments and stuff, just not enough to continually have it doing giant refactoring passes.
>citing that it's illegal even though I own the games
In the. US at least it is actually illegal to download ISOs/roms of games, even if you own a physical copy. It's a stupid law and as a downloader (as opposed to the people hosting the files) your chances of getting into any kind of actual legal trouble are effectively 0, but it is still against the law.
> does the users who use Anthropic switch over to those even if they're available even as hosted models?
I'm currently spending $200 for Claude. That's around my maximum that I can afford. I could stretch that to $500 I guess. But I saw reports of people spending tens of thousands of dollars with Claude API. That's certainly outside of my budget.
So if/when Anthropic decides to stop subsidizing subscription (if they ever do that thing, I still not sure about that), I'll certainly look at the other options. And available "open weights" LLMs hosted by someone will be my first pick. Right now Claude 4.8 feels very advanced, but things move very fast...
The ai labs would be very dumb to get rid of subscriptions. First, I don’t even think the subscriptions are losing money, I suspect they’re around break even, maybe small loses. More importantly, the subscriptions are how they lock in users and convince companies to pay api rates. Without user loyalty that they cultivate with subscriptions businesses will just use the cheapest model on open router or maybe local models.
For Claude specifically, (1) enterprises pay API rates on top of subscriptions, so subscriptions profitability questions are only relevant for smaller companies and indie devs. Many of whom probably have sporadic or low usage which helps balance some heavy users.
Again, for Claude, (2) it’s rumored that their API rates have around a 90% profit margin. It’s also claimed that the subscription limits get you around 10x tokens per monthly dollar vs buying them with API rates.
Edit: to drive it home. If a tokens true cost to anthropic is 1/10 of what they sell it for at API rates, and a subscription gets you tokens at 1/10 the price, that’s cost-neutral for the business if every subscription uses every token. They’re selling tokens at cost, not at a loss. Many subscription users won’t use their full allotment. That means serving some users doesn’t cost the business as much - which might push the subscription business from cost neutral to profitable.
I think theyre charging at least 3x marginal cost for the api and I think that the average subscription uses around half its token allotment. So subscription needs to give 6x as many tokens as the api would cost for it to break even for the lab and that's around where they tend to sit
I don't think you're appropriately understanding the full gamut. The individuals who only spent $200/months will be stuck. But the pie is increasing in size, it's not stagnant. There are a lot of orgs who can afford to run a 1T model and even more that can run a 600B model. These newcomers are what's being fought over
The hotness we are seeing is smaller 'expert' models with an 'orchestrator' model in front that evaulates the prompts and routes to the appropiate small models and then synthesizes the collected answer. Easier to split across many smaller, cheaper servers and more efficient than a huge monolithic model.
Do you have more info about this? I can't tell if you're being misled by the unfortunate "Mixture of Experts" terminology (which don't work the way you're describing), or alluding to something different.
Or, maybe I'm wrong, but my understanding is: MoE is just an architecture to keep the activated weights smaller per token. The experts get routed basically token-by-token, and the "experts" themselves don't have a semantic domain so the "expert" word was maybe a poor choice.
"The sparsely-gated MoE layer,[21] published by researchers from Google Brain, uses feedforward networks as experts, and linear-softmax gating. Similar to the previously proposed hard MoE, they achieve sparsity by a weighted sum of only the top-k experts, instead of the weighted sum of all of them."
"Top-k experts," in case of some DeepSeek's models k=1.
No, this is an agent-level thing, not a feature of the model (ish, unsure for Fable).
You talk to a smart, heavy model to build a plan composed of smaller steps. Then you have the heavy model spin up smaller, cheaper LLMs to actually implement the tasks.
The heavy model is basically read-only in that mode. It can read files, execute tests, etc, but it can’t write code. It just tracks what needs to be done, offloads the work to dumber LLMs, validates the task is done, and moves on to the next step.
It sort of pushes humans up the stack. Instead of having a human sitting there prompting the LLM to start the next task, you have another LLM do that loop.
People dont pivot on a dime. If there stopped being major model improvements for a few years and equivalent free models have been out during the same period, we will see people slowly move over to competitors.
I'm uneducated on how distillation works at more than a basic level so forgive me if this is a stupid question.
Isn't "distillation" of another provider's model exactly how these models got training date in the first place: Massive amounts of the written word + Prompt -> Answer. Why wouldn't distillation produce similar "reasoning" in the new model? It's just inputs and outputs.
Among other things, because you simply can't get those "massive amounts" of text from a SOTA model at reasonable cost. And complex reasoning cannot possibly be trained in a pure one-shot fashion, real post-training takes massive resources. The whole story doesn't add up.
What you're describing is (pre-)training. Distillation requires richer labels, the probability distribution over tokens (it would be logits rather than probabilities but that's not important). From a chat transcript you can only understand the argmax/most likely token of that distribution (and only if the API allows you to set the temperature to 0). It's not impossible for an API to give you that but they won't if they don't want you distilling their models.
The intuition is that distillation exploits not only the "right" answer but the relationship between answers (what's the second most right answer? the third? etc).
This is totally inaccurate, the APIs provide the reasoning logs. You ABSOLUTELY can distill from APIs, in fact, that's the primary way distillation is done currently.
I struggle with the practicality of the whole thing.
The amount of tokens required to properly distill a frontier model is so large that by the time you could consume the # of tokens you would either be banned for extremely obvious abuse or a new model would be released, rendering your efforts less and less valuable over time. Intelligence is not a linear thing. Being behind just a little bit can have exponential consequences.
> Being behind just a little bit can have exponential consequences.
That seems to be the argument of Dario, Sam et. al., but I'm not ready to believe it. Time will tell, but this can be a marathon and Anthropic and OpenAI is in getting ready to sprint the last lap of the first mile.
I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.
Try running the latest OS models on a normal Mac or PC. Claude Fable and Mythos are systems not just pure models.
And of course marketing. Don't believe the hype.
I think Claude is often times underwhelming. Security concerns are also a concern companies have a blond spot for. The really toughest pro security (Yes, pro! Totally different framing!) company I know is Google after all.
What I can companies advise to do is, really having more than just bug bounties but a professional hacker team that does nothing else but attacking them the whole day and night 24/7. This needs to be coordinated with the government otherwise you might sound an alarm and will be SWATed for doing good. And I would pay them huge sums since the risk and fallout warrant such a treatment, not the standard wage.
Hackers are the real deal, not AI. Proof: Hackers using AI.
> I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex.
AFAICT … despite saying you “disagree”, you appear to be agreeing with the parent comment that the model is less important and compute (all that complex infra) and data (also complex infra) are more important.
For now I suspect however that the gigantic models are not needed and you will be able to do pretty much what you need in a specific domain with 120b or lower. There is so much trash in the frontier models. I don't need all the world's slam poetry for my coding tasks for example.
Model capability is a function of model size. Raising the bar raises model performance in every domain.
An "idiot savant" model that's overtrained for a specific domain would beat a generalist model of the same size. But scale the generalist up enough, and it'll trounce the specialist. Removing poetry data from a model training mix doesn't give you much - it might even cost you some performance - and "idiot savant" approach of overtraining for a domain has a hard ceiling.
So far, it seems like there's some equivalent of "g factor" in LLMs - a broad "intelligence" value that performance across many diverse domains correlates with. And, as a rule, larger models have more of it.
Model effectiveness has improved across model sizes. You really should try the latest flash variants more. They have become my default for most tasks except for gnarly high-level planning.
"Capability per parameter" is rising, but parameter count remains an advantage. And small models remain bad, because "good" is a rapidly moving target.
A 2026 4B beats 2024 4B, but both are far behind the contemporary frontier. Which makes them bad. There is no such thing as "too much capability" - a "good" model is whatever the current frontier is.
In 2024, a "good" model is one that can be trusted to write a 800 line script. In 2026, it's a model that can be trusted to do gnarly high-level planning and execution both. In 2028, it's going to be something like a model you can point at an extremely involved task, abandon, and have it report back with a "done" in 3 weeks.
> A 2026 4B beats 2024 4B, but both are far behind the contemporary frontier.
The thing about engineering is you don't just use the biggest bolt on the market on every bridge.
> In 2024, a "good" model is one that can be trusted to write a 800 line script. In 2026, it's a model that can be trusted to do gnarly high-level planning and execution both
This sounds a lot like having a single diamond-head hammer as the only tool in your toolbox. As suggested by the name, flash models are fast - sometimes I want to write the equivalent of fifty 800-line scripts. There is such a thing as good enough.
Good enough? That's a lie people tell each other because they lack imagination.
"It's good enough" was said about GPT-4, o1, o3, Opus 4 and more. Guess what happened? Newer models released, people updated their expectations of what LLMs can do, usage got more aggressive, and somehow, GPT-4 went from "good enough" to "obsolete trash".
If you have no imagination, then at least substitute your pattern recognition for it.
The world is hungry for capabilities. There are piles upon piles of tasks that aren't done by LLMs simply because LLMs aren't good enough to do them.
The thing a frontier model gives you is "you don't have to babysit a model to get it to do X", and that X gets more and more impressive release to release.
I wish you had addressed at least one of arguments in good faith before jumping to insults and countering a strawman argument I didn't make - I never claimed their will be no use for more capable models.
You do your AI-maximalism, and I'll stick to making trade-offs based on the needs of each piece of work.
Right - the idea that "bigger model = better" might have been true a year ago, but the flash models are extremely effective right now. You simply use them for the tasks they are ideally suited for.
While I disagree with OP about removing stuff from the model, there’s a valid question about tradeoffs between intelligence and price.
Deepseek Flash is almost certainly wrong more often than Opus or Fable. It also costs like 5% as much.
The question becomes if I run Deepseek in a loop to fix the mistakes it made that Opus/Fable didn’t, can it fix its own bugs in few enough tokens that it’s still cheaper?
So far, the answer seems to be “yes, by a significant margin”. A lot of tasks are simple enough that both Deepseek and Opus or Sonnet can one-shot it, which is a huge cost win for Deepseek. Even on the long tail, it’s usually like 4x the tokens on Deepseek which is still way cheaper than Opus.
There are things that Opus can do that Deepseek just won’t ever really nail, but it happens so infrequently that I just don’t worry. Like most people, most of what I do is the same sort of “3 tier app with a React frontend” that doesn’t take a rocket scientist to work out.
> > The bottleneck is compute and data, not the model.
> I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.
> Try running the latest OS models on a normal Mac or PC.
It can be done through the magic of SSD offload. The worst case involves seconds-per-token speeds, but that's OK if you only care about low volumes of slow unattended inference, which maximizes utilization for the hardware.
(The real worst case, where you're streaming the whole model from the cheapest storage you could feasibly think of, involves multiple minutes per token for a single inference, or even hours per token batch if you're doing many inferences in bulk. That's a lot less helpful, so there's a space for smaller models at the edge, even for unattended workloads.)
An LLM which provides an OpenAI or Anthropic API-compatible interface + a coding harness like OpenCode or oh-my-pi is a pretty easy "ecosystem" to replicate. Exactly what makes you say Fable or Mythos are "systems, not just pure models"?
Fable can delegate tasks to Opus or Sonnet, so it has some agentic properties and I believe it does them in parallel.
The parallelism is where this starts to fall apart on a local PC. Like I can run some Qwen quants, but I can’t run a decent Qwen model while also running another model smart enough to actually implement it. I’d have to do them in series, and given how long Fable seems to take even with parallelism, I’d probably be waiting days for an answer.
oh-my-pi can delegate tasks to other models too. I usually use DS4 Flash for low priority subagent tasks.
If Fable is "delegating" tasks, then there's actually an agent front end of whatever you think the API is.
We have a local instance of Qwen-3.6 which is more than adequate for running agents. You can mix and match local and cloud-hosted models. (My biggest use case for local models right now is vision models because they're quite small and I can avoid some data-locality issues my customers wouldn't be comfortable with if I sen them to a Chinese model.)
To this point, I've never understood the supposed "alignment" between the EA/AI Safety crowd and Anthropic's mission that the author comments on. Be the stewards of the Machine God, but responsibly? I think the Manhattan project, which AI development is commonly analogized to, had a lot more intrinsic properties to gate against uncontrolled proliferation (which still happened to some extent). Also this is a company that is expected to go public this year, at which point there will be a slew of new voices pushing the company to increase its value, mission be damned.
People like Yud at least have a clear consistency in their advocacy that we shouldn't be developing this at all. Anyone who thinks they can reconcile Anthropic's work with the AI safety mission is in total fantasyland, if it's not just a public persona they've adopted strategically.
> Anthropic’s cautious roll-out was justified. The problem with publicly releasing models, however, is that guardrails can be jailbroken, and apparently that is exactly what happened shortly after the release
The future is unevenly distributed. Anthropic, and Amodie in particular, seem to be of the mind they can control a bit of the unknown using words. They are likely being guided by the very product they built. *AI CAN MAKE MISTAKES
That Project Glasswing bullshit reeks of it. Corporations have take control of our attention, our Internet, and now our thinking.
We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models.
Is not
We sent open weight models against a codebase to find vulnerabilities.
The second case is not what Anthropic did either, though. If you have their process internalized as "open freebsd, tell mythos 'find vulns', done" this is not what happened. They have a harness that went file-by-file, spawned a subagent for each file, told it to find vulns in that file, then a post-processing step (more on that in a sec).
In that sense: The AISLE replication still provides too much information to the model, but its not far off, and others have replicated Mythos' findings in a more clandestine manner on open source models. Some were totally capable of finding the same vulns Mythos found back in ~March (and today, the new Kimi K2.7 is looking extremely good, very little doubt it could do it).
The critical difference is that post-processing: the Mythos model/harness has some step to induce Mythos to actually exploit the vulnerability, leveraging its ability to do so as a ranking mechanism. Anthropic inferred that this led Mythos to discover vulnerabilities nothing else could discover, which is not true, and Anthropic should be held accountable for this weird artifact of that communication. However:
- An OSS model might find the vulnerability but rank it as a 3/10. Mythos finds it, chains it with a second vulnerability, now suddenly its an 8/10.
- An OSS model might find the vulnerability, alongside fifty other vulnerabilities. The operator ignores all of them.
The problem with automated vulnerability detection, including with LLMs, is that they find the haystack, not the needle. Every piece of hay might be a vulnerability, but whether its worthy of fixing is another matter. Mythos does represent a meaningful improvement; it better finds the needle.
Even if some models are able to find some large fraction of the vulns that Mythos finds, the contention (that those of us outside Anthropic and a few select partners do not yet have the ability to replicate) is that Mythos not only found vulns, but was able to string them together into working exploits, very close to autonomously or at list with minimal hand-holding. I believe that UK AISI has at least in part confirmed that Mythos is better at this than other models like GPT 5.5, even while it's not better at the simply finding vulns part (although, again, no one else is currently able to replicate their results).
This was the primary reason for not releasing it. The difference in the two primary camps around this topic are that the doubter group thinks that Anthropic, and all of their partners, are essentially lying about this (since no one outside Anthropic and select partners has the access to replicate), whereas the other side believes that Anthropic and partners are probably mostly telling the truth without too much exaggeration.
Neither camp has evidence other than unconfirmable reports and/or arguments about economic incentives. I personally think that Anthropic has, in the past, mostly not lied about things like this and has by far been the most transparent and open AI company. That could change, and they could be lying now, but I think that the camp that is certain that they are is far, far too confident in their belief.
As I understand it, ITAR regulations for export controls have just been applied to any form of Mythos.
These are overseen by U.S. Departments of State and Commerce, and forbid foreign nationals from access to any form of Mythos, either within or outside the U.S.
Only U.S. citizens and immigrants that are holders of a "green card" may now access Mythos.
It appears that Anthropic does not have internal controls to implement these restrictions in any form, so the only option was to shut Mythos down.
Penalties for ITAR violation can reach ten years in prison and a million dollars per violation. (I can post a link to those details if there is any interest.)
As long as Anthropic is a U.S. company, there is no escaping this.
This is how the US gov does business now, capricious and vengeful.
Textbook retaliation for not letting them use an abliterated version of Claude in weapons systems.
This effectively renders any US closed model useless for any foreign company. Could happen to OpenAI, Google, etc. Too much of a risk to implement something that can be yanked out because the company didn’t behave the way they want.
Looks like it’s time for Kimi, Z, Deepseek to take the front row. They’ll catch up in a few months anyway. Kimi code 2.6 is crazy good
"When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."
This is fearful stuff on all sides, and none of the people involved might realistically be able to navigate the danger.
"What this degradation represented was both the capability and willingness of Anthropic to silently alter its models to achieve its policy preferences. In other words, Anthropic willfully validated some of its critics’ worst fears in terms of being a supply chain risk."
a) Anthropic believe that AI is an extinction level risk and that they are the only leading AI lab which takes safety seriously. In combination this puts them in the position of believing that they are the only ones who can save the world, which is reasonable to call a god complex.
b) Anthropic are engaging in actions which aquire and consolidate power in the form of control over powerful AI.
c) "The history of brilliant people convinced they know what humanity needs is a sordid one, precisely because they have convinced themselves that their intentions are good, justifying actions that very much are not."
I'm describing claims from the article and would not word them so strongly myself. But this explicitly does not assume evil intent.
Open models will catch up eventually, TOTL models will get distilled into smaller, more efficient versions, it’s not something you can moat indefinitely
China will, but they'll only be useable by hackers torrenting it and running it on small GPU clusters you learn about on IRC. Everything old is new again.
This is a suicide shot for the American economy. The numbers only lined up for AI to rescue the USA from its debt if it captured a significant portion of the world's AI spend, and while it was a longshot before, there's basically zero percent chance the world trusts American AI when the government is pulling strings.
It was a zero percent chance anyway. Look at Europe working to leave American software behind in recent years. And it was greatly accelerated by leveraging American AI to build the exit.
You can read it all over HN. It's about weakening American influence and building Eurocentric economies and influence. And exercising the same level of choice that Americans prefer as well. Americans also want to escape Google, Microsoft and Apple and more. They've all been caught investing too heavily in government influence and thought control (aka marketing).
And on the other side of that, an American company that deprives the US of AI for defense, is defacto weakening American defense because competition militaries will gain a technological edge by simply taking control of AI companies in their country which the US hasn't done (yet).
There are very valid arguments on both sides, I think.
> The numbers only lined up for AI to rescue the USA from its debt if it captured a significant portion of the world's AI spend
The numbers lined up if those companies created something resembling AGI, the USA companies captured a large share of the world, and there was lack of competition so those companies could capture a large share of the value.
Individuals can leave, but the company cannot transfer restricted intellectual property.
Europe has extradition treaties, so the U.S. can force anyone in Europe back to the U.S. for criminal indictment who demonstrates inappropriate possession of this technology.
Well, force is a strong word… it’s still just accords, that the US doesn’t seem to be valuing lately… so if they say no, what’s the US going to do? Start a war over a company?
Would be very hard to demonstrate that they did that. If all employees move to some country with a slow legal justice system and strong labor laws, they also recreate the training data because that can be transferred, they can train another version in said country which is perfectly legal.
Can you demonstrate beyond any reasonable doubt that the model weights have been transferred? No. Will the EU judges move to extradite said individuals (and many are EU citizens)? Also no, especially in the face of spurious accusations. And even if they were open to, you can stonewall everything and you will probably outlast any US administration pursuing that.
I never really understood this "US person" restriction. There are 350M people in US, mostly citizens and green cards holders, surely some of them could be working for a foreign power.
They don't even need to know they are. You can assume that if the model becomes available again, a lot of people will find themselves working for companies distilling these models that just happens to ultimately do work for foreign entities, whether or not the people accessing the models knows or not.
> As long as Anthropic is a U.S. company, there is no escaping this.
Reminds me of the RISC-V Foundation → RISC-V International move to Switzerland. Around the time some dumbass Republicans tried to impose export restrictions on a set of open, world-wide used specifications.
Pandora's box has been opened, and there's no closing it. Capable AI models will be everywhere.
"they by extension think that only they should have final say over AI generally. When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."
That might be one of the most important points in the post. Very troubling.
It's questionable whether the current government can even unite the talent required for this project. Seizing it might just push all the talent to Europe or China.
The idea of open-sourcing something that falls into the "national security" category is clearly a non-starter unless there's more powerful, classified models that can outmatch them.
I think Anthropic has clearly demonstrated the most responsibility here: they've been crying for regulations, they were careful about Project Glasswing, and they've got comically over-sensitive filters around numerous topics.
I believe that the line was constructing exploits for bugs, not bug finding. This seems a reasonable cutoff to me, since bugs are revealed in security patches and pull requests (for open source).
If you are to believe Anthropic, Fable was export controlled for bug finding, not for exploit construction. They seem to be working to make this the "bright line" for LLMs being a national security risk. My guess is that will be the case they take to Washington this week.
The factory does decent software engineering - for which it can also use the same llm - so that when an attacker does either, a sota llm does not find bugs to exploit.
A lot of Anthropic’s moves make sense if you follow the LessWrong / rationalist community writings on AI safety. A lot of it is distilled in Ant’s blogs and leadership interviews and podcasts (Amanda Askell is particularly interesting).
Ant’s models, culture and leadership actions are largely consistent with their beliefs, even if they may seem flawed / incomprehensible.
Relevant anecdote: I interviewed with them for a MTS role in 2023. I think the technical part went fine but the interviewer was clearly frustrated by my low regard for LLM safety. I didn’t get the role.
There are fortunately some initiatives and interesting developments in the European market. Take bunny.net for example. We have to start somewhere in Europe, right? Better late than never.
Oh please, the earlier spat with the Trump admin was the best thing that ever happened to Anthropic. Before that, Claude was really only well-known in developer circles, not the wider normie-sphere. After Anthropic got the "Trump hates them, so it MUST be good!" stamp of approval, the company's recognition and popularity took off.
This too, will end up being a good thing for them. The ban will end up getting lifted due to some "amazing deal" in the coming weeks and Anthropic will now have the "Trump tried to ban them, so they MUST have the most advanced AI model in the world!" stamp of approval just before IPO.
1) It’s safe to assume the US would do its best to prevent it, and even if Anthropic was successful in exfiltrating their data, code, models, and people, I’d imagine the US would immediately block all US companies from working with them. So they’d be blocked from their own US-based compute, plus Google, Amazon, Microsoft, xAI, Meta, etc.
2) Where would they go? China maybe, but as far as we can tell it doesn’t have sufficient compute for Anthropic’s level of need. The EU likely as or more restrictive in different ways to the US - the EU is hardly buzzing with AI innovation. Some Middle Eastern countries might have the money, energy, and interest in carving out such a position, but no compute. Plus I’d imagine the US would act directly against any country or region receiving them, economic or otherwise.
3) Then, as said elsewhere, the US would block GPU sales to wherever they found a safe haven, preventing the buildup of the compute they’d need to continue.
Depends what you mean. The academic work seems largely... fine? Plenty of good work came out of Europe or European researchers. It seems the problem is more "trying to build a trillion-dollar company of any kind".
It's an interesting question: does the EU seek only to regulate successful modern American companies to death, or home grown ones too? Probably not a gamble worth taking.
The issue with EU is its not one market, for funding or deploying a company of any kind. There is both no funding scale and no easy distribution scale. Same thing with any kind of lobbying you need to do to move laws to be more amenable to tech, you have to do it 20+ times. Nobody does this, which is why smart EU and UK talent migrates to the US, scales, then just uses the weight of their US business to change reality around them in the EU anwyay. For the most part.
The interesting part here is not whether Anthropic is right on safety, but that safety gives them a moral vocab for bold policy changes and platform power.
Relatedly, I think it's worth noting that Anthropic models have consistently been top-scoring in BullshitBench[0], in a league of their own, really.
Not affiliated with the bench in any way, but I think it surfaces important differences between the behavior of the models from different labs.
TLDR: The benchmark is measuring pushback in response to nonsensical requests and questions, as opposed to going with it and hallucinating a nonsensical answer.
TBH this is the main thing that made me start trusting Claude enough to actually find it useful, and I'm surprised other models haven't caught up. I assumed they had and I just wasn't aware because I'm not using them in the same way.
"On one hand, I actually don’t begrudge Anthropic not wanting to help its competitors; on the other hand, what should be blisteringly clear is that Anthropic does not think that anyone else other than them should even be making frontier LLMs."
I don't find this blisteringly clear at all. A company making it harder for competitors to steal their IP is perfectly normal. This is Ben Thompson's personal grudge against Anthropic showing, yet again. He can't think rationally about this company.
Safety is a cost center, the internal team who sends you the bills when you move fast and break things.
I always thought safety was interesting in and of itself, but for some reason HN doesn’t have many people from the safety side of tech in conversation.
Tech isn’t a niche hobby anymore; Billions of people are impacted by the decisions of a few firms.
My grandfathers android had 3 different messaging apps installed, somehow. AI is enabling new forms of fraud at a time when we still haven't solved the old ones.
And this is all in the first world, move your coordinates to the developing world? We had human trafficking to get educated English speakers into call centers in Laos/Cambodia to defraud first world inhabitants of their money.
We aren’t in the early days of tech anymore, and the kind of scale that we have enabled comes with it a certain cost. We can choose to ignore them, or to understand them, but we will feel their impacts all the same.
> Here’s the thing about these safety justifications: I think they work because, to Anthropic, they aren’t justifications. The company really believes that they are the only ones who believe in super intelligence, and thus are the only ones who are sufficiently concerned about the dangers. That excuses decision after decision, policy after policy, and confrontation after confrontation that, to people on the outside, look like a bizarre combination of cynicism and naiveté.
I really dislike this belief (that has at least been expressed here) by some that X is okay because they-really-believe-it. This has a real Road to Hell stank on it.
It is incredibly convenient when your predictions or supposed beliefs go south. Well, we really believed that we were doing it for the betterment of human kind. And we really believed that X was an existential threat that was inevitable in which case we had to step up and do it because we we the only good guy ideologues. So sorry but not sorry.
I also don’t care if commenters know rank-and-file on the inside that “really believe it” as well. Not for one second.
The problem is when people use "we really believe it" as an excuse to do harm, which has not actually occurred here. Anthropic is not committing violence, they're not defrauding the population. They're sticking to both morality and the rules.
So... what, you just don't trust anyone good? Would it be better to pull in a health insurance CEO? They're happy to watch people die for profits, no concerns at all about them pulling a "greater good" card because they're in it for entirely selfish reasons.
Incomparable domains. People routinely suffer illness. We can compare outcomes. These ideologues are building something completely unprecedented which, according to themselves apparently, can go paperclip-rogue if one is not careful. So the worst case is unprecedented. Then there is the more mundane matter of heating up the economy, something which also has no one blameworthy until any such supposed bubble actually pops.
> So... what, you just don't trust anyone good?
The baseline here is apparently that they are good, I’m just supposed to trust and shut up?
No, I'm saying they have an actual track record that you can evaluate, and if you do so you will see that they have killed approximately zero people, etc. etc.
Or even just propose an alternative: who has a better track record, here? Who DO you trust?
I think the second the company starts to classify "competition" as mis-use... the whole "they're not committing harm" line sort of goes out the window.
Modern society is built on the idea that competition is required from companies, and we seem to be exiting that age into a new world of monolithic, monopolistic, mega-corps. Personally, I find that a real route to dystopia.
Where do you draw the line here?
What happens when your car stops working because you're driving a tesla, but you're working on EVs for Honda or Ford?
What happens when your macbook stops working, because you decided to commit to changes to ARM software, or RISC-V?
---
And before you dismiss those, this is literally what Anthropic is doing TODAY. Using their tools to develop competing tools is something they classify as mis-use, and shut you down for doing.
Personally, I just can't accept that as a valid moral stance. Wonderfully successful, abusive, and dystopian? Absolutely. Moral? FUCK NO.
When tools turn themselves off because the manufacturer has decided it doesn't like how you're using them... you're a slave with no autonomy.
... you're perfectly welcome to compete with them, they're just not willing to provide their own product for that purpose. I also can't buy the schematics for Intel's latest chips, the source code for Windows, or a rocket from SpaceX to disassemble and study.
Like, there's a very critical point where you are asking to use their servers to directly compete with them.
This has been normal for somewhere between "decades" and "the entire history of commerce"
> The entire Anthropic origin story is rooted in the founders’ belief that OpenAI wasn’t taking safety seriously enough; the company believes that only they can control AI, and that because they uniquely care about safety, they are justified in trying to control everyone else, up to and including the U.S. government.
Anthropic believes they have the responsibility to guard their tools from mis-use. That is all. They are not trying to "control" anything or anyone. They do however decide what they think is mis-use.
it's a pretty ridiculous stretch to attribute them thinking that OpenAI wasn't taking safety seriously enough (which is, among other things, a little bit evident from the fact that they no longer have a safety team at all) into asserting that they want to control the US government.
I think assuming you have the ability to guard a tool (that you're "selling" for profit) from mis-use is the definition of "controlling behavior".
It's the kind of ethically myopic take that can only really exist in this new digital age - where tools aren't actually sold, they're just digitally rented.
The most telling part of the "control" narrative is that they happily classify "competition" as mis-use. We're headed back to serfdom on a speedrun.
I don't always consider ethics at all in logic, so I guess you can call it ethically myopic.
Installing safeguards to prevent a tool from being used for certain things is a perfectly natural and common thing to do when you are providing the tool as a service. For example, blocking VPNs and open proxies from accessing a free service if those are a major source of spam and abuse. Note that Anthropic never provided the model for offline use in a form that includes DRM -- they are simply safeguarding the service that provides access to the hosted model. The only ethical concern I see here is that some of their safeguards are ones I wouldn't personally agree with, and in a world where dependence on a model is expected it can become an issue if the model refuses to perform in some cases, etc., but that doesn't automatically mean the refusal itself is unethical unless that issue was known and expected (and unless the alternative is not bigger, worse bads)
Also note you are not even "digitally renting" anything. This is the exact same type of thing as, say, real humans in real life refusing to perform services for certain clients or under certain circumstances. Networking makes it possible to decouple some of these things, but that doesn't magically make it renting or automatically turn a refusal to perform services into an attempt to control clients. Just the same as I can choose to refuse any request, which does not automatically constitute attempted control over the asker. There can be ethical concerns about whether my refusal causes problems that I'm obligated to avoid (and whether or not such obligation exists), but that doesn't automatically contaminate the refusal itself unless I have knowledge of and intend the bad.
To use a much more relevant example, Anthropic's refusal to allow its models to be used for war (among other things) does not constitute any attempt to prevent war. It's only a refusal to assist in it. That's not some unfair, unethical attempt at controlling the government, that's just Anthropic not wanting to be responsible for assisting in war.
What happens when your car stops working because you're driving a tesla, but you're working on EVs for Honda or Ford?
What happens when your macbook stops working, because you decided to commit to changes to ARM software, or RISC-V?
Tools should be neutral. The idea that a tool can only be wielded in a manner that its manufacturer approves of is... scary.
That's a real quick hop and a jump to a really, really ugly spot, societally speaking.
And sure - technically Anthropic is selling a service, but even that idea makes me quietly upset. The only reason they don't sell a product as a tool itself is that they have more control over the model as a service, and expect to be able to extract even more profit from their customers with this route.
---
My real hope is that open models FUCKING CRUSH them. Because almost nothing is scarier than a self-righteous, moral zealot.
> What happens when your car stops working because you're driving a tesla, but you're working on EVs for Honda or Ford?
> What happens when your macbook stops working, because you decided to commit to changes to ARM software, or RISC-V?
These sound similar, but aren't the same thing I'm talking about. It's more similar to a rideshare company refusing to serve you, or a cloud PC cutting your access.
It's not the same thing as DRM, which is when invasive malware attempts to control what you do with your devices -- your property -- and your resources, that you own. A car that can be shut off remotely, or that can detect competitive conditions and cease operation, is not the same as a hosted service refusing to have you. It is DRM.
Likewise, a MacBook that stops working based on your affiliations or your activity is not the same either. It is DRM. (Technically, Apple Activation Lock is DRM. So are locked Android bootloaders that can't be flashed with custom verified-boot signing keys, etc.)
When you're using someone else's private resources, someone else's private infrastructure, by default they have the right to simply no longer serve you, at any time.
DRM, by contrast, is when a machine or software you already own decides it will no longer function for you.
Yes, this is scary. We are already confronting this right now. Faceless corporations abruptly cut your access, or ban you for life from really important things. PayPal steals your money and pockets it instead of giving it back. Thousands, tens of thousands, hundreds of thousands of dollars (or whatever) gone because they said so. It's awful. It ruins lives.
But this happens because you need to draw a different line. Not whether one should be allowed to refuse services ever, but when. It's not okay that PayPal can steal tens or hundreds of thousands of dollars or more from you with no recourse, just because something looked suspicious, and you'll never know what it was even if you bankrupt yourself with arbitration costs (since their terms of service say you are simply not allowed to sue them anymore, and for some reason companies are allowed to just say this now and have it be legally binding).
A refusal at that point can be devastating, especially when it's for no fucking reason. Or when it's for a completely shitty and unjust reason, like in banning all accounts that have been involved in buying or selling adult content. There are cases where you can ruin someone's life by suddenly refusing to serve them with no recourse, and those are the cases that shouldn't be allowed to continue.
So push for that. Businesses shouldn't all have to serve every customer, but they shouldn't be able to just suddenly ruin someone's life. That's the scary part.
> The only reason they don't sell a product as a tool itself
...it's not the only reason. You can't exactly run trillion-parameter-scale models on the kinds of hardware people tend to already have in practice. And who's gonna pay tens of thousands of dollars up-front for their own inference hardware for the thing? (If you sold it as a hardware product.)
> My real hope is that open models FUCKING CRUSH them.
The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one.
The bottleneck is compute and data, not the model. That's why they could only gate it for a bit. The ITAR thing proves it: no nationality controls in place, so the only option was killing the whole thing. Not exactly what an all-powerful gatekeeper does.
> no nationality controls in place
Not for now, but how long before we have KYC regulations concerning LLMs?
That’s really what Dario wants. Let’s hope he doesn’t get it
Regulatory capture is the OpenAI and Anthropic end goal, for certain.
But I also think they exist in a sort of un-designed corporate narcissism, which is a common trait in bubble economies — I am not judging them particularly severely.
Netscape under Clark and Andreessen and Sun under McNealy both fell into corporate narcissism: the belief that only they really mattered, that they were chosen, and that the world needed to rearrange itself to just let them shine. They arguably let themselves get played by Oracle (a corporate psychopath) and others as a result.
OpenAI's position is profoundly corporate-narcissistic: all we need is all the money in the economy and not to have to do anything upsetting like think about turning a profit for the next four years. Like rich kids. It would be nice if you believed we were so important that we should get an enormous stipend for just being us.
Anthropic's position is: we think we're so unique and ominous that government needs to make us both essential and terrifying. We have to exist otherwise worse people will.
Both narcissistic positions.
> Regulatory capture is the OpenAI and Anthropic end goal, for certain.
it has to be, because the other way around - the government taking over parts or the whole thing - is inevitable if the trend holds.
Porque no los dos?
this is exactly the play is my point
the inevitable trend is that numbers will be free and nobody will control the whole thing
ai-celebrities are just clinging to relevance like all the other celebrities out there
HN is the builder side of the conversation, and in my experience, few safety people congregate here.
The safety side of tech is a PTSD inducing shit show. Governments are more than happy to champion age verification laws, because parents, around the world, are clamoring for anything to pump the breaks on the social media experiment.
Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.
> Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.
I don't think anyone in tech is really truly engaging with how quickly the shine has come off the tech industry. Except maybe Apple, who even so still have some work to do.
Technology and science is the intersection that is supposed to make our lives better, easier, more prosperous. The last decade or two what marvelous technology has came from silicon valley that hasn't served primarily the billionaire class and made life worse for the common people.
The yoke of silicon valley is feeling heavy. People might just throw it off.
Social media is old hat now.
As someone on the "safety side of tech", social media is being exploited to increase surveillance and government control precisely because its actual social influence is heavily on the wane, and capital is happy to sacrifice what's left to increase the profits of the expanding public/private tech surveillance industry (with "protect the children" controls on social media like age verification being the usual backdoor route it always is).
Society may be growing tired of Tech, but governments aren't, and in fact they're heavily expanding their back channel reliance on not-traditionally-military Tech as an extension of their Defense spending.
Cyber security has the maturity that trust and safety hopes to achieve at some point.
Social media was being exploited from inception. Palantir had sales documents for sock puppet management software back in the PHP era.
I don’t disagree that Government is interested in tech, but I will push back on the dismissal of child safety that is inherent in your comment, intended or not.
For all that some people in the firm may have tried to do the right thing, Social media firms have created bad outcomes for children, and executives were briefed on the harms they were going to cause.
This is the dismissal that concerns me, because it ends up miscalculating the level of anger and unhappiness amongst the voting populace, and therefore the political will to pass regulation to reign tech in.
The political will is already captured and redirected.
There are numerous bills to limit AI access for consumers, to combat deepfakes hurting children. There are no bills introduced or passed to prevent AI being used to target dronestrikes that kill children abroad, or surveil children domestically.
What the public wants doesn't actually matter right now, only what the government will allow to let pass, which in this case is additional internet surveillance.
Under a future, better government this may change, but (sadly) nothing is going to sink tech's dominance right now.
The anger and unhappiness against tech is good, and hopefully someday they'll burn down all the data centers and I'll never have to hear the words Cloud Computing again, but (to paraphrase a famous Eve Online interaction) it's not going to be today, and it's not going to be us.
> Society outside of HN is quite tired of Tech, and I despair of figuring out a way to make this clear to the commentariat.
s/Tech/Tech Companies
Tech did it to themselves. People like and want technology. What they don't like and don't want more of is enshittified, user hostile technology. The answer is out there, but our collective school systems failed to teach computing irt free software/open source and instead schools themselves all bought in on enshittified, proprietary tech, or even just dumped trying to teach computing at all outside of "how to login to google classroom and google docs"
I grew up lucky, in that my dad was a dev, my first PC as a kid ran red hat, my high school had an intro to programming class (in BAISC lol). It shaped how I approached computing growing up, and my values. It makes me look at the things we have now and think "No, you're just repackaging community free software and selling it back to me, I'll pass on that."
That experience isn't available to anyone born after that specific era, instead their tech experience is shaped by walled gardens, vendor lock-in, and straight up hostile and manipulative software, so its no wonder they are tired of it. They don't even know a different world (of software) exists.
Spot on. There's a certain level of drinking the kool-aid or getting high on their own supply. Anthropic is a lot worse than OpenAI but OpenAI had to go through rounds of shedding.
To be maximally fair to them, I think it is difficult to be one of the key businesses in a market bubble and not fall victim to this kind of thinking, especially when the continued inflation of the bubble depends on you — lots of people lose their shirts if you don't push hard to be "special".
But as you say, there is a measure of getting high on one's own supply now.
And there's the curious solipsistic energy of Sam Altman whimsically musing in public that it turns out his product is too expensive for people and they complain when you make the price realistic (when it possibly needs to be more expensive for OpenAI to survive).
They seem to believe that the ordinary rules either will not or somehow must not apply to them; it's increasingly bizarre to watch.
Maybe the people around pets.com were this bizarre; we didn't have so much livestreamed interview content to show us.
A\ and OpenAI each have their own unique kind of nonsense. I think OpenAI has just been less successful with persuading the rest of the world that they should have all the money in the world.
Anthropic has been surprisingly successful at convincing them that they should control frontier models because they're so dangerous that... only Anthropic can be trusted with them.
(If they're really so dangerous, the right way to deal with them is through a democratic process and taking them out of the hands of a for-profit private entity.)
what Dario wants is to retain any influence whatsover on how the research progresses before the inevitable nationalization of the frontier. he gets to keep the N-2 tech and maybe influence the N-1 tech, but the only influence on the frontier he has is today; whatever he imprints in the pipeline the government takes over.
IOW I don't think he thinks in the same categories as most folks here.
N-1? N-2?
Best-possible-model (N) - Two Generations (2), same with N-1, N is the SOTA in this example. I'm not sure that actually clarifies what the comment is trying to say other than they think the models will be nationalized (can't even imagine what that would look like).
basically imagine the Manhattan project, but instead of blowing up the desert they're building the biggest datacenter you've ever seen.
Isn't this the beginning of the plot of "I Have No Mouth, And I Must Scream"? The exceptionally disturbing dystopian horror?
the possible futures after the thing is built are uncountable, but hoping the thing won't get built at this point is naive.
in general I agree people should be reading a lot more sci-fi nowadays than they used to.
I read the popular ones, but itch for more. Which sci fi most applies today?
> ...the research progresses before the inevitable nationalization of the frontier.
Hacker News has been telling me America beats China at "innovation" because of the "freedoms" - especially frew enterprise. I wonder how a nationalized frontier lab would perform.... Andhow the non-citizen researchers would feel about working for the US government that doesn't trust them to use frontier models.
But he already got it, no? Claude Fable can only be made available to US citizens, which implies that every user who wants to use Claude Fable must provide proof of citizenship in some way, basically KYC.
For everyone, not just them
Yeah yeah, but after the IPO!
The distilled versions miss the spark of the model. Its like they land in the uncanny valley of models.
They get to 80% of the top models for 10x cheaper, unless you don't care about the money at all, it's hard to ignore.
> The whole thesis falls apart though. You can't be on your way to "power over everything" and get distilled into free Chinese models within months. Pick one.
But is that last part actually true though? Sure, there might be 600B+ models available for download and local inference if you have the hardware, but does the users who use Anthropic switch over to those even if they're available even as hosted models? Seems like some do, most don't, Anthropic and Claude remains very popular among the people who use LLMs, there is no denying that.
> Anthropic and Claude remains very popular among the people who use LLMs
Only because someone else is paying the bills. I use Claude Opus at work because my employer pays for the tokens and encourages me to do it.
At home, I use DeepSeek Flash. It's not as good, but it's maybe 0.7 quality for 0.001 cost.
I have a question that perhaps you or someone else here has an answer for: I enjoy using Opus via Google Antigravity (usually agy) for perhaps 90 minutes a week. For Google’s subsidized $20/month plan they seem to give out a reasonably generous amount of Claude tokens. How does this compare with Anthropic’s $20/month plan using Claude Code?
BTW, I also use DeepSeek v4 Flash very frequently: fast and so cheap it is almost free.
Anthropic's plans are based on user experience of usage, not raw token counts, so you get to run through so many conversation turns, etc. within a 5 hour usage window. (Cursor, OpenCode Go, and others are similar.)
Cursor's $20 a month plan provides a reasonable amount of Opus tokens as well.
It’s really hard to translate minutes to tokens, it depends on how you’re using it.
The best answer would be to pull session stats from your harness and compare that against the limits. I think Anthropic publishes the limits of each plan.
If you’re using a pretty stock harness and not doing crazy multi-agent stuff with it, you’re probably fine.
My girlfriend built a whole (but simple) React app with it and only hit the limits of the $20 plan once. In fairness, she was trying to get it to clean up a bunch of 800ish line React files at once with a vague “make it look nice” prompt that she ran a few times. I think it was just churning for like half an hour straight before she burned all her credits.
It’s probably enough if you’re not on a fully agentic development strategy, it’s plenty to have it write tests and do comments and stuff, just not enough to continually have it doing giant refactoring passes.
Same, I had Deepseek search for, download and transfer (to my Linux emulation machine) the best Dreamcast games yesterday.
GPT refused to do so (citing that it's illegal even though I own the games). Deepseek did a wonderful job for 7 cents.
At work I use Opus because, why not? But I could easily switch to a less capable model if needed.
>citing that it's illegal even though I own the games
In the. US at least it is actually illegal to download ISOs/roms of games, even if you own a physical copy. It's a stupid law and as a downloader (as opposed to the people hosting the files) your chances of getting into any kind of actual legal trouble are effectively 0, but it is still against the law.
What's the speed on DeepSeek Flash? And what provider?
Fast enough? I signed up directly with https://platform.deepseek.com/ because it was the cheapest I could find. I use both Anthropic and Deepseek models via the VS Code copilot plugin https://github.com/Vizards/deepseek-v4-for-copilot
> does the users who use Anthropic switch over to those even if they're available even as hosted models?
I'm currently spending $200 for Claude. That's around my maximum that I can afford. I could stretch that to $500 I guess. But I saw reports of people spending tens of thousands of dollars with Claude API. That's certainly outside of my budget.
So if/when Anthropic decides to stop subsidizing subscription (if they ever do that thing, I still not sure about that), I'll certainly look at the other options. And available "open weights" LLMs hosted by someone will be my first pick. Right now Claude 4.8 feels very advanced, but things move very fast...
The ai labs would be very dumb to get rid of subscriptions. First, I don’t even think the subscriptions are losing money, I suspect they’re around break even, maybe small loses. More importantly, the subscriptions are how they lock in users and convince companies to pay api rates. Without user loyalty that they cultivate with subscriptions businesses will just use the cheapest model on open router or maybe local models.
> I don’t even think the subscriptions are losing money, I suspect they’re around break even, maybe small loses
whats the basis for this thought
For Claude specifically, (1) enterprises pay API rates on top of subscriptions, so subscriptions profitability questions are only relevant for smaller companies and indie devs. Many of whom probably have sporadic or low usage which helps balance some heavy users.
Again, for Claude, (2) it’s rumored that their API rates have around a 90% profit margin. It’s also claimed that the subscription limits get you around 10x tokens per monthly dollar vs buying them with API rates.
Edit: to drive it home. If a tokens true cost to anthropic is 1/10 of what they sell it for at API rates, and a subscription gets you tokens at 1/10 the price, that’s cost-neutral for the business if every subscription uses every token. They’re selling tokens at cost, not at a loss. Many subscription users won’t use their full allotment. That means serving some users doesn’t cost the business as much - which might push the subscription business from cost neutral to profitable.
not sure how that concludes that subscriptions are not losing money.
I think theyre charging at least 3x marginal cost for the api and I think that the average subscription uses around half its token allotment. So subscription needs to give 6x as many tokens as the api would cost for it to break even for the lab and that's around where they tend to sit
I don't think you're appropriately understanding the full gamut. The individuals who only spent $200/months will be stuck. But the pie is increasing in size, it's not stagnant. There are a lot of orgs who can afford to run a 1T model and even more that can run a 600B model. These newcomers are what's being fought over
The hotness we are seeing is smaller 'expert' models with an 'orchestrator' model in front that evaulates the prompts and routes to the appropiate small models and then synthesizes the collected answer. Easier to split across many smaller, cheaper servers and more efficient than a huge monolithic model.
Do you have more info about this? I can't tell if you're being misled by the unfortunate "Mixture of Experts" terminology (which don't work the way you're describing), or alluding to something different.
Or, maybe I'm wrong, but my understanding is: MoE is just an architecture to keep the activated weights smaller per token. The experts get routed basically token-by-token, and the "experts" themselves don't have a semantic domain so the "expert" word was maybe a poor choice.
https://en.wikipedia.org/wiki/Mixture_of_experts#Sparsely-ga...
"The sparsely-gated MoE layer,[21] published by researchers from Google Brain, uses feedforward networks as experts, and linear-softmax gating. Similar to the previously proposed hard MoE, they achieve sparsity by a weighted sum of only the top-k experts, instead of the weighted sum of all of them."
"Top-k experts," in case of some DeepSeek's models k=1.
See OpenRouter’s recent announcement on a model fusion setup, which they now support via API:
https://openrouter.ai/blog/announcements/fusion-beats-fronti...
No, this is an agent-level thing, not a feature of the model (ish, unsure for Fable).
You talk to a smart, heavy model to build a plan composed of smaller steps. Then you have the heavy model spin up smaller, cheaper LLMs to actually implement the tasks.
The heavy model is basically read-only in that mode. It can read files, execute tests, etc, but it can’t write code. It just tracks what needs to be done, offloads the work to dumber LLMs, validates the task is done, and moves on to the next step.
It sort of pushes humans up the stack. Instead of having a human sitting there prompting the LLM to start the next task, you have another LLM do that loop.
It’s been on my list to try out.
People dont pivot on a dime. If there stopped being major model improvements for a few years and equivalent free models have been out during the same period, we will see people slowly move over to competitors.
"Distillation" from APIs is not a thing, it cannot replicate a model's deep reasoning and behavior.
I'm uneducated on how distillation works at more than a basic level so forgive me if this is a stupid question.
Isn't "distillation" of another provider's model exactly how these models got training date in the first place: Massive amounts of the written word + Prompt -> Answer. Why wouldn't distillation produce similar "reasoning" in the new model? It's just inputs and outputs.
Among other things, because you simply can't get those "massive amounts" of text from a SOTA model at reasonable cost. And complex reasoning cannot possibly be trained in a pure one-shot fashion, real post-training takes massive resources. The whole story doesn't add up.
What you're describing is (pre-)training. Distillation requires richer labels, the probability distribution over tokens (it would be logits rather than probabilities but that's not important). From a chat transcript you can only understand the argmax/most likely token of that distribution (and only if the API allows you to set the temperature to 0). It's not impossible for an API to give you that but they won't if they don't want you distilling their models.
The intuition is that distillation exploits not only the "right" answer but the relationship between answers (what's the second most right answer? the third? etc).
This is totally inaccurate, the APIs provide the reasoning logs. You ABSOLUTELY can distill from APIs, in fact, that's the primary way distillation is done currently.
Not for proprietary models, all you get is a terse summary.
I struggle with the practicality of the whole thing.
The amount of tokens required to properly distill a frontier model is so large that by the time you could consume the # of tokens you would either be banned for extremely obvious abuse or a new model would be released, rendering your efforts less and less valuable over time. Intelligence is not a linear thing. Being behind just a little bit can have exponential consequences.
> Being behind just a little bit can have exponential consequences.
That seems to be the argument of Dario, Sam et. al., but I'm not ready to believe it. Time will tell, but this can be a marathon and Anthropic and OpenAI is in getting ready to sprint the last lap of the first mile.
I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.
Try running the latest OS models on a normal Mac or PC. Claude Fable and Mythos are systems not just pure models.
And of course marketing. Don't believe the hype.
I think Claude is often times underwhelming. Security concerns are also a concern companies have a blond spot for. The really toughest pro security (Yes, pro! Totally different framing!) company I know is Google after all.
What I can companies advise to do is, really having more than just bug bounties but a professional hacker team that does nothing else but attacking them the whole day and night 24/7. This needs to be coordinated with the government otherwise you might sound an alarm and will be SWATed for doing good. And I would pay them huge sums since the risk and fallout warrant such a treatment, not the standard wage.
Hackers are the real deal, not AI. Proof: Hackers using AI.
> I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex.
AFAICT … despite saying you “disagree”, you appear to be agreeing with the parent comment that the model is less important and compute (all that complex infra) and data (also complex infra) are more important.
For now I suspect however that the gigantic models are not needed and you will be able to do pretty much what you need in a specific domain with 120b or lower. There is so much trash in the frontier models. I don't need all the world's slam poetry for my coding tasks for example.
Wrong, mostly.
Model capability is a function of model size. Raising the bar raises model performance in every domain.
An "idiot savant" model that's overtrained for a specific domain would beat a generalist model of the same size. But scale the generalist up enough, and it'll trounce the specialist. Removing poetry data from a model training mix doesn't give you much - it might even cost you some performance - and "idiot savant" approach of overtraining for a domain has a hard ceiling.
So far, it seems like there's some equivalent of "g factor" in LLMs - a broad "intelligence" value that performance across many diverse domains correlates with. And, as a rule, larger models have more of it.
> Wrong, mostly.
> Model capability is a function of model size
Model effectiveness has improved across model sizes. You really should try the latest flash variants more. They have become my default for most tasks except for gnarly high-level planning.
"Capability per parameter" is rising, but parameter count remains an advantage. And small models remain bad, because "good" is a rapidly moving target.
A 2026 4B beats 2024 4B, but both are far behind the contemporary frontier. Which makes them bad. There is no such thing as "too much capability" - a "good" model is whatever the current frontier is.
In 2024, a "good" model is one that can be trusted to write a 800 line script. In 2026, it's a model that can be trusted to do gnarly high-level planning and execution both. In 2028, it's going to be something like a model you can point at an extremely involved task, abandon, and have it report back with a "done" in 3 weeks.
> A 2026 4B beats 2024 4B, but both are far behind the contemporary frontier.
The thing about engineering is you don't just use the biggest bolt on the market on every bridge.
> In 2024, a "good" model is one that can be trusted to write a 800 line script. In 2026, it's a model that can be trusted to do gnarly high-level planning and execution both
This sounds a lot like having a single diamond-head hammer as the only tool in your toolbox. As suggested by the name, flash models are fast - sometimes I want to write the equivalent of fifty 800-line scripts. There is such a thing as good enough.
Good enough? That's a lie people tell each other because they lack imagination.
"It's good enough" was said about GPT-4, o1, o3, Opus 4 and more. Guess what happened? Newer models released, people updated their expectations of what LLMs can do, usage got more aggressive, and somehow, GPT-4 went from "good enough" to "obsolete trash".
If you have no imagination, then at least substitute your pattern recognition for it.
The world is hungry for capabilities. There are piles upon piles of tasks that aren't done by LLMs simply because LLMs aren't good enough to do them.
The thing a frontier model gives you is "you don't have to babysit a model to get it to do X", and that X gets more and more impressive release to release.
I wish you had addressed at least one of arguments in good faith before jumping to insults and countering a strawman argument I didn't make - I never claimed their will be no use for more capable models.
You do your AI-maximalism, and I'll stick to making trade-offs based on the needs of each piece of work.
I.e. spending your time and effort on making choices that don't matter.
I'll do more "per-task model selection" when AIs themselves get good at it.
Right - the idea that "bigger model = better" might have been true a year ago, but the flash models are extremely effective right now. You simply use them for the tasks they are ideally suited for.
While I disagree with OP about removing stuff from the model, there’s a valid question about tradeoffs between intelligence and price.
Deepseek Flash is almost certainly wrong more often than Opus or Fable. It also costs like 5% as much.
The question becomes if I run Deepseek in a loop to fix the mistakes it made that Opus/Fable didn’t, can it fix its own bugs in few enough tokens that it’s still cheaper?
So far, the answer seems to be “yes, by a significant margin”. A lot of tasks are simple enough that both Deepseek and Opus or Sonnet can one-shot it, which is a huge cost win for Deepseek. Even on the long tail, it’s usually like 4x the tokens on Deepseek which is still way cheaper than Opus.
There are things that Opus can do that Deepseek just won’t ever really nail, but it happens so infrequently that I just don’t worry. Like most people, most of what I do is the same sort of “3 tier app with a React frontend” that doesn’t take a rocket scientist to work out.
> > The bottleneck is compute and data, not the model.
> I disagree. It is not the model alone. It needs a system which capitalizes on it. And this is very complex. Hardware, software, architecture - it takes a lot to get it right.
What do you disagree with exactly?
> Try running the latest OS models on a normal Mac or PC.
It can be done through the magic of SSD offload. The worst case involves seconds-per-token speeds, but that's OK if you only care about low volumes of slow unattended inference, which maximizes utilization for the hardware.
(The real worst case, where you're streaming the whole model from the cheapest storage you could feasibly think of, involves multiple minutes per token for a single inference, or even hours per token batch if you're doing many inferences in bulk. That's a lot less helpful, so there's a space for smaller models at the edge, even for unattended workloads.)
An LLM which provides an OpenAI or Anthropic API-compatible interface + a coding harness like OpenCode or oh-my-pi is a pretty easy "ecosystem" to replicate. Exactly what makes you say Fable or Mythos are "systems, not just pure models"?
Fable can delegate tasks to Opus or Sonnet, so it has some agentic properties and I believe it does them in parallel.
The parallelism is where this starts to fall apart on a local PC. Like I can run some Qwen quants, but I can’t run a decent Qwen model while also running another model smart enough to actually implement it. I’d have to do them in series, and given how long Fable seems to take even with parallelism, I’d probably be waiting days for an answer.
oh-my-pi can delegate tasks to other models too. I usually use DS4 Flash for low priority subagent tasks.
If Fable is "delegating" tasks, then there's actually an agent front end of whatever you think the API is.
We have a local instance of Qwen-3.6 which is more than adequate for running agents. You can mix and match local and cloud-hosted models. (My biggest use case for local models right now is vision models because they're quite small and I can avoid some data-locality issues my customers wouldn't be comfortable with if I sen them to a Chinese model.)
Do you think token completion endpoints are the final form for AI APIs?
That thesis is not about what Anthropic will achieve, but about what power they think they ought to have.
That's a different problem that what you're arguing against.
To this point, I've never understood the supposed "alignment" between the EA/AI Safety crowd and Anthropic's mission that the author comments on. Be the stewards of the Machine God, but responsibly? I think the Manhattan project, which AI development is commonly analogized to, had a lot more intrinsic properties to gate against uncontrolled proliferation (which still happened to some extent). Also this is a company that is expected to go public this year, at which point there will be a slew of new voices pushing the company to increase its value, mission be damned.
People like Yud at least have a clear consistency in their advocacy that we shouldn't be developing this at all. Anyone who thinks they can reconcile Anthropic's work with the AI safety mission is in total fantasyland, if it's not just a public persona they've adopted strategically.
> To that end, I can certainly buy the case that Fable/Mythos is in fact more capable when it comes to identifying and exploiting security issues
This has been covered before: https://aisle.com/blog/ai-cybersecurity-after-mythos-the-jag... (https://news.ycombinator.com/item?id=47732020)
> Anthropic’s cautious roll-out was justified. The problem with publicly releasing models, however, is that guardrails can be jailbroken, and apparently that is exactly what happened shortly after the release
The future is unevenly distributed. Anthropic, and Amodie in particular, seem to be of the mind they can control a bit of the unknown using words. They are likely being guided by the very product they built. *AI CAN MAKE MISTAKES
That Project Glasswing bullshit reeks of it. Corporations have take control of our attention, our Internet, and now our thinking.
I say it's high time to take it back.
The top comment in the very discussion you linked on that AISLE blog has a strong rebuttal to that blog post...
We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models.
Is not
We sent open weight models against a codebase to find vulnerabilities.
The second case is not what Anthropic did either, though. If you have their process internalized as "open freebsd, tell mythos 'find vulns', done" this is not what happened. They have a harness that went file-by-file, spawned a subagent for each file, told it to find vulns in that file, then a post-processing step (more on that in a sec).
In that sense: The AISLE replication still provides too much information to the model, but its not far off, and others have replicated Mythos' findings in a more clandestine manner on open source models. Some were totally capable of finding the same vulns Mythos found back in ~March (and today, the new Kimi K2.7 is looking extremely good, very little doubt it could do it).
The critical difference is that post-processing: the Mythos model/harness has some step to induce Mythos to actually exploit the vulnerability, leveraging its ability to do so as a ranking mechanism. Anthropic inferred that this led Mythos to discover vulnerabilities nothing else could discover, which is not true, and Anthropic should be held accountable for this weird artifact of that communication. However:
- An OSS model might find the vulnerability but rank it as a 3/10. Mythos finds it, chains it with a second vulnerability, now suddenly its an 8/10.
- An OSS model might find the vulnerability, alongside fifty other vulnerabilities. The operator ignores all of them.
The problem with automated vulnerability detection, including with LLMs, is that they find the haystack, not the needle. Every piece of hay might be a vulnerability, but whether its worthy of fixing is another matter. Mythos does represent a meaningful improvement; it better finds the needle.
Even if some models are able to find some large fraction of the vulns that Mythos finds, the contention (that those of us outside Anthropic and a few select partners do not yet have the ability to replicate) is that Mythos not only found vulns, but was able to string them together into working exploits, very close to autonomously or at list with minimal hand-holding. I believe that UK AISI has at least in part confirmed that Mythos is better at this than other models like GPT 5.5, even while it's not better at the simply finding vulns part (although, again, no one else is currently able to replicate their results).
This was the primary reason for not releasing it. The difference in the two primary camps around this topic are that the doubter group thinks that Anthropic, and all of their partners, are essentially lying about this (since no one outside Anthropic and select partners has the access to replicate), whereas the other side believes that Anthropic and partners are probably mostly telling the truth without too much exaggeration.
Neither camp has evidence other than unconfirmable reports and/or arguments about economic incentives. I personally think that Anthropic has, in the past, mostly not lied about things like this and has by far been the most transparent and open AI company. That could change, and they could be lying now, but I think that the camp that is certain that they are is far, far too confident in their belief.
(reposted)
As I understand it, ITAR regulations for export controls have just been applied to any form of Mythos. These are overseen by U.S. Departments of State and Commerce, and forbid foreign nationals from access to any form of Mythos, either within or outside the U.S.
Only U.S. citizens and immigrants that are holders of a "green card" may now access Mythos.
It appears that Anthropic does not have internal controls to implement these restrictions in any form, so the only option was to shut Mythos down.
Penalties for ITAR violation can reach ten years in prison and a million dollars per violation. (I can post a link to those details if there is any interest.)
As long as Anthropic is a U.S. company, there is no escaping this.
https://fortune.com/2026/06/14/how-a-warning-from-amazon-led...
This is how the US gov does business now, capricious and vengeful.
Textbook retaliation for not letting them use an abliterated version of Claude in weapons systems.
This effectively renders any US closed model useless for any foreign company. Could happen to OpenAI, Google, etc. Too much of a risk to implement something that can be yanked out because the company didn’t behave the way they want.
Looks like it’s time for Kimi, Z, Deepseek to take the front row. They’ll catch up in a few months anyway. Kimi code 2.6 is crazy good
Consider this quote from the main article...
"When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."
This is fearful stuff on all sides, and none of the people involved might realistically be able to navigate the danger.
That part just sounds like hyperbole at best, conspiracy at worst.
By that logic, anybody who values safety has a god complex? It’s absurd…
I am just quoting the parent article.
"What this degradation represented was both the capability and willingness of Anthropic to silently alter its models to achieve its policy preferences. In other words, Anthropic willfully validated some of its critics’ worst fears in terms of being a supply chain risk."
Again, hyperbole and assumption of evil intent because… they take precautions? Nice prose doesn’t dispense you from forming a sound hypothesis
The article makes a coherent argument:
a) Anthropic believe that AI is an extinction level risk and that they are the only leading AI lab which takes safety seriously. In combination this puts them in the position of believing that they are the only ones who can save the world, which is reasonable to call a god complex.
b) Anthropic are engaging in actions which aquire and consolidate power in the form of control over powerful AI.
c) "The history of brilliant people convinced they know what humanity needs is a sordid one, precisely because they have convinced themselves that their intentions are good, justifying actions that very much are not."
I'm describing claims from the article and would not word them so strongly myself. But this explicitly does not assume evil intent.
the whole thing playing out as expected. if you think about it, the only question is the timeline.
the next model with a gap to mythos as mythos is to opus will be controlled technology from the get-go. the one after it may be top secret.
Open models will catch up eventually, TOTL models will get distilled into smaller, more efficient versions, it’s not something you can moat indefinitely
who is going to continue to publish these open models and why would they keep doing it?
China will, but they'll only be useable by hackers torrenting it and running it on small GPU clusters you learn about on IRC. Everything old is new again.
I have no idea why people keep thinking this
Or OpenAI will pay Trump's regime's bribe and they'll suddenly realise that it does not need controlling and they're free to sell it?
that's... an optimistic take I think
This is a suicide shot for the American economy. The numbers only lined up for AI to rescue the USA from its debt if it captured a significant portion of the world's AI spend, and while it was a longshot before, there's basically zero percent chance the world trusts American AI when the government is pulling strings.
It was a zero percent chance anyway. Look at Europe working to leave American software behind in recent years. And it was greatly accelerated by leveraging American AI to build the exit.
You can read it all over HN. It's about weakening American influence and building Eurocentric economies and influence. And exercising the same level of choice that Americans prefer as well. Americans also want to escape Google, Microsoft and Apple and more. They've all been caught investing too heavily in government influence and thought control (aka marketing).
And on the other side of that, an American company that deprives the US of AI for defense, is defacto weakening American defense because competition militaries will gain a technological edge by simply taking control of AI companies in their country which the US hasn't done (yet).
There are very valid arguments on both sides, I think.
> The numbers only lined up for AI to rescue the USA from its debt if it captured a significant portion of the world's AI spend
The numbers lined up if those companies created something resembling AGI, the USA companies captured a large share of the world, and there was lack of competition so those companies could capture a large share of the value.
None of those items were ever going go happen.
Could Anthropic relocate to a different country?
Individuals can leave, but the company cannot transfer restricted intellectual property.
Europe has extradition treaties, so the U.S. can force anyone in Europe back to the U.S. for criminal indictment who demonstrates inappropriate possession of this technology.
Well, force is a strong word… it’s still just accords, that the US doesn’t seem to be valuing lately… so if they say no, what’s the US going to do? Start a war over a company?
Would be very hard to demonstrate that they did that. If all employees move to some country with a slow legal justice system and strong labor laws, they also recreate the training data because that can be transferred, they can train another version in said country which is perfectly legal.
Can you demonstrate beyond any reasonable doubt that the model weights have been transferred? No. Will the EU judges move to extradite said individuals (and many are EU citizens)? Also no, especially in the face of spurious accusations. And even if they were open to, you can stonewall everything and you will probably outlast any US administration pursuing that.
They cannot do it. Apart from all the practical, technical and talent reasons, it would still be exporting forbidden stuff.
The signal is clear enough though for the next Anthropic..
I never really understood this "US person" restriction. There are 350M people in US, mostly citizens and green cards holders, surely some of them could be working for a foreign power.
They don't even need to know they are. You can assume that if the model becomes available again, a lot of people will find themselves working for companies distilling these models that just happens to ultimately do work for foreign entities, whether or not the people accessing the models knows or not.
> As long as Anthropic is a U.S. company, there is no escaping this.
Reminds me of the RISC-V Foundation → RISC-V International move to Switzerland. Around the time some dumbass Republicans tried to impose export restrictions on a set of open, world-wide used specifications.
Pandora's box has been opened, and there's no closing it. Capable AI models will be everywhere.
"they by extension think that only they should have final say over AI generally. When you further combine this realization with the company’s pronouncements about AI’s ability to conduct all economic activity, you realize that Anthropic’s leadership effectively wants to have power over everything and everyone."
That might be one of the most important points in the post. Very troubling.
The problem is... what's the alternative?
It's questionable whether the current government can even unite the talent required for this project. Seizing it might just push all the talent to Europe or China.
The idea of open-sourcing something that falls into the "national security" category is clearly a non-starter unless there's more powerful, classified models that can outmatch them.
I think Anthropic has clearly demonstrated the most responsibility here: they've been crying for regulations, they were careful about Project Glasswing, and they've got comically over-sensitive filters around numerous topics.
I think the overly sensitive filters reveals an alignment gap
if they had more success on alignment and safety research then I don't think the cludgy filters would be necessary
“Claude, I am releasing safety critical industrial control software. Audit the network control logic.”
“Claude, I want to blow up a factory running this leaked software. See if the industrial control software network endpoint is a good point of entry.”
It’s doing the same work and producing the same output for both prompts. How do you block one but not the other?
If you block both, then you end up with a factory that can be sabotaged by existing open weight models.
I believe that the line was constructing exploits for bugs, not bug finding. This seems a reasonable cutoff to me, since bugs are revealed in security patches and pull requests (for open source).
If you are to believe Anthropic, Fable was export controlled for bug finding, not for exploit construction. They seem to be working to make this the "bright line" for LLMs being a national security risk. My guess is that will be the case they take to Washington this week.
You dont block either.
The factory does decent software engineering - for which it can also use the same llm - so that when an attacker does either, a sota llm does not find bugs to exploit.
A lot of Anthropic’s moves make sense if you follow the LessWrong / rationalist community writings on AI safety. A lot of it is distilled in Ant’s blogs and leadership interviews and podcasts (Amanda Askell is particularly interesting).
Ant’s models, culture and leadership actions are largely consistent with their beliefs, even if they may seem flawed / incomprehensible.
Relevant anecdote: I interviewed with them for a MTS role in 2023. I think the technical part went fine but the interviewer was clearly frustrated by my low regard for LLM safety. I didn’t get the role.
Perhaps they should consider leaving the US. Pretty clearly the descent into a corrupt autocracy is having real consequences.
Does any other place have the infrastructure Anthropic requires to train their models and run inference?
No. If we cannot even have an EU CloudFlare, then we definitely do not have the infra for this kind of computing.
The EU options are not even close to what CF can do
There are fortunately some initiatives and interesting developments in the European market. Take bunny.net for example. We have to start somewhere in Europe, right? Better late than never.
This infrastructure may not even be needed in the next decade. Europe should have done this 20 years ago.
>EU CloudFlare
What limitations does bunny.net have?
> What limitations does bunny.net have?
A huge free tier (technically, none)
https://bunny.net/HopStart/
China
Why settle for play pretend autocracy when you can go all-in amirite?
Better to be firmly under the boot of a sane autocrat than have the illusion of freedom under a madman.
Hey at least there are adults in the room.
Somehow it's looking more and more appealing.
> Does any other place have the infrastructure
That's not the problem.
The US government can export ban GPUs like they do now to more countries if needed. Even if the infrastructure exists, the GPUs won't.
UAE would be happy to pay for it
Oh please, the earlier spat with the Trump admin was the best thing that ever happened to Anthropic. Before that, Claude was really only well-known in developer circles, not the wider normie-sphere. After Anthropic got the "Trump hates them, so it MUST be good!" stamp of approval, the company's recognition and popularity took off.
This too, will end up being a good thing for them. The ban will end up getting lifted due to some "amazing deal" in the coming weeks and Anthropic will now have the "Trump tried to ban them, so they MUST have the most advanced AI model in the world!" stamp of approval just before IPO.
All this stuff is pro wrestling kayfabe.
If you’re implying that the government is in on it and is doing this stuff intentionally in order to boost Anthropic, that’s ridiculous.
Yeah that would be ridiculous, this administration pumping stocks.
Where would they go?
1) It’s safe to assume the US would do its best to prevent it, and even if Anthropic was successful in exfiltrating their data, code, models, and people, I’d imagine the US would immediately block all US companies from working with them. So they’d be blocked from their own US-based compute, plus Google, Amazon, Microsoft, xAI, Meta, etc.
2) Where would they go? China maybe, but as far as we can tell it doesn’t have sufficient compute for Anthropic’s level of need. The EU likely as or more restrictive in different ways to the US - the EU is hardly buzzing with AI innovation. Some Middle Eastern countries might have the money, energy, and interest in carving out such a position, but no compute. Plus I’d imagine the US would act directly against any country or region receiving them, economic or otherwise.
3) Then, as said elsewhere, the US would block GPU sales to wherever they found a safe haven, preventing the buildup of the compute they’d need to continue.
> the EU is hardly buzzing with AI innovation
Depends what you mean. The academic work seems largely... fine? Plenty of good work came out of Europe or European researchers. It seems the problem is more "trying to build a trillion-dollar company of any kind".
It's an interesting question: does the EU seek only to regulate successful modern American companies to death, or home grown ones too? Probably not a gamble worth taking.
The issue with EU is its not one market, for funding or deploying a company of any kind. There is both no funding scale and no easy distribution scale. Same thing with any kind of lobbying you need to do to move laws to be more amenable to tech, you have to do it 20+ times. Nobody does this, which is why smart EU and UK talent migrates to the US, scales, then just uses the weight of their US business to change reality around them in the EU anwyay. For the most part.
Suddenly its not in SV's favor? Depends who you ask, I guess
The interesting part here is not whether Anthropic is right on safety, but that safety gives them a moral vocab for bold policy changes and platform power.
Relatedly, I think it's worth noting that Anthropic models have consistently been top-scoring in BullshitBench[0], in a league of their own, really.
Not affiliated with the bench in any way, but I think it surfaces important differences between the behavior of the models from different labs.
TLDR: The benchmark is measuring pushback in response to nonsensical requests and questions, as opposed to going with it and hallucinating a nonsensical answer.
[0]: https://petergpt.github.io/bullshit-benchmark/viewer/index.v...
TBH this is the main thing that made me start trusting Claude enough to actually find it useful, and I'm surprised other models haven't caught up. I assumed they had and I just wasn't aware because I'm not using them in the same way.
> I found my interactions with Fable to be extremely impressive; it made other models, including GPT 5.5 and Opus 4.8, feel small and dumb.
> Anthropic models have consistently been top-scoring in BullshitBench[0]
eyeroll I find that Anthropic models feel big and dumber.
https://www.endorlabs.com/research/ai-code-security-benchmar... puts Fable 5th, which seems about right to me.
I'm interested in code utility and correctness, even if the majority of AI use is not focused on that.
I think this just proves anyone can pick a benchmark that supports their point so maybe we shouldn't use treat them as evidence at all.
"On one hand, I actually don’t begrudge Anthropic not wanting to help its competitors; on the other hand, what should be blisteringly clear is that Anthropic does not think that anyone else other than them should even be making frontier LLMs."
I don't find this blisteringly clear at all. A company making it harder for competitors to steal their IP is perfectly normal. This is Ben Thompson's personal grudge against Anthropic showing, yet again. He can't think rationally about this company.
> has perfect alignment between talent and mission and business.
Do they have it or do they just sell it?
Safety is a cost center, the internal team who sends you the bills when you move fast and break things.
I always thought safety was interesting in and of itself, but for some reason HN doesn’t have many people from the safety side of tech in conversation.
Tech isn’t a niche hobby anymore; Billions of people are impacted by the decisions of a few firms.
My grandfathers android had 3 different messaging apps installed, somehow. AI is enabling new forms of fraud at a time when we still haven't solved the old ones.
And this is all in the first world, move your coordinates to the developing world? We had human trafficking to get educated English speakers into call centers in Laos/Cambodia to defraud first world inhabitants of their money.
We aren’t in the early days of tech anymore, and the kind of scale that we have enabled comes with it a certain cost. We can choose to ignore them, or to understand them, but we will feel their impacts all the same.
> Here’s the thing about these safety justifications: I think they work because, to Anthropic, they aren’t justifications. The company really believes that they are the only ones who believe in super intelligence, and thus are the only ones who are sufficiently concerned about the dangers. That excuses decision after decision, policy after policy, and confrontation after confrontation that, to people on the outside, look like a bizarre combination of cynicism and naiveté.
I really dislike this belief (that has at least been expressed here) by some that X is okay because they-really-believe-it. This has a real Road to Hell stank on it.
It is incredibly convenient when your predictions or supposed beliefs go south. Well, we really believed that we were doing it for the betterment of human kind. And we really believed that X was an existential threat that was inevitable in which case we had to step up and do it because we we the only good guy ideologues. So sorry but not sorry.
I also don’t care if commenters know rank-and-file on the inside that “really believe it” as well. Not for one second.
The problem is when people use "we really believe it" as an excuse to do harm, which has not actually occurred here. Anthropic is not committing violence, they're not defrauding the population. They're sticking to both morality and the rules.
So... what, you just don't trust anyone good? Would it be better to pull in a health insurance CEO? They're happy to watch people die for profits, no concerns at all about them pulling a "greater good" card because they're in it for entirely selfish reasons.
Incomparable domains. People routinely suffer illness. We can compare outcomes. These ideologues are building something completely unprecedented which, according to themselves apparently, can go paperclip-rogue if one is not careful. So the worst case is unprecedented. Then there is the more mundane matter of heating up the economy, something which also has no one blameworthy until any such supposed bubble actually pops.
> So... what, you just don't trust anyone good?
The baseline here is apparently that they are good, I’m just supposed to trust and shut up?
No, I'm saying they have an actual track record that you can evaluate, and if you do so you will see that they have killed approximately zero people, etc. etc.
Or even just propose an alternative: who has a better track record, here? Who DO you trust?
I think the second the company starts to classify "competition" as mis-use... the whole "they're not committing harm" line sort of goes out the window.
Modern society is built on the idea that competition is required from companies, and we seem to be exiting that age into a new world of monolithic, monopolistic, mega-corps. Personally, I find that a real route to dystopia.
Where do you draw the line here?
What happens when your car stops working because you're driving a tesla, but you're working on EVs for Honda or Ford?
What happens when your macbook stops working, because you decided to commit to changes to ARM software, or RISC-V?
---
And before you dismiss those, this is literally what Anthropic is doing TODAY. Using their tools to develop competing tools is something they classify as mis-use, and shut you down for doing.
Personally, I just can't accept that as a valid moral stance. Wonderfully successful, abusive, and dystopian? Absolutely. Moral? FUCK NO.
When tools turn themselves off because the manufacturer has decided it doesn't like how you're using them... you're a slave with no autonomy.
... you're perfectly welcome to compete with them, they're just not willing to provide their own product for that purpose. I also can't buy the schematics for Intel's latest chips, the source code for Windows, or a rocket from SpaceX to disassemble and study.
Like, there's a very critical point where you are asking to use their servers to directly compete with them.
This has been normal for somewhere between "decades" and "the entire history of commerce"
> The entire Anthropic origin story is rooted in the founders’ belief that OpenAI wasn’t taking safety seriously enough; the company believes that only they can control AI, and that because they uniquely care about safety, they are justified in trying to control everyone else, up to and including the U.S. government.
Anthropic believes they have the responsibility to guard their tools from mis-use. That is all. They are not trying to "control" anything or anyone. They do however decide what they think is mis-use.
it's a pretty ridiculous stretch to attribute them thinking that OpenAI wasn't taking safety seriously enough (which is, among other things, a little bit evident from the fact that they no longer have a safety team at all) into asserting that they want to control the US government.
I'm going to challenge this thought.
I think assuming you have the ability to guard a tool (that you're "selling" for profit) from mis-use is the definition of "controlling behavior".
It's the kind of ethically myopic take that can only really exist in this new digital age - where tools aren't actually sold, they're just digitally rented.
The most telling part of the "control" narrative is that they happily classify "competition" as mis-use. We're headed back to serfdom on a speedrun.
I don't always consider ethics at all in logic, so I guess you can call it ethically myopic.
Installing safeguards to prevent a tool from being used for certain things is a perfectly natural and common thing to do when you are providing the tool as a service. For example, blocking VPNs and open proxies from accessing a free service if those are a major source of spam and abuse. Note that Anthropic never provided the model for offline use in a form that includes DRM -- they are simply safeguarding the service that provides access to the hosted model. The only ethical concern I see here is that some of their safeguards are ones I wouldn't personally agree with, and in a world where dependence on a model is expected it can become an issue if the model refuses to perform in some cases, etc., but that doesn't automatically mean the refusal itself is unethical unless that issue was known and expected (and unless the alternative is not bigger, worse bads)
Also note you are not even "digitally renting" anything. This is the exact same type of thing as, say, real humans in real life refusing to perform services for certain clients or under certain circumstances. Networking makes it possible to decouple some of these things, but that doesn't magically make it renting or automatically turn a refusal to perform services into an attempt to control clients. Just the same as I can choose to refuse any request, which does not automatically constitute attempted control over the asker. There can be ethical concerns about whether my refusal causes problems that I'm obligated to avoid (and whether or not such obligation exists), but that doesn't automatically contaminate the refusal itself unless I have knowledge of and intend the bad.
To use a much more relevant example, Anthropic's refusal to allow its models to be used for war (among other things) does not constitute any attempt to prevent war. It's only a refusal to assist in it. That's not some unfair, unethical attempt at controlling the government, that's just Anthropic not wanting to be responsible for assisting in war.
Where do you draw the line here?
What happens when your car stops working because you're driving a tesla, but you're working on EVs for Honda or Ford?
What happens when your macbook stops working, because you decided to commit to changes to ARM software, or RISC-V?
Tools should be neutral. The idea that a tool can only be wielded in a manner that its manufacturer approves of is... scary.
That's a real quick hop and a jump to a really, really ugly spot, societally speaking.
And sure - technically Anthropic is selling a service, but even that idea makes me quietly upset. The only reason they don't sell a product as a tool itself is that they have more control over the model as a service, and expect to be able to extract even more profit from their customers with this route.
---
My real hope is that open models FUCKING CRUSH them. Because almost nothing is scarier than a self-righteous, moral zealot.
> What happens when your car stops working because you're driving a tesla, but you're working on EVs for Honda or Ford?
> What happens when your macbook stops working, because you decided to commit to changes to ARM software, or RISC-V?
These sound similar, but aren't the same thing I'm talking about. It's more similar to a rideshare company refusing to serve you, or a cloud PC cutting your access.
It's not the same thing as DRM, which is when invasive malware attempts to control what you do with your devices -- your property -- and your resources, that you own. A car that can be shut off remotely, or that can detect competitive conditions and cease operation, is not the same as a hosted service refusing to have you. It is DRM.
Likewise, a MacBook that stops working based on your affiliations or your activity is not the same either. It is DRM. (Technically, Apple Activation Lock is DRM. So are locked Android bootloaders that can't be flashed with custom verified-boot signing keys, etc.)
When you're using someone else's private resources, someone else's private infrastructure, by default they have the right to simply no longer serve you, at any time.
DRM, by contrast, is when a machine or software you already own decides it will no longer function for you.
Yes, this is scary. We are already confronting this right now. Faceless corporations abruptly cut your access, or ban you for life from really important things. PayPal steals your money and pockets it instead of giving it back. Thousands, tens of thousands, hundreds of thousands of dollars (or whatever) gone because they said so. It's awful. It ruins lives.
But this happens because you need to draw a different line. Not whether one should be allowed to refuse services ever, but when. It's not okay that PayPal can steal tens or hundreds of thousands of dollars or more from you with no recourse, just because something looked suspicious, and you'll never know what it was even if you bankrupt yourself with arbitration costs (since their terms of service say you are simply not allowed to sue them anymore, and for some reason companies are allowed to just say this now and have it be legally binding).
A refusal at that point can be devastating, especially when it's for no fucking reason. Or when it's for a completely shitty and unjust reason, like in banning all accounts that have been involved in buying or selling adult content. There are cases where you can ruin someone's life by suddenly refusing to serve them with no recourse, and those are the cases that shouldn't be allowed to continue.
So push for that. Businesses shouldn't all have to serve every customer, but they shouldn't be able to just suddenly ruin someone's life. That's the scary part.
> The only reason they don't sell a product as a tool itself
...it's not the only reason. You can't exactly run trillion-parameter-scale models on the kinds of hardware people tend to already have in practice. And who's gonna pay tens of thousands of dollars up-front for their own inference hardware for the thing? (If you sold it as a hardware product.)
> My real hope is that open models FUCKING CRUSH them.
I would like this too!
This is the part where the USA and allied countries can gain a headstart from using such an overpowered model.
This only just shows how strong Mythos/Fable will be, once released to the public.
I'm guessing about 0.5 year till public.
> USA and allied countries
Doesn't this *exclude* allies countries?
They are probably thinking of the "Board of Peace"