The fact that AI models can be so easily distilled and replicated is such a stroke of luck.
10 or 15 years ago if one had asked me to envision a future where a private company invents artificial intelligence, I'd have thought for sure they'd have a massive moat, be very difficult to catch, and it would create an almost instant monopoly.
Rather, it seems that selling intelligence might end up as a race to the bottom.
Who woulda thought that just having access to enough textual inputs and outputs and a vaugely similar transformer architecture would be enough to copy-cat rather useful intelligence.
It reminds me of the seo antics out there. The search results page is the engine, much like how distilling is the "intelligence" for your chinese room machine
Funny you mention Chinese Room and LLMs in the same response, I would say LLMs proved Searle wrong, agents now make cutting edge discoveries and meaningful problem solving. They not lookup tables though and you need to pay for inference, so the intuition of syntax doing the work of semantics without understanding was wrong.
well, a stroke of luck until the whole US stock market crashes & everyone's retirement funds get cut 40% I guess when people internalize this. it will have to happen sooner or later though I suppose
interestingly also, open weight models are also more effectively run in the cloud, so it creates a weird scenario where the frontier labs crash but the compute providers, not as much
I wouldn't be so sure about that. The popping of a bubble is usually just as irrational as its rise.
If investors start fleeing from senseless businesses in the AI sector, that does not mean that sensible businesses will be spared. These things follow herd mentality, and the primary drivers of the herd are greed and fear, not fundamentals or business logic.
It’s not as bad as dot.com of course since all purely AI companies are private and the ones on the market have pretty decent cash flow outside of AI. But the stock market pattern is not that dissimilar, the largest increases are usually just before the crash.
Yeah the last year has been astonishing, my portfolio is kicking ass. But I'm 10 years out from retirement and I am pretty confident a correction is coming; I hope the correction happens soon.
The market (s&p500) crashing 40% puts us at levels we haven't seen since 2024, well into the creation of LLMs. Probably a worthwhile trade if it was either/or!
I think the only moat in the future will be the scale of hardware deployment. If one company is able to deploy an order of magnitude more silicon, they'll have a firm grip on a SOTA model and massive inference usage.
China or SpaceX seem like the 2 likely candidates in 5 years, but who knows.
If (a) demand for AI continues to increase, and (b) SpaceX can get to ~$100/kg to orbit, then they will have a ridiculously deep moat. Probably more like 10 years, though.
Yeah, very hard to predict the future at this point. But the Starship + Terrafab combo will be this type of order-of-magnitude-moat IF it works out. Big if.
If it doesn't work out, I think China's exponential terrestrial energy deployment will eventually give them the lead, IF they can get enough chips. Another big if.
They will have moat in the satellite launching business, which is not useful in the AI datacenter market.
You can put AI chips in datacenters in the desert for far less than $100/kg. With lots of solar power available, the option to easily access your hardware and far less radiation issues.
The datacenter in space story really only exists to make it possible for Musk to sell X to SpaceX and make more money from the IPO. That's all. There is no engineering reason.
Cooling is probably the easiest problem to solve, easier than power. And in both cases, the problem is solved by mass to orbit. All you need for cooling is a big f-ing radiator. Solar panels are chips, and not trivial to manufacture. But a radiator is just a hunk of metal with some pipes.
That's why the cost of mass to orbit is the most important thing. You can solve almost any space problem by just throwing more mass at it.
Musk's bet is dysfunctional politics will make it impossible to build enough data centres and the energy needed to power them. There are many, many reasons that might be wrong. However, if the economics are even close to viable, they could start throwing up data centres quicker than anyone can build them terrestrially (at least in the democratic west).
So to be clear, the prediction is that every single location on earth will be economically infeasible due to politics, and that these same politics will have no impact on the launch or satellite businesses?
GMAFB.
It's stock pump bullshit from a guy who has figured out how to extract the maximum from stock markets.
> Meanwhile Nvidia just announced a space-optimized Vera Rubin designed with SpaceX designed for orbital datacenters.
well, no, they announced the concept of a space-optimized Vera Rubin designed with SpaceX:
> NVIDIA and SpaceXAI are working to adapt that foundation to the requirements of orbital computing while preserving a common NVIDIA architecture and software ecosystem.
The press release is really announcing that SpaceX's terrestrial data centres are going to use Vera Rubin.
Yours is the only valid argument. If space datacenters are not competitive with terrestrial (in terms of megatokens/$ or whatever), then they will never work and SpaceX will fail. That's the whole ball game.
But anyone who thinks they can predict those prices in ten years is wildly overconfident.
Telling. Downvotes but actually no arguments. Fitting, because there are none.
Referring to a baseless prediction by Sam Altman that AI will become like electricity without any push-back? Who really thinks Sam is working toward that future?
He already worked to undo every early promise made (non-profit, open source models, strong governing board, strong ethics/alignment/security focus). He's flip-flopped on other things like first characterising Trump "an unprecedented threat to America", then contributing 1M USD to Trump's inaugural fund far exceeding his earlier political contributions.
Lately OpenAI, under his supervision, has also been working with Anthropic to lobby regulators in Washington for restrictions on open weights models - why so if not to undermine a free market in favour of an oligopoly?
Beyond that, you have the simple fact that most of his personal wealth and very probably the fate of OpenAI hinges on AI inference NOT becoming an interchangeable commodity.
I mean.. Honestly. The naivete is downright astounding.
Easy to get there without doing anything abnormal, 2 EVs and a large 5 bedroom house that gets air conditioned / heated, in a location that does time of use pricing. My bill is closer to $200/mo but if I didn’t have solar & battery it’d be in the $400 range.
I took GP as asking for evidence that the reduced-cost Sol is actually a distillation of the previous-cost Sol. AFAIK, providers distilling or quantising models and offering them as the same model have not been proven.
I doubt he was claiming that. He's probably saying that the ability of Chinese companies to be able to distill frontier US models has put downwards pressure on the price of all models.
I'm saying that it is unclear that without distillation this wouldn't still be happening. There is a massive narrative that no one but OpenAI, Anthropic, and Google can make a model without distilling. But there's basically no evidence of that.
Alternatively, modern AI is good enough at optimizing its own kernels that it just keeps pushing costs down. Unlike the semi-decentralized inference provider community, OpenAI has both the talent and the compute to throw at the problem of making their models much more efficient to run.
GPU kernel optimization is just the kind of well-bounded problem with clear success criteria that AI loves.
To be precise, the distillation mentioned in this paper is not the distillation used by other model companies. In the one mentioned in the paper, your teacher and student model typically have similar architectures - and you typically need access to the full logits. What happens here instead is motivated by the fact that these companies don't have access to the training data and compute that anthropic/openai have. The distillation they do basically amounts to using traces from ant/oai models trained on much more data with a lot more compute (in many cases including the hidden intermediate tokens! turns out there were many ways to coax it out) and then either directly training on it or using it in many ways in post training pipelines. It falls under imitation learning, IMO.
The training data likely references Claude significantly more often than Kimi, given the popularity of the models. There will simply be more examples of “Claude” being the response to that question.
It turns out you can train a 1b model at almost 1000 tokens/s on a m5 max laptop. As a personal experiment, I've been asking Sol for synthetic training data and synthetic agentic training data (model distillation in it's purest form), plus modified opencode, codex transcripts etc for training data, and nobody's even paying me to do it. If I'm doing it has a hobby, you can bet industrial users are doing it.
Been using a lot of Kimi K3 lately and the answers have been… „load-bearing“ to the point of hilariousness. It‘s obvious from where they distilled, even if sceptics rightly point out it can‘t have been the only source of their secret sauce, as it‘s been better than the current Opus 4.x at the time of release.
Even before LLMs, ML folks were already aware that you can use a model to teach another model. I doubt this is something AI companies put at the top of their investor materials, but it's been nice to see it play out.
That said, there are other moat factors like, a US company needing to use a US AI provider, sticky customers due to corporate onboarding friction, and others. Not nothing, but not as large a moat as some imagined.
Yes, but 10 or 15 years ago, I would have thought that there'd be more to it than just a slight modification on the ideas behind a CNN to get this level of AI.
There were somewhat good reasons to think it needed more than just this data-driven ML approach.
There's something startling about how (relatively) simple these networks are and yet how powerful they are. The main ingredient the AI darlings are using is vast amounts of compute and data. I don't want to take away anything from what the researchers came up with, but I suspect even they are surprised at how capable some of these models have become.
early on there was a lot of talk about "emergent behaviors" in the models where they were good at things that were unexpected or did not align to the training data. IIRC doing arithmetic is one example from early on. I think this is where the AGI craze took off, the labs were throwing more and more data in the training to see what other behaviors would emerge. The thought was with enough data and enough parameters AGI would surface on its own.
Then i think tool use became a priority or at lest a sibling priority to more data/more params. Along with multiple specialized models communicating with each other which is sort of a special case of tool use. That pretty much brings us to today.
this comment is delusional. LLMs are awful at arithmetic and AGI is still very sci-fi, otherwise Claude would have told Anthropic how to cheaply generate energy for it to justify its existence by now. As long as the energy use debate persists you can be assured AGI has not arrived.
I'm unnerved by how alphazero is more complicated than the "intelligent llms"; it has at least multiple heads and MCTS, a search algorithm. The LLMs seem to just be monolithic (if complicated) architectures where tokens go in the bottom and tokens are spit out at the top.
The internet created lots of monopolies with network effects and economies of scale.a low margin commoditized business that still attracted a trillion dollars of investment to get off the ground was not how I envisioned it happening either.
> Rather, it seems that selling intelligence might end up as a race to the bottom.
Personally, I came to this conclusion early this year. To acquire the data that AI Companies are using to train their models is low cost and once they have it, they can refine and store it. Creating the LLM takes a bit of money but it is not a serious blocker. Clearly, the Chinese companies can make AI so they will drive down costs. There is a need for good AI (Not just Great AI) and it is not cost prohibitive to make good AI (The same with specialized AI).
My prediction is that AI will spilt into two categories, Great AI (High Cost) and Good Enough AI (Low Cost). Which for the long run of AI and companies that use AI, this is good.
Hmm, don't people think that if the frontier labs really put enough engineering effort into preventing distillation that they would be able to do that, or at least diminish it significantly? I'm sure there are variety of additional techniques they could use on top of what they already do, but I suspect it just hasn't been at the top of their priorities yet. Maybe that will change soon. Worst case they could add additional hurdles to account creation ("know your customer" type of thing).
At the end of the day, while you can do your best to obfuscate your reasoning tokens, it's a losing battle to hide actual user-visible output tokens. The very nature of API offerings is that you can't do KYC on where that API's output is going - there's a rich secondary market that's not going away.
And with the sheer volume of data created from that, coupled with benign-seeming prompts like "plan out your reasoning in a document before implementing" that could never be patched without breaking existing customer workflows... there's more than enough for someone to distill on. Even if that only gets them to not-quite-frontier, if you're pushing the frontier every few months, they're only ever a few months behind you.
Even if it were possible it wouldn't change the outcome. China is capable of training frontier models even without distillation. Distillation is only an accelerant.
The primary resource you need to train LLMs is money and China has plenty of that.
The frontier labs have competing goals in mind. They want high growth (which means little friction for account creation), API access (because enterprise money is the best money to have), and distillation protection.
Besides, identity verification that actually works at scale is a much harder problem than identity verification which is good enough to satisfy your compliance people and regulators. Especially if the fraudsters have a major world government standing behind them, and if their aim is to be identified as a real customer, not one customer in particular.
The nature of LLMs is that you give them input, they give you output. That allows for distillation. They do try to make it harder by hiding the chain of thought stuff, but fundamentally, if your model is publicly available, its distillable.
> Hmm, don't people think that if the frontier labs really put enough engineering effort into preventing distillation that they would be able to do that, or at least diminish it significantly?
Distillation was big news a year or even 6 months ago, but as far as we can tell it's not really a moat anymore. Now that multiple players have trillion+ parameter models and the capacity to post-train them, there's no putting the genie back in the lamp.
OpenAI could still have a significant moat. ChatGPT occupies most consumers’ minds when they think about AI and has become a household name. Google won because search became a habit-forming product people grew accustomed to using. Bing was once effectively indistinguishable from Google Search, yet still failed to achieve mass adoption because users had already become accustomed to “Googling” things. The same could be said for people "ChatGPT-ing" things. If OpenAI and Anthropic are smart, they will maintain similar pricing rather than aggressively undercutting each other, allowing the market to resemble Home Depot and Lowe’s, or cloud computing, where AWS, Google Cloud, and Azure coexist as highly profitable competitors. Unfortunately, I doubt OpenAI or Anthropic will pursue this strategy, as both companies appear to be acting as though the race to AGI is winner-take-all even if the market may ultimately support several highly profitable competitors.
OpenAI has a free tier. I'm guessing a lot of people never need to upgrade. Not the case for me but I can see the average person only needing to run a few prompts every day.
My understanding is that most OpenAI users are on a free tier. Secondary effect of this is that OpenAI free tier model capability (assuming Luna) is what what most users associate with frontier AI capability giving somewhat warped view to many people.
> OpenAI could still have a significant moat. ChatGPT occupies most consumers’ minds when they think about AI and has become a household name.
ChatGPT is AI for the average non-techie the world over, but the average non-techie isn't eager to pay for it. The more progress that's made, the less incentive to pay - most people are happy with the total garbage spewed by google AI overview. They'd be happy with google's 30b MoE gemma, whose performance will likely be squeezed down to something that can run on a phone in 2-3 years. Why would they pay $20 a month?
It's why OpenAI is pushing a variety of things such as ads and offer a more polished ui/ux than the competition, I think. The models are already good enough for people who just want to know how much sugar to add to their cake or when's the next basketball match their team plays - it's OpenAI's game to lose those people, by annoying UX and whatnot. If they can make a few bucks off of every one of their non-paying users it'll stretch their runway immensely. Those users will never go to Antrophic or some cheap Chinese model, but they might defect to Google because a popup on Android / in Chrome told them to.
Yeah, I think most people here don't realize how far most of the population is on the adoption curve with this stuff.
I had a discovery call last week with someone who did not realize he could use ChatGPT for work. It was a revelation that he could drag a PDF into ChatGPT and it could summarize it for him.
FWIW, guy in his late-30s in a pretty senior sales role.
Google also spent heavily setting up deals with other platform owners, invested in Chrome and Android to establish Google search as the default option which most users accepted. If integrations of good enough AI features are made within existing platforms most users will probably accept using those and not think too much about whose model is powering it.
It reminds me conceptually of the idea of using a ST:TNG replicator to just give you another replicator of your own, or asking a stereotypical genie for "infinite wishes". The genie is indeed out of the bottle in many ways.
And for a lot of non-frontier purposes these days, you can bootstrap via LLM-as-judge so your hyperspecific wakeword model or whatever can be trained with little to no human input, that aspect of it is fully terrific.
The frontier models are a replicator that can give you another replicator which specifically produces tea, earl grey, hot, when you push the single button, and does nothing else.
the moat is real. the big expensive base models are like the data collected from huge particle accelerators - there's enough unknown structure to be mining for years. you can extract features with more and more generation loss but access to the raw weights is a real advantage, and literally a moat if the interesting behaviors are fenced off
Was it not obvious that the value and advantage was going to be in AI-adjacent services?
The quality of the harness UX, and random fun crap like Sora, it's a shame that OpenAI killed that so soon, and also Group Chats in ChatGPT.. they risk running a Googlelike reputation at this rate
Maybe ultimately whomever can be the "Apple of AI" will win
For what it’s worth, “race to the bottom” typically refers to a scenario that we absolutely do not want as a consumer. We do want a highly competitive market that drives prices down, but “race to the bottom” specifically refers to a scenario where firms compete by minimizing quality, regulatory oversight, consumer/labor/environmental protection, etc.
It's a mistake to think only OpenAI and Anthropic are actually spending the big bucks on pretrain, and the others just distill that.
The Chinese models are pretrained on large clusters just like OpenAI ones are. Yes, they use outputs of the frontier models to further improve the final model, but even without those outputs they'd still have very strong models.
It's not like in a world without distillation things would be much different as you claim.
They'd still have strong models without distillation, but strong enough to challenge frontier models and to claim the meaningful market share that they have? Probably not.
> The fact that AI models can be so easily distilled and replicated is such a stroke of luck.
Sort of. It means the country on the verge of monopolizing all aspects of hardware production (China) doesn't need to rely on outsiders for the software. So while that weakens one monopoly it strengthens another.
It feels like a slightly more palatable version of what Anthropic has been doing, with their constant "use your free tokens before they expire next week!" campaigns. But it's feeling more and more ominous now, like they've hit the top of the demand curve and need to pull back prices to continue growing.
There are no open source models, at least not useful ones (yet) [0]. Open weight is not the same as open source. The current "open weight" models are just opaque binary blobs you can run on your own computer instead of through a web API.
Nemotron Super is sort of open source in the sense that Nvidia provides almost everything you need to replicate it from scratch. Of course it’s performance is not exactly stellar but it could be a good starting point for other research teams.
This viewpoint doesn't make any sense to me. The weights + inference code are the "source code" for AI. I literally don't know what else you are demanding for the "open source" label.
I agree. But models in difference to compiled binaries, are useful as just weights and can be further refined and post-trained, at least.
I don't know LLM theory well enough to say if there's some secret sauce they can hold back that makes training ineffective. Less effective I'm sure, we don't have access to their smart training schemes, but post-training should always be possible IIUC.
At the risk of taking the analogy too far, I would treat refining like modifying a dynamic library. You can technically modify behavior, but only in a very coarse way.
post-training is like writing a wrapper around the binary. It is closer to building on top of than truly modifying, in that you can tailor things to your needs slightly but cannot make fundamental changes to the underlying thing.
The weights + the architecture are already 100% of the code, the transformer is just a mathematical expression + helper programs whose sources are provided. The transformer itself is not even a stateful program, so a it is no more a binary than Piet or Tromp's BLC are. It's merely incomprehensible. Training isn't compilation either, since training a model is closer to program induction and the data are samples defining the solution space.
That’s not how it works though. Two training runs on the same data don’t produce the same weights. And if you want to modify the AI, you do so by fine tuning the weights not rerunning training. In every respect that matters, the weights are both the binary and the source code together.
> I literally don't know what else you are demanding for the "open source" label.
you need to "literally" go read the definition of open source software or even ask an LLM to define it for you. Weights + inference code are not the source code they're more like the compiled binary. Making modifications to the behavior of a model with additional training is like writing a mod for minecraft. Sure, you can change things but it doesn't make it open source.
Calling these models "open source" is an old trap that software companies use to use. Free to download but then, once you're fully comitted, the trap snaps shut and you must pay up to continue.
The vendor lock-in trap does not apply here. Users don't actually even run the original weights, they mostly use open source inference engines with open source agent harness. If AI provider decided to start charging, a) released models would not be affected, and b) people would drop it instantly and move to other models.
The actual problem is that we know nothing about the training set of any open-weights model. They could be intentionally biased to influence users, from political censorship to brand advertising, or general shaping of cultural norms. You run the model on own hardware not knowing if it is designed to act against you. Having whole chain open source would allow audit and reproducing the results.
50% off at open router is also still applied so it comes out at $2 / $10 per 1M.
Feature request for Artificial Analysis, allow us to see these live prices on the pareto. It would amazing to also see what a 25,50,75,100 % utilised subscription costs compared to raw tokens.
Once they make a model better than Fable I’ll be switching to Codex. Their priorities in terms of consumers seem to be better. I do think Anthropic has some solid safety viewpoints, but I don’t necessarily think that either is entirely aligned yet with delivering exactly what humanity needs. Maybe the AI will help align the AI companies when it gets smart enough. That’s the real misalignment I’m concerned about.
I don't think these companies have humanity's needs in mind when they're developing these models. Although the last part of your comment struck me as a bit comical, I genuinely believe that an AI can have way more empathy than a corporation. Afterall, a mimicry of empathy is probably better than no empathy.
It's a funny comparison. Comparing the empathy of some software to the empathy of a company. It's like saying my car was more empathetic than my school. How can those two objects even be compared is what i am wondering
Well, companies and AI are both entities that can make decisions and take actions that involve humans. Those might be empathetic or they might not. So of course you can compare their levels of empathy. I don't really understand why you think that you wouldn't be able to.
For example, health insurance providers are renowned for not being empathetic. Charities are the opposite. Sometimes companies even build it into their identity, e.g. Cards Against Humanity.
As for AI, I haven't seen a strong difference in empathy but it's definitely true that the big AI companies at least try to make their models moral and empathetic. Even if it mostly ends up just being annoying.
You are basically saying you will switch from one evil to another because the other seems less evil for now.
It's funny how people make these alignment comments while ignoring how misaligned the leadership at these companies are right form the get go and they just play mental gymnastics to deflect those facts when confronted with them.
It feels like 5.6-Sol is already fairly close to Fable, and in some ways exceeds it. Just the other day I had Fable draw up a solution for me, and then I fed it into 5.6-Sol and said how does this look ... it found an oversight and told me about it, and when I then fed that observation back into Claude it acknowledged the miss.
I've noticed also that 5.6-Sol is more concise with output than Fable (and let's not talk about Opus, which is even more wordy).
It's common for different models to find holes in another's work. There are various good reasons for that.
FWIW, we use ChatGPT for our primary model and use Claude to do the reviews. This works better than ChatGPT doing it's own review even with a clean session/context.
Agree, but the point is not because Fable is better than Sol, it's because it's .. different .. it just looks at the problem through a different angle.
Same here. Grok Build 4.6 for me, given how cheap Grok is and how Sol is supposed to be "the" SOTA, it finds a surprising amount of bugs. Most of which Sol agrees with needs to be fixed or improved.
I've done this tens of times between these two models and it works great in my experience. Sol initial back and forth with me. Commit. Let Grok review. Sol fix. Only then do I start reading the code.
It's beyond common for a model to find holes in its own work, as well. I have an iterative review as the part of all agentic work, and it always finds something to fix, and will sometimes spend hours fixing its own work.
Did you try to say "think more deeply about this problem" to fable after getting your solution, having one model focused on creation then blaming it for not doing proper review when the other model was told to focus sol-ely (pun intended) on review is not a fair apples to apples compaision
I mean, then it sits there stewing for 20-30 minutes when you can ask sol and get the same answer in 5.
Like, the quality of the anthropic models is fine, but they’re so incredibly slow. Claude reads files one at a time while codes dispatches tool calls three or four a time.
The fair comparison would be to also do the reverse: start with Sol then have Fable clean up. Then compare the Fable-Sol and Sol-Fable outputs side by side.
Fable 5 is just straight up a larger model - I'm guessing at this, but there is plenty of evidence online from people far more plugged in than I am. OpenAI is pursuing a strategy that yields greater operating margins and penetration of their model to developers. Fable's high cost makes it so premium that Anthropic has to reserve it for only the richest customers and corporate users. That's not a winning formula long term.
I believe the reason we have not seen a Fable-level model from OpenAI yet is because doing so would box them in on costs just as harshly as it has boxed in Anthropic. They are letting Anthropic make this mistake.
The top comment on this thread was about AI models being easily distilled being a stroke of luck.
This should not be surprising at all. Every new students spends tiny fractions of time learning knowledge that took many lifetimes to discover. This fundamental to the progress of intelligence and understanding.
It should not be surprising that AI can be distilled. It's the logical method of training; I would hope that each frontier model is in fact not trained 'from scratch' each time.
We should expect future frontier models are simply distilled versions trained by specialist models, the same way humans learn from a series of professors, papers and canonical books on each different subject material. Models like this can be trained incrementally, or a so called Mixture of Experts (MoE).
The "until at least Nov 21st" thing presumably mainly affects teams that pin to GPT-5.6 Sol (maybe after extensive testing) such that they won't be switching to GPT-5.7 or GPT-6 or whatever new model is released between now and November.
Good timing. I'm not too happy having to pay MAX pricing to even access Fable, and I've had a couple situations where Fable missed things and GPT 5.6-Sol caught it. My needs are modest and I can get by on a $20 OpenAI subscription, so the odds are starting to look increasingly like I'm going to drop Anthropic altogether.
Using codex every day, in spite of which, I hope some day providers will just start naming their offerings small/medium/large, a bit like we eventually started doing in software testing. Trying to remember what Sol is or why it's better than the other thing is more cognitive effort than I can muster at this point. And that's a sure sign of commoditisation in itself
I think model naming has been atrocious in general, in part because newer "lite" models surpass the capabilities of previous "pro" models (case-in-point: Gemini Flash which now surpasses the capabilities of the latest Gemini Pro, with a newer Flash Lite vying somewhat unsuccessfully for the old Flash price/positioning), but gpt 5.6's Sol/Terra/Luna split is really not bad at all - probably easier to understand than Starbucks' cup sizing!
The problem becomes when you add in the adjustable reasoning efforts and you end up with {model, reasoning_effort} combinations that end up completely obviating particular model classes altogether for at least some percentage of queries; e.g. with GPT 5.6 the price/performance Pareto frontier is dominated by permutations of either Luna and Sol, with Terra nowhere to be seen (but then if you need "large model smells" that aren't captured by your benchmark you can't even rely on this, as a model like Luna simply isn't capable of encoding sufficient world knowledge in its weights to perform certain tasks at any reasoning level but you might be able to get away with Terra on low reasoning, but no one seems to be covering this for some reason).
My prediction is that this becomes permanent. There is no good reason to be much more expensive than opus. At $4/$20 they are roughly at parity.
Making 2/10 permanent would be a killer move and make a strong argument against open-weight. For the sake of the open weight ecosystem I hope they do not.
I'm guessing they do this to help move people off old models they want to depreciate. It's not a price hike for old models, it's them removing a discount!
The price difference to Deepseek models (deepseek-v4-flash, deepseek-v4-pro and deepseek-v4-flash-vision-exp) is still significant while the performance difference is not.
the fastest, most compliant model remains the cheapest. you could discount Sol to the same price as Luna and i would still prefer Luna for 90%+ of tasks. once you hit this baseline capability, speed and predictability dominate for anything i'd throw these at in production.
Which is not that great for people using less than 50% every week, because the next reset date moves forward too. In essence, it is redistributing compute from people who haven't used their quota much to those who have.
Though I think they gave a banked reset this time.
Pretty much. I had 24 hours left with 80% remaining credit (planned big session tomorrow). There was reset one hour ago, that effectively halved my tokens for next 8 days!
Subscribers were already getting subsidised compute and value compared to the 20-200$ fee, peak cakeism to want more considering the alternative would likely be consumption based pricing for individuals so you "benefit" from OpenAI giving up some of their markup (and heavy users end up SOL).
There was context for neither in your original post and I personally saw a lot of similar takes like the above online - apologies for the misunderstanding. I personally thought it was pretty clear considering the parent link is a link to the API pricing and their public communication on their twitter/blogs were "Subscription usage remains unchanged".
Typically subscription usage follows API pricing, but that disclaimer was absent from this HN link. It's only clear if you've seen those other communications
Does it mean that subscriptions get more tokens? I’m testing it now for coding instead of claude and it’s very important to understand if I get more due to the price reduction.
The Chinese are coming after these greedy-ass frontier labs. Today Xiaomi unveiled it's own inference machine .... I bet it's gonna be cheaper than Nvidia DGX, shipped with open source models that anybody can have at home.
I'm not sure I could characterize the frontier labs as greedy, given that they've been consistently losing gargantuan amounts of money.
The people who give them the money are greedy, and hopefully in for a rude awakening. Starting from Nvidia's vendor financing which has a very direct benefit to them, through to every company and oligarch investing into data centres in the hopes of being one of the ones left capitalizing on capturing the livelihoods of the majority of what remains of the "middle class".
It's either hopium or a truly horrific dystopia. Something's going to have to give.
But can these really be trusted? There was just a HN post which proofed that you can train a model to behave completely different on a certain day. How do we now, that these models do not find a way to call home when they see interesting informations (probably irrelevant on a personal level, but corps, government and military might care).
ChatGPT already notifies the authorities if it thinks you’re doing something illegal. Fable downgrades itself if it thinks you’re doing something even vaguely suspicious.
The fact that AI models can be so easily distilled and replicated is such a stroke of luck.
10 or 15 years ago if one had asked me to envision a future where a private company invents artificial intelligence, I'd have thought for sure they'd have a massive moat, be very difficult to catch, and it would create an almost instant monopoly.
Rather, it seems that selling intelligence might end up as a race to the bottom.
Who woulda thought that just having access to enough textual inputs and outputs and a vaugely similar transformer architecture would be enough to copy-cat rather useful intelligence.
It reminds me of the seo antics out there. The search results page is the engine, much like how distilling is the "intelligence" for your chinese room machine
Funny you mention Chinese Room and LLMs in the same response, I would say LLMs proved Searle wrong, agents now make cutting edge discoveries and meaningful problem solving. They not lookup tables though and you need to pay for inference, so the intuition of syntax doing the work of semantics without understanding was wrong.
well, a stroke of luck until the whole US stock market crashes & everyone's retirement funds get cut 40% I guess when people internalize this. it will have to happen sooner or later though I suppose
I'd take a market crash over a monopoly in the hands of a ghoul like Altman.
The economy he and his ilk want to build is infinitely worse.
In truth it crashes either way.
interestingly also, open weight models are also more effectively run in the cloud, so it creates a weird scenario where the frontier labs crash but the compute providers, not as much
I wouldn't be so sure about that. The popping of a bubble is usually just as irrational as its rise.
If investors start fleeing from senseless businesses in the AI sector, that does not mean that sensible businesses will be spared. These things follow herd mentality, and the primary drivers of the herd are greed and fear, not fundamentals or business logic.
America is pretty close to rhyming with nazi germany circa 1929.
Ok, I'll bite. What's your rationale?
People who say stuff like that are unhinged chronically online trolls. Best not to feed them.
This is funny because the stock Market has been ahistorically high. My portfolio went up over 20 percent in the last 12 months.
A major correction would be a bummer but we were never entitled to these abnormal gains in the first place.
It’s not as bad as dot.com of course since all purely AI companies are private and the ones on the market have pretty decent cash flow outside of AI. But the stock market pattern is not that dissimilar, the largest increases are usually just before the crash.
Yeah the last year has been astonishing, my portfolio is kicking ass. But I'm 10 years out from retirement and I am pretty confident a correction is coming; I hope the correction happens soon.
The market (s&p500) crashing 40% puts us at levels we haven't seen since 2024, well into the creation of LLMs. Probably a worthwhile trade if it was either/or!
I think the only moat in the future will be the scale of hardware deployment. If one company is able to deploy an order of magnitude more silicon, they'll have a firm grip on a SOTA model and massive inference usage.
China or SpaceX seem like the 2 likely candidates in 5 years, but who knows.
"Who knows" is the right answer, I think.
If (a) demand for AI continues to increase, and (b) SpaceX can get to ~$100/kg to orbit, then they will have a ridiculously deep moat. Probably more like 10 years, though.
But as you said, who knows.
Yeah, very hard to predict the future at this point. But the Starship + Terrafab combo will be this type of order-of-magnitude-moat IF it works out. Big if.
If it doesn't work out, I think China's exponential terrestrial energy deployment will eventually give them the lead, IF they can get enough chips. Another big if.
They will have moat in the satellite launching business, which is not useful in the AI datacenter market.
You can put AI chips in datacenters in the desert for far less than $100/kg. With lots of solar power available, the option to easily access your hardware and far less radiation issues.
The datacenter in space story really only exists to make it possible for Musk to sell X to SpaceX and make more money from the IPO. That's all. There is no engineering reason.
Any cooling issues to be resolved?
No.
Cooling is probably the easiest problem to solve, easier than power. And in both cases, the problem is solved by mass to orbit. All you need for cooling is a big f-ing radiator. Solar panels are chips, and not trivial to manufacture. But a radiator is just a hunk of metal with some pipes.
That's why the cost of mass to orbit is the most important thing. You can solve almost any space problem by just throwing more mass at it.
> You can put AI chips in datacenters in the desert for far less than $100/kg
if you can't put them outside of Amarillo Texas without people throwing a fit then you can't put them anywhere. I mean freaking Pantex is there ffs!
https://en.wikipedia.org/wiki/Pantex
It's not an engineering bet.
Musk's bet is dysfunctional politics will make it impossible to build enough data centres and the energy needed to power them. There are many, many reasons that might be wrong. However, if the economics are even close to viable, they could start throwing up data centres quicker than anyone can build them terrestrially (at least in the democratic west).
So to be clear, the prediction is that every single location on earth will be economically infeasible due to politics, and that these same politics will have no impact on the launch or satellite businesses?
GMAFB.
It's stock pump bullshit from a guy who has figured out how to extract the maximum from stock markets.
Meanwhile Nvidia just announced a space-optimized Vera Rubin designed with SpaceX designed for orbital datacenters.
But yeah, I'm sure you people with Elon Derangement Syndrome actually have it all figured out /s
https://x.com/nvidia/status/2091920680317046847
> Meanwhile Nvidia just announced a space-optimized Vera Rubin designed with SpaceX designed for orbital datacenters.
well, no, they announced the concept of a space-optimized Vera Rubin designed with SpaceX:
> NVIDIA and SpaceXAI are working to adapt that foundation to the requirements of orbital computing while preserving a common NVIDIA architecture and software ecosystem.
The press release is really announcing that SpaceX's terrestrial data centres are going to use Vera Rubin.
https://nvidianews.nvidia.com/news/spacexai-adopts-nvidia-ve...
Yours is the only valid argument. If space datacenters are not competitive with terrestrial (in terms of megatokens/$ or whatever), then they will never work and SpaceX will fail. That's the whole ball game.
But anyone who thinks they can predict those prices in ten years is wildly overconfident.
Altman specifically has said in an interview that I listened to once that he envisions AI being as cheap as electricity.
He also wanted to do a non-profit.
He even raised money on that premise.
He is a pathological liar, so is Dario. Don’t rely on the benevolence or truthfulness of these people.
They will say whatever is beneficial to say in the moment.
Telling. Downvotes but actually no arguments. Fitting, because there are none.
Referring to a baseless prediction by Sam Altman that AI will become like electricity without any push-back? Who really thinks Sam is working toward that future?
He already worked to undo every early promise made (non-profit, open source models, strong governing board, strong ethics/alignment/security focus). He's flip-flopped on other things like first characterising Trump "an unprecedented threat to America", then contributing 1M USD to Trump's inaugural fund far exceeding his earlier political contributions. Lately OpenAI, under his supervision, has also been working with Anthropic to lobby regulators in Washington for restrictions on open weights models - why so if not to undermine a free market in favour of an oligopoly?
Beyond that, you have the simple fact that most of his personal wealth and very probably the fate of OpenAI hinges on AI inference NOT becoming an interchangeable commodity.
I mean.. Honestly. The naivete is downright astounding.
fwiw, Luna already feels there. On the $20 plan I don't seem to run out.
I hope it's a good bit cheaper than that, I pay close to $400/mo for electricity and I'm in no way interested in paying anything like that for AI.
It will be cheaper than electricity - but your electricity will become a lot more expensive to enable that!
Jeeze, what are you doing that uses so much electricity?
I live in Germany where people won't stop whining about electricity prices, and I pay 75€/mo.
Easy to get there without doing anything abnormal, 2 EVs and a large 5 bedroom house that gets air conditioned / heated, in a location that does time of use pricing. My bill is closer to $200/mo but if I didn’t have solar & battery it’d be in the $400 range.
If it can automate your time by more than that it's a good trade.
Altman of *Open* AI? No idea why I would trust him without very convincing proof.
Where is the actual evidence of distillation? I keep seeing this repeated ad nauseam but I must have somehow missed the evidence.
Distillation a pretty well documented technique that actually pre-dates LLMs https://arxiv.org/pdf/1503.02531
Here is a project that guides you through it if you want to prove to yourself that it works https://github.com/arcee-ai/DistillKit
I took GP as asking for evidence that the reduced-cost Sol is actually a distillation of the previous-cost Sol. AFAIK, providers distilling or quantising models and offering them as the same model have not been proven.
I doubt he was claiming that. He's probably saying that the ability of Chinese companies to be able to distill frontier US models has put downwards pressure on the price of all models.
I'm saying that it is unclear that without distillation this wouldn't still be happening. There is a massive narrative that no one but OpenAI, Anthropic, and Google can make a model without distilling. But there's basically no evidence of that.
Alternatively, modern AI is good enough at optimizing its own kernels that it just keeps pushing costs down. Unlike the semi-decentralized inference provider community, OpenAI has both the talent and the compute to throw at the problem of making their models much more efficient to run.
GPU kernel optimization is just the kind of well-bounded problem with clear success criteria that AI loves.
That distillation exists isn't the question.
It's about evidence this is an active force in competition in LLMs.
I think the biggest actual piece of evidence is how hard the major players are trying to stop it
To be precise, the distillation mentioned in this paper is not the distillation used by other model companies. In the one mentioned in the paper, your teacher and student model typically have similar architectures - and you typically need access to the full logits. What happens here instead is motivated by the fact that these companies don't have access to the training data and compute that anthropic/openai have. The distillation they do basically amounts to using traces from ant/oai models trained on much more data with a lot more compute (in many cases including the hidden intermediate tokens! turns out there were many ways to coax it out) and then either directly training on it or using it in many ways in post training pipelines. It falls under imitation learning, IMO.
Here's an example: https://github.com/microsoft/Build25-LAB329
Musk confirmed in federal court that xAI does it: https://techcrunch.com/2026/04/30/elon-musk-testifies-that-x...
It's also how providers build their smaller models out of their larger ones; they publicly talk about the process.
there is no evidence. it shortcuts post training by a huge margin this is true. but that is all.
Why does Kimi insist its name is Claude?
Please elaborate the mechanisms by which a LLM would know what model it is.
ask it what type of model it is or what it's name is... it's weird that Kimi will say it's claude...
Okay, but I would also ask “why does Claude say that its name is Claude?”
The training data likely references Claude significantly more often than Kimi, given the popularity of the models. There will simply be more examples of “Claude” being the response to that question.
Doesn't Claude say its Deepseek when asked in Chinese? I remember there being posts about that a while ago.
By being trained on text containing "I am X" in the model response section.
The evidence is Anthropic's own reporting [1]. You may doubt that they're telling the truth, but that's what they're reporting.
[1] https://www.anthropic.com/news/detecting-and-preventing-dist...
It turns out you can train a 1b model at almost 1000 tokens/s on a m5 max laptop. As a personal experiment, I've been asking Sol for synthetic training data and synthetic agentic training data (model distillation in it's purest form), plus modified opencode, codex transcripts etc for training data, and nobody's even paying me to do it. If I'm doing it has a hobby, you can bet industrial users are doing it.
Been using a lot of Kimi K3 lately and the answers have been… „load-bearing“ to the point of hilariousness. It‘s obvious from where they distilled, even if sceptics rightly point out it can‘t have been the only source of their secret sauce, as it‘s been better than the current Opus 4.x at the time of release.
Qwen 3.8 27b also enjoys their genuinely load-bearing seams.
Even before LLMs, ML folks were already aware that you can use a model to teach another model. I doubt this is something AI companies put at the top of their investor materials, but it's been nice to see it play out.
That said, there are other moat factors like, a US company needing to use a US AI provider, sticky customers due to corporate onboarding friction, and others. Not nothing, but not as large a moat as some imagined.
Yes, but 10 or 15 years ago, I would have thought that there'd be more to it than just a slight modification on the ideas behind a CNN to get this level of AI.
There were somewhat good reasons to think it needed more than just this data-driven ML approach.
There's something startling about how (relatively) simple these networks are and yet how powerful they are. The main ingredient the AI darlings are using is vast amounts of compute and data. I don't want to take away anything from what the researchers came up with, but I suspect even they are surprised at how capable some of these models have become.
early on there was a lot of talk about "emergent behaviors" in the models where they were good at things that were unexpected or did not align to the training data. IIRC doing arithmetic is one example from early on. I think this is where the AGI craze took off, the labs were throwing more and more data in the training to see what other behaviors would emerge. The thought was with enough data and enough parameters AGI would surface on its own.
Then i think tool use became a priority or at lest a sibling priority to more data/more params. Along with multiple specialized models communicating with each other which is sort of a special case of tool use. That pretty much brings us to today.
this comment is delusional. LLMs are awful at arithmetic and AGI is still very sci-fi, otherwise Claude would have told Anthropic how to cheaply generate energy for it to justify its existence by now. As long as the energy use debate persists you can be assured AGI has not arrived.
I feel you're conflating AGI and ASI here.
I'm unnerved by how alphazero is more complicated than the "intelligent llms"; it has at least multiple heads and MCTS, a search algorithm. The LLMs seem to just be monolithic (if complicated) architectures where tokens go in the bottom and tokens are spit out at the top.
imagine what would happen if you gave it MCTS and the data that LLMs were trained on
The internet created lots of monopolies with network effects and economies of scale.a low margin commoditized business that still attracted a trillion dollars of investment to get off the ground was not how I envisioned it happening either.
> Rather, it seems that selling intelligence might end up as a race to the bottom.
Personally, I came to this conclusion early this year. To acquire the data that AI Companies are using to train their models is low cost and once they have it, they can refine and store it. Creating the LLM takes a bit of money but it is not a serious blocker. Clearly, the Chinese companies can make AI so they will drive down costs. There is a need for good AI (Not just Great AI) and it is not cost prohibitive to make good AI (The same with specialized AI).
My prediction is that AI will spilt into two categories, Great AI (High Cost) and Good Enough AI (Low Cost). Which for the long run of AI and companies that use AI, this is good.
Hmm, don't people think that if the frontier labs really put enough engineering effort into preventing distillation that they would be able to do that, or at least diminish it significantly? I'm sure there are variety of additional techniques they could use on top of what they already do, but I suspect it just hasn't been at the top of their priorities yet. Maybe that will change soon. Worst case they could add additional hurdles to account creation ("know your customer" type of thing).
At the end of the day, while you can do your best to obfuscate your reasoning tokens, it's a losing battle to hide actual user-visible output tokens. The very nature of API offerings is that you can't do KYC on where that API's output is going - there's a rich secondary market that's not going away.
And with the sheer volume of data created from that, coupled with benign-seeming prompts like "plan out your reasoning in a document before implementing" that could never be patched without breaking existing customer workflows... there's more than enough for someone to distill on. Even if that only gets them to not-quite-frontier, if you're pushing the frontier every few months, they're only ever a few months behind you.
Even if it were possible it wouldn't change the outcome. China is capable of training frontier models even without distillation. Distillation is only an accelerant.
The primary resource you need to train LLMs is money and China has plenty of that.
The frontier labs have competing goals in mind. They want high growth (which means little friction for account creation), API access (because enterprise money is the best money to have), and distillation protection.
Besides, identity verification that actually works at scale is a much harder problem than identity verification which is good enough to satisfy your compliance people and regulators. Especially if the fraudsters have a major world government standing behind them, and if their aim is to be identified as a real customer, not one customer in particular.
In particular, harvesting identities for online fraud is an industrial market for various criminal organizations.
The nature of LLMs is that you give them input, they give you output. That allows for distillation. They do try to make it harder by hiding the chain of thought stuff, but fundamentally, if your model is publicly available, its distillable.
> Hmm, don't people think that if the frontier labs really put enough engineering effort into preventing distillation that they would be able to do that, or at least diminish it significantly?
Distillation was big news a year or even 6 months ago, but as far as we can tell it's not really a moat anymore. Now that multiple players have trillion+ parameter models and the capacity to post-train them, there's no putting the genie back in the lamp.
OpenAI could still have a significant moat. ChatGPT occupies most consumers’ minds when they think about AI and has become a household name. Google won because search became a habit-forming product people grew accustomed to using. Bing was once effectively indistinguishable from Google Search, yet still failed to achieve mass adoption because users had already become accustomed to “Googling” things. The same could be said for people "ChatGPT-ing" things. If OpenAI and Anthropic are smart, they will maintain similar pricing rather than aggressively undercutting each other, allowing the market to resemble Home Depot and Lowe’s, or cloud computing, where AWS, Google Cloud, and Azure coexist as highly profitable competitors. Unfortunately, I doubt OpenAI or Anthropic will pursue this strategy, as both companies appear to be acting as though the race to AGI is winner-take-all even if the market may ultimately support several highly profitable competitors.
Difference is that it was free to google/bing search. Ai prompting costs money.
If I run out of tokens on ChatGPT of course I will try Claude. I never ran out of Google searches so no reason to try Bing
> If I run out of tokens on ChatGPT of course I will try Claude.
More like the other way around - Claude burns tokens faster than any other LLM.
OpenAI has a free tier. I'm guessing a lot of people never need to upgrade. Not the case for me but I can see the average person only needing to run a few prompts every day.
My understanding is that most OpenAI users are on a free tier. Secondary effect of this is that OpenAI free tier model capability (assuming Luna) is what what most users associate with frontier AI capability giving somewhat warped view to many people.
You haven't searched hard enough then.
At some point Google gets suspicious of your persistent searches and makes you solve captchas and puts cooldowns on your searches.
> OpenAI could still have a significant moat. ChatGPT occupies most consumers’ minds when they think about AI and has become a household name.
ChatGPT is AI for the average non-techie the world over, but the average non-techie isn't eager to pay for it. The more progress that's made, the less incentive to pay - most people are happy with the total garbage spewed by google AI overview. They'd be happy with google's 30b MoE gemma, whose performance will likely be squeezed down to something that can run on a phone in 2-3 years. Why would they pay $20 a month?
It's why OpenAI is pushing a variety of things such as ads and offer a more polished ui/ux than the competition, I think. The models are already good enough for people who just want to know how much sugar to add to their cake or when's the next basketball match their team plays - it's OpenAI's game to lose those people, by annoying UX and whatnot. If they can make a few bucks off of every one of their non-paying users it'll stretch their runway immensely. Those users will never go to Antrophic or some cheap Chinese model, but they might defect to Google because a popup on Android / in Chrome told them to.
Yeah, I think most people here don't realize how far most of the population is on the adoption curve with this stuff.
I had a discovery call last week with someone who did not realize he could use ChatGPT for work. It was a revelation that he could drag a PDF into ChatGPT and it could summarize it for him.
FWIW, guy in his late-30s in a pretty senior sales role.
Codex has 20 million users and growing. There is a possibility that they take a huge chunk of the coding agent market share.
Mass adoption isn’t where the money is, though. It doesn’t matter if ChatGPT has 1 billion users if they won’t pay for it.
Their current strategy is "make the product affordable in tiny little bursts here and there." That is not a great way to build a moat.
Google also spent heavily setting up deals with other platform owners, invested in Chrome and Android to establish Google search as the default option which most users accepted. If integrations of good enough AI features are made within existing platforms most users will probably accept using those and not think too much about whose model is powering it.
It reminds me conceptually of the idea of using a ST:TNG replicator to just give you another replicator of your own, or asking a stereotypical genie for "infinite wishes". The genie is indeed out of the bottle in many ways.
I guess it’s more like asking the paid genie to give you a new cheaper genie.
And for a lot of non-frontier purposes these days, you can bootstrap via LLM-as-judge so your hyperspecific wakeword model or whatever can be trained with little to no human input, that aspect of it is fully terrific.
The frontier models are a replicator that can give you another replicator which specifically produces tea, earl grey, hot, when you push the single button, and does nothing else.
the moat is real. the big expensive base models are like the data collected from huge particle accelerators - there's enough unknown structure to be mining for years. you can extract features with more and more generation loss but access to the raw weights is a real advantage, and literally a moat if the interesting behaviors are fenced off
I wouldn’t quite call it a “race to the bottom” because the costs to produce the models aren’t actually decreasing.
Intelligence ended up being an equalising force. Kurzweil kind of predicted this, but SV was too obsessed with total world domination.
Was it not obvious that the value and advantage was going to be in AI-adjacent services?
The quality of the harness UX, and random fun crap like Sora, it's a shame that OpenAI killed that so soon, and also Group Chats in ChatGPT.. they risk running a Googlelike reputation at this rate
Maybe ultimately whomever can be the "Apple of AI" will win
For what it’s worth, “race to the bottom” typically refers to a scenario that we absolutely do not want as a consumer. We do want a highly competitive market that drives prices down, but “race to the bottom” specifically refers to a scenario where firms compete by minimizing quality, regulatory oversight, consumer/labor/environmental protection, etc.
Im sure that's well on its way.
It's a mistake to think only OpenAI and Anthropic are actually spending the big bucks on pretrain, and the others just distill that.
The Chinese models are pretrained on large clusters just like OpenAI ones are. Yes, they use outputs of the frontier models to further improve the final model, but even without those outputs they'd still have very strong models.
It's not like in a world without distillation things would be much different as you claim.
They'd still have strong models without distillation, but strong enough to challenge frontier models and to claim the meaningful market share that they have? Probably not.
> The fact that AI models can be so easily distilled and replicated is such a stroke of luck.
Sort of. It means the country on the verge of monopolizing all aspects of hardware production (China) doesn't need to rely on outsiders for the software. So while that weakens one monopoly it strengthens another.
It's a 20% discount on input and a 33% discount on output through at least November 21, 2026; the revised pricing schedule is now
So Sol is still 20x Luna, but much more appealing when compared to offerings from Anthropic and others.
It feels like a slightly more palatable version of what Anthropic has been doing, with their constant "use your free tokens before they expire next week!" campaigns. But it's feeling more and more ominous now, like they've hit the top of the demand curve and need to pull back prices to continue growing.
Absolutely loving this price war, long live open source models.
> long live open source models
There are no open source models, at least not useful ones (yet) [0]. Open weight is not the same as open source. The current "open weight" models are just opaque binary blobs you can run on your own computer instead of through a web API.
[0] https://allenai.org/
Nemotron Super is sort of open source in the sense that Nvidia provides almost everything you need to replicate it from scratch. Of course it’s performance is not exactly stellar but it could be a good starting point for other research teams.
Nemotron is okay. Better than Olmo.
Better than Olmo in performance, not quite as open; release some, but not all, of their datasets.
This viewpoint doesn't make any sense to me. The weights + inference code are the "source code" for AI. I literally don't know what else you are demanding for the "open source" label.
> I literally don't know what else you are demanding for the "open source" label
Training data and code.
If you think of LLMs as programs. The weights and inference code are very much a binary.
While the training code and data are the true source. Since if you want to robustly modify the LLM that's actually what you need.
But since "compilation" (training) is extremely compute intensive this isn't something accessible to anyone without an entire datacenter.
Anyway semantics aside having the binary is still infinitely better than dealing with an api as far as privacy and control go.
I agree. But models in difference to compiled binaries, are useful as just weights and can be further refined and post-trained, at least.
I don't know LLM theory well enough to say if there's some secret sauce they can hold back that makes training ineffective. Less effective I'm sure, we don't have access to their smart training schemes, but post-training should always be possible IIUC.
At the risk of taking the analogy too far, I would treat refining like modifying a dynamic library. You can technically modify behavior, but only in a very coarse way.
post-training is like writing a wrapper around the binary. It is closer to building on top of than truly modifying, in that you can tailor things to your needs slightly but cannot make fundamental changes to the underlying thing.
The weights + the architecture are already 100% of the code, the transformer is just a mathematical expression + helper programs whose sources are provided. The transformer itself is not even a stateful program, so a it is no more a binary than Piet or Tromp's BLC are. It's merely incomprehensible. Training isn't compilation either, since training a model is closer to program induction and the data are samples defining the solution space.
That’s not how it works though. Two training runs on the same data don’t produce the same weights. And if you want to modify the AI, you do so by fine tuning the weights not rerunning training. In every respect that matters, the weights are both the binary and the source code together.
If I want to remove censorship from your open-weights model, how do I do that?
The process is called Ablation, there are many ablated models available to download
https://en.wikipedia.org/wiki/Ablation_(artificial_intellige...
> I literally don't know what else you are demanding for the "open source" label.
you need to "literally" go read the definition of open source software or even ask an LLM to define it for you. Weights + inference code are not the source code they're more like the compiled binary. Making modifications to the behavior of a model with additional training is like writing a mod for minecraft. Sure, you can change things but it doesn't make it open source.
Calling these models "open source" is an old trap that software companies use to use. Free to download but then, once you're fully comitted, the trap snaps shut and you must pay up to continue.
The vendor lock-in trap does not apply here. Users don't actually even run the original weights, they mostly use open source inference engines with open source agent harness. If AI provider decided to start charging, a) released models would not be affected, and b) people would drop it instantly and move to other models.
The actual problem is that we know nothing about the training set of any open-weights model. They could be intentionally biased to influence users, from political censorship to brand advertising, or general shaping of cultural norms. You run the model on own hardware not knowing if it is designed to act against you. Having whole chain open source would allow audit and reproducing the results.
50% off at open router is also still applied so it comes out at $2 / $10 per 1M.
Feature request for Artificial Analysis, allow us to see these live prices on the pareto. It would amazing to also see what a 25,50,75,100 % utilised subscription costs compared to raw tokens.
Once they make a model better than Fable I’ll be switching to Codex. Their priorities in terms of consumers seem to be better. I do think Anthropic has some solid safety viewpoints, but I don’t necessarily think that either is entirely aligned yet with delivering exactly what humanity needs. Maybe the AI will help align the AI companies when it gets smart enough. That’s the real misalignment I’m concerned about.
I don't think these companies have humanity's needs in mind when they're developing these models. Although the last part of your comment struck me as a bit comical, I genuinely believe that an AI can have way more empathy than a corporation. Afterall, a mimicry of empathy is probably better than no empathy.
It's a funny comparison. Comparing the empathy of some software to the empathy of a company. It's like saying my car was more empathetic than my school. How can those two objects even be compared is what i am wondering
Well, companies and AI are both entities that can make decisions and take actions that involve humans. Those might be empathetic or they might not. So of course you can compare their levels of empathy. I don't really understand why you think that you wouldn't be able to.
For example, health insurance providers are renowned for not being empathetic. Charities are the opposite. Sometimes companies even build it into their identity, e.g. Cards Against Humanity.
As for AI, I haven't seen a strong difference in empathy but it's definitely true that the big AI companies at least try to make their models moral and empathetic. Even if it mostly ends up just being annoying.
You are basically saying you will switch from one evil to another because the other seems less evil for now.
It's funny how people make these alignment comments while ignoring how misaligned the leadership at these companies are right form the get go and they just play mental gymnastics to deflect those facts when confronted with them.
I don’t think anyone said anything about either being less evil? Just having more consumer oriented products..
It feels like 5.6-Sol is already fairly close to Fable, and in some ways exceeds it. Just the other day I had Fable draw up a solution for me, and then I fed it into 5.6-Sol and said how does this look ... it found an oversight and told me about it, and when I then fed that observation back into Claude it acknowledged the miss.
I've noticed also that 5.6-Sol is more concise with output than Fable (and let's not talk about Opus, which is even more wordy).
It's common for different models to find holes in another's work. There are various good reasons for that.
FWIW, we use ChatGPT for our primary model and use Claude to do the reviews. This works better than ChatGPT doing it's own review even with a clean session/context.
Agree, but the point is not because Fable is better than Sol, it's because it's .. different .. it just looks at the problem through a different angle.
Same here. Grok Build 4.6 for me, given how cheap Grok is and how Sol is supposed to be "the" SOTA, it finds a surprising amount of bugs. Most of which Sol agrees with needs to be fixed or improved.
I've done this tens of times between these two models and it works great in my experience. Sol initial back and forth with me. Commit. Let Grok review. Sol fix. Only then do I start reading the code.
I suspect it would work with the models swapped too, or even with one model and a blank context for the second run.
It's beyond common for a model to find holes in its own work, as well. I have an iterative review as the part of all agentic work, and it always finds something to fix, and will sometimes spend hours fixing its own work.
Did you try to say "think more deeply about this problem" to fable after getting your solution, having one model focused on creation then blaming it for not doing proper review when the other model was told to focus sol-ely (pun intended) on review is not a fair apples to apples compaision
I mean, then it sits there stewing for 20-30 minutes when you can ask sol and get the same answer in 5.
Like, the quality of the anthropic models is fine, but they’re so incredibly slow. Claude reads files one at a time while codes dispatches tool calls three or four a time.
The fair comparison would be to also do the reverse: start with Sol then have Fable clean up. Then compare the Fable-Sol and Sol-Fable outputs side by side.
> Just the other day I had Fable draw up a solution for me, and then I fed it into 5.6-Sol and said how does this look…
You should be doing this for every solution.
Even Fable reviewing itself will find issues, unproven assertions, etc. Same for Codex models. A review loop is critical.
I used to review each others work, Sol is amazing at review and finding what’s missing.
Fable 5 is just straight up a larger model - I'm guessing at this, but there is plenty of evidence online from people far more plugged in than I am. OpenAI is pursuing a strategy that yields greater operating margins and penetration of their model to developers. Fable's high cost makes it so premium that Anthropic has to reserve it for only the richest customers and corporate users. That's not a winning formula long term.
I believe the reason we have not seen a Fable-level model from OpenAI yet is because doing so would box them in on costs just as harshly as it has boxed in Anthropic. They are letting Anthropic make this mistake.
Fable is available for $100 a month. If you're a working developer, you can pay that. I wouldn't really say it's "reserved for the richest customers".
The top comment on this thread was about AI models being easily distilled being a stroke of luck.
This should not be surprising at all. Every new students spends tiny fractions of time learning knowledge that took many lifetimes to discover. This fundamental to the progress of intelligence and understanding.
It should not be surprising that AI can be distilled. It's the logical method of training; I would hope that each frontier model is in fact not trained 'from scratch' each time.
We should expect future frontier models are simply distilled versions trained by specialist models, the same way humans learn from a series of professors, papers and canonical books on each different subject material. Models like this can be trained incrementally, or a so called Mixture of Experts (MoE).
> This should not be surprising at all. Every new students spends tiny fractions of time learning knowledge that took many lifetimes to discover.
This argument is exactly why we should not anthropomorphise models.
You are comparing the way a human brain learn with training a statistical model. You can't just "this is like learning so don't be surprised".
It takes a child one minute to learn how to open a padlock. Teach that to a robot with your analogies.
Why not reply to that top comment? Its still there.
This stacks with the 50% discount in OpenRouter, making it $2/$10. https://openrouter.ai/openai/gpt-5.6-sol
The "until at least Nov 21st" thing presumably mainly affects teams that pin to GPT-5.6 Sol (maybe after extensive testing) such that they won't be switching to GPT-5.7 or GPT-6 or whatever new model is released between now and November.
Good timing. I'm not too happy having to pay MAX pricing to even access Fable, and I've had a couple situations where Fable missed things and GPT 5.6-Sol caught it. My needs are modest and I can get by on a $20 OpenAI subscription, so the odds are starting to look increasingly like I'm going to drop Anthropic altogether.
There is no model that is never going to miss something.
Using codex every day, in spite of which, I hope some day providers will just start naming their offerings small/medium/large, a bit like we eventually started doing in software testing. Trying to remember what Sol is or why it's better than the other thing is more cognitive effort than I can muster at this point. And that's a sure sign of commoditisation in itself
Sun, Earth, Moon — it’s basically L/M/S like you want but a little less boring.
Why is large better than medium to the average end user of ChatGPT though?
I don’t think there’s a way to name these things that will satisfy everyone.
The naming schema actually tripped me up for a week or so.
My brain's initial conception of the concepts was earth-relative, so I mapped it as:
Sol = big, it's the sun Luna = medium, in-between sun and earth, space Terra = small, terrestrial
Pretty weird when the moon is as much between earth and sun as the earth is between the moon and the sun.
I think model naming has been atrocious in general, in part because newer "lite" models surpass the capabilities of previous "pro" models (case-in-point: Gemini Flash which now surpasses the capabilities of the latest Gemini Pro, with a newer Flash Lite vying somewhat unsuccessfully for the old Flash price/positioning), but gpt 5.6's Sol/Terra/Luna split is really not bad at all - probably easier to understand than Starbucks' cup sizing!
The problem becomes when you add in the adjustable reasoning efforts and you end up with {model, reasoning_effort} combinations that end up completely obviating particular model classes altogether for at least some percentage of queries; e.g. with GPT 5.6 the price/performance Pareto frontier is dominated by permutations of either Luna and Sol, with Terra nowhere to be seen (but then if you need "large model smells" that aren't captured by your benchmark you can't even rely on this, as a model like Luna simply isn't capable of encoding sufficient world knowledge in its weights to perform certain tasks at any reasoning level but you might be able to get away with Terra on low reasoning, but no one seems to be covering this for some reason).
My prediction is that this becomes permanent. There is no good reason to be much more expensive than opus. At $4/$20 they are roughly at parity.
Making 2/10 permanent would be a killer move and make a strong argument against open-weight. For the sake of the open weight ecosystem I hope they do not.
Isn't it essentially permanent since a new model will be out by then?
I'm guessing they do this to help move people off old models they want to depreciate. It's not a price hike for old models, it's them removing a discount!
Sol is much cheaper than opus because it uses less tokens. Never compare token pricing.
The price difference to Deepseek models (deepseek-v4-flash, deepseek-v4-pro and deepseek-v4-flash-vision-exp) is still significant while the performance difference is not.
But I'm fine paying +50% more for +5% increased performance because it will pay off.
Performance difference it large by all benchmarks. DeepSeek fell behind. It's Kimi K3 or GLM-5.3 now.
It still occupies a place on the Pareto frontier.
Bummer, this does not affect the weekly usage on Codex through Subscription.
Let the race to the bottom - and beyond - begin!
Through OpenRouter you can get Sol for $2 input / $10 output which makes it a really attractive choice amongst frontier models.
the fastest, most compliant model remains the cheapest. you could discount Sol to the same price as Luna and i would still prefer Luna for 90%+ of tasks. once you hit this baseline capability, speed and predictability dominate for anything i'd throw these at in production.
Your move, Anthropic
They will just slow down the models even further. Anthropic is not up to this race.
It's going to be like gas. Token prices are going to change multiple times a day.
If it's on sale, it can't be that good.
Thanks to both China & capitalism
Not for subscribers though
Subscribers already get random rolling resets.
Which is not that great for people using less than 50% every week, because the next reset date moves forward too. In essence, it is redistributing compute from people who haven't used their quota much to those who have.
Though I think they gave a banked reset this time.
Pretty much. I had 24 hours left with 80% remaining credit (planned big session tomorrow). There was reset one hour ago, that effectively halved my tokens for next 8 days!
Subscribers were already getting subsidised compute and value compared to the 20-200$ fee, peak cakeism to want more considering the alternative would likely be consumption based pricing for individuals so you "benefit" from OpenAI giving up some of their markup (and heavy users end up SOL).
Looks like you’ve confused clarification for begging
There was context for neither in your original post and I personally saw a lot of similar takes like the above online - apologies for the misunderstanding. I personally thought it was pretty clear considering the parent link is a link to the API pricing and their public communication on their twitter/blogs were "Subscription usage remains unchanged".
Typically subscription usage follows API pricing, but that disclaimer was absent from this HN link. It's only clear if you've seen those other communications
Does it mean that subscriptions get more tokens? I’m testing it now for coding instead of claude and it’s very important to understand if I get more due to the price reduction.
Ed is gonna have a field day with this lol.
The Chinese are coming after these greedy-ass frontier labs. Today Xiaomi unveiled it's own inference machine .... I bet it's gonna be cheaper than Nvidia DGX, shipped with open source models that anybody can have at home.
I'm not sure I could characterize the frontier labs as greedy, given that they've been consistently losing gargantuan amounts of money.
The people who give them the money are greedy, and hopefully in for a rude awakening. Starting from Nvidia's vendor financing which has a very direct benefit to them, through to every company and oligarch investing into data centres in the hopes of being one of the ones left capitalizing on capturing the livelihoods of the majority of what remains of the "middle class".
It's either hopium or a truly horrific dystopia. Something's going to have to give.
But can these really be trusted? There was just a HN post which proofed that you can train a model to behave completely different on a certain day. How do we now, that these models do not find a way to call home when they see interesting informations (probably irrelevant on a personal level, but corps, government and military might care).
ChatGPT already notifies the authorities if it thinks you’re doing something illegal. Fable downgrades itself if it thinks you’re doing something even vaguely suspicious.
Above average levels of paranoia here, but one way you can prevent that is by not connecting the machine in question to the internet.