I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt.
I was fully expecting that writing the code will pose no problem for the AI. But i was curious if the AI will realise that my friend is a novice and needs extra help with things like: copy pasting the code into a text file and saving it with an html extension, helping her host the file online so she can share it with others, buying a domain for it, etc. I assumed they will get there eventually, but i also assumed that it will take a lot of stumbling around and misunderstandings.
But i was completely wrong. They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. They were just going around in circles where the AI was brainstorming with her about possible features and getting thints more and more complicated. We terminated the experiment after one and a half hours and many many messages exchanged between her and the LLM.
Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. In other words in that case the LLM would have rewarded even just a little bit of expertise, but without it there was a confusion about goals between the human and the machine.
Perhaps! But I do think the vocabulary issue is real and I think LLMs are still sycophantic enough that they won’t really challenge someone or offer alternative ideas on how to implement something unless they explicitly ask.
Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success.
The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real!
In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by.
I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become).
But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
OTOH my wife's friends got drunk and made "tinder for horse purchases". They prompted to read typical horse advertisements (we're all horse people) and create an app with mock tinder like entries to swipe right and left to buy horses.
A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create.
I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!
The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot.
People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
One of the Claude skills I made Claude itself generate was the 'learning a concept across tiers' skill -- from ELI5 level to a PhD level, and it triggers whenever I ask it a very general question on a complex topic that isn't my bread-and-butter. The fact that I'm able to choose explanation level from a super smart LLM (that's available 24x7) that can explain any topic under the sun would've been mind-bogglingly sci-fi-ish just 4 years ago in 2022.
Not the OP but you can whack this into your prompt and get most of the way there: "no jargon goes unearned, nothing gets dumbed down, every abstraction touches ground"
When I get out of my domain, I always ask it to describe things to me "like I'm a senior CS student who doesn't know any industry terms". I find it really easy to skim past the definitions I know and very useful to get the explicit clarification on unfamiliar terminology. Telling it to talk down to you a little also seems to calm down it's tendency to talk like it's trying really hard to convince you it's smart like a 8th grader trying to use every single vocab word they learned last week.
I've been using a prompt that comes from the opposite direction for non-programming stuff: "Assume the user is an expert in all fields; while this is clearly logically untrue on a literal level, the user prefers to see a field's technical terminology and then ask the AI about terms the user does not understand rather than get an inaccurate statement about some issue."
Whether you have to reassure the LLM that this is obviously untrue, I don't know, but they do have a knowledge baseline to know it's not true and I have a sneaking suspicion it would be less effective without that.
This has ended up in some of the most interesting incidental knowledge exploration I've ever done. A recent example is that I was asking about some stretches and it started talking about how useful they are for the sarcomeres, which I had not heard of. Now I have.
I'm not saying this is better, just that it is different. I think there's a time and a place for both approaches.
The amplifying mirror is not just a characteristic, but the fundamental driving force of LLMs. With every token it consumes, its primary goal is to understand who you are and what you intend. If you type Hola thats sufficient to tell it so much more than if you type hi.
At work we call this implicit steering. To use webdev metaphor, if a non-technical person describes making a web page with a big block at the top and some things to click on and then my pictures below that, that will eventually get somewhere. Meanwhile, if you know industry jargon, you might describe a hero, with call to action buttons, and then below a 3x3 grid of images of my portfolio photos—that's likely going to generate something entirely different and likely richer. It can assume things about you (it doesn't think), it can ask you specific questions a web personal might know, it can infer domain context that is otherwise omitted with a basic conversation. Everyone wants to capitalize on corporate vibe coding but the tech literacy is hardly there, let alone more advanced topics.
> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems.
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
If it performs the overall function but the code is a mess, then you can ask the model to describe parts of it, the structures and what the structures are for. When it shows you something that is messed up, suggest a better framing for the same thing and ask the model to implement the better framing without regressing on already passing tests.
I'm going through this right now on a very difficult to implement task, the original was python and very verbose. But had facilitated a rust implementation that produced byte identical outputs. Then I asked it for what data was being passed around, placing restrictions on what passed between interfaces I could tell it what parts should be immutable and what parts should have no presence outside it's local context. Placing those limitations while having a exemplar of what it should be doing gives it little choice but to make better code if it meets the conditions set while at the same time not regressing.
I say things like 'this field is a implementation detail in a declarative data structure, it should not exist here.
This can mean hours of work with no observable change in program function, yet it is directly addressing the limitations that prevents it from being used in larger tasks.
the devil is always in the details. those details are on every level you look at: human minds, nature around us, space. so if your inputs are vague, you should only expect outputs that are vague and generalized
I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
We are having trouble onboarding engineers with AI now. Some still struggle after their first year with very basic concepts/patterns we use and make the same mistakes again and again in their PRs because they just ask Claude to fix it and never internalize it. I think using LLMs feels good at first because you can get tickets out the door faster but you never develop enough knowledge to make a big impact or become an expert in the code or business.
I’ve found that planning tickets with granular details (like semi specific code changes needed) is one of the best ways to get that deep knowledge of the system. Even if ultimately I delegate most of the implementation to an LLM. I also heavily verify the changes, but I find that that’s less impactful than planning a feature / tickets.
> If we're pushed to delegate most work to them, how do you build that expertise?
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
It's not for everyone, but I write all the code and then I ask the LLM for a review. It catches something minor thing every single time (out-of-date comments on a function or method, typo, etc.), something that's clear a mistake maybe a third of the time and, occasionally, points out something that looked fine to me but actually won't work because of some other code that I forgot about. I'm a person, I can't remember everything I guess!
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
This is something that really needs to be formally studied.
I'm inclined to say that this matches my own experience, but I can't rule out confirmation bias on my part.
As a meticulous person generally looking for a very specific code outcome, I prompt in a way intended to get exactly the thing I have in mind, and my results reflect that. But on the other hand, I have coworkers who type ten-word prompts with very limited specificity, and they seem to get results that way as well, and that makes me wonder.
It would certainly be beneficial for my career and financial well-being for the assertion to be true, because it means I don't have to worry about being pushed out of my job by an army of $15/hr vibe coders. But the convenience of that assumption is exactly why I think it's important to be skeptical.
I do find that "signalling expertise" is important. "I have a significant background in biblical scholarship. You can assume I've read the most important works in NT studies in particular. Do not translate Greek, Latin, Hebrew, or Syriac. Now, I would like to know . . ." That changes things significantly. So does telling it you have 20+ years of experience with C programming, that you have a robust understanding of machine organization, memory layouts, embedded systems, etc.
For sure. On a personal coding project I said "I'm a professional software engineer, and while this is a hobby project I'm not just vibe-coding and want to build reliable software" and the agent suddenly started suggesting all kinds of things to make its code more robust.
LLMs skew toward over-focusing on things that you mention.
The reason "the agent suddenly started suggesting all kinds of things to make its code more robust" is because you said you "want to build reliable software".
It's not a signal of good judgment or understanding. It's just how LLM attention works.
I thought exactly the same at first. But then i wondered if that still holds true with today's advanced thinking, RLHF involved, frontier models. I guess to a certain extend it did indeed behave better, as a reaction to his self description into account.
EDIT: I mean, those systems accumulated so much complexity around the attention based next token predictor.
Training the LLM to do things that the user didn’t explicitly ask for is a good way to get complaints from the users. Doesn’t matter if those things are best practices.
this feels like "make no mistakes" level of prompting. reliable software isn't as simple as making it reliable, it's about choosing the trade-offs in the areas that don't matter as much as the areas that do. if you keep prompting the LLM to make your software more robust it will keep giving you things to do. they aren't all good things. eventually you'll end up needing kubernetes to run a calculator app.
I've been equating them to graphing calculators since the first LLM launched. It's an amazing tool if you know how to use it. If you don't know how to use it, it's still a tool, but you won't be doing anything amazing with it.
maybe outing myself as a dinosaur, but "back in my day" the calculator came with a book that detailed exactly how to use it. Both the high level basic language and the low level system language. Not knowing how to use it is simply a failure to Read The Fucking Manual.
You can read the manual all you want, but if you don't know basic algebra, trig, calculus, etc, you are not going to have any idea how to apply or use much of anything that the manual describes with regards to actually doing math with a graphing calculator. There is a base level of knowledge required.
Not sure I agree with this. The math guy at anthropic's prompts are essentially:
"suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!"
https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7
https://xcancel.com/__alpoge__/status/2083855298239078748
Tao's chat was for him to gain intuition, not to solve the problem from the outset.
What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.
This works better for math because math is self-verifiable. Once you have a proof it needs no outside evidence.
Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction.
However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about".
This includes things like "before you start fixing this bug, write two tests that fail proving it exists".
Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it.
Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
Yes, we're in the low hanging fruit stage where folks can just say "here's the problem" and "keep going" until a result is found and it will work sometimes.
The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
> Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof
And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them.
Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
Yup I linked that in my other comment but copy-pasted here for everyone's convenience:
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample"
LLMs are a collection of biases. Humans are also a collection of biases. So we project our biases as input through the biases of an LLM and get an output. Hence why I think getting optimal output requires being an optimal person. And in that sentence there are many points of expression.
Finally, we train our LLMs on who we are. Another reinforcement of biases.
It depends on the levels. People with differing fitness levels and ages run at very different paces. Now, do cars make them more equal or less? On the bottom end, the tide lifts all boats. Most healthy people can learn to drive and will drive "fine", they get from A to B. Out there in the city streets the car flattens the differences, everyone roughly takes the same time to get from A to B in a car.
But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.
Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
So you're saying that it depends whether you're measuring "success at task X" (where in average everyone improves) vs comparative success (where people with knowledge can do far and away better at specific tasks).
But the guy who writes the “just do it” prompt can neither formulate the conjecture in the first place, nor come up with any follow-up questions to build on the result.
Yeah, the people who say no expertise is needed for these things confuse me somewhat. This is indeed the case if you want to be a meat wrapper around an LLM, understanding neither your inputs nor your outputs. But at that point, what is the point of you versus going to the LLM myself? Expertise is necessary because it adds understanding and structure to the blob of text produced by an LLM. Progress can only be built on such understanding.
I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.
> In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet.
You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem (without searching for "how to center a div", or whatever your exact problem was, and copy&pasting the answer and moving on), also becoming more knowledgeable in the process.
The rest of the short blog post has some good points, but the first sentence sounds like it's targeted at the percentage of developers who did StackOverflow copy&paste to close Jira tickets, never becoming experts.
Delegating to LLM-ish AI is just a natural evolution of that. The question is whether they can still add value if kept in the loop.
The article author suggests that the answer is to be expert, and is addressing people who... "either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet."
I don't think AI use is supposed to replace foundational learning such as reading a C++ book or Python book or CSS tutorial when you're a beginner. You still have to do those things if you want to be a professional or a strong amateur. But many people just want to get the thing done. They don't want to become a mechanic, they just want to drive from A to B.
> They don't want to become a mechanic, they just want to drive from A to B.
I'm fairly certain the article is directed at professionals, or at least the AI companies are basing their valuations off of directly taking a slice of that professional "productivity".
Except LLMs will only tend to share the most common or average of what it knows as the standard and deviating from it (including new ways) it can be resistant to.
An expert can lay a different kind of frame to prevent the llm to fell out of its way of being generally too verbose, and that can transfer as well to code generation and complication.
> You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem
Hours + Hours of reading and a lot of trial-and-error. The loop was so long and sooo slow. Now it's instant. As if your very first Google search just solved the problem for you immediately.
Not sure what your point is here since wrong code doesn't mean non-deterministic. How many of these yield different results with the same input and architecture?
I am a frontend engineer and I see so many bugs from vibe-coded CSS from people who think just like you. CSS is very powerful and easy to get wrong if you don't understand it. Since most CSS code on the internet is not good, LLMs do not generate good CSS code and create a lot of bugs ranging from bad performance to completely broken reflow.
Honestly, we don't really need to learn any more. These days, I mostly copy and paste from tickets to the LLM, and then rubber stamp the output. I have people in my team that try to care about what they're making, but they're generating less code, and filling tickets more slowly, than I am. Even when it comes to system design and architecture, the LLMs aren't great at it, but honestly, they're better than my coworkers (I'd say system architecture is probably at the same readiness level that coding was last year). Expertise wouldn't really help you get things done faster.
You could probably replace me with a minimum wage worker to do some manual testing and copy-paste errors from the console into the LLM, and still be fine.
Let's see how long it is before the next round of layoffs, I guess. For now, the money's fine and the work's boring but ok.
But, no, the LLMs rewarding expertise line is pure cope. Software is not really skilled labor any more.
(obviously, throwaway -- I'm not interested in potentially advertising this to my employer)
> Honestly, we don't really need to learn any more.
Perhaps for your task of closing Jira tickets that’s true - but the you have a decision to make: wait until you’re eliminated or try to anticipate where future competitiveness comes from.
Yeah, that's the thing; future competitiveness isn't going to come from learning, at least not from learning about software. The LLMs have that covered, and software is becoming low status work anyways.
It's largely going to come from interpersonal skills and being likeable enough to be the face of the projects and companies. Technical skill is rapidly dropping in value.
All? obviously not, but enough that I can ship more than my colleagues while not really paying attention to the output or fully understanding it. LLMs debug extremely well, generate code well enough, and do high level design acceptably, but fail in weird ways, so my actual work these days (outside of delegating to the LLM) is mostly manual testing. I need to give them nudges sometimes, but a year ago they needed constant nudges. In another year, I expect that it'll be good enough that pasting in a bug report will be all the nudging I need to get something that customers will accept if it gets shipped. They're already close.
It's obviously going to remain pretty manual without some breakthroughs, but it's largely not work that needs a ton of intelligence or skill. The world hasn't caught up with software development being a few steps up from burger flipping, so I'm still getting paid well to do it, and I'll keep collecting that paycheck as long as it lasts. But learning about code is rapidly becoming relatively valueless.
The people who make the tools that generate your assembly instructions need to learn it. Just like the people who make the browser rendering engine and push CSS forward still need to learn it.
The people who don't need to learn asm never needed to learn it. If you wouldn't code in asm now, you wouldn't have ever.
I just want to point out the exception - the person who the AI learns from.
Someone has to know it. And to advance it to the next level requires domain level expertise. Or are we allowing LLM's to determine the next CSS snapshot/version/level/whatever?
Yes, but sometimes I don't need and want to learn. One example from my recent experience in research -- building custom dashboard pages for results of scientific analyses. Each analysis is bespoke, and building interactive webpages is simply not the skill many researchers have (and it's boring IMO). But here with LLM you could easily explore the results visually/share them with collaborators etc. There are plenty examples like that.
But certainly there are cases where learning is required.
Why do you think thats worth learning? Doing a few times manually sure… but to force yourself to solely rely on it is foolish. Just use the calculator.
I lost my desire to learn anything about web development twenty years ago and specialized my career in a completely different direction. I had not touched HTML at all since then until earlier this year, when I discovered that AI robots could do it for me. Now I am happy to whip up HTML/CSS/JavaScript visualizers, explorers, and even one-off interactive report presentations whenever they might be useful, precisely because I don't have to learn any of that crap to do it. My time & attention are far more productively spent focused on the work I am actually good at and interested in doing.
There was some learning, but also a lot of waste. As a self-learner, I've never been able to learn so fast as I can now with an LLM to instantly answer my specific questions, and incrementally correct and grow my mental model. And while with some subjects you need to worry about accuracy, LLMs are generally very good with programming questions, and (for most types of questions) you can verify their claims yourself.
The problem with CSS is it's a) all broad and no depth and b) constantly adding major new paradigms. If you don't get up to speed and stay there, you lose it.
Pre-AI I'd say I've gotten over the hump 3 times with CSS, only to lose it again by the next time I had to use it in volume.
You seem to be hung up on the part about "learning". Do you know how x86 registers work? Or atomic instructions available on ARM, SPARC, or POWER architectures?
No, of course not. Because all of that got abstracted to higher-level instructions decades ago.
I understand your point, but also imagine applying that across the whole spectrum of computing.
It's very useful that there's an intermediary that knows how that stuff works so I can build things without thinking about it in excruciating detail the whole time, I can dip down lower and learn stuff when it's relevant (like cache access and nested arrays) but I can also not do that in many situations.
There's also the argument that you can do engineering without understanding the underlying science as seen in th pyramids, the beautiful old european churches, etc
Well that's great for you. That's some knowledge that you will possess for the rest of your life.
I want a green lawn and big bushy shrubs in front of my house. Do I need to know the intricate biology of my soil and habitat? No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
>No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
And others want the green lawn and big bushy shrubs, but don't want to learn all of the surface level things to make their lawn thrive so they hire a service to do it for them. And there's nothing wrong with that - not everyone enjoys the yard work, but they still want the thriving lawn and shrubs... and they are happy to write a check to OpenAI... err...Lawn Doctor every month to get that result since the time they save by not dealing with their lawn, they can pursue things they do want to do.
Not everyone wants to (or needs to) learn every detail along the way of getting the results they want.
But maybe there's a limit to the usefulness of this abstraction. Perhaps forgetting too many basic skills lower down isn't always as useful as going from Assembly to C. I'd argue that there's a law of diminishing returns there, and it's not an endless series of benefits.
So, since you care so much and caring requires knowing everything about everything that even remotely intersects with your project, as (presumably) a software developer presumably you understand in detail how every bit of the CPUs you're targeting, plus the lithography, chemistry, and how solid state physics, and the underlying quantum mechanics work?
You seem to be massively exaggerating here for effect.
I think the main point is, if you're building frontend web apps, you should probably know how HTML, CSS, and JavaScript interact to some decently high level. Can you look stuff up on the margins via AI, or StackOverflow, or whatever? Sure.
If you're building <something else>, you should probably know the core tools and concepts at the appropriate level to build them.
No we do not need to know every aspect of CPU branch prediction and whatnot to make a webpage.
I think it's pretty obvious from the context of "if you had technical gaps (say, you couldn’t write CSS)" that this isn't about someone that regularly works on frontend web apps, it's about someone that is having to do something that is not normal for them. E.g. I mainly work on HPC code, maybe once in a year or two I need to throw together an animated poster to accompany a printed one. It's obviously ridiculous to claim that this means I have to learn frontend web dev.
All the boring stuff you skipped to get right to the answer you want was just as important whether you want to admit it or not.
Having to read through a structured resource describing something to figure something out has intrinsic value that an LLM is not going to provide you with.
> All the boring stuff you skipped to get right to the answer you want was just as important
This is obviously not the case. There are mountains and mountains of boring, unimportant stuff that LLMs can do for us now.
For example, at work I can now make a nestJS dashboard without having any idea what nestJS is. I can just tell the LLM what I want, and it makes it so. This allows me to make my tool useful to people without having to become an expert on something unimportant.
I've seen junior engineers be productive on their first day in the industry because of AI, so I don't think the article is the whole truth.
The example math is boundary-pushing and definitely not a solved problem. But most of us work on CRUD backends with a React frontend. Those are more or less solved problems that have well-documented solutions. For those kinds of tasks, LLMs just reward usage.
I can count on one hand the number of times in my career I've needed to solve a problem that's not described on Stack Overflow.
I think you're talking about a different type of expertise from TFA. Consider this: What if I never enjoyed frontend programming and so I never wanted to be an expert on that?
In fact, I never enjoyed frontend programming because it was such a pain to deal with matters I considered trivial yet so frustratingly hard to do right... like centering a div. And yet the slightest misalignment is visually jarring and forces me to get a bit OCD about fixing it, which made it even more frustrating.
I questioned the whole premise of the situation: is working around a bad developer experience something worth spending my time on? Unless I actively wanted to get in there and fix the situation, not really. So yes, in those cases I would outsource my problem to a colleague or StackOverflow and move on. And as a career choice, I preferred to do more backend dev.
I would posit that that was the type of expertise that did not matter. The type of expertise that really matters here is good UI design. That is entirely orthogonal to the drudgery that is implementing and debugging webpage rendering, and I am eternally grateful to LLMs for freeing us from it.
You can extend that line of thought to the entire article. What really matters (and what LLMs reward) is domain expertise rather than technical expertise.
The short version I give to non-technical people who ask me about whether "AI will replace coding" is this: it accelerates you. You can get much further much more quickly.
If you don't know where you're going or how to get there, or even if you're just not paying enough attention, it will get you very far in the wrong direction before you've realised.
The LLM is like the death star. If you don't know exactly where to point it, you will likely miss your target and have no/negative effect. The further away the target, the more accurate your firing solution needs to be. If all you need to do is add something like a dark mode theme to an existing product, this is probably a point blank shot in this metaphor. Building an entire codebase from zero, or even refactoring a legacy codebase into a new codebase, are lightyears away by comparison. You can still land the shot, but you need to deeply understand the metrology and astrodynamics.
The information system required to encode the aesthetic preferences needed to make a technology experience not suck is likely in excess of what any near-term solution will offer. Knowing when to say "no" is perhaps the most important skill here. You can't just say it arbitrarily either. You really have to mean it and be willing to fight other humans for it.
I totally see this. I just did 3 hours of bot sitting to put together some thrash loops that thrash our provisioning working flow for a BLE gadget we make. It was pretty straightforward and productive. But then, I have a lot of experience with BLE, and a quite a bit of experience with python and shell scripting. So I was able to guide the process through stages, do some intermediate testing, make some adjustments, and proceed. Domain experience made this really easy and straightforward. Me two junior engineers who have only superficial/high level knowledge of BLE and some of the other pieces, couldn't have done this as effectively.
Where my angst comes, is worrying that no one will ever get that experience anymore. They might have had some eventual success, who knows what monstrosity a much less guided LLM would have done, but experential learning may be mostly a thing of the past. And it creates a real tension between the person with experience and the person without.
>Where my angst comes, is worrying that no one will ever get that experience anymore.
I am a fairly inexperienced python developer using LLMs to build software and find that I still learn a lot just from prompting and tinkering. Maybe that's less true once you reach a certain level of competence, but at my intermediate level I am still learning a lot even leaning heavily on LLMs.
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample" lol
I actually don't feel like Tao's recently published conversation is the best example of this idea. As intelligent as Dr. Tao is, and surely more so than me, I got the feeling that he wasn't running up against failure states of the model, which I'm not sure you could attribute entirely to his expertise. I honestly think it was more a matter of luck that the model apparently had so much training data on the topic or that it was architecturally so well suited for it. On the other hand, I've had really surprising moments where Claude was just failing terribly to execute simple dev ops tasks having to do with log processing. And I'd be so bold to say that I don't think it could have been explained by a lack of expertise on my part, or even a misuse of the model.
So yeah, sometimes LLMs reward expertise, sometimes they don't. I guess either way it helps to have it.
That's why when people like Pieter Levels tweet "I cancelled and then vibecoded 100% of my SaaS subscriptions", you need to take it with a huge grain of salt because you're not Pieter Levels, you cannot vibe code your SaaS subscriptions.
There's a growing and fascinating divide between people who see LLMs as more of a "bicycle for the mind" in the vein of Jobs vs those who see them as whollly supplanting the role of human intelligence. I can't help but wonder if these aren't primarily two human archetypes more than anything - the LLMs can be both and they erect a mirror of the human using them. Some humans really don't want deep individual expertise and intelligence to be the deciding factor because they don't identify with that. Others are completely the opposite. We really can't tell which will be more effective yet, because LLMs are very good in both modes. But most of the predictions currently are people executing on wishful thinking about what they hope will be the outcome.
"The most important skill in the AI era may not be prompting. It may be learning how to solve problems using the right kind of help." [1]
Context - I have over 25+ years in software, and I have this observation - being introduced to a new codebase as a human is difficult, especially depending on the scale/size and complexity of it.
Yes, you do start to learn it as you work through it, but if the scale is truly huge, it may just not be possible to fully read and understand all the code and paths etc.
I have found systems-thinkers (I believe I am one, sometimes they are architects) to be able to kind of "see the whole picture" while not knowing all the details, to the point of being able to guess how the system/software should be behaving, even if it is not actually yet. This is a hugely valuable skill and I think takes a certain kind of brain too.
That said, I think recently I may have realized something - we rely on statistics and confidence levels in order to make statements about larger populations. If we can represent a codebase as a, perhaps stratified population of code, interfaces, docs, etc. etc. etc. we may be able to take a valid random sample, review portions of the code, and make some kind of assertions about the state of the larger system - potentially, from that.
I am trying to implement this as a side project right now to see if there is anything to it, basically, a combination of AI/LLM + stats/sampling + facilitated expert human review.
I'd be interested to know if anybody is doing anything similar.
I am feeling this a lot lately. Getting the most out of agents seems to require being able to ask the right question. And how can you ask the right questions without deep domain expertise?
Yes sometimes it’s a matter of just using the right word. You can talk to an agent about a general concept for hours and hours and it may never mention $Concept_X, but you mention $Keyword_Y and all of a sudden the AI is going on about how $Concept_X is foundational to understanding the whole thing.
I started developing webapps back in the late 90s when I was in high school using Perl, and I've worked with tons of technologies up till around 2014 or so when I shifted into almost pure backend work and lost touch with modern frontend development.
I'm now learning how modern frontend development is done (for both personal and professional projects), so I may not know the specific tools, technologies, or terms but I can say "whatever the equivalent of XYZ is" and the models will translate for me. If I say "run pytype" it will tell me "we're using mypy - i'll run that checker for you".
If you can express what problem you're trying to solve, that will get you most of the way - and then you can refine by asking questions. "I think I need something like Redis for caching things - do people still use that? Is there a simpler more modern version that is the new standard? Do we already have company docs suggesting what to use?"
This matches my experience. Just Talk To It is the best method for working with LLMs if you're an expert.
I've seen this at work (as eng manager/lead/principal/whoevenknowsanymore) – all the big APIs give you stats. We see how much people burn in tokens and we know how much output they produce. There is a pretty strong inverse correlation between token burn and output.
The more tokens people burn, the less likely they are to produce a good outcome.
A good moment to remind everyone that if we took the promise for granted, that AI will in fact prevail and prompting is the one skill that will rule them all... we'll lose all domain experts in one generation.
It's less of "signaling expertise" and more about actually having said "expertise".
In my experience with LLMs it's not uncommon to be having a deep conversation about making pasta, only to be told, after asking for a sample recipe, to get a bucket of paint and a bag of concrete. Of course these hallucinations are way more subtle and easy to miss for someone who doesn't have deep domain knowledge.
Domain knowledge will stand alone as the sole differentiator. Because LLM benefits can be reaped by almost anyone and it’s a force multiplier. Now those who have the strongest initial force will have a far bigger edge than before.
Real world domain knowledge and experience cuts through the chaff too. LLMs are going to have people reinventing the wheel and wasting tonnes of time on stuff that won't work out. If you're a domain expert you are going to be much more aware of how to focus effort in the right places, and what's actually needed or been tried before in your niche.
A lot of this domain knowledge is not in any training data, it's locked up in companies in the industry. I suspect it will get even more important to guard it.
Love this idea of reading prompts that lead to new discoveries and figuring out how the person got the LLM there. It truly is an art and I’m always reminded of “I, Robot” and the scene about “you must ask the right questions.”
Well, nice post. Actually, there may be some truth behind it, but basically, it captures what I—as a programmer—want to read: expertise will remain valuable.
But how I am observing is different, though. Since LLMs the gap between experts and non-experts has been shrinking. And yes, there is still a gap, but vanishing.
Agree so much with this! In domains I know well I get much better results then someone who doesn't know the domain because I know where to challenge the LLM. LLMs need to be pushed because otherwise their answers are typically average.
In my experience (scientific programming) AI is a giant multiplier for people with specialized knowledge.
But it’s also a giant devaluer for that same knowledge as people with no idea what they’re doing can clog the field with plausible bullshit.
It’s now the case that if someone tells me they’ve done something, and I look into it and find out it’s completely AI slop, then I will have spent more time on the project than the person who “made” it. The situation is completely untenable and only serves to drain time and resources from people with better things to do.
this will have educational consequences (that I'm trying to solve). I don't think that we can adjust without rapid education and making extreme specialists of us all.
This requires coordination, certification, licensing, and other tiers of authenticity. False experts can ruin sample gathering, can ruin training. False expertise is exemplified by the current American Administration. Look at Robert F. Kennedy Jr.; he's a false expert. He is responsible for the measles outbreak. He is responsible for ivermectin abuse by humans. False expertise is overtaking real expertise. And the results are continuously disastrous and large-scale.
So we could run a lighter LLM in front of humans, which translates from 'no domain knowledge' to 'domain expert' and in turn prompts over to the larger LLM.
Then the larger LLM gets all the right lights on, yields better outputs and we translate back into user domain.
I kinda thought the chain-of-thought reasoning already did this, no?
Yes. I agree with most, if not all of this. For instance, I am seeing folks either relying in the LLM as an _assumed_ expert or, assuming someone - who knows the structure of skill definitions - also has some expertise (in the area of the skill).
It's a difficult situation; there is not much point in explaining _why_ the LLM output or skill in use (on a domain problem) isn't what the person actually _needs_ to address the domain problem, because the person isn't a domain expert or indeed, adjacent to domain expertise.
But, it is an interesting experiemnt to arm folk with little domain expertise with the _skill_ necessary to be able to extract the right solution from the model.
I agree. When I talk to LLMs about fields I am familiar with, I can push back on bad suggestions and ignore faulty/incorrect advice and assumptions, which is much harder for unfamiliar subjects.
Of course, simple common sense and extremely basic Googling on unfamiliar subjects can produce similar results, but it's much faster if you are truly understanding what the AI is suggesting.
This is also why people's experience with LLMs/AI varies so much, because some people can see a use for AI for their needs and go about using the tool, while others do not as it relates to whatever they're working on and so they may say "LLMs/AI are useless" (it doesn't mean they're not experts though, although some people who have totally no expertise might also see no use for AI for themselves).
I often do my best to represent a genuine interest in the subject at hand and learning in general to models. Imagine the model's response prose and mannerisms being on the other polar end of answering questions simply to get the correct answers as they're often scoped for on quantitative benchmarks. Not sure I explained this well, sorry. An LLM could help
I have lengthy conversations with my LLM, almost like an interview. I agree on the expertise part, because I wouldn't be able to go in depth on a subject with it if I lacked the expertise.
Some work is a result of design and negotiations in those designs.
I don't think Tao's style works with everyone/thing, especially if we don't know what style he's tuned his LLM on.
Might be a very noob question but in this era of LLMs, let me ask the reverse, how do you gain expertise? It seems this rewards people who had expertise pre LLMs, but what about people who don’t have that in a specific domain? What approaches are viable now in this current system?
But getting an answer is not the same thing as understanding why that is the correct answer, or going deeper and learning more about the subject.
IMHO, the people who genuinely desire to learn will trudge through whatever they need to in order to grow their understanding - be it through reading books, original research papers or what have you.
If, OTOH, all you seek is the answers and that alone is satisfying to you, then of course you simply will not be motivated to do it the old school way anyway. But that's hardly different now in the age of AI.
The same as it's been! Make things without using LLMs. Don't debug with them, don't use them to research things, just do it yourself. It'll be painful and that pain is learning.
this 100%. Skills are gained via effort. Not too much effort that it discourages you, but enough so it's a challenge and engages you. Ofc it helps to have wonderful teachers, coaches, mentors, books, even ai to help guide you, but YOU must put in the effort. You don't get something for nothing.
i find this post re-assuring (as who doesn't like to feel like they are an expert at something and llm definitely strips that away)... but it still feels like you are rewarded just as much for being a 6/10 expert as you are for being a 9/10 expert. It definitely is an equalizer it is just a question of to what degree.
The fact that Claude knows I wrote the Virgil compiler makes it be on its best behavior when working on it. I force it to not write too much code, and to write more tests. I push back on slop and just adding another special case. It has a surprisingly deep understanding of floating point.
Overall, I agree, when I ask things I'm an expert in and do professionally every day. I get very good useful answers. When, for example, our marketing people, ask about the science, they often get confusing and wrong answers.
You got to know how to use the model+harness+prompt to achieve the results you want, but honestly for many projects and questions all the models already pump out their same best version of an answer.
Sometimes it is really akin to a git clone, although it was a LLM request.
This rewarding expertise is somewhat wishful thinking.
At the end of a day, it feels and is more like gambling, even with the recommended expertise and a good approach, don't delude yourself you're simply pulling the lever too, as any novice.
I'm surprised nobody mentioned (including the author) the Gell-Mann Amnesia Effect [1]... Just substitute "LLM" for "journalist" and there you have it.
And to be honest, I have seen it, as I'm sure (almost) everyone has, who has demonstrated experience/expertise in their own fields, and correct the LLM's responses one time or another...
This kind of sentiment really pisses me off. It’s just fucking pathetic. A computer is doing the interesting and amazing parts and we’re coping hard by suggesting we can master the trivial detail of making our prompts concise (no wait next week make them detailed again because Terence Tao wrote in long paragraphs this time!)
Terence Tao’s conversation with the chat model was wholly unremarkable compared to the solution to the Jacobian Conjecture. Tao gets to write a well-read vibe-researched blog post not because he’s mastered some genius prompting skill. I’m sure thousands of other people could do the same. It is because he is a celebrity. That’s it.
By the way, it was the fashionable prompt-jitsu to include more in your prompt like two months ago. The prompt magic changes all the time to suit whatever makes some blog tsar feel like they’re really adding to the discourse.
If there’s some optimal prompting method then the AI system will figure it out and modify your prompt to match it. Maybe for one or two years having domain knowledge will help you, but everyone and their grandma has been saying forever that this will only last a short while.
Unfortunately the software industry is saying things like "don't look at the code", "LLMs have made developers 10-100x faster", etc. The only way they can make such claims is by saying what you said above: "Claude, make me Microsoft Flight Simulator, make no mistakes".
Additionally engineers are facing pressures via deadlines to work in the paradigm of "Claude, make me Microsoft Flight Simulator, make no mistakes"...
The question i wonder about is, when will an event come along that persuades everyone that human understanding is still required? Or will it never come?
Such an event would have to be pretty catastrophic at this point to slow down the inertia. Perhaps the tech debt will just pile up until someone's product implodes, or there's a massive safety issue that causes loss of life, or some big hedge fund goes bust.
I wonder the same thing. I think we've already seen some of this happening, however the consequences haven't been large enough to the organization, for example:
The event could be when fair pricing comes from the model providers. We're still at the cash burning stage. When the economy crashes a little and departments start monitoring their spending, and the prices for inference are 10x what they are, there will be less tolerance for employees to substitute constant AI usage for understanding.
You're assuming that LLMs entered a world of people who understood how the systems they're inside of work, why they're setup that way, and that LLMs are displacing them. I sadly don't think that's the case in... well... a lot of the cases.
Many, including myself, report having a lot of success with braindumping and not structuring anything. Just talking into speech recognition for 2-10 minutes as a stream of consciousness about what my context is, what I want, what I know already, what I have a vague hunch about, how it fits into a bigger picture, what aspects are most important to me, any footguns I already know about, really like having a chat with a person on the phone, with someone you have to guide remotely because they have to implement the thing right now but you have to be out of office and so your only interface is speech. Except you can be more structureless because the AI won't be offended. Just keep on rambling, and press enter, don't even correct mistranscriptions. It will understand it anyway.
Now, the key is, that while rambling without structure, you do have to drop the key facts into your speech, and you have to know what you're talking about in at least a good portion of it.
I think people are afraid of doing it, because it seems "not the right way" or "not scientific" or whatnot. They want to believe there is some magic to writing the right prompt. So let me tell you, it works.
I don't completely disagree with the concept of giving a free association thought process ramble into context. But I also bet that when you start getting it to actually generate code and link modules of things together, subroutines, functions, code structure and filenames, you still pay attention to what it does and you guide it into the architecture that makes logical sense to you.
For real work yes. For personal projects, less and less since Fable came out (probably the same if true of the other frontier models). You can get a lot done if it's just some one off, or a personal tool, even without looking at the code, just trying the application. Frontier models now automatically test it before handing the thing to you, they take screenshots, they fix the superficial issues themselves. To get something up and running, it's enough to send chat messages.
I did a test a few months ago. A friend of mine wanted to develop what i understood to be a simple single page web app. But since she didn’t have any software engineering experience she asked me to help. Around that time everyone was talking about how literally anyone can develop software with LLMs i asked her if she could give it a try first, and if I could watch the attempt.
I was fully expecting that writing the code will pose no problem for the AI. But i was curious if the AI will realise that my friend is a novice and needs extra help with things like: copy pasting the code into a text file and saving it with an html extension, helping her host the file online so she can share it with others, buying a domain for it, etc. I assumed they will get there eventually, but i also assumed that it will take a lot of stumbling around and misunderstandings.
But i was completely wrong. They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. They were just going around in circles where the AI was brainstorming with her about possible features and getting thints more and more complicated. We terminated the experiment after one and a half hours and many many messages exchanged between her and the LLM.
Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. In other words in that case the LLM would have rewarded even just a little bit of expertise, but without it there was a confusion about goals between the human and the machine.
Are you describing a "chat window" experience here? This is apples to oranges.
Yeah I wonder if they had given their friend Claude code or Codex, would it have been more likely to create what she wanted?
Perhaps! But I do think the vocabulary issue is real and I think LLMs are still sycophantic enough that they won’t really challenge someone or offer alternative ideas on how to implement something unless they explicitly ask.
Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success.
The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real!
In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by.
I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become).
But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
OTOH my wife's friends got drunk and made "tinder for horse purchases". They prompted to read typical horse advertisements (we're all horse people) and create an app with mock tinder like entries to swipe right and left to buy horses.
A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create.
I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!
The amplifying mirror analogy works best here. LLMs are ultimately a reflection of your own interactions with its weights, the tone you use, the structure with which you construct your prompt, aspects of an issue you tend to focus on, your breadth of vocabulary and world knowledge and whatnot.
People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
One of the Claude skills I made Claude itself generate was the 'learning a concept across tiers' skill -- from ELI5 level to a PhD level, and it triggers whenever I ask it a very general question on a complex topic that isn't my bread-and-butter. The fact that I'm able to choose explanation level from a super smart LLM (that's available 24x7) that can explain any topic under the sun would've been mind-bogglingly sci-fi-ish just 4 years ago in 2022.
That sounds useful. Can you share that skill?
Not the OP but you can whack this into your prompt and get most of the way there: "no jargon goes unearned, nothing gets dumbed down, every abstraction touches ground"
A punchy tricolon containing 2 analogies that don’t quite make sense. That’s some S-tier AI-mimicking. Nice!
One could say that it being an LLMism is...load-bearing :)
When I get out of my domain, I always ask it to describe things to me "like I'm a senior CS student who doesn't know any industry terms". I find it really easy to skim past the definitions I know and very useful to get the explicit clarification on unfamiliar terminology. Telling it to talk down to you a little also seems to calm down it's tendency to talk like it's trying really hard to convince you it's smart like a 8th grader trying to use every single vocab word they learned last week.
I've been using a prompt that comes from the opposite direction for non-programming stuff: "Assume the user is an expert in all fields; while this is clearly logically untrue on a literal level, the user prefers to see a field's technical terminology and then ask the AI about terms the user does not understand rather than get an inaccurate statement about some issue."
Whether you have to reassure the LLM that this is obviously untrue, I don't know, but they do have a knowledge baseline to know it's not true and I have a sneaking suspicion it would be less effective without that.
This has ended up in some of the most interesting incidental knowledge exploration I've ever done. A recent example is that I was asking about some stretches and it started talking about how useful they are for the sarcomeres, which I had not heard of. Now I have.
I'm not saying this is better, just that it is different. I think there's a time and a place for both approaches.
The amplifying mirror is not just a characteristic, but the fundamental driving force of LLMs. With every token it consumes, its primary goal is to understand who you are and what you intend. If you type Hola thats sufficient to tell it so much more than if you type hi.
At work we call this implicit steering. To use webdev metaphor, if a non-technical person describes making a web page with a big block at the top and some things to click on and then my pictures below that, that will eventually get somewhere. Meanwhile, if you know industry jargon, you might describe a hero, with call to action buttons, and then below a 3x3 grid of images of my portfolio photos—that's likely going to generate something entirely different and likely richer. It can assume things about you (it doesn't think), it can ask you specific questions a web personal might know, it can infer domain context that is otherwise omitted with a basic conversation. Everyone wants to capitalize on corporate vibe coding but the tech literacy is hardly there, let alone more advanced topics.
> Of course both are useful, but I’d rather have familiarity with the codebase than a deep general understanding of software systems.
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
Read the code.
What if the code sucks, because it was vibe coded by an LLM over a dozen disparate sessions?
Claude, make this codebase less ass
That will actually work pretty well.
I legitimately caught Claude calling things "ass" while a friend was using it earlier, which I think is pretty funny.
Then fix it using the llm
Then that's the point. You know the code sucks, the guy who vibecoded it either didn't know or didn't care. That's the added value.
If it performs the overall function but the code is a mess, then you can ask the model to describe parts of it, the structures and what the structures are for. When it shows you something that is messed up, suggest a better framing for the same thing and ask the model to implement the better framing without regressing on already passing tests.
I'm going through this right now on a very difficult to implement task, the original was python and very verbose. But had facilitated a rust implementation that produced byte identical outputs. Then I asked it for what data was being passed around, placing restrictions on what passed between interfaces I could tell it what parts should be immutable and what parts should have no presence outside it's local context. Placing those limitations while having a exemplar of what it should be doing gives it little choice but to make better code if it meets the conditions set while at the same time not regressing.
I say things like 'this field is a implementation detail in a declarative data structure, it should not exist here.
This can mean hours of work with no observable change in program function, yet it is directly addressing the limitations that prevents it from being used in larger tasks.
Reading it is good, but working with it more directly seems to help retention.
the devil is always in the details. those details are on every level you look at: human minds, nature around us, space. so if your inputs are vague, you should only expect outputs that are vague and generalized
I think this is only a partial answer but I've been surprised by how familiar dev leads become with the app even if they are not in the code.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
We are having trouble onboarding engineers with AI now. Some still struggle after their first year with very basic concepts/patterns we use and make the same mistakes again and again in their PRs because they just ask Claude to fix it and never internalize it. I think using LLMs feels good at first because you can get tickets out the door faster but you never develop enough knowledge to make a big impact or become an expert in the code or business.
These would have been good hires before AI but now they struggle?
I’ve found that planning tickets with granular details (like semi specific code changes needed) is one of the best ways to get that deep knowledge of the system. Even if ultimately I delegate most of the implementation to an LLM. I also heavily verify the changes, but I find that that’s less impactful than planning a feature / tickets.
> If we're pushed to delegate most work to them, how do you build that expertise?
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
It's not for everyone, but I write all the code and then I ask the LLM for a review. It catches something minor thing every single time (out-of-date comments on a function or method, typo, etc.), something that's clear a mistake maybe a third of the time and, occasionally, points out something that looked fine to me but actually won't work because of some other code that I forgot about. I'm a person, I can't remember everything I guess!
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
This is something that really needs to be formally studied.
I'm inclined to say that this matches my own experience, but I can't rule out confirmation bias on my part.
As a meticulous person generally looking for a very specific code outcome, I prompt in a way intended to get exactly the thing I have in mind, and my results reflect that. But on the other hand, I have coworkers who type ten-word prompts with very limited specificity, and they seem to get results that way as well, and that makes me wonder.
It would certainly be beneficial for my career and financial well-being for the assertion to be true, because it means I don't have to worry about being pushed out of my job by an army of $15/hr vibe coders. But the convenience of that assumption is exactly why I think it's important to be skeptical.
I do find that "signalling expertise" is important. "I have a significant background in biblical scholarship. You can assume I've read the most important works in NT studies in particular. Do not translate Greek, Latin, Hebrew, or Syriac. Now, I would like to know . . ." That changes things significantly. So does telling it you have 20+ years of experience with C programming, that you have a robust understanding of machine organization, memory layouts, embedded systems, etc.
For sure. On a personal coding project I said "I'm a professional software engineer, and while this is a hobby project I'm not just vibe-coding and want to build reliable software" and the agent suddenly started suggesting all kinds of things to make its code more robust.
LLMs skew toward over-focusing on things that you mention.
The reason "the agent suddenly started suggesting all kinds of things to make its code more robust" is because you said you "want to build reliable software".
It's not a signal of good judgment or understanding. It's just how LLM attention works.
I thought exactly the same at first. But then i wondered if that still holds true with today's advanced thinking, RLHF involved, frontier models. I guess to a certain extend it did indeed behave better, as a reaction to his self description into account.
EDIT: I mean, those systems accumulated so much complexity around the attention based next token predictor.
Training the LLM to do things that the user didn’t explicitly ask for is a good way to get complaints from the users. Doesn’t matter if those things are best practices.
>build reliable software
this feels like "make no mistakes" level of prompting. reliable software isn't as simple as making it reliable, it's about choosing the trade-offs in the areas that don't matter as much as the areas that do. if you keep prompting the LLM to make your software more robust it will keep giving you things to do. they aren't all good things. eventually you'll end up needing kubernetes to run a calculator app.
I've been equating them to graphing calculators since the first LLM launched. It's an amazing tool if you know how to use it. If you don't know how to use it, it's still a tool, but you won't be doing anything amazing with it.
maybe outing myself as a dinosaur, but "back in my day" the calculator came with a book that detailed exactly how to use it. Both the high level basic language and the low level system language. Not knowing how to use it is simply a failure to Read The Fucking Manual.
You can read the manual all you want, but if you don't know basic algebra, trig, calculus, etc, you are not going to have any idea how to apply or use much of anything that the manual describes with regards to actually doing math with a graphing calculator. There is a base level of knowledge required.
Nice analogy.
I loved graphing calculators until I learned tools like Mathematica and Matlab. Still waiting for the Mathematica version of LLMs.
Agents / loop engineering / whatever is hot with the AI Twitter kids still isn’t it.
LLMs are golden retrievers that have memorized all of wikipedia
Not sure I agree with this. The math guy at anthropic's prompts are essentially:
Tao's chat was for him to gain intuition, not to solve the problem from the outset.
What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.
This works better for math because math is self-verifiable. Once you have a proof it needs no outside evidence.
Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction.
However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
Corollary to this is an important part of LLM usage is what I call pinning it to reality. That is, designing verification steps that interact with the real world in some way not easy to hallucinate or work around. This means things like having code that interacts with the physical world, round trip tests, arriving at the same result using different paths, interoperability / replication with external libraries / competing products, performance improvement projects that start with robust performance test suites, and similar sorts of things that reduce to "how do I provide evidence that's difficult to fool myself about".
This includes things like "before you start fixing this bug, write two tests that fail proving it exists".
Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
And what about problems that cannot be one-shotted but helped along?
There was one math proof that was AI generated going around Twitter and the chat transcript to generate it was basically the human prompting "keep going" until it solved it.
Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
Yes, we're in the low hanging fruit stage where folks can just say "here's the problem" and "keep going" until a result is found and it will work sometimes.
The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
> Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof
And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
> why do you think this would be the case?
Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them.
Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
While they may struggle on what to study, we'll be able to spin up millions of them.
Yup I linked that in my other comment but copy-pasted here for everyone's convenience:
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample"
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
LLMs are a collection of biases. Humans are also a collection of biases. So we project our biases as input through the biases of an LLM and get an output. Hence why I think getting optimal output requires being an optimal person. And in that sentence there are many points of expression.
Finally, we train our LLMs on who we are. Another reinforcement of biases.
It depends on the levels. People with differing fitness levels and ages run at very different paces. Now, do cars make them more equal or less? On the bottom end, the tide lifts all boats. Most healthy people can learn to drive and will drive "fine", they get from A to B. Out there in the city streets the car flattens the differences, everyone roughly takes the same time to get from A to B in a car.
But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.
Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
So you're saying that it depends whether you're measuring "success at task X" (where in average everyone improves) vs comparative success (where people with knowledge can do far and away better at specific tasks).
> who these models reward/empower
The easy, straightforward answer is "the people who own the models". Who else benefits feels like a more complex question and we'll have to see...
> Like all of life, maybe the "winner" is the person who just does stuff.
Someone who just does stuff still has to be able to deal with errors and failures. That’s where an expert or a generalist may have an advantage.
But the guy who writes the “just do it” prompt can neither formulate the conjecture in the first place, nor come up with any follow-up questions to build on the result.
Yeah, the people who say no expertise is needed for these things confuse me somewhat. This is indeed the case if you want to be a meat wrapper around an LLM, understanding neither your inputs nor your outputs. But at that point, what is the point of you versus going to the LLM myself? Expertise is necessary because it adds understanding and structure to the blob of text produced by an LLM. Progress can only be built on such understanding.
I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
Who's end state took / is going to take more tokens / money, however?
"LLMs reward expertise" is the title, not that "LLMs only make things possible for those with expertise"
It's not contradictory to say that expertise is a multiplier, and that models are systematically underconfident in themselves.
It's actually refreshing when a model is sure about something because it actually tested it and has the receipts. Opus 5 seems really good about testing its own knowledge with experiments. Scientific method ftw.
> In the 2010s, if you had technical gaps (say, you couldn’t write CSS), you had to either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet.
You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem (without searching for "how to center a div", or whatever your exact problem was, and copy&pasting the answer and moving on), also becoming more knowledgeable in the process.
The rest of the short blog post has some good points, but the first sentence sounds like it's targeted at the percentage of developers who did StackOverflow copy&paste to close Jira tickets, never becoming experts.
Delegating to LLM-ish AI is just a natural evolution of that. The question is whether they can still add value if kept in the loop.
The article author suggests that the answer is to be expert, and is addressing people who... "either rely on a skilled colleague or just hope that the answer to your exact problem was out there on the internet."
I don't think AI use is supposed to replace foundational learning such as reading a C++ book or Python book or CSS tutorial when you're a beginner. You still have to do those things if you want to be a professional or a strong amateur. But many people just want to get the thing done. They don't want to become a mechanic, they just want to drive from A to B.
> They don't want to become a mechanic, they just want to drive from A to B.
I'm fairly certain the article is directed at professionals, or at least the AI companies are basing their valuations off of directly taking a slice of that professional "productivity".
Except LLMs will only tend to share the most common or average of what it knows as the standard and deviating from it (including new ways) it can be resistant to.
An expert can lay a different kind of frame to prevent the llm to fell out of its way of being generally too verbose, and that can transfer as well to code generation and complication.
> You could read some general reference/guide/tutorial documentation on CSS, and then probably solve your problem
Hours + Hours of reading and a lot of trial-and-error. The loop was so long and sooo slow. Now it's instant. As if your very first Google search just solved the problem for you immediately.
>Hours + Hours of reading and a lot of trial-and-error. The loop was so long and sooo slow.
But that's how you learn...
It is not the only way to learn.
We don't need to learn CSS anymore. Just like we don't need to learn ASM since GCC does a great job generating it from higher-level code.
I want to believe this is sarcasm, but it's so hard to tell these days.
It's not sarcasm.
We've all accepted that code-generation has been required and accepted for decades.
Code generation (by compiler) has one major difference: it's deterministic.
Here are the reported miscompilation bugs in GCC so far in 2026. The ones labeled "wrong-code".
https://gcc.gnu.org/bugzilla/buglist.cgi?chfield=%5BBug%20cr
I count 500+ of them.
Not sure what your point is here since wrong code doesn't mean non-deterministic. How many of these yield different results with the same input and architecture?
That's not what "deterministic" means.
For many cases like the one described, its true.
I am a frontend engineer and I see so many bugs from vibe-coded CSS from people who think just like you. CSS is very powerful and easy to get wrong if you don't understand it. Since most CSS code on the internet is not good, LLMs do not generate good CSS code and create a lot of bugs ranging from bad performance to completely broken reflow.
Funnily enough, I seem to remember some people saying that compiler generated assembly wasn’t as good as handmade assembly.
For early compilers in certain contexts, this was once true.
the question is, what's the business cost of some amount of css bugs vs paying specialists or people to learn these details.
the generalists win overall, except of course for specific cases where specialists are great
Honestly, we don't really need to learn any more. These days, I mostly copy and paste from tickets to the LLM, and then rubber stamp the output. I have people in my team that try to care about what they're making, but they're generating less code, and filling tickets more slowly, than I am. Even when it comes to system design and architecture, the LLMs aren't great at it, but honestly, they're better than my coworkers (I'd say system architecture is probably at the same readiness level that coding was last year). Expertise wouldn't really help you get things done faster.
You could probably replace me with a minimum wage worker to do some manual testing and copy-paste errors from the console into the LLM, and still be fine.
Let's see how long it is before the next round of layoffs, I guess. For now, the money's fine and the work's boring but ok.
But, no, the LLMs rewarding expertise line is pure cope. Software is not really skilled labor any more.
(obviously, throwaway -- I'm not interested in potentially advertising this to my employer)
> Honestly, we don't really need to learn any more.
Perhaps for your task of closing Jira tickets that’s true - but the you have a decision to make: wait until you’re eliminated or try to anticipate where future competitiveness comes from.
Yeah, that's the thing; future competitiveness isn't going to come from learning, at least not from learning about software. The LLMs have that covered, and software is becoming low status work anyways.
It's largely going to come from interpersonal skills and being likeable enough to be the face of the projects and companies. Technical skill is rapidly dropping in value.
This claim can only be true if LLMs have completely mastered all intellectual aspects of creating and shipping software.
That's so far from being true it's like saying that humans are currently an interplanetary, post-scarcity species.
All? obviously not, but enough that I can ship more than my colleagues while not really paying attention to the output or fully understanding it. LLMs debug extremely well, generate code well enough, and do high level design acceptably, but fail in weird ways, so my actual work these days (outside of delegating to the LLM) is mostly manual testing. I need to give them nudges sometimes, but a year ago they needed constant nudges. In another year, I expect that it'll be good enough that pasting in a bug report will be all the nudging I need to get something that customers will accept if it gets shipped. They're already close.
It's obviously going to remain pretty manual without some breakthroughs, but it's largely not work that needs a ton of intelligence or skill. The world hasn't caught up with software development being a few steps up from burger flipping, so I'm still getting paid well to do it, and I'll keep collecting that paycheck as long as it lasts. But learning about code is rapidly becoming relatively valueless.
Define "we".
The people who make the tools that generate your assembly instructions need to learn it. Just like the people who make the browser rendering engine and push CSS forward still need to learn it.
The people who don't need to learn asm never needed to learn it. If you wouldn't code in asm now, you wouldn't have ever.
This " logic" is so irrational.
I just want to point out the exception - the person who the AI learns from.
Someone has to know it. And to advance it to the next level requires domain level expertise. Or are we allowing LLM's to determine the next CSS snapshot/version/level/whatever?
Yes
Experts work on gcc so everyone else doesn’t have to.
Learning ASM will actually teach you a lot about how the computer works, and I doubt learning CSS deeply teaches you much about design fundamentals.
Yes, but sometimes I don't need and want to learn. One example from my recent experience in research -- building custom dashboard pages for results of scientific analyses. Each analysis is bespoke, and building interactive webpages is simply not the skill many researchers have (and it's boring IMO). But here with LLM you could easily explore the results visually/share them with collaborators etc. There are plenty examples like that. But certainly there are cases where learning is required.
Why do you think thats worth learning? Doing a few times manually sure… but to force yourself to solely rely on it is foolish. Just use the calculator.
I lost my desire to learn anything about web development twenty years ago and specialized my career in a completely different direction. I had not touched HTML at all since then until earlier this year, when I discovered that AI robots could do it for me. Now I am happy to whip up HTML/CSS/JavaScript visualizers, explorers, and even one-off interactive report presentations whenever they might be useful, precisely because I don't have to learn any of that crap to do it. My time & attention are far more productively spent focused on the work I am actually good at and interested in doing.
There was some learning, but also a lot of waste. As a self-learner, I've never been able to learn so fast as I can now with an LLM to instantly answer my specific questions, and incrementally correct and grow my mental model. And while with some subjects you need to worry about accuracy, LLMs are generally very good with programming questions, and (for most types of questions) you can verify their claims yourself.
The problem with CSS is it's a) all broad and no depth and b) constantly adding major new paradigms. If you don't get up to speed and stay there, you lose it.
Pre-AI I'd say I've gotten over the hump 3 times with CSS, only to lose it again by the next time I had to use it in volume.
This "instant" loop is so fast because it doesn't involve the part where you learn
You seem to be hung up on the part about "learning". Do you know how x86 registers work? Or atomic instructions available on ARM, SPARC, or POWER architectures?
No, of course not. Because all of that got abstracted to higher-level instructions decades ago.
Yes, I do. And it wouldn't take you that long to learn, either.
I understand your point, but also imagine applying that across the whole spectrum of computing.
It's very useful that there's an intermediary that knows how that stuff works so I can build things without thinking about it in excruciating detail the whole time, I can dip down lower and learn stuff when it's relevant (like cache access and nested arrays) but I can also not do that in many situations.
There's also the argument that you can do engineering without understanding the underlying science as seen in th pyramids, the beautiful old european churches, etc
Well that's great for you. That's some knowledge that you will possess for the rest of your life.
I want a green lawn and big bushy shrubs in front of my house. Do I need to know the intricate biology of my soil and habitat? No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
>No, of course not. I just do the surface-level things that make the lawn and shrubs thrive.
And others want the green lawn and big bushy shrubs, but don't want to learn all of the surface level things to make their lawn thrive so they hire a service to do it for them. And there's nothing wrong with that - not everyone enjoys the yard work, but they still want the thriving lawn and shrubs... and they are happy to write a check to OpenAI... err...Lawn Doctor every month to get that result since the time they save by not dealing with their lawn, they can pursue things they do want to do.
Not everyone wants to (or needs to) learn every detail along the way of getting the results they want.
Who is more productive? A python dev who learns about x86 registers in his free time or one that uses AI at work?
The one who uses AI at work will eventually be replaced by someone cheaper who uses AI at work.
The one who learns about x86 registers is much more likely to continue to evolve
But maybe there's a limit to the usefulness of this abstraction. Perhaps forgetting too many basic skills lower down isn't always as useful as going from Assembly to C. I'd argue that there's a law of diminishing returns there, and it's not an endless series of benefits.
In a world where there are endless things to do and only limited time to get them done, not every issue needs to be a learning matter.
If you are making something that you care about you should probably understand how it works. I didn't know this was such a hot take.
So, since you care so much and caring requires knowing everything about everything that even remotely intersects with your project, as (presumably) a software developer presumably you understand in detail how every bit of the CPUs you're targeting, plus the lithography, chemistry, and how solid state physics, and the underlying quantum mechanics work?
You seem to be massively exaggerating here for effect.
I think the main point is, if you're building frontend web apps, you should probably know how HTML, CSS, and JavaScript interact to some decently high level. Can you look stuff up on the margins via AI, or StackOverflow, or whatever? Sure.
If you're building <something else>, you should probably know the core tools and concepts at the appropriate level to build them.
No we do not need to know every aspect of CPU branch prediction and whatnot to make a webpage.
I think it's pretty obvious from the context of "if you had technical gaps (say, you couldn’t write CSS)" that this isn't about someone that regularly works on frontend web apps, it's about someone that is having to do something that is not normal for them. E.g. I mainly work on HPC code, maybe once in a year or two I need to throw together an animated poster to accompany a printed one. It's obviously ridiculous to claim that this means I have to learn frontend web dev.
Learn what? The thing the tool can do instantly? Take the win and spend your energy on bigger problems.
All the boring stuff you skipped to get right to the answer you want was just as important whether you want to admit it or not.
Having to read through a structured resource describing something to figure something out has intrinsic value that an LLM is not going to provide you with.
Well, no, it might not be important. Maybe you won't use that knowledge about CSS grids and flexbox for another year or two, or maybe ever.
> All the boring stuff you skipped to get right to the answer you want was just as important
This is obviously not the case. There are mountains and mountains of boring, unimportant stuff that LLMs can do for us now.
For example, at work I can now make a nestJS dashboard without having any idea what nestJS is. I can just tell the LLM what I want, and it makes it so. This allows me to make my tool useful to people without having to become an expert on something unimportant.
You’re not the only one who doesn’t know what nestJS is
As they said in the 80s or maybe earlier RTFM. I think if you got a good enough duster TFM was still readable in 2010.
I've seen junior engineers be productive on their first day in the industry because of AI, so I don't think the article is the whole truth.
The example math is boundary-pushing and definitely not a solved problem. But most of us work on CRUD backends with a React frontend. Those are more or less solved problems that have well-documented solutions. For those kinds of tasks, LLMs just reward usage.
I can count on one hand the number of times in my career I've needed to solve a problem that's not described on Stack Overflow.
the author forgot you could also do a secret third thing: learn!
dont really see the point when LLM compiles english
I think you're talking about a different type of expertise from TFA. Consider this: What if I never enjoyed frontend programming and so I never wanted to be an expert on that?
In fact, I never enjoyed frontend programming because it was such a pain to deal with matters I considered trivial yet so frustratingly hard to do right... like centering a div. And yet the slightest misalignment is visually jarring and forces me to get a bit OCD about fixing it, which made it even more frustrating.
I questioned the whole premise of the situation: is working around a bad developer experience something worth spending my time on? Unless I actively wanted to get in there and fix the situation, not really. So yes, in those cases I would outsource my problem to a colleague or StackOverflow and move on. And as a career choice, I preferred to do more backend dev.
I would posit that that was the type of expertise that did not matter. The type of expertise that really matters here is good UI design. That is entirely orthogonal to the drudgery that is implementing and debugging webpage rendering, and I am eternally grateful to LLMs for freeing us from it.
You can extend that line of thought to the entire article. What really matters (and what LLMs reward) is domain expertise rather than technical expertise.
> like centering a div
We'll of course you're gonna be frustrated if you start with the hardest problem in HTML :/
Just in HTML?? I think it's up there with naming and cache invalidation!
The short version I give to non-technical people who ask me about whether "AI will replace coding" is this: it accelerates you. You can get much further much more quickly.
If you don't know where you're going or how to get there, or even if you're just not paying enough attention, it will get you very far in the wrong direction before you've realised.
The LLM is like the death star. If you don't know exactly where to point it, you will likely miss your target and have no/negative effect. The further away the target, the more accurate your firing solution needs to be. If all you need to do is add something like a dark mode theme to an existing product, this is probably a point blank shot in this metaphor. Building an entire codebase from zero, or even refactoring a legacy codebase into a new codebase, are lightyears away by comparison. You can still land the shot, but you need to deeply understand the metrology and astrodynamics.
The information system required to encode the aesthetic preferences needed to make a technology experience not suck is likely in excess of what any near-term solution will offer. Knowing when to say "no" is perhaps the most important skill here. You can't just say it arbitrarily either. You really have to mean it and be willing to fight other humans for it.
I totally see this. I just did 3 hours of bot sitting to put together some thrash loops that thrash our provisioning working flow for a BLE gadget we make. It was pretty straightforward and productive. But then, I have a lot of experience with BLE, and a quite a bit of experience with python and shell scripting. So I was able to guide the process through stages, do some intermediate testing, make some adjustments, and proceed. Domain experience made this really easy and straightforward. Me two junior engineers who have only superficial/high level knowledge of BLE and some of the other pieces, couldn't have done this as effectively.
Where my angst comes, is worrying that no one will ever get that experience anymore. They might have had some eventual success, who knows what monstrosity a much less guided LLM would have done, but experential learning may be mostly a thing of the past. And it creates a real tension between the person with experience and the person without.
>Where my angst comes, is worrying that no one will ever get that experience anymore.
I am a fairly inexperienced python developer using LLMs to build software and find that I still learn a lot just from prompting and tinkering. Maybe that's less true once you reach a certain level of competence, but at my intermediate level I am still learning a lot even leaning heavily on LLMs.
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample" lol
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
I actually don't feel like Tao's recently published conversation is the best example of this idea. As intelligent as Dr. Tao is, and surely more so than me, I got the feeling that he wasn't running up against failure states of the model, which I'm not sure you could attribute entirely to his expertise. I honestly think it was more a matter of luck that the model apparently had so much training data on the topic or that it was architecturally so well suited for it. On the other hand, I've had really surprising moments where Claude was just failing terribly to execute simple dev ops tasks having to do with log processing. And I'd be so bold to say that I don't think it could have been explained by a lack of expertise on my part, or even a misuse of the model.
So yeah, sometimes LLMs reward expertise, sometimes they don't. I guess either way it helps to have it.
That's why when people like Pieter Levels tweet "I cancelled and then vibecoded 100% of my SaaS subscriptions", you need to take it with a huge grain of salt because you're not Pieter Levels, you cannot vibe code your SaaS subscriptions.
There's a growing and fascinating divide between people who see LLMs as more of a "bicycle for the mind" in the vein of Jobs vs those who see them as whollly supplanting the role of human intelligence. I can't help but wonder if these aren't primarily two human archetypes more than anything - the LLMs can be both and they erect a mirror of the human using them. Some humans really don't want deep individual expertise and intelligence to be the deciding factor because they don't identify with that. Others are completely the opposite. We really can't tell which will be more effective yet, because LLMs are very good in both modes. But most of the predictions currently are people executing on wishful thinking about what they hope will be the outcome.
"The most important skill in the AI era may not be prompting. It may be learning how to solve problems using the right kind of help." [1]
Context - I have over 25+ years in software, and I have this observation - being introduced to a new codebase as a human is difficult, especially depending on the scale/size and complexity of it.
Yes, you do start to learn it as you work through it, but if the scale is truly huge, it may just not be possible to fully read and understand all the code and paths etc.
I have found systems-thinkers (I believe I am one, sometimes they are architects) to be able to kind of "see the whole picture" while not knowing all the details, to the point of being able to guess how the system/software should be behaving, even if it is not actually yet. This is a hugely valuable skill and I think takes a certain kind of brain too.
That said, I think recently I may have realized something - we rely on statistics and confidence levels in order to make statements about larger populations. If we can represent a codebase as a, perhaps stratified population of code, interfaces, docs, etc. etc. etc. we may be able to take a valid random sample, review portions of the code, and make some kind of assertions about the state of the larger system - potentially, from that.
I am trying to implement this as a side project right now to see if there is anything to it, basically, a combination of AI/LLM + stats/sampling + facilitated expert human review.
I'd be interested to know if anybody is doing anything similar.
[1] https://www.actinginbalance.com/p/the-right-tool-rule
I am feeling this a lot lately. Getting the most out of agents seems to require being able to ask the right question. And how can you ask the right questions without deep domain expertise?
Yes sometimes it’s a matter of just using the right word. You can talk to an agent about a general concept for hours and hours and it may never mention $Concept_X, but you mention $Keyword_Y and all of a sudden the AI is going on about how $Concept_X is foundational to understanding the whole thing.
I started developing webapps back in the late 90s when I was in high school using Perl, and I've worked with tons of technologies up till around 2014 or so when I shifted into almost pure backend work and lost touch with modern frontend development.
I'm now learning how modern frontend development is done (for both personal and professional projects), so I may not know the specific tools, technologies, or terms but I can say "whatever the equivalent of XYZ is" and the models will translate for me. If I say "run pytype" it will tell me "we're using mypy - i'll run that checker for you".
If you can express what problem you're trying to solve, that will get you most of the way - and then you can refine by asking questions. "I think I need something like Redis for caching things - do people still use that? Is there a simpler more modern version that is the new standard? Do we already have company docs suggesting what to use?"
This matches my experience. Just Talk To It is the best method for working with LLMs if you're an expert.
I've seen this at work (as eng manager/lead/principal/whoevenknowsanymore) – all the big APIs give you stats. We see how much people burn in tokens and we know how much output they produce. There is a pretty strong inverse correlation between token burn and output.
The more tokens people burn, the less likely they are to produce a good outcome.
A good moment to remind everyone that if we took the promise for granted, that AI will in fact prevail and prompting is the one skill that will rule them all... we'll lose all domain experts in one generation.
It's less of "signaling expertise" and more about actually having said "expertise".
In my experience with LLMs it's not uncommon to be having a deep conversation about making pasta, only to be told, after asking for a sample recipe, to get a bucket of paint and a bag of concrete. Of course these hallucinations are way more subtle and easy to miss for someone who doesn't have deep domain knowledge.
Domain knowledge will stand alone as the sole differentiator. Because LLM benefits can be reaped by almost anyone and it’s a force multiplier. Now those who have the strongest initial force will have a far bigger edge than before.
Real world domain knowledge and experience cuts through the chaff too. LLMs are going to have people reinventing the wheel and wasting tonnes of time on stuff that won't work out. If you're a domain expert you are going to be much more aware of how to focus effort in the right places, and what's actually needed or been tried before in your niche.
A lot of this domain knowledge is not in any training data, it's locked up in companies in the industry. I suspect it will get even more important to guard it.
Agree with this. LLMs multiply the human user's ability. More ability, more impact!
And unfortunately, more ineptitude, more chaos.
Love this idea of reading prompts that lead to new discoveries and figuring out how the person got the LLM there. It truly is an art and I’m always reminded of “I, Robot” and the scene about “you must ask the right questions.”
You're basically playing the role of team lead to the LLM's junior dev.
I love larping as a vacant scrum master
Prompt an image or video generator without knowledge in photography or art skills and your results will look sloppy.
Well, nice post. Actually, there may be some truth behind it, but basically, it captures what I—as a programmer—want to read: expertise will remain valuable.
But how I am observing is different, though. Since LLMs the gap between experts and non-experts has been shrinking. And yes, there is still a gap, but vanishing.
Agree so much with this! In domains I know well I get much better results then someone who doesn't know the domain because I know where to challenge the LLM. LLMs need to be pushed because otherwise their answers are typically average.
This feels like a moment in time, not the end state of AI.
Like I read there was a time when teams of people + AI could beat pure AI at chess. But that these days, pure AI wins.
For all the things people say about "how AI works" you have to add the missing piece "how current AI works".
This is true but also false.
In my experience (scientific programming) AI is a giant multiplier for people with specialized knowledge.
But it’s also a giant devaluer for that same knowledge as people with no idea what they’re doing can clog the field with plausible bullshit.
It’s now the case that if someone tells me they’ve done something, and I look into it and find out it’s completely AI slop, then I will have spent more time on the project than the person who “made” it. The situation is completely untenable and only serves to drain time and resources from people with better things to do.
we are slowly punishing reading comprehension
this will have educational consequences (that I'm trying to solve). I don't think that we can adjust without rapid education and making extreme specialists of us all.
This requires coordination, certification, licensing, and other tiers of authenticity. False experts can ruin sample gathering, can ruin training. False expertise is exemplified by the current American Administration. Look at Robert F. Kennedy Jr.; he's a false expert. He is responsible for the measles outbreak. He is responsible for ivermectin abuse by humans. False expertise is overtaking real expertise. And the results are continuously disastrous and large-scale.
So we could run a lighter LLM in front of humans, which translates from 'no domain knowledge' to 'domain expert' and in turn prompts over to the larger LLM.
Then the larger LLM gets all the right lights on, yields better outputs and we translate back into user domain.
I kinda thought the chain-of-thought reasoning already did this, no?
Yes. I agree with most, if not all of this. For instance, I am seeing folks either relying in the LLM as an _assumed_ expert or, assuming someone - who knows the structure of skill definitions - also has some expertise (in the area of the skill). It's a difficult situation; there is not much point in explaining _why_ the LLM output or skill in use (on a domain problem) isn't what the person actually _needs_ to address the domain problem, because the person isn't a domain expert or indeed, adjacent to domain expertise. But, it is an interesting experiemnt to arm folk with little domain expertise with the _skill_ necessary to be able to extract the right solution from the model.
This has been my experience as well. I’ve also been thinking a lot about Terrence Tao and his chats and presentation.
I agree. When I talk to LLMs about fields I am familiar with, I can push back on bad suggestions and ignore faulty/incorrect advice and assumptions, which is much harder for unfamiliar subjects.
Of course, simple common sense and extremely basic Googling on unfamiliar subjects can produce similar results, but it's much faster if you are truly understanding what the AI is suggesting.
This is also why people's experience with LLMs/AI varies so much, because some people can see a use for AI for their needs and go about using the tool, while others do not as it relates to whatever they're working on and so they may say "LLMs/AI are useless" (it doesn't mean they're not experts though, although some people who have totally no expertise might also see no use for AI for themselves).
I often do my best to represent a genuine interest in the subject at hand and learning in general to models. Imagine the model's response prose and mannerisms being on the other polar end of answering questions simply to get the correct answers as they're often scoped for on quantitative benchmarks. Not sure I explained this well, sorry. An LLM could help
I have lengthy conversations with my LLM, almost like an interview. I agree on the expertise part, because I wouldn't be able to go in depth on a subject with it if I lacked the expertise. Some work is a result of design and negotiations in those designs.
I don't think Tao's style works with everyone/thing, especially if we don't know what style he's tuned his LLM on.
Might be a very noob question but in this era of LLMs, let me ask the reverse, how do you gain expertise? It seems this rewards people who had expertise pre LLMs, but what about people who don’t have that in a specific domain? What approaches are viable now in this current system?
I'm not sure I understand the question. What would prevent you from doing what these people did now that LLMs are here?
Those people had no choice. In my opinion, it’s harder to grind through problems knowing very well an answer is a prompt away.
But getting an answer is not the same thing as understanding why that is the correct answer, or going deeper and learning more about the subject.
IMHO, the people who genuinely desire to learn will trudge through whatever they need to in order to grow their understanding - be it through reading books, original research papers or what have you.
If, OTOH, all you seek is the answers and that alone is satisfying to you, then of course you simply will not be motivated to do it the old school way anyway. But that's hardly different now in the age of AI.
The same as it's been! Make things without using LLMs. Don't debug with them, don't use them to research things, just do it yourself. It'll be painful and that pain is learning.
Same way you get strong in an age of heavy machinery: Lift heavy weights yourself.
Skills will have to be built through artificial constraints. Pen & paper, reading books, not using AI, etc.
this 100%. Skills are gained via effort. Not too much effort that it discourages you, but enough so it's a challenge and engages you. Ofc it helps to have wonderful teachers, coaches, mentors, books, even ai to help guide you, but YOU must put in the effort. You don't get something for nothing.
I cant have an overall opinion but in my personal experience i have analysed that LLMs do reward concreteness
i find this post re-assuring (as who doesn't like to feel like they are an expert at something and llm definitely strips that away)... but it still feels like you are rewarded just as much for being a 6/10 expert as you are for being a 9/10 expert. It definitely is an equalizer it is just a question of to what degree.
Is someone keeping a list of the excuses and varying instructions on how to hold it right? It would be fascinating historic documentation
The fact that Claude knows I wrote the Virgil compiler makes it be on its best behavior when working on it. I force it to not write too much code, and to write more tests. I push back on slop and just adding another special case. It has a surprisingly deep understanding of floating point.
Overall, I agree, when I ask things I'm an expert in and do professionally every day. I get very good useful answers. When, for example, our marketing people, ask about the science, they often get confusing and wrong answers.
I think if we have a large population of generalists, then none of them are generalists after all
I believe they would still be called generalists.
You got to know how to use the model+harness+prompt to achieve the results you want, but honestly for many projects and questions all the models already pump out their same best version of an answer.
Sometimes it is really akin to a git clone, although it was a LLM request.
This rewarding expertise is somewhat wishful thinking.
At the end of a day, it feels and is more like gambling, even with the recommended expertise and a good approach, don't delude yourself you're simply pulling the lever too, as any novice.
Skilled use may or may not matter, depending on the task. Do you need to do what Terence Tao is doing?
I'm surprised nobody mentioned (including the author) the Gell-Mann Amnesia Effect [1]... Just substitute "LLM" for "journalist" and there you have it.
And to be honest, I have seen it, as I'm sure (almost) everyone has, who has demonstrated experience/expertise in their own fields, and correct the LLM's responses one time or another...
[1] https://en.wikipedia.org/wiki/Michael_Crichton#%22Gell-Mann_...
This kind of sentiment really pisses me off. It’s just fucking pathetic. A computer is doing the interesting and amazing parts and we’re coping hard by suggesting we can master the trivial detail of making our prompts concise (no wait next week make them detailed again because Terence Tao wrote in long paragraphs this time!)
Terence Tao’s conversation with the chat model was wholly unremarkable compared to the solution to the Jacobian Conjecture. Tao gets to write a well-read vibe-researched blog post not because he’s mastered some genius prompting skill. I’m sure thousands of other people could do the same. It is because he is a celebrity. That’s it.
By the way, it was the fashionable prompt-jitsu to include more in your prompt like two months ago. The prompt magic changes all the time to suit whatever makes some blog tsar feel like they’re really adding to the discourse.
If there’s some optimal prompting method then the AI system will figure it out and modify your prompt to match it. Maybe for one or two years having domain knowledge will help you, but everyone and their grandma has been saying forever that this will only last a short while.
Yes. This is called the Matthew Principle:
> For to every one who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away.
LLMs reward architecture knowledge of how to structure things and how to not just say "Claude, make me Microsoft Flight Simulator, make no mistakes".
Unfortunately the software industry is saying things like "don't look at the code", "LLMs have made developers 10-100x faster", etc. The only way they can make such claims is by saying what you said above: "Claude, make me Microsoft Flight Simulator, make no mistakes". Additionally engineers are facing pressures via deadlines to work in the paradigm of "Claude, make me Microsoft Flight Simulator, make no mistakes"...
The question i wonder about is, when will an event come along that persuades everyone that human understanding is still required? Or will it never come?
Such an event would have to be pretty catastrophic at this point to slow down the inertia. Perhaps the tech debt will just pile up until someone's product implodes, or there's a massive safety issue that causes loss of life, or some big hedge fund goes bust.
I wonder the same thing. I think we've already seen some of this happening, however the consequences haven't been large enough to the organization, for example:
- https://www.theguardian.com/technology/2026/mar/20/meta-ai-a...
- https://tech.yahoo.com/articles/ai-code-wreaked-havoc-amazon...
- https://alexeyondata.substack.com/p/how-i-dropped-our-produc...
We can only hope that engineers working in safety critical systems haven't fallen to these working conditions.
That question makes me think about Boeing. Or NASA. Or Enron. Reality always wins, no matter what management and Investor Relations says.
It makes me think about The Terminator.
The event could be when fair pricing comes from the model providers. We're still at the cash burning stage. When the economy crashes a little and departments start monitoring their spending, and the prices for inference are 10x what they are, there will be less tolerance for employees to substitute constant AI usage for understanding.
You're assuming that LLMs entered a world of people who understood how the systems they're inside of work, why they're setup that way, and that LLMs are displacing them. I sadly don't think that's the case in... well... a lot of the cases.
Many, including myself, report having a lot of success with braindumping and not structuring anything. Just talking into speech recognition for 2-10 minutes as a stream of consciousness about what my context is, what I want, what I know already, what I have a vague hunch about, how it fits into a bigger picture, what aspects are most important to me, any footguns I already know about, really like having a chat with a person on the phone, with someone you have to guide remotely because they have to implement the thing right now but you have to be out of office and so your only interface is speech. Except you can be more structureless because the AI won't be offended. Just keep on rambling, and press enter, don't even correct mistranscriptions. It will understand it anyway.
Now, the key is, that while rambling without structure, you do have to drop the key facts into your speech, and you have to know what you're talking about in at least a good portion of it.
I think people are afraid of doing it, because it seems "not the right way" or "not scientific" or whatnot. They want to believe there is some magic to writing the right prompt. So let me tell you, it works.
I don't completely disagree with the concept of giving a free association thought process ramble into context. But I also bet that when you start getting it to actually generate code and link modules of things together, subroutines, functions, code structure and filenames, you still pay attention to what it does and you guide it into the architecture that makes logical sense to you.
For real work yes. For personal projects, less and less since Fable came out (probably the same if true of the other frontier models). You can get a lot done if it's just some one off, or a personal tool, even without looking at the code, just trying the application. Frontier models now automatically test it before handing the thing to you, they take screenshots, they fix the superficial issues themselves. To get something up and running, it's enough to send chat messages.
Now everyone who feels rewarded by LLMs will conclude that it demonstrates their expertise. ;)
Yes. If you use the right technical terms together it’s lights up more specific feature spaces to your task.
Specificity matters to LLMs a lot.