We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong."
Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
This is all complicated by the fact that we're also losing our grasp on reality from the other direction because we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
My job improved significantly since I managed to get into their (managers) thick skulls to tell Claude to not be verbose. I'm tired of 1200-word tickets for "put the totals in the automated emails".
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
Works for a demo, but quickly stagnates. Future models may of course push the limits further, but right now it quickly goes sideways without someone knowledgeable at the helm.
You don't design things with never ending details though. If your developing a feature there's a limited scope to it. If your designing the direction of a large project you're working with higher level concepts.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
> The point is that AI is aware of constraints and can manage a round it.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
I have also tried developing games with them. I find them very weak at these things.
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
Before, if I would start working / thinking on a problem and discover it was harder than anticipated, that often was a signal that the feature may not be worth it. Because implementing it doesn't fit the current model of how things work or similar. Then it was back to the drawing table and find a better way of solving the underlying issue.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
I first brought this up late last year – that there used to exist a kind of selection pressure against both developers and against features which would prevent certain functionality from ever being shipped, specifically:
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
In my personal project I've found having a hard lines of code limit has been a good limit on complexity. Makes me really ask if I need things.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
So exactly true! With ideas people unbridled, the painting-yourself-in-a-corner tendency that was always there has gone exponential, add to it the not-completely-aligned incentives, and the oh-so-clever impressive writer of long prose LLMs just end up churning. The real value add may end up being the lesson in honesty and humility to us all...
I feel like there is also the "Nerd Factor" to consider which is this unrelenting passion to type away at a screen all day, whether that is yesterday's code or tomorrow's prompt. Few non-technical people have the attention span to suffer such monotony. (Even most devs don't have that to varying extents)
It's only monotone from someone looking, not from someone doing. If you look at digital painting tutorial, you'll see a lot of timelapse, because it's a very slow process where you spend the majority of time correcting stuff. But if you're the one doing it, each individual brush stroke is the result of a conscious decision.
Working on a passion project or working through the hairy details of a complex algorithm or whatever can be fun. But that's not really what the job is like most of the time.
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
>My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
The fact that they sometimes can't means you can't have someone who is not a software engineer. Otherwise there is no way to verify whether the LLM wrote correct code or not.
Exactly. Expertise is frequently required to steer, correct, rework, know what is good and what isn't. This happens 500 times a day on both the micro level and the architecture level.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper - if all that's left for most of us is slicing and dicing of what has been already been implemented, then LLM can reliably take over.
> Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Because great insight and expertise stems from foundational knowledge.
I am a huge hockey fan. In the NHL, the players do not stop practicing the fundamentals once they make it to the NHL. They practice the fundamentals even more. Many practice the same drills as youth leagues -- stick handling, passing back and forth, shooting, edgework, shooting, rebound control, etc..
The best of the NHL might not hit the hardest, have the most accurate shot, or skate the fastest. What separates them from the rest of the lot is that they are fundamentally better than everyone else.
> there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms
Transit routing is still moving fast in this area. We're still figuring out the best ways to return thousands-by-thousands transit time matrices with query-time truck dimensions and traffic updates. It can't be the only field!
I think it follows the path of the spreadsheet. For a long while, only geeks and finance or accountants or data monkeys used spreadsheets. Then, it was such that anyone could create a spreadsheet. Vlookup was something you learned early in school or on your home computer tinkering around. There’s still some modeling gurus out there but largely everyone is developing solutions using spreadsheets everyday. And they’re doing it rather autonomously even prior to AI helping. We can talk about how they’re ugly and crappy spreadsheets but they generally solve the problem the user had.
With AI, people can build and collaborate on applications much more complex with much less technical knowledge. It might be ugly and crappy but I bet they’ll be mostly autonomous and not need to work through their IT team, or go through the hell of PM and requirements. If I know my requirements, I don’t need you. Hell, I can just start building and add requirements as I come across them. It’s not a major risk to the project like it used to be.
It will, and is, going much faster than the spreadsheet did.
staying within free tiers has led to some really great optimizations for me
took complex pub/sub event architectures down to the events postgres can do on its own, and a system architecture that works with that payload. am aware of the limits with this approach and stress potential on db
This I solemnly swear. To take my paycheck with gratitude, even if my employer makes three times as much based on my output. To take whatever technological disruption that graces my path as an immaterial and impersonal force. To politely and discreetely bow out when the statistics soup trained on my fellow colleagues makes me unemployable and worthless. Amen.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature".
We are currently in the "centaur" phase where a human-AI combination produces the best output, but I think some people are betting on the fact that the AI only product will eventually outperform the centaur. And with the cost of tokens falling thanks to fierce competition from the Chinese open weight models, it's definitely possible that those that bet on AI early will reap payoffs in lower expenditures for more output.
Honestly not sure which side I land on that bet but I definitely can't rule it out.
We've gone through this with chess, and the days when a human can do anything to help AI there are long past; the human can only drag it down. It seems to me to be a form of the bitter lesson. I expect this will happen in every field where we don't add guardrails to require humans in the network. I have a hard time seeing regulation happening around this in the US but vanity may also serve, will CEOs be satisfied ruling armies of virtual assistants, or do they need big buildings full of human thralls?
This is not a silly question, because it's kind of silly:
Has an AI ever actually gone through all of the steps necessary to become a chess master? Go to competitions, raise in the ranks, take a plane to the masters, organize support and all the real life details necessary? Playing chess is just the small formal part here, and all of the obvious things humans obviously do, well...
Another thing is, I have been playing Chinese chess a bit lately and, yes, Pikafish is stronger than any human player, but if you actually share a board state or list of moves with an LLM they will fail to accurately interpret the state of the board, let alone give good advice on what moves to make. Doesn't really prove or disprove the claim that LLMs will replace human programmers but it does show that the word "AI" is papering over a lot of specifics of particular technologies.
It's like a tool that has caused a fan out of both good and bad outcomes.
Good: you can refactor your codebase at will, throw out legacy cruft by the megabyte, improve build/CI time, and simplify ruthlessly. Not to mention kick out new features in simple and consistent ways that align with what a user actually wants.
Or you could add megabytes of vibe coded crap, solutions that add a ton of mass but don't actually solve the problem at hand (seriously!), implement abstractions that are logically inconsistent with the rest of the system, etc.
Both of these are happening right now, and I think the latter is happening at a rate far higher than the former. But our fundamental dynamics are still at play - the ball of mud is still a ball of mud, even if AI lets you make it 100x bigger. The problem just gets more entrenched.
Eventually AI will learn how to simplify code, understand coupling, etc - and hopefully it will just iron out problems as it goes. I think we're a long way away from that. But this is uncharted territory, and I don't think anyone really knows. I certainly don't hear anyone focusing on that as a target of optimization however.
My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue, and perhaps we can re-learn these lessons yet again. It kind of feels like GitHub might be the first example of this.
> My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue
If you worked in SV like I did from 2010-2020 you know that the exact opposite of that will happen. I saw hundreds of companies successfully scale out of their garbage stack such as Facebook and PHP.
The only thing that matters is the problem you’re solving and the quality of the code is irrelevant.
> A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
I had a recent experience with this. Although it was months and not weeks and there were other factors into play such as the market etc but the last company i was in had a poor engineering culture with inexperienced engineers equipped with AI output some of the lowest quality features for both customer-facing products and internal dev tools. A lot of customers churned, and im sure quality us part of the reason.
We have a product guy on the team who was in a deeply not technical role before AI who is trying to do the “hey Claude, read this Jira ticket, implement” thing.
It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
The spacer between the keyboard and the chair still matters in my experience.
Not everyone has this, but I always felt that a significant percentage of the value I bring is in immediately recognizing what you shouldn’t do.
I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand.
Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the consequences. It’s high fives and pats on the back, until I’m cleaning up the mess.
This lands. Llms are bad precisely at following what not to do. They work best off of positive constraints.
I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps. I call it a bias field, pushes them toward hopefully the happy and harmonious (with the rest of the system) paths. Obviously this is only partial and imperfect enforcement, but if it's applied to everything consistently it naturally encodes some self-consistency and harmony.
> I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps.
I’ve done the same but it’s a moving target as models advance and I find half of my points are ignored until I’m prompting “No wtf why are you still trying to symlink the global Python executable just use the virtual environment that’s already activated”.
Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time. There’s no way my hobby fiddling will outpace whatever gets released in the next couple months.
In the meantime, real linting does work pretty well, if you can write a detector for whatever antipattern you find LLMs fall into (like multiline comments).
> Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time.
I get patches into the biggest AI projects all the time to make inference faster on my 3090 and all you have to do is read the contribution docs and open a small PR.
It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI.
You can trivially outpace what big companies are doing it’s insane, why do you think so many innovations are coming out of scrappy Chinese labs? They are willing to engage instead of being defeatist about it.
> It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI. You can trivially outpace what big companies are doing it’s insane
Not my core competence and I’m not interested in making it a core competence while the field is still rapidly changing. I find harness engineering interesting, but not lucrative enough to push one of the many very important other things on my plate to the backburner.
I don’t think Python language developers are super geniuses either, and I’ve never submitted code for consideration.
Oh I'm certain I'm not doing it optimally or close to best practices. It's just an example I had in my head related to my comment.
Although, given opus 4.8->5.0 experience within claude code, I don't know if I'd agree they necessarily improve things with time. But that's just my experience. I should probably read their release notes, maybe just a tweak to my claude.md would help
fucking is it? AI labs certainly want you to think so but they're running out of time and money. they've been telling me that the advancements of the last three months have made this a reality for the last five years. it's starting to feel a lot like Elon promising fully automated taxi services by next year... 11 years ago
For the completion to work, the source text needs to be ‘good’. That’s a basic kernel of how the thing works. Even with a perfect oracle autocomplete if the source text is ‘bullshit’, the output is too.
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
I've had some success with: here is this android bug ticket in a product that I don't know much about, fix it! And it works sometimes. I'm still technical, so it isn't the same, but it was crazy that AI could just solve some problems automagically without me know much about the system being modified.
Probably worth not getting too comfortable. It's only a matter of time before a new generation of product people can do this. Also, I have said this before, instead of 1 product person and 10 engineers on a team, you may have 2-3 product+eng people and 3 engineers, etc.
You have it mixed up. This guy is “the new generation of product people”. The old generation had to be technical enough to be able to grok the systems that they ‘producted’ over. It’s exactly the ‘new generation’ that LLM themselves right into Dunning-Kruger.
> It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
Remove the dev from the equation. Suppose you need to produce circumstances for the slop machine to succeed on its own (imagine a loop).
What is missing? A good enough description of the current behavior, desired changes and how it should work? Understanding of the business domain or the system in question, its architectural choices? Enough guardrails and linters and tests to ensure that the solution will indeed be testable and work end to end? Cause a lot of the things that will help human devs will also help LLMs make their output more workable - for example ADRs in the repo and custom in-house tools to ensure they follow architectural conventions and practices (or just something like ArchUnit). Realistically, how close to a finished implementation must you get and how far backwards from that can you go before its output degrades too much?
I tried figuring that out myself on a personal project where I mostly want what I want in GitHub issues and then feed them to Kepler, which spawns OpenCode in worktrees and also gives me a UI to follow the execution. Most models, even SOTA, occasionally implement things wrong, but it's suprising how far you can take things when you've spent a dozen hours planning out what you want the end result to be vs not to be (talking about the goals for an entire system, maybe an hour of planning for a more bounded type of task).
In general, when I’ve worked on projects where I’ve spent that much time planning, having a magic AI box that wrote perfect code wouldn’t have helped that much. When you spec every everything out like that, writing the code is the most trivial part of this whole process.
That led to a reduction in knowledge of assembly in the average programmer but the people who specialize in it haven't gotten any worse at it.
The result was a generation of programmers who make useful software while very few of them understand the machine they program. You could easily make either a positive or negative value judgement about that result.
The people who swizzle registers on GPU drivers need that skill, and there's no evidence that they lost it. There's a reason why we use the stack metaphor for a programming ecosystem. You trust the lower layers below the one you work in.
People are fundamentally short-term thinkers. Just imagine spraying neurotoxins on your lawn. You're "saved" from the "terror" of noticing more than one kind of grass, but of course you're incrementally increased the cancer risk for yourself, your children, and who knows who else. This is just one example; people do this constantly. Plastic fleece puts an untold amount of poison into the environment, but "I need to be warm RIGHT NOW." etc. Or lead paint. "It's only toxic when it breaks down, and that won't be for decades!"
All the issues you mention here have a death horizon: the people engaged in the behaviour will be dead when its effects are most acute. That they engage in it then isnt very mysterious.
Boris: "I don't prompt Claude anymore. I have loops prompting Claude and figuring what to do".
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
They work on research problems you can define a clear solution criteria for.
The kinds of business software I work on don't have those characteristics. If I needed something like a utils library, I think I could easily have Claude write the whole thing and not read the code.
What puzzles me is this: research means that we are exploring something that has not done before, yet using Claude to generate code means slicing and dicing what has been done many times before. So, I'm not sure how to make sense of both at the same time: Anthropic is pushing the boundary of AI, yet all the knowledge and engineering in form of code can be generated from the previous code?
> No, they haven't figured out anything. Just vibe coding it with their bleeding dege models.
I'm sure you were saying "bleeding edge" but my first impression was to translate dege to mean degen/degenerate which made this so much more fun to read.
Engineers, free from writing thousands of lines of boilerplate, now have more time to review code and be more efficient in that armed with LLMs. E.g. instead of asking a question about the code and waiting for the author to respond, I can ask an LLM and get the answer almost immediately. Or, if I see some strange code being changed, I am always curious why it was there in the first place and, instead of tracing it through git blame over formatting, renaming, and moving files around dirs I can delegate this to an LLM. If I have doubts about it working in some environment I can, again, make an LLM test it out instead of asking the author etc. etc.
And no, telling Claude to implement a Jira ticket is not worth $200K/year. Checking if it has not done something stupid and correcting it when it's trying to - is.
> We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
Our jobs have never been to write code. We’ve been saying for decades that LoC isn’t a rational way to measure engineering output, and now we have our chance to structurally change that system before processes re-solidify. In fact, I think the flexibility to adopt new systems and the experience and foresight to choose a path that is better than the status quo without throwing out everything we’ve learned is going to be what sets companies apart and makes individual careers over the next few years.
> we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
There’s an important criticism here IMO - the relationship between “business” and “engineering” is changing drastically. It’s going to be a challenge to set the expectation that just because Marketing was able to vibe-code a prototype in a day, actual implementation may well take weeks or months. Engineering should be considering things like security, scalability, and systems integration that aren’t a concern for Marketing - that’s why we’re being paid!
> So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
I think it’s an extremely heterogenous landscape right now. Where I work I’m struggling mostly with organization - keeping the (literal) dozens of inbound features that come in every day straight long enough to hook up an agent harness, review, validate, and deploy. At other companies the issues seem to revolve around their Agile-based processes. Or maybe it’s non-technical vibe-coders and their expectations. Or maybe it’s executive leadership flirting with AI psychosis.
Everything is in flux. It’s stressful and exciting, and I’m thankful to be around for it, even if I am in my 40s at this point and looking at the core skills I’ve built rapidly become worth exponentially less. It’s a huge opportunity for growth.
> My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
The problems:
1. The market is/has been saturated with engineers/developers earning well into the six-figures who are basically decent at writing code but not so good at anything else. So sorting out who is really worth these salaries is not going to be easy and the people who decide often lack the knowledge and skills to determine it.
2. You need to successfully make this argument to people who control budgets and more and more of them just aren't going to buy it.
> the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it
The word "engineering" means doing things using a repeatable process to get predictable results. If you can get predictable results (e.g. guaranteeing the absence of data integrity issues, security issues, anything that will cause downtime, etc.) without looking at the code, you're still doing the work of engineering.
Your job as an engineer is to choose high-value problems to work on, solve them in the correct way, and guarantee that the functional and non-functional requirements are met. If you no longer need to read the code to guarantee that it has the right set of ilities, then I'm not sure that's obviously a bad thing, as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
> as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
Perfect example of begging the question.
Vibe coding works great as long as vibe coding works great.
Personally, I'd love to see how you enforce security and long-term maintainability without looking at the code cuz we need to add that to our build pipelines ASAP.
Mythical man month should still apply for AI coding. If you look at the code you're shipping, and you can't quickly edit it by hand, or fix a bug with it, you will be in trouble if the models are down, and your website is down at the same unfortunate time.
1. I think LLMs will end up pretty dramatically shrinking MTTR, a lot of that will be tooling to proactively resolve problems as soon as they start, but a lot of it is that agents are very good at finding and fixing problems by correlating data, something humans can do but I think we're going to be slower at
2. "The models are down" doesn't seem like a very realistic problem unless you have a single provider/set of endpoints. I do not recommend this setup but I guess if you have a single point of failure that is a risk.
// The need for ongoing friction in long-term skill formation.
The subtitle of the story tells it all.
There are some people who seek out friction. Think about an athlete or a hardcore nerd.
The best engineers are ones who were fascinated with computers and learning as kids and persued it at every opportunity. Found their own friction in other words.
For those kinds of people, friction-seeking is the constant and what LLMs did is moved the point of where the friction occurs.
For example - the best engineers I've worked with didn't necessarily have lots of experience coding in assembly because that kind of friction was no longer necessary. But they could solve hard problems (and if a problem really required assembly they could go learn it)
What I think will be hit much harder by AI is the low tier engineer. Someone who was never truly curious and committed to it, for whom it was just a job. For example a typical offshore ticket pusher kind of person. That kind of person never went out to find friction and that's the kind of thing that's never going to fly again - if I want mediocre or average, the LLMs are sufficient
I mean if you think about it, that sounds like a miserable existence to begin with.
I would not rule out that some people will use this.. uh.. disruption, to start pursuing something that actually brings them joy.
Of course, bills need to be paid and it's not all that simple. But at least there might be such a silver lining.
I think this expresses what I've been trying to form in my mind.
I've been trying to map the LLM advancements and the current state of software development onto prior technological improvements. History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
For the people that fall in love with a single abstraction layer or don't have an interest in learning new paradigms, when their known pattern is abstracted away, they're condemned to being left behind, either unwilling or unable to adapt.
I don't think any industry is free from this, any person in any industry/profession over a period of 20 years or more has likely had to undergo massive adjustments as technology changed their field.
We're not unique, but that doesn't stop it from feeling so jarring when it happens to us
> History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
But there's no abstraction layer that ends up getting lifted. When I use a library like SDL or a standard like POSIX, I don't tend to look at the underlying implementation. Instead I work with the high level concepts that they come up with. There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
I've learned something at every abstraction layer in computing from electronics (hardware), theory of computation (software) to high level programming languages with their paradigms. Same with several domains embodied by libraries. LLM tooling is more like shamanic ritual than engineering.
Yeah, a bespoke program that does exactly what I need it to do, at a speed that I had forgotten was possible on computers, with customization that is an exact fit to me, at a cost that is smaller than a rounding error.
I get that LLMs struggle with the old paradigm of a single piece of software meant to serve every conceivable use case of every conceivable user, but I kinda hope that paradigm dies.
> There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
This is the same “but the ai can’t even do fingers” argument all over again, and it will age just a well.
The whole vibe coding thing only really started taking hold with the release of opus 4.5. That wasn’t even a year ago. you absolutely can be quality software without looking at the code. The only thing up for debate in my mind is how big that software can get before you need to look under the hood.
But that’s dictated by current model capabilities. And I don’t see any reason to believe that they will suddenly stop improving on this front.
Exactly this. You still have to understand the abstraction layer the LLM is working at in order to be able to form an opinion about the quality of the code. (If you DON’T care about the quality of the code, there’s always been ways to get an extra 1000% short-term speed improvement by just not giving a shit.)
AI isn’t “a code abstraction layer” any more than Jira is, or the engineer themselves is.
I think coding experts who are only experts at writing code will also be hit pretty hard. Every problem given to them is solved by writing code, and only code, their way and only their way. I think people like that are having a very hard time relinquishing control regardless of the quality and correctness of the LLM output. If it didn't come from them, or conform to their conventions and style, then it's wrong. These are the people that seem to be terrific developers but then when asked to be a team lead and bring a group of developers to their level completely fail.
Curiosity is really key, and I find that LLMs diminish curiosity in people that might otherwise have it because the allure of the answer is often a lot stronger than the allure of the friction. I guess that's just the distilling of the industry over time, but that could certainly lead to a shortage of expertise.
If that is representative of your personal experience, then I'm saddened to hear it. I believe your article is an argument from incredulity [1]. You may not personally know anyone who thrives in learning from and applying knowledge with AI. But being unaware that it is even possible is not evidence that expertise will inevitably collapse.
The snake eating it's own tail for llm software development has really been met with a shoulder shrug whenever it gets brought up. At best you might have a small cohort of developers that don't cook their brains with AI and their reward for that appears to be having to review terrible AI code written by people who have cooked their brains.
Maybe I’m naive, or misguided, or something else, but I decided almost 2 years ago that I’m going to bank on not using LLMs to do my job. I still write code by hand everyday. I don’t use AI to write code or lookup docs, or anything else. I just figure that if we really, truly, indisputably hit the point of AI assisted coding being mandatory, I can learn it and get up to speed in a few weeks (or at worst a couple months), whereas uncooking my brain is way way more expensive and time consuming and difficult. We’ll see how it plays out, but so far I don’t feel left behind at all really.
As a tech educator I 100% agree. LLMs are not going to become a "new compiler" where we don't have to worry about the code any more. There's a reason we trust deterministic systems.
I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've found it helps a lot: https://github.com/AnthonyPAlicea/skills/blob/main/skills/do...
One way or another, there will be a skill reckoning.
they probably mean deterministic in the sense of traditional computer programs consisting of if-else decision points and ordinary cpu logic, as opposed to numerical models (which can of course be deterministic too, as you pointed out)
Then I think I'd rephrase you: There's a reason we trust trusted systems.
:)
Even their fans agree, LLMs are inherently unreliable. The fact some people trust them regardless is due to a deep flaw in human psychology that I believe has not been significantly exploited by any previous tech. There will be tears.
There are 2 levels of determinism that matter here IMO. Level 1, given the same input you get the same output. Level 2, you know what to change on the input to get a specific change on the output.
LLMs are currently capable of level 1, but not capable of level 2. Trust follows what is actually deterministic.
Trust should, but instead is being gulled by those incapable LLMs. And I don't see that changing, given the typical human's seemingly unlimited vulnerability to this con.
It absolutely might make mistakes. But the skill requires all questions to be based on lines of code with specific line numbers which the human needs to review. That's intentional to serve as a safeguard against hallucinations.
It doesn't matter if the LLM is not deterministic.. the OUTPUT of typed code is deterministic in the sense that it can be seen to be type safe, compiled, and run appropriate unit tests on it.
If I use two LLM's to create some chunk of code and they both do it slightly differently but they both compile and pass appropriate tests.. it honestly doesn't matter if the LLM itself is not deterministic in exactly what it's going to output.
I would also argue- doesn't that make sense? You give two human coders the same task and they are also going to come up with slightly different results.
We don't run our own unit tests on compiled code because it generally can be trusted. LLMs are an extraordinarily leaky abstraction.
Saying "as long as it works and tests pass" suggests that we can test for every possible scenario. We can't. And tests can be flawed on top of it. Which is why an LLM is no more a compiler than a human coder (as you say) is.
I strongly agree with the concept that cognitive friction is the engine of learning.
First and foremost, it's an issue of "dependency": if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself.
A historical example that brought about a similar shift is this: when the production process moved from the craftsman's mind and hands to the Fordist factory (and the assembly line), the skill of building things shifted from human craftsmanship to anonymous, structured processes.
Bit by bit, traditional artisans lost their knowledge and "know-how." Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
> cognitive friction is the engine of learning.
I agree 100% too.
This is why in order to understand new codebase or to ramp up to new projects, I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way. Then I print them on a paper and read it using a pencil/pen and take notes.
Because of my math training, I am in the habit of slowing down to read textbook style texts which helps.
Exactly the type of usage I would hope to see advocated for in the future. As I wrote, when leveraged with Socratic methods and a sort of "interactive documentation", they can really boost one's understanding. Of course, this is predicated on the individual having enough information to know the right questions to ask.
> "...if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself."
For a bit of extra unease, consider that these people will retain their right to vote, both in making business decisions as employees, in shareholder meetings as shareholders, and in government elections as voters, despite their atrophied reasoning abilities. Who will wind up whispering what to vote for in the ears of these reverse centaurs?
> Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that can not be automated. Thus, operators will not practice skills as part of their ongoing work. Their work now also includes exhausting monitoring tasks. Thus, rather than needing less training, operators need to be trained more to be ready for the rare but crucial interventions
It believe the most likely consequence of AI is to promote generalists: people who have a domain expertise, can work cross discipline, and has enough programming knowledge to keep the LLMs on track. I don't think 'pure' software engineers will end up being as highly valued as the last decade though I also think that will be true for other disciplines as well. Just as an example, in signal processing, it's not uncommon to have a person designing the general algorithm and another person dedicated to implementing the algorithm in the embedded system. With the quality of coding agents, it's not really necessary to have both of those people anymore. A person who is moderately experienced in both can do the job now.
This seems sadly very true.
Just yesterday I was implementing some slightly tricky javascript (not my main language) on hover show n neighbours to each side, and if a deficit on either side, expand to the other side. After about 20 minutes of struggling to get the offsets just right, I succumbed to just asking an agent to do it.
I'm sure I'd still be able to do it, but was saddened that I didn't get it as quickly as I think I used to be able to. Atrophy might already be in play.
TBH it was already pretty bad. There is a stark difference between the best and the average in my experience. The top, say, ten percent of coders are vastly better than anyone else when it comes to anything but boilerplate glue code(which is still needed and is better done by average coders anyway).
This is speaking from my experience as a systems/c/c++ guy. If you are a js web frontend guy, python, or whatever I have no idea if this applies to you.
Yea, it does apply. Honestly I'd argue that as you move up the abstraction layer cake, it applies even more. If a team gets a tool which 2x's their output (maybe a higher level language vs writing in C), then the difference in absolute output someone who produces 1 widget per day and 5 widgets per day becomes substantial. If you're given a tool which 10x's your output in a day, then it's even bigger. And of course you can have devs on a team who produce -.5 widgets per day.
Think about the worst colleague you've had, now give them a tool which 10x's their output :)
Writing code in programming languages is a skill/necessity created by us to instruct computers what we want them to do
Initially in the 60s, this was done by connecting circuits one way or another (think ENIAC). Then we devised "programmable" computers and devised a bunch of codes (computer code instructions) that abstracted away those cables.
The we created Programming Languages to further abstract away the hardware complexity, and to be able to write down our wishes in a way that is more transferable between us humans, but that is still computable by machines.
But with LLMs and neural networks, at some point these abstractions wont be necessary.
The computers will still be making computations, but the way we tell them what we want is going to evolve.
Not to detract from your point, but there were software-programmable computers in the 50s. Check out Usagi Electric's videos about the Bendix G-15, where he loads software from punched tape.
We are spending sooo much for our intellectual time and energy on discussions about alchemy-like experiments with different models, agents, harnesses, orchestrations.
And there are huge amounts of time, energy, and writing sunk into arguing about AI or trying to convince people of the danger or problems.
Sadly all of this is taking time away that could be spent on real progress and learning about coding/FP/PL/algorithms/type theory/etc.
I'm looking forward to having the same experience as COBOL engineers did after retirement: companies backing up the brinks truck to anyone with the skills to keep the lights on.
now that's a phrase I have only heard in one other context: as an alleged quote from former Boston Celtics player Isaiah Thomas (who was really about 5'7 despite his official listed 5'9; yes, professional basketball player in NBA). This was around 2017 when he had just come off a career year and was soon due for a contract extension in the low 9 figures. He got injured and bounced around for a few years before eventually leaving the league, and never got that payday.
Yeah, every COBOL job paid pretty terrible relative to other types of work (like web dev).
Sure there might be a dozen individuals making over $300k but most COBOL devs live in the deep south making less than $100k for their entire careers. Fine salary, but that line of work is severely limited and the salaries do not reflect how valuable that work is.
Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design. I don't think AI atrophies those skills / knowledge as much as the coding language and I think in many cases actually improves them, especially if you are reading the code that is produced and at least understanding the flow of it.
I agree that AI reliance is hurting engineer's language understanding, but I don't know how much we should care.
I don't think that's quite correct; every CS program I've seen involves writing code, quite a lot of it. What they don't teach is /software engineering/: working large codebases, maintaining code, dealing with legacy code, long-term collaboration, working with product managed and designers, etc. That's stuff that you mostly end up having to pickup in the real world outside school.
I did B.S. in software engineering instead computer science in college, and there were classes that indeed taught the industry software engineering process like you mentioned.
I often think back to the early GPT days before agents as maybe the last time I will deeply learn a technical subject. I think that was when I actually learned the most as I had to stay completely in the loop. I had recently started a new job and had to work with k8s for the first time. Using GPT to help me implement new services and help me diagnose and fix issues with kubectl taught me so much.
Now with agents I don't have any insights into what its doing with kubectl and would have no reason to learn how it works (other than my own curiosity).
I agree that the human no longer gains expertise, but I am not sure that will matter in the long run.
The end result is a total and complete reliance on AI providers. It's not really a secret; that's why I started the article with the direct quote from Altman. They would love to see a world where nobody is able to do any type of development work without an LLM subscription enabled...even though they also admit that these models perform best when they are steered by someone who is highly capable and knowledgeable in the first place. They're powerful, but they are also VERY fallible.
Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.
In some ways, it feel like I'd be "stuck" at my 2024 capabilities if I didn't push myself constantly, and I'm really glad for every hour I spent learning/applying new stuff before that, because now it feels way harder.
So far I think I have learned far more from LLMs than I've lost to them. I forget some syntax, definitely. But I now reach for a much wider range of tools that I have become familiar with because of LLMs.
Yeah, I have to say some of this deskilling argument sounds like Socrates complaining about the invention of writing ruining people's ability to truly own the text. I really don't mourn my lost assembly skills or my lost C skills. And I'm not missing the details of whatever web framework du jour my Claude is vibing for me on my extremely useful dataviz internal tools; and I have learned how to rein the agents in to not ruin my high-performance Go code and yet debug concurrency issues. Maybe coding qua coding is what some people find their summum bonum, but for me programming is instrumentive - to build real things in the world that do things to make something better. When I want pure virtuosity of intellectual construction, I read maths papers in arxiv.
(Not to say that the current state of the art with LLMs merits borrowing money for a multi-trillion spend on hardware that will be obsolete in 5 years, but for me that is a different issue).
The master decrying the invention that resulted in more wisdom for more people than anything prior:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." (quoting from https://www.historyofinformation.com/detail.php?id=3439)
Yeah. Thought in Socrates' defense I think the jury is still out on writing, in the scale of human existence it's still very new. Give it a hundred thousand years or so.
I posted this elsewhere, but that story has a lot more nuance.
Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
I fall into the category of senior engineers who benefit from LLMs for all the reasons mentioned in this post. I find it's possible to agree completely with sentiments like this and still feel as if this is all written in the sand below the high tide line, and ten years from now nobody will care about this.
Horsemanship and sailing were both specialized skills of high value to society, and now they're not. But in each case there was probably a liminal period, when being an accomplished horseman or sailor was still valuable, even as motors were taking over. Eventually that period ended, as the new generations without those skills found ways to get by with cars and motorboats.
This analogy breaks down because computers aren't going away unlike horse drawn carriages or sailing vessels. Computers aren't being replaced by something. It's that fewer people will understand how they work while the efficiency and reach of computers extends into areas of life previously unreachable due to the scale of having humans build it or the refusal of humans.
I could wax philosophical about what that leads to but enough people already have.
Great read that brings some rigor (or at least cites rigor) to what has otherwise been anecdotal.
> then what is the path for one to become an expert so they can effectively use these tools?
The article briefly mentions guard rails and I think on the same note we need the AI agents we use for coding to encourage learning and comprehension in its users. Right now it’s completely voluntary and on the onus of engineers to put in the effort to learn since, as the article states, the friction that led to some degree of required learning is gone.
I don’t think there’s a big societal problem here, except that the agentic programming developers may be out of a job like the rest of us. If AI continues improving it will just keep taking over whatever skills are involved in the current AI coding meta are. If it stops improving then the meta will stabilize and after 10 years everyone will have longevity in it.
I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
Depends on what you are programming. I have been using it to build things that have never existed before, that I have wanted for over a decade. Experience gives one more imagination.
> I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
I do not think anyone could do the job I do with AI without my decades of experience, is my point. For now.
The AI programming meta that keeps evolving (last month it was loops) seems like a relatively trivial skill to adopt compared to learning what it’s like to do high quality work and to participate in a team, and these kinds of things. If the AI stops improving then everyone will master the AI skill set that everyone else is using, and people will acquire maturity and experience over time, like always. This is my point.
There have always been opportunities for those who seek out the less traveled paths.
If all your peers are only accepting what LLMs tell them then you should do the opposite. Spend some of your time writing code by hand, especially when working at a new layer of the system or on new things you've never looked at before.
While the agents churn out more CRUD crap in the background spend your attention learning how everything works.
I do way more experiments with way more things by taking advantage of LLMS. When I have them do research I use that as a starting point for my own reading (but don't blindly trust them).
I find I am learning more faster.
Change is an opportunity. Use it to your advantage. Let other people while and moan while you get things done.
I think we now entering this state where code is cheap but good well made programs will be rarer then ever, a lot of crap can be made and a lot of programs made without ever thinking the problem through properly will occur.
It might be like how an archeologist since the invention of plastic can date the period of the soil as post plastic, if one could cut open the software stack of the coming systems that will be built in the near future one would could data that code as post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
I am not Anti-AI but I think it's going to be interesting and I am surprised at how bad a program ends up when one tries to "vibe code", though often ends up working, though in isolation and when used with discipline (that the tools themselves psychologically make it harder to do) can produce some very good code and being able to use loops to solve difficult problems or problems that simply would have require banging ones head against the problem many times is very profitable.
Yes every time I tried to understand ffmpeg I got a little of it for a little bit and then lost it by 2 days or so. Just getting LLM's to do their thing for ffmpeg is great. the other one is socat an utility I used before but LLM's can do magic with it.
> post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
This is an accurate description of 90% of corporate software pre-LLM.
LLMs are introducing a unique kind of bloat and dysfunction but it's not like they are replacing pristine code. Most software that is subject to market pressures and is generating a profit is painful to look at and difficult to work in.
I wonder if we will eventually see the same drop in general knowledge we saw with manual driving (in some parts of the world), cursive handwriting, clothing repair, etc. Of course, coding was never as prevalent as these practices in the general population, so it might be harder to measure, but I feel we will definitely see it taught less in schools, harder to find resources, etc. in the coming decades, becoming more of an underground hobby.
It is guaranteed to happen. The key question is when is it a problem? We've lost lots of knowledge as a society but we're still going. Some skills decline in usefulness, others gain utility with the passage of time. Tale as old as time.
I would hope as long as there is a demand, there will be those willing to fill it. There never would be a problem because the skill set would stick around as long as there is the need for it. At least that has been the historical trend.
We still have horse farriers for show horses or for family pets. It is a niche there for those that need it, but nowhere near as common a job as it once was. It will likely be like this. Those that like doing it will stick to it, but we will not see Indeed listing jobs for it as much. Of course with less demand, less will have the skills, so those that do have them can definitely command quite the power when they are called on for something.
I hope that this will expose the true value of education. It's a new scale perhaps, but not a new problem. We sit in classrooms even though books and YouTube videos make available the teachings of far more illustrious teachers than the one in front of us. We watch things written out agonizingly slowly in chalk. We learned to do algebra and calculus even though we had our TI-86s and Mathematica. We learn in Haskell and Scheme even though those are rarely used later. We write toy compilers and write essays and repeat experiments with known outcomes in labs. We do it to learn how to think, how to build. We just have to be even more deliberate about it now.
I’m still resisting the AI flood. I use it but I do read code without its help. I also make sure I have the muscle memory of stuffs that will be tested in interviews — somehow big techs have forgotten that maybe they shouldn’t test people on coding problems.
So basically I’m still doing things manually for everything that preserves my edge for interviews and as a technical person. But I use AI to generate diagrams and documentation, as well as operation scripts that no one is supposed to memorize.
Does anyone else feel like the web has become so "readable" it is illegible?
I have such a hard time quickly reading pages like TFA. Headings, bullets, line-spacing, width, link hover animations, even the font. Even reader mode defaults aren't great for me.
I can totally see where that could be the case. I'm sure there have been breakthroughs already, but it's not mainstream news. The mountain of AI slop is all over the mainstream though. You have to wonder why.
> This applied friction is directly what builds "developer intuition" (or "taste"). The Germans have a great word for this: Fingerspitzengefühl (fingertip feeling). It’s the muscle memory that triggers when a developer looks at something and thinks, “yeah...this is probably going to cause problems.”
No, we do not have that word for that.
Yes, we have that word. No, it does not mean that.
Any LLM proofreading could've told the author that, for that matter.
__
What Fingerspitzengefühl actually means could be described as "tact", a precise approach to something and general attention to detail that leads to success.
Or.. the lack of all of that leading to blunt failure.
__
I would of course comment with more Fingerspitzengefühl for the emotional needs of the author if the writing wasn't just a sales funnel for their courses and whatever else (+ posted by them themselves.)
Alternate take: How many people have known how to program in assembly for the last 30 years or so? It's been a skill on the decline. Good compilers made that happen.
My personal antidote is to write unassisted Zig in my spare time (though C would be fine as well). While I can't back with data I certainly feel sharper.
The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!
This happened before with “real” engineering. Engineers a hundred years ago used to be proficient with slide rules and mental math and 3D assembly by hand. Products designed in that era often lasted a hundred years, like Singer sewing machines and Lada cars and 500 year old churches. Now things aren’t built like they used to be, and few people can rebuild civilization by hand like they did. But the world moves on fine!
It's different though because for structural engineering finite element models (set up correctly) will give you a deterministic, mathematically correct answer.
In fact FEM is arguably more deterministic than the old school pen and paper methods (yield line analysis etc.) which often rely more on correct judgement of the structural engineer doing the analysis.
Substantiveness of this article notwithstanding, I do want to point out that the author also runs (or is planning on running) a programming course which to me may be a slight conflict of interest. I think the author may altogether be well-intentioned, but it IS something that I'm going to keep in the back of my mind as I consume this content.
when the tool becomes a crutch you end up with a limp.
at work, i can tell that people are not thinking and losing the ability to thinking hard.
I keep coming back to what LLMs "really" are. Recursive autocomplete, it keeps choosing the best fit for the next token. This really demystifies it for me. It also reinforces that is not "reasoning", so there's no hidden mental model.
>at work, i can tell that people are not thinking and losing the ability to thinking hard.
its also wild how quickly and willingly many people handed this over. the apathy and desire for connivence runs extremely deep among my coworkers. we are at the point where people are not even running their local dev servers and clicking around to ensure their work is functionality correct, they just ingest the ticket, dump it into the model, commit the output, and say they're the most productive they've ever been.
reminds of college when everyone would use chegg, but knew it was a shortcut.
the difference now is it is shamelessly encouraged.
one interesting topic is code reviews of AI code. code reviews have always been a bottleneck on the team and there's lots of trust and rubber stamping. I don't believe people are reading the code they ship.
I will say that it is a beautiful tool if you turn it on its head. if you don't quite understand something, it will never tire of answering your questions. It is a great time to need handholding.
Writing code was never the job. It was a small part of the job that can now be delegated to AI.
Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage. You know `Engineering`. Mechanical and Electrical engineers didn't go away because of CAD.
> Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage.
Besides your first point, the rest can be delegated to AI as well. And you don't need to be a "software engineer" to spec out requirements and user needs.
This is happening at my company: all of these software phases are being delegated to AI. And the people without any understanding of software have absolutely no idea what they're doing. Features no one needs are being made in minutes. It hasn't been long enough to really tell how that's turning out. Everything seems to be in endless development.
Then you make it write code in the tickets for the tricky stuff.
Have it reference relevant documentation/APIs.
Then there is no lost knowledge.
(Still, sometimes it sneaks in some "helpful" belt and suspenders, but for the most part, I know everything that is going on in the code base with this method.)
The boss says we should do it this way. The boss hasn’t got a clue about how to do it. But he knows what he wants, more or less. And we can fill in the details. And we don’t make a stink about him not knowing exactly how things work. That doesn’t serve anyone. And he still thinks that he knows how to do it. Because he tells us how to do it and it gets done.
At some point it won’t be realistic to ask a human to solve a technical problem any more. I’m not suggesting it will happen soon but I can at least imagine that day coming. Whereas before modern “AI”, that possibility wasn’t even on my radar.
This just feels like someone whining about how ORMs result in $#1tty data access code because lazy programmers don't take the time to learn how to use a database.
When that happens, (developer thinks using an ORM means they don't need to know how to use a database,) the project eventually fails. It's the same thing with AI; just like companies learned that they need to make sure developers actually know how to use a database, companies will learn that their engineers actually need to know how to program.
I am not personally worried on this front. It is a near-certainty that something catastrophic will happen in the not-too-distant future because of the issues Lars raises in the essay. Not until that moment occurs will there be a broader industry-wide reckoning. Most likely it will get expressed in breathless language like "you'll never believe the crazy gains we got from adding tests!" (or whatever).
I feel pretty good asserting that, as always, a few smart leaders will recognize early that quality and stability are the new game now that we have speed 'for free.' And then there will be incumbents whose eventual demise will merely be accelerated.
Speaking on a handful of these that I'm familiar enough with to opine on:
- Photography hasn't erased other visual arts. Both require a separate skillset; both have their uses and their enthusiasts. Photography benefits from having a lower skill floor, in my experience doing it, and this in turn has led to commercial photography eclipsing commercial painting (barring things like painting buildings) as a viable business model, largely.
I'm not sure if we're better or worse off as a society for having gone through that shift.
- Television is arguably not a net positive in society (I won't say net negative yet, but there are some reasonable arguments for that position). It enables a form of passive consumption that allows bad arguments to come through as persuasive through the gradual (on the scale of months and years) numbing of critical appraisal of the media being consumed.
This has more- and less-pronounced areas. In the US (which is geographically close to me and which has an outsize proportion on my own country's culture), Fox News is an excellent example of how TV can erode critical thinking.
- Smartphones arguably reproduce and amplify this problem, and tack on others: they make it increasingly easy to dissociate from daily life, and make it easier to continue consuming content from actors increasingly interested in steering conversations in one direction or another (on the most benign end, it's influencers trying to get you to buy things, which obviously has negative economic impacts on an individual scale; on the more overtly harmful end, they serve as a hardware channel for less-regulated or unregulated extreme and hateful content).
Just like with TV, smartphones have clear benefits and utility in daily life. They aren't abjectly bad inventions, but they have a serious societal toll, and it's difficult to ignore that.
- In aviation, the first generation of (relatively modern) automation did lead to a high number of accidents and loss of life. American Airlines flight 965, in 1995, is a well-studied case of an over-reliance on flight automation before it was widely understood among pilots what the limits of these early-modern incarnations of autoflight were.
In the wake of that accident, many aviation regulators instituted or began requiring that manufacturers and airlines formalize training and operational procedures that protect thoroughly against this sort of cognitive gap.
I think that last example (and many aviation accidents) are an interesting example to study in the context of software. Aviation's extremely and conspicuously responsive to accidents, often requiring procedural and hardware changes as a result of these. They can be as simple as a software patch, or as complex as the redesign of a component because a human factors analysis found that it could be confusing under stress.
In software, we like to think that we're a serious industry, sometimes. We spent much of the 2010s going in what I found to be a good direction: the development of IaC, the popularization and standardization of tools like Git, registries for vulnerabilities like the NVD, and even the development of languages like Rust which address historical technical shortfalls of high-performance, low-level languages like C; but we always fell short, industry-wide, on matching these toolchain upgrades with technically-rigorous procedures.
And so what? How do you think people are going to be working on software in 10 years. They're going to use coding agents and harnesses that work much better than the current ones (although I don't think we're getting anything but incremental improvement from here on out). They will run on commodity hardware. I first got paid for writing code in 1999, I've worked at publicly traded software companies and failed startups, in every language out there. And if LLMs "atrophy" the expertise I built up over those years, then so what? What is lost, exactly? I stopped programming in C++ about 10 years ago when Rust seemed to get good. I no longer remember a lot of the specialized knowledge I had in C++. My life goes on. I am now becoming an expert at building software with agents. The problem is still the same, you still have to apply engineering principles. You just don't have to write the code.
I was thinking about this anti AI sentiment and how it is related to American individualism. In socialist countries like China there has never been a protest related to data centres or job loss etc. unlike in the USA. Because the large scale impact of AI on the country is exclusively a net positive.
The reduction of the individuals cost of labour by also reduces perceived egalitarianism which is the mental model used by ultra capitalists to allow themselves to sleep at night whilst homeless people beg for food on the streets.
The long term impact of AI on society is exclusively a net positive if utilised in the correct way. But most of the westerners lack faith in government fiscal policy and think that they are going to be left destitute when they can no longer provide for themselves. When in fact UBI is going to be the most likely outcome.
They are myopically focusing on the short term labour market effects of AI on their own profession instead of realising the long term utopian societal outcome.
"We need to ban AI because I'm going to lose my job!" - This is only said by someone who is winning in their society and wants to maintain the status quo not by those who slipped through the cracks.
It's surprisingly late for him to notice. But wait til you think through the consequences of the already collapsing human capacity for extended written composition, which is orders of magnitude more significant.
It kinda seems like wishful thinking on our part as software engineers that one day we'll be able to go "Hah, miss me now?" but that just isn't the writing that I see on the wall.
Over the past year, I've AI-generated two large pieces of software over 2000 commits without having much depth in either domain, and my ability to deliver value as a veteran software engineer has only shrunk over the months or it can be trivially extracted into a reusable markdown file like "design bar: ensure things 'by construction' where possible", a lesson I learned viscerally over decades.
Because I can still reason and dig into things and ask questions and have the domain explained to me out of curiosity, the same way I could build anything that was originally out of my depth.
But what I've noticed is that the need for any corrective power has gone or is going to nil and I'm mainly doing directional work. And even then, I can constantly have sota models "rank the top options and recommend one", and it's almost always the right way forward.
I used to read every line of every plan that LLMs generated, but over time I've realized I have less and less to correct or, more impressively, the LLM had foresights I never considered. The same thing happens with implementation.
Periodically I can spawn a bunch of agents to evaluate the system adversarially to find improvements, and the findings have been so good that it's evident that soon I can just automate that too.
In other words, the skills I need to excel here are more curiosity and patience than tech expertise -- the things that got me into software engineering in the first place since that was the only way to build things.
You got it. You may well be right, but from my standpoint it's hard to see how you'd know if you weren't. Yeah, the models fill in vagueness with detail sourced from "what's likely and helpful". (In one of my particular domains, that's "use SHAP values for everything everywhere".) Yeah, that detail's increasingly unlikely to be "wrong" in a way that you could explicitly argue. Is it what an expert would come up with? Is it the best it could be? Is there something else that we would have done had we been forced to think it through? Will it ever be _surprising_?
I know, all irrelevant little philosophical roadbumps. The artifact satisfies the buyer. I think I'm going to have to step away from this industry pretty soon, at least for a while.
Ever since I was a teenager, I started to develop cyclical bouts of existential crises.
While I have been feeling better for quite some time, I feel another starting to brew. While I was never one of the fabled 10x or rockstar developers, I still derived a lot of enjoyment and and sense of purpose from my work.
With the utility of LLMs becoming more ubiquitous by the day, what purpose can I provide anymore? I struggled greatly to get where I am today, and while it might not be impressive to any of my fellow programmers, I am still proud to be a programmer.
I obtained a degree in CS because I found the entire field to be fascinating. I had great passion for learning many of the subjects. What I learned in my degree changed my life forever. I seriously went from one of the, "What's the point in learning most math if you never use it?" to being absolutely infatuated with the subject. (I'm still bad at math, but that does not detract from my pleasure).
With the utmost certainty, I did not push through all my struggles just to prompt LLMs all day. Maybe's its my insecurities manifesting, but I want people to say, "You did a great job" and not "Claude did a great job." I understand the Buddhist were likely correct. My suffering really does come from my attachment, but at the same time, I do not want to let it go.
I have no opposition to using LLMs as tools. I think a fine balance can be achieved. LLMs can even be exceptional learning tools when used appropriately.
If programming is just a means to an end to you, then you will receive no ill judgements from me. We all have different motives in life. But programming has always meant something more to me.
As always, the innovators will thrive and become the new experts in the field. Everyone else will die out.
It's impossible to buy the FUD that expertise will somehow vanish because people use AI. People come in a variety of capabilities, challenges, and personal interests. Expertise varies from person to person. Those with personality traits that are more likely to take risks and persist against adversity will be the innovators and experts. We've seen this played out countless times.
You have to ignore everything you know intrinsically but can't articulate about innovation, technology, and evolution to drink this FUD Kool-Aid.
I'm seeing it happen in real time. I've seen several folks claim they "vibe-coded" something and upon review it's some seriously low quality garbage. Unfortunately it's just "functional enough" where it's encouraging the one who talked to the robot to continue thinking they can code for some reason.
It is important to ensure that any LLM code is named and shamed.
One interesting feature of vibe coding is that it allows you to write code in specialized areas where you have no expert knowledge. That makes the code effectively impossible to review in any useful way.
For example, I have a vibe coded Python monstrosity which I use for stitching together scans of large format negatives. The enlarging lens that I use for scanning has strong chromatic aberration in the red channel but is otherwise very sharp†. I got Codex to figure out some way of using the green and blue channels to sharpen the luminance while preserving the color information from the red channel. I sort of vaguely understand how this works, but not nearly well enough to review all the horrid numpy code that implements it.
For my use case, there is really no reason to look at the code, as I am only interested in the output. As disconcerting as it is, I feel that this is going to become more and more normalized. Looking at the code that your LLM generates will become like looking at the assembly output of your C compiler.
† This makes a lot of sense, if you think about the color of a darkroom safelight...
Coding expertise was on a downward trend way before AI arrived. Does nobody remember how most "engineers" were literally copying and pasting snippets from StackOverflow they didn't understand, and the snippets usually had tons of bugs? How many of you know Assembly well? How many of you can decompile a program, or write your own compiler or VM? How many of you could pass a computer science course (that you didn't take at Uni)?
We've been hearing about the demise of skills since before the invention of the loom. The only skills we've truly lost are for things nobody uses or makes anymore. What actually happens is, we develop a new tool that's better than the old tool, and we get good at using the new tool. We then have a class of people whose job it is to maintain the new tools. We don't need that many of them because the whole point is to eliminate that labor.
Your jobs as software engineers are going away, it's that simple. Your new job is to use the AI tools to make products. Somebody else's job will be to make the AI a better software engineer.
I hate to day this, but most coding work isn't hard at all - for a human or AI. There are a lot of important skills around coding, but for most people, much of the time, they are not doing any coding that is particularly challenging, difficult, it novel.
If you have a truly hard problem - sure write some code or take a hybrid approach. Still, let's not pretend this is what most people are tasked with.
1. function trampolines, such as in libraries like ATL (Microsoft's COM template library);
2. hand-optimized encoders/decoders for things like ffmpeg;
3. niche applications such as Ben Eater's 6502 breadboard computer (using a modified WozMon and Microsoft BASIC);
4. reading ASM output from C/C++ code in things like the GodBolt site for performance comparisons/analysis;
5. reverse engineering old DOS games;
6. people working on things like compilers (generating optimized assembly), JIT engines, or other code generation -- though this will typically be machine code, possibly with assembly instruction comments;
and other performance critical places where assembly can speed up that part of the code.
ASM to a higher level language is a jump to a clearly defined higher level language through a deterministic compile step. The jump away to LLMs is something completely different.
Keep in mind also that LLMs are currenctly heavily subsidized. Once VCs money are gonna run out, you will see the real price, and your 200K per year dev is probably nothing.
I learned BASIC, Pascal and x86 Assembly language by the age of 14-15. None of which I remember (nor care to). The strict instructions (aka language) we gave the compiler was not the point. Those experiences taught me how to think logically, and over time develop a taste/structure for how software should be built, as well as foot guns to avoid. I don't know how you teach that without coding, but the "coding expertise" was always a means to an end. LLMs are just another abstraction layer above programming languages, machine language, etc.
Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:
> LLMs are just another abstraction layer above programming languages, machine language, etc.
Programming languages are (largely) deterministic. LLMs are not. The boundary between an abstract concept and something executable sits comfortably in the mind of the programmer in non-LLM contexts. This boundary shifts when using an LLM and becomes shared between the programmer and the matrix multiplication machine.
The "layer of abstraction" analogy is therefore technically true but essentially false. "Just another" papers over how seismic of a paradigm shift all of this is.
That's the whole scam. It's a ransom with more steps.
First they fuck up your codebase with so much slop that no human does (or could) understand it.
Now if you want to keep making bug fixes, you're on the hook for whatever Anthropic wants to charge you.
The solution is to make AI developers the scapegoat. Fire the people at the top of the AI leaderboards, with malice. Otherwise, lazy AI-addict developers are going to destroy what remains of your company's IP.
"Mental math skills" literally did collapse tho??? "Math skills" to write proofs (which a calculator cannot do) is a fundamentally different skill set than that "math skills" of mechanically doing arithmetic (which a calculator does do).
Seems like critical reading skills have already collapsed.
This is exactly the right analogy. Granted LLMs are a more sophisticated calculator and engineering is still adapting to this wonky/evolving technology. But to pretend it's a coming catastrophe is to ignore every past parallel development - where this has not happened.
Math skills are at their lowest point ever in modern society, and many are still advocating for the reduction in reliance on calculators for this reason.
Actually, they have for a large majority of the population. And fun fact: that debate is still going on, and the Mathematical Association of America (MAA) enacted a rule prohibiting calculator use in AMC.
Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...
Not really the same, since your average calculator doesn't take over executive function. The user thought and decided what needs to happen, down to one extremely well-specified task, which is so unambiguous that it can be completed by rote and without context.
This is more like paying someone else to read your math homework, generate answers, and maybe even submit them on your behalf. If that had been the (average) student experience, math skills really would have collapsed.
A more apt analogy is the diminishing understanding of assembly language and machine instruction sets. I love programming and something got lost in both transitions, first to high level languages and now to coding agents. I am quite sure some contexts will require programming by hand for a long while as there are still people who are writing amd64/avx512/sass by hand. I hope in time we will find new ways to enjoy solving problems with this new programming modality. By now, I am pretty sure it is here to stay and evolve from its current crude form.
100%
We're already seeing this at the enterprise level. Companies have dictates from leadership that "if you're writing code manually, you're doing it wrong."
Okay, that kind of works for a while. We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
This is all complicated by the fact that we're also losing our grasp on reality from the other direction because we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
My job improved significantly since I managed to get into their (managers) thick skulls to tell Claude to not be verbose. I'm tired of 1200-word tickets for "put the totals in the automated emails".
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
Nor even €50k/year.
Two things are true:
1) The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to *write*.
2) My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
That last one, "given the time constraints", is a place where my experience is still useful. The AI is as lazy (or as optimised for fast wins) as the humans whose examples it was trained upon; but an LLM costs so little that the answer is always "do it right" rather than "do it fast". A lot of people don't know what "do it right" even looks like, having only ever known e.g. websites that take 5 seconds to load because of all the adverts and analytics, and never the world where machines with 1% of the CPU and no GPU at all could fit a fully playable first person shooter in the same memory footprint as that page.
At least, I hope this "is a place where my experience is still useful"; I may just be telling myself a nice story, same as all the other people through history who have found themselves obsoleted when the automation came for them.
Ai can work through all of those questions in number 2 and design around it.
I was just doing that yesterday designing a game demo lol
Works for a demo, but quickly stagnates. Future models may of course push the limits further, but right now it quickly goes sideways without someone knowledgeable at the helm.
You don't design things with never ending details though. If your developing a feature there's a limited scope to it. If your designing the direction of a large project you're working with higher level concepts.
The point is that AI is aware of constraints and can manage a round it. You've got to keep in mind the size of a typical software project and plan. Most people aren't writing a kernel. They're writing some backend tool and need a project that fits into a couple of quarters with a handful of people at most involved.
> The point is that AI is aware of constraints and can manage a round it.
Only if you instruct it with constraints. Otherwise, it's happy to implement whatever workaround it needs. But it still takes a dev to know what those constraints are and why they're needed.
I have also tried developing games with them. I find them very weak at these things.
Often they can't figure out how to test the thing, so "does this attempted solution even work at all?" is its guess from reading the source code, and sometimes I've even caught them writing "tests" which are a regex on the source code, not functionality.
They know almost nothing about "is this solving the right problem?", they're trained to assume the user is right, not to frame-challenge.
My general experience both in game and non-game projects is that it will be lazy by default rather than solving things correctly. Sometimes I spot this from their responses, other times I only notice with manual testing.
FWIW: The highest paid devs I know excel at 2, and aren't even that good at 1.
Before, if I would start working / thinking on a problem and discover it was harder than anticipated, that often was a signal that the feature may not be worth it. Because implementing it doesn't fit the current model of how things work or similar. Then it was back to the drawing table and find a better way of solving the underlying issue.
But an LLM just happily chugs along and does it, no one feels the friction, which means you never stop up to think if you're solving the right problem, or in the right way. I'm already seeing this bite us in the ass, as you get hacks upon hacks.
I first brought this up late last year – that there used to exist a kind of selection pressure against both developers and against features which would prevent certain functionality from ever being shipped, specifically:
- Functionality which exceed the technical ability or knowledge of the developer which built it; and
- Functionality which would require an excessive number of changes that time/cost became a constraint (likely because it wasn't an incremental change but a significant rework, or just a bad fit with the existing product).
Sometimes I had the displeasure of joining a company where you could see developers hit these blockers and tried to fight through them (likely under pressure from management) rather than avoid them. And in the process they created a mess of buggy, half-functional spaghetti code which someone else later had to resolve.
Today however, people can use agents to smash through these blockers and ship an incomprehensible amount of crap. And what's worse is they're celebrated by management who don't know any better and see this all as a productivity win with no downside.
I suppose there are two ways to look at this – some would argue that engineers don't need to understand what they're building in detail anymore so non of this matters. Instead they can always use agents to explain what's going on and prompt them to fix any issues that come up.
Then there's another camp which might argue that agents will fundamentally have all the same limitations as humans, and that at some point a codebase will grow too complex that it exceeds even the limits of an agent's knowledge retention or intelligence. Or a codebase may just grow so large that it costs 1 million tokens to make even a simple change.
Unless agents start saying, "there's no way I'm shipping that" like humans used to I don't really see how we avoid the latter scenario... Complexity simply must have limits even if agents allow the bounds of those limits to grow.
At some point models will have to take control or we risk hitting these limits in irrecoverable ways since complexity far exceed that of what a human can reason about well.
If this wasn't true then "build me all the software" would be a reasonable prompt. Because why wouldn't we just get an agent to build everything we could ever possibly need? It's obvious that in the limit there will be limits in knowledge, intelligence and cost.
Whether it's humans or agents, someone needs to manage complexity. That is the most important thing a good SWE used to do. It's why technology selection mattered, it's why good architecture mattered, it's why clean code mattered.
There is a limit, after a few 10,000's of LOC ai projects start disintegrating and becoming unmanageable.
10,000 LOC, hell my LLM can do that in a day!
Current project has over 60,000 LOC, I find using an antagonistic agent code and architecture reviews are really handy for keeping things grounded.
I believe tinygrad still caps the total LoC in their repository to 10k
In my personal project I've found having a hard lines of code limit has been a good limit on complexity. Makes me really ask if I need things.
One theory I have floating around in my head is that if a whole code base was microservices and micro front ends that were all less than 5,000 loc then you could fit the whole thing into a 100k token AI context window when working on it. And being few lines of code would force them to be somewhat simple
So exactly true! With ideas people unbridled, the painting-yourself-in-a-corner tendency that was always there has gone exponential, add to it the not-completely-aligned incentives, and the oh-so-clever impressive writer of long prose LLMs just end up churning. The real value add may end up being the lesson in honesty and humility to us all...
I feel like there is also the "Nerd Factor" to consider which is this unrelenting passion to type away at a screen all day, whether that is yesterday's code or tomorrow's prompt. Few non-technical people have the attention span to suffer such monotony. (Even most devs don't have that to varying extents)
It's only monotone from someone looking, not from someone doing. If you look at digital painting tutorial, you'll see a lot of timelapse, because it's a very slow process where you spend the majority of time correcting stuff. But if you're the one doing it, each individual brush stroke is the result of a conscious decision.
I'm not sure; I find the job pretty boring.
Working on a passion project or working through the hairy details of a complex algorithm or whatever can be fun. But that's not really what the job is like most of the time.
Aye. This is true for many professions, I think.
I'm trying to write a novel, have been for a while now; one thing I hear from professional writers is that one should only do this if you are prepared to re-read whole thing six times before you even send it to the editor, and when you do send it to the editor, you and they will likely spend several months repeatedly re-reading the whole thing as you respond to a long list of changes they give you.
>My actual job also included non-coding work: Does this attempted solution even work at all? Is this solving the right problem? Even if it's a valid solution to the right problem, is it the best solution given the time constraints?
This is not non-coding work. This is coding.
this is software development
There's a reason it's called Software Engineering
Coding < Development < Engineering.
My claim is just that LLMs are basically fine now with the "coding" part. Not so much with development, definitely not with engineering.
Sir, this is a Wendy’s
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
That is certainly not true. LLMs can't write code worth a damn still, and you have to babysit them to make sure they aren't doing stupid stuff. A human is still by far the best choice for doing programming, and when trend-chasing companies realize the damage they've done to their businesses they will stop pushing LLMs so hard.
"LLMs can't write code worth a damn"
I think it depends on the - ahem - context. In my experience they usually can, sometimes they can't.
But generally their sense of software architecture remains abysmal, so even if their writing is ok, you need to have architectural authority the very least to guide them.
The fact that they sometimes can't means you can't have someone who is not a software engineer. Otherwise there is no way to verify whether the LLM wrote correct code or not.
Exactly. Expertise is frequently required to steer, correct, rework, know what is good and what isn't. This happens 500 times a day on both the micro level and the architecture level.
> The coding part of my career is over. LLMs are capable of doing everything I've ever been paid to write.
Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Engineers have done amazing work of abstracting away the hard algorithms and data structures. In the meantime, there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms. In contrast, getting LLM to work is a new field, so it requires tons of new implementations: KV caches, speculative decoding, all kinds of variants of attention like FlashAttention, all kinds of parallel processing techniques, RL pipelines, post-training pipelines, and etc. It's just that the field is so concentrated that only luck few get to work on them.
So, maybe it's not LLM per se that removes the need of writing code. It is the maturity of the software engineering that has done so. It's just that LLM fills the last gap: making knowledge transfer so much faster and cheaper - if all that's left for most of us is slicing and dicing of what has been already been implemented, then LLM can reliably take over.
> Remember we used to spend enormous amount of time in school and in our spare time studying computer science? Algorithms, operating systems, compilers, and etc. All kinds of insights. All kinds of fun. All kinds of hard engineering. Yet, how much time do we really need to spend in our day-to-day work implementing or using the algorithms and etc that we have learned?
Because great insight and expertise stems from foundational knowledge.
I am a huge hockey fan. In the NHL, the players do not stop practicing the fundamentals once they make it to the NHL. They practice the fundamentals even more. Many practice the same drills as youth leagues -- stick handling, passing back and forth, shooting, edgework, shooting, rebound control, etc..
The best of the NHL might not hit the hardest, have the most accurate shot, or skate the fastest. What separates them from the rest of the lot is that they are fundamentally better than everyone else.
> there has been little progress or few new fields in the past 10 years or so in business that ask for implementation of new algorithms
Transit routing is still moving fast in this area. We're still figuring out the best ways to return thousands-by-thousands transit time matrices with query-time truck dimensions and traffic updates. It can't be the only field!
I think it follows the path of the spreadsheet. For a long while, only geeks and finance or accountants or data monkeys used spreadsheets. Then, it was such that anyone could create a spreadsheet. Vlookup was something you learned early in school or on your home computer tinkering around. There’s still some modeling gurus out there but largely everyone is developing solutions using spreadsheets everyday. And they’re doing it rather autonomously even prior to AI helping. We can talk about how they’re ugly and crappy spreadsheets but they generally solve the problem the user had.
With AI, people can build and collaborate on applications much more complex with much less technical knowledge. It might be ugly and crappy but I bet they’ll be mostly autonomous and not need to work through their IT team, or go through the hell of PM and requirements. If I know my requirements, I don’t need you. Hell, I can just start building and add requirements as I come across them. It’s not a major risk to the project like it used to be.
It will, and is, going much faster than the spreadsheet did.
staying within free tiers has led to some really great optimizations for me
took complex pub/sub event architectures down to the events postgres can do on its own, and a system architecture that works with that payload. am aware of the limits with this approach and stress potential on db
This I solemnly swear. To take my paycheck with gratitude, even if my employer makes three times as much based on my output. To take whatever technological disruption that graces my path as an immaterial and impersonal force. To politely and discreetely bow out when the statistics soup trained on my fellow colleagues makes me unemployable and worthless. Amen.
> That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
When you factor in overhead and benefits, many companies were regularly paying that much for someone (many someones) to "read this Jira ticket and implement the feature".
We are currently in the "centaur" phase where a human-AI combination produces the best output, but I think some people are betting on the fact that the AI only product will eventually outperform the centaur. And with the cost of tokens falling thanks to fierce competition from the Chinese open weight models, it's definitely possible that those that bet on AI early will reap payoffs in lower expenditures for more output.
Honestly not sure which side I land on that bet but I definitely can't rule it out.
We've gone through this with chess, and the days when a human can do anything to help AI there are long past; the human can only drag it down. It seems to me to be a form of the bitter lesson. I expect this will happen in every field where we don't add guardrails to require humans in the network. I have a hard time seeing regulation happening around this in the US but vanity may also serve, will CEOs be satisfied ruling armies of virtual assistants, or do they need big buildings full of human thralls?
At some point the whims of the CEOs don’t win. The shareholders demand more growth.
Demand from whom, if not the CEO?
The board, presumably
Of course the demand will be directed at the CEO and they will have to provide results or be replaced.
Well, maybe. But most fields of human endeavor aren't very much like a board game with clear, unchanging rules and a single, unambiguous objective.
Have we really though?
This is not a silly question, because it's kind of silly:
Has an AI ever actually gone through all of the steps necessary to become a chess master? Go to competitions, raise in the ranks, take a plane to the masters, organize support and all the real life details necessary? Playing chess is just the small formal part here, and all of the obvious things humans obviously do, well...
I don't agree.
Chess is a perfect information game with objective, easily measurable outcomes.
Show me a software business that looks like that and I'll agree with your hypothesis.
Otherwise I don't believe your metaphor proves much of anything.
Another thing is, I have been playing Chinese chess a bit lately and, yes, Pikafish is stronger than any human player, but if you actually share a board state or list of moves with an LLM they will fail to accurately interpret the state of the board, let alone give good advice on what moves to make. Doesn't really prove or disprove the claim that LLMs will replace human programmers but it does show that the word "AI" is papering over a lot of specifics of particular technologies.
Maybe some day, not in the short term. I've seen output of bad developers using Fable and after that I sleep safe.
A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
It's like a tool that has caused a fan out of both good and bad outcomes.
Good: you can refactor your codebase at will, throw out legacy cruft by the megabyte, improve build/CI time, and simplify ruthlessly. Not to mention kick out new features in simple and consistent ways that align with what a user actually wants.
Or you could add megabytes of vibe coded crap, solutions that add a ton of mass but don't actually solve the problem at hand (seriously!), implement abstractions that are logically inconsistent with the rest of the system, etc.
Both of these are happening right now, and I think the latter is happening at a rate far higher than the former. But our fundamental dynamics are still at play - the ball of mud is still a ball of mud, even if AI lets you make it 100x bigger. The problem just gets more entrenched.
Eventually AI will learn how to simplify code, understand coupling, etc - and hopefully it will just iron out problems as it goes. I think we're a long way away from that. But this is uncharted territory, and I don't think anyone really knows. I certainly don't hear anyone focusing on that as a target of optimization however.
My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue, and perhaps we can re-learn these lessons yet again. It kind of feels like GitHub might be the first example of this.
> My hope is that we'll see a number of companies collapse as they scale - with basically no hope of rescue
If you worked in SV like I did from 2010-2020 you know that the exact opposite of that will happen. I saw hundreds of companies successfully scale out of their garbage stack such as Facebook and PHP.
The only thing that matters is the problem you’re solving and the quality of the code is irrelevant.
> A bad or below average dev with AI will run your product and company into the ground in a matter of weeks.
I had a recent experience with this. Although it was months and not weeks and there were other factors into play such as the market etc but the last company i was in had a poor engineering culture with inexperienced engineers equipped with AI output some of the lowest quality features for both customer-facing products and internal dev tools. A lot of customers churned, and im sure quality us part of the reason.
We have a product guy on the team who was in a deeply not technical role before AI who is trying to do the “hey Claude, read this Jira ticket, implement” thing.
It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
The spacer between the keyboard and the chair still matters in my experience.
Not everyone has this, but I always felt that a significant percentage of the value I bring is in immediately recognizing what you shouldn’t do.
I have a hard enough time explaining why “yet another bespoke application on some unmanaged azure resources” is a bad idea when they have more reasonable alternatives at hand.
Now these goofs can (very nearly) press a button and do it anyway, with no comprehension of the consequences. It’s high fives and pats on the back, until I’m cleaning up the mess.
This lands. Llms are bad precisely at following what not to do. They work best off of positive constraints.
I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps. I call it a bias field, pushes them toward hopefully the happy and harmonious (with the rest of the system) paths. Obviously this is only partial and imperfect enforcement, but if it's applied to everything consistently it naturally encodes some self-consistency and harmony.
I would be interested in see that. I have reems of rules I use with Claude.
> I have a design principles + tech preferences doc I force llms "lint" their approach against. It's not perfect but it helps.
I’ve done the same but it’s a moving target as models advance and I find half of my points are ignored until I’m prompting “No wtf why are you still trying to symlink the global Python executable just use the virtual environment that’s already activated”.
Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time. There’s no way my hobby fiddling will outpace whatever gets released in the next couple months.
In the meantime, real linting does work pretty well, if you can write a detector for whatever antipattern you find LLMs fall into (like multiline comments).
> Anyway, companies are pouring billions into improving AI tooling user experience so most of what I do manually I just anticipate to be a waste of time.
I get patches into the biggest AI projects all the time to make inference faster on my 3090 and all you have to do is read the contribution docs and open a small PR.
It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI.
You can trivially outpace what big companies are doing it’s insane, why do you think so many innovations are coming out of scrappy Chinese labs? They are willing to engage instead of being defeatist about it.
> It’s not a cabal of super geniuses. We are literally in the Wright Brothers era of AI. You can trivially outpace what big companies are doing it’s insane
Not my core competence and I’m not interested in making it a core competence while the field is still rapidly changing. I find harness engineering interesting, but not lucrative enough to push one of the many very important other things on my plate to the backburner.
I don’t think Python language developers are super geniuses either, and I’ve never submitted code for consideration.
Oh I'm certain I'm not doing it optimally or close to best practices. It's just an example I had in my head related to my comment.
Although, given opus 4.8->5.0 experience within claude code, I don't know if I'd agree they necessarily improve things with time. But that's just my experience. I should probably read their release notes, maybe just a tweak to my claude.md would help
Yes, there is a post that made the rounds not long ago entitled Taste Is All That's Left: https://notashelf.dev/posts/taste-is-all-thats-left
Knowing what not to do has always been important, but it is perhaps more important than ever.
But isn't it only a matter of time before the AI generated autocomplete will contain the correct solution? What then?
fucking is it? AI labs certainly want you to think so but they're running out of time and money. they've been telling me that the advancements of the last three months have made this a reality for the last five years. it's starting to feel a lot like Elon promising fully automated taxi services by next year... 11 years ago
next year I PROMISE!
Come on bro, PhD level intelligence!
For the completion to work, the source text needs to be ‘good’. That’s a basic kernel of how the thing works. Even with a perfect oracle autocomplete if the source text is ‘bullshit’, the output is too.
Or, slightly changing this. The source text needs to speak the correct vocabulary and language to produce a good completion. See the chat where Terry Tao is doing maths with an LLM. There’s _no way in hell_ I could get to his output because I just have no idea, and can’t speak the language.
Same with any field.
I've had some success with: here is this android bug ticket in a product that I don't know much about, fix it! And it works sometimes. I'm still technical, so it isn't the same, but it was crazy that AI could just solve some problems automagically without me know much about the system being modified.
Probably worth not getting too comfortable. It's only a matter of time before a new generation of product people can do this. Also, I have said this before, instead of 1 product person and 10 engineers on a team, you may have 2-3 product+eng people and 3 engineers, etc.
You have it mixed up. This guy is “the new generation of product people”. The old generation had to be technical enough to be able to grok the systems that they ‘producted’ over. It’s exactly the ‘new generation’ that LLM themselves right into Dunning-Kruger.
> It doesn’t work for the vast majority of tickets he attempts because he doesn’t have the necessary understanding to even start thinking about if the solution that the autocomplete generates is even remotely workable. And that’s with fancy dev loops and whatnot.
Remove the dev from the equation. Suppose you need to produce circumstances for the slop machine to succeed on its own (imagine a loop).
What is missing? A good enough description of the current behavior, desired changes and how it should work? Understanding of the business domain or the system in question, its architectural choices? Enough guardrails and linters and tests to ensure that the solution will indeed be testable and work end to end? Cause a lot of the things that will help human devs will also help LLMs make their output more workable - for example ADRs in the repo and custom in-house tools to ensure they follow architectural conventions and practices (or just something like ArchUnit). Realistically, how close to a finished implementation must you get and how far backwards from that can you go before its output degrades too much?
I tried figuring that out myself on a personal project where I mostly want what I want in GitHub issues and then feed them to Kepler, which spawns OpenCode in worktrees and also gives me a UI to follow the execution. Most models, even SOTA, occasionally implement things wrong, but it's suprising how far you can take things when you've spent a dozen hours planning out what you want the end result to be vs not to be (talking about the goals for an entire system, maybe an hour of planning for a more bounded type of task).
In general, when I’ve worked on projects where I’ve spent that much time planning, having a magic AI box that wrote perfect code wouldn’t have helped that much. When you spec every everything out like that, writing the code is the most trivial part of this whole process.
Decades ago, enterprises said, "If you're writing assembly you're doing it wrong".
That led to a reduction in knowledge of assembly in the average programmer but the people who specialize in it haven't gotten any worse at it.
The result was a generation of programmers who make useful software while very few of them understand the machine they program. You could easily make either a positive or negative value judgement about that result.
I came here to rephrase the headline as "Punch card expertise is going to collapse from compiler reliance."
Yes, it's true... but who cares? We lost that skill because we don't need it anymore.
English is becoming a very popular programming language.
The people who swizzle registers on GPU drivers need that skill, and there's no evidence that they lost it. There's a reason why we use the stack metaphor for a programming ecosystem. You trust the lower layers below the one you work in.
People are fundamentally short-term thinkers. Just imagine spraying neurotoxins on your lawn. You're "saved" from the "terror" of noticing more than one kind of grass, but of course you're incrementally increased the cancer risk for yourself, your children, and who knows who else. This is just one example; people do this constantly. Plastic fleece puts an untold amount of poison into the environment, but "I need to be warm RIGHT NOW." etc. Or lead paint. "It's only toxic when it breaks down, and that won't be for decades!"
Why would programming be any different?
All the issues you mention here have a death horizon: the people engaged in the behaviour will be dead when its effects are most acute. That they engage in it then isnt very mysterious.
That doesnt clearly apply here
The problem is further away when the next quarter, which for some companies, might as well be in the year 3000.
Up to now, manual coding has produced mountains of code, faster than humans can un derstand it! Already!
The nincompoops at the top think that if mountains of code can appear by snapping your fingers, they will be less of a problem.
If the mountain doesn't work, no problem; since it was so cheap to produce, just scrap it and snap you fingers again. The next one will be better.
Comparatively evaluating 2, 3, ... mountains of code is harder than trying to understand one.
Boris: "I don't prompt Claude anymore. I have loops prompting Claude and figuring what to do".
Boris: "I haven’t written a line of code by hand in, I think, eight months now… Claude Code, 100% written by Claude Code".
Boris: "There’s no manually written code anywhere at the company… All of the SQL is written by models. Everything is just built by the models... Claude instances communicate with each other (e.g., over Slack) in autonomous loops"
This does not sound like they review the code either. So, either the frontier labs like Anthropic have figured out something that very few companies could replicate, or they are being incredibly deceptive. I don't know which is true.
They work on research problems you can define a clear solution criteria for.
The kinds of business software I work on don't have those characteristics. If I needed something like a utils library, I think I could easily have Claude write the whole thing and not read the code.
As always, the essential skill in engineering is not providing the right answer it's about asking the right questions.
What puzzles me is this: research means that we are exploring something that has not done before, yet using Claude to generate code means slicing and dicing what has been done many times before. So, I'm not sure how to make sense of both at the same time: Anthropic is pushing the boundary of AI, yet all the knowledge and engineering in form of code can be generated from the previous code?
Judging by how it feels being a customer and using their products, I can confidently say:
1. Yes, we know, and can tell.
2. No, they haven't figured out anything. Just vibe coding it with their bleeding dege models.
> No, they haven't figured out anything. Just vibe coding it with their bleeding dege models.
I'm sure you were saying "bleeding edge" but my first impression was to translate dege to mean degen/degenerate which made this so much more fun to read.
Engineers, free from writing thousands of lines of boilerplate, now have more time to review code and be more efficient in that armed with LLMs. E.g. instead of asking a question about the code and waiting for the author to respond, I can ask an LLM and get the answer almost immediately. Or, if I see some strange code being changed, I am always curious why it was there in the first place and, instead of tracing it through git blame over formatting, renaming, and moving files around dirs I can delegate this to an LLM. If I have doubts about it working in some environment I can, again, make an LLM test it out instead of asking the author etc. etc.
And no, telling Claude to implement a Jira ticket is not worth $200K/year. Checking if it has not done something stupid and correcting it when it's trying to - is.
> We are indeed producing a shit-ton of code, but the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it. That sounds great until you realize that "hey Claude, read this Jira ticket and implement the feature in this code base" isn't really worth $200K/year.
My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
Our jobs have never been to write code. We’ve been saying for decades that LoC isn’t a rational way to measure engineering output, and now we have our chance to structurally change that system before processes re-solidify. In fact, I think the flexibility to adopt new systems and the experience and foresight to choose a path that is better than the status quo without throwing out everything we’ve learned is going to be what sets companies apart and makes individual careers over the next few years.
> we have leadership air dropping AI generated manifestos on the product owners and product owners having to use AI to transmute all that shit into 1,500 word Jira tickets that are 10% necessary feature work and 90% LLM boilerplate.
There’s an important criticism here IMO - the relationship between “business” and “engineering” is changing drastically. It’s going to be a challenge to set the expectation that just because Marketing was able to vibe-code a prototype in a day, actual implementation may well take weeks or months. Engineering should be considering things like security, scalability, and systems integration that aren’t a concern for Marketing - that’s why we’re being paid!
> So now you have software engineers whose job has changed radically to the point that the hardest part about being a software engineer is just filtering through AI generated artifacts from all directions just to try to get a feature out the door.
I think it’s an extremely heterogenous landscape right now. Where I work I’m struggling mostly with organization - keeping the (literal) dozens of inbound features that come in every day straight long enough to hook up an agent harness, review, validate, and deploy. At other companies the issues seem to revolve around their Agile-based processes. Or maybe it’s non-technical vibe-coders and their expectations. Or maybe it’s executive leadership flirting with AI psychosis.
Everything is in flux. It’s stressful and exciting, and I’m thankful to be around for it, even if I am in my 40s at this point and looking at the core skills I’ve built rapidly become worth exponentially less. It’s a huge opportunity for growth.
> My argument here is that what is worth $200k+ is the ability to distinguish the changes that must get thorough, critical review from those that need only a couple of specific things verified and those that require no manual review at all.
The problems:
1. The market is/has been saturated with engineers/developers earning well into the six-figures who are basically decent at writing code but not so good at anything else. So sorting out who is really worth these salaries is not going to be easy and the people who decide often lack the knowledge and skills to determine it.
2. You need to successfully make this argument to people who control budgets and more and more of them just aren't going to buy it.
> the reality is that engineers are pumping out code faster than the humans can understand and (honestly) review it
The word "engineering" means doing things using a repeatable process to get predictable results. If you can get predictable results (e.g. guaranteeing the absence of data integrity issues, security issues, anything that will cause downtime, etc.) without looking at the code, you're still doing the work of engineering.
Your job as an engineer is to choose high-value problems to work on, solve them in the correct way, and guarantee that the functional and non-functional requirements are met. If you no longer need to read the code to guarantee that it has the right set of ilities, then I'm not sure that's obviously a bad thing, as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
> as long as the ilities you're enforcing result in a codebase that is sustainably secure and maintainable over the long term.
Perfect example of begging the question.
Vibe coding works great as long as vibe coding works great.
Personally, I'd love to see how you enforce security and long-term maintainability without looking at the code cuz we need to add that to our build pipelines ASAP.
Mythical man month should still apply for AI coding. If you look at the code you're shipping, and you can't quickly edit it by hand, or fix a bug with it, you will be in trouble if the models are down, and your website is down at the same unfortunate time.
1. I think LLMs will end up pretty dramatically shrinking MTTR, a lot of that will be tooling to proactively resolve problems as soon as they start, but a lot of it is that agents are very good at finding and fixing problems by correlating data, something humans can do but I think we're going to be slower at 2. "The models are down" doesn't seem like a very realistic problem unless you have a single provider/set of endpoints. I do not recommend this setup but I guess if you have a single point of failure that is a risk.
// The need for ongoing friction in long-term skill formation.
The subtitle of the story tells it all.
There are some people who seek out friction. Think about an athlete or a hardcore nerd.
The best engineers are ones who were fascinated with computers and learning as kids and persued it at every opportunity. Found their own friction in other words.
For those kinds of people, friction-seeking is the constant and what LLMs did is moved the point of where the friction occurs.
For example - the best engineers I've worked with didn't necessarily have lots of experience coding in assembly because that kind of friction was no longer necessary. But they could solve hard problems (and if a problem really required assembly they could go learn it)
What I think will be hit much harder by AI is the low tier engineer. Someone who was never truly curious and committed to it, for whom it was just a job. For example a typical offshore ticket pusher kind of person. That kind of person never went out to find friction and that's the kind of thing that's never going to fly again - if I want mediocre or average, the LLMs are sufficient
I mean if you think about it, that sounds like a miserable existence to begin with. I would not rule out that some people will use this.. uh.. disruption, to start pursuing something that actually brings them joy.
Of course, bills need to be paid and it's not all that simple. But at least there might be such a silver lining.
I think this expresses what I've been trying to form in my mind.
I've been trying to map the LLM advancements and the current state of software development onto prior technological improvements. History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
For the people that fall in love with a single abstraction layer or don't have an interest in learning new paradigms, when their known pattern is abstracted away, they're condemned to being left behind, either unwilling or unable to adapt.
I don't think any industry is free from this, any person in any industry/profession over a period of 20 years or more has likely had to undergo massive adjustments as technology changed their field.
We're not unique, but that doesn't stop it from feeling so jarring when it happens to us
> History is littered with similar cases where the abstraction layer ends up getting lifted, and people struggle with getting accustomed to working at that higher abstraction level.
But there's no abstraction layer that ends up getting lifted. When I use a library like SDL or a standard like POSIX, I don't tend to look at the underlying implementation. Instead I work with the high level concepts that they come up with. There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
I've learned something at every abstraction layer in computing from electronics (hardware), theory of computation (software) to high level programming languages with their paradigms. Same with several domains embodied by libraries. LLM tooling is more like shamanic ritual than engineering.
>and everyone knows the quality of the result.
Yeah, a bespoke program that does exactly what I need it to do, at a speed that I had forgotten was possible on computers, with customization that is an exact fit to me, at a cost that is smaller than a rounding error.
I get that LLMs struggle with the old paradigm of a single piece of software meant to serve every conceivable use case of every conceivable user, but I kinda hope that paradigm dies.
> There's no such things with AI tooling. The most similar is when fully vibing software and everyone knows the quality of the result.
This is the same “but the ai can’t even do fingers” argument all over again, and it will age just a well.
The whole vibe coding thing only really started taking hold with the release of opus 4.5. That wasn’t even a year ago. you absolutely can be quality software without looking at the code. The only thing up for debate in my mind is how big that software can get before you need to look under the hood.
But that’s dictated by current model capabilities. And I don’t see any reason to believe that they will suddenly stop improving on this front.
With AI you learn contributor, project, and product management.
Exactly this. You still have to understand the abstraction layer the LLM is working at in order to be able to form an opinion about the quality of the code. (If you DON’T care about the quality of the code, there’s always been ways to get an extra 1000% short-term speed improvement by just not giving a shit.)
AI isn’t “a code abstraction layer” any more than Jira is, or the engineer themselves is.
I think coding experts who are only experts at writing code will also be hit pretty hard. Every problem given to them is solved by writing code, and only code, their way and only their way. I think people like that are having a very hard time relinquishing control regardless of the quality and correctness of the LLM output. If it didn't come from them, or conform to their conventions and style, then it's wrong. These are the people that seem to be terrific developers but then when asked to be a team lead and bring a group of developers to their level completely fail.
Curiosity is really key, and I find that LLMs diminish curiosity in people that might otherwise have it because the allure of the answer is often a lot stronger than the allure of the friction. I guess that's just the distilling of the industry over time, but that could certainly lead to a shortage of expertise.
If that is representative of your personal experience, then I'm saddened to hear it. I believe your article is an argument from incredulity [1]. You may not personally know anyone who thrives in learning from and applying knowledge with AI. But being unaware that it is even possible is not evidence that expertise will inevitably collapse.
[1]: https://en.wikipedia.org/wiki/Argument_from_incredulity
The snake eating it's own tail for llm software development has really been met with a shoulder shrug whenever it gets brought up. At best you might have a small cohort of developers that don't cook their brains with AI and their reward for that appears to be having to review terrible AI code written by people who have cooked their brains.
Completely unsustainable.
Maybe I’m naive, or misguided, or something else, but I decided almost 2 years ago that I’m going to bank on not using LLMs to do my job. I still write code by hand everyday. I don’t use AI to write code or lookup docs, or anything else. I just figure that if we really, truly, indisputably hit the point of AI assisted coding being mandatory, I can learn it and get up to speed in a few weeks (or at worst a couple months), whereas uncooking my brain is way way more expensive and time consuming and difficult. We’ll see how it plays out, but so far I don’t feel left behind at all really.
As a tech educator I 100% agree. LLMs are not going to become a "new compiler" where we don't have to worry about the code any more. There's a reason we trust deterministic systems.
I've been worried about this a lot, I even created an agent skill called do-i-understand that's designed for novice devs (and experienced too, because atrophy) where the LLM asks you questions about the PR you're about to submit. I've found it helps a lot: https://github.com/AnthonyPAlicea/skills/blob/main/skills/do...
One way or another, there will be a skill reckoning.
A deterministic LLM is possible. Would you trust it?
they probably mean deterministic in the sense of traditional computer programs consisting of if-else decision points and ordinary cpu logic, as opposed to numerical models (which can of course be deterministic too, as you pointed out)
Model inference is just that same logic but faster.
I think the point is determined. No-one can determine what the LLM might do.
I mean in the traditional software sense. We trust autopilots to fly planes. Would you get on a plane flown by an LLM?
Then I think I'd rephrase you: There's a reason we trust trusted systems.
:)
Even their fans agree, LLMs are inherently unreliable. The fact some people trust them regardless is due to a deep flaw in human psychology that I believe has not been significantly exploited by any previous tech. There will be tears.
I think two things contribute to the trust in LLMs:
1. Decades of generally reliable systems (I trust my calculator because it always says 1+1=2) has trained people to believe what computers say.
2. LLMs "speak" with great confidence, which has, as you say, a psychological effect.
There are 2 levels of determinism that matter here IMO. Level 1, given the same input you get the same output. Level 2, you know what to change on the input to get a specific change on the output.
LLMs are currently capable of level 1, but not capable of level 2. Trust follows what is actually deterministic.
Trust should, but instead is being gulled by those incapable LLMs. And I don't see that changing, given the typical human's seemingly unlimited vulnerability to this con.
Why do you trust the non-deterministic LLM with your skill?
It absolutely might make mistakes. But the skill requires all questions to be based on lines of code with specific line numbers which the human needs to review. That's intentional to serve as a safeguard against hallucinations.
It doesn't matter if the LLM is not deterministic.. the OUTPUT of typed code is deterministic in the sense that it can be seen to be type safe, compiled, and run appropriate unit tests on it.
If I use two LLM's to create some chunk of code and they both do it slightly differently but they both compile and pass appropriate tests.. it honestly doesn't matter if the LLM itself is not deterministic in exactly what it's going to output.
I would also argue- doesn't that make sense? You give two human coders the same task and they are also going to come up with slightly different results.
We don't run our own unit tests on compiled code because it generally can be trusted. LLMs are an extraordinarily leaky abstraction.
Saying "as long as it works and tests pass" suggests that we can test for every possible scenario. We can't. And tests can be flawed on top of it. Which is why an LLM is no more a compiler than a human coder (as you say) is.
The exact same can be said for human code... so what's your point?
I strongly agree with the concept that cognitive friction is the engine of learning.
First and foremost, it's an issue of "dependency": if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself.
A historical example that brought about a similar shift is this: when the production process moved from the craftsman's mind and hands to the Fordist factory (and the assembly line), the skill of building things shifted from human craftsmanship to anonymous, structured processes.
Bit by bit, traditional artisans lost their knowledge and "know-how." Today, having a piece of furniture in our home depends on a massive production and supply chain; the "average" person no longer has the ability to build it themselves.
The exact same thing is happening to software.
We are the (now "former") software craftsmen.
> cognitive friction is the engine of learning. I agree 100% too.
This is why in order to understand new codebase or to ramp up to new projects, I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way. Then I print them on a paper and read it using a pencil/pen and take notes.
Because of my math training, I am in the habit of slowing down to read textbook style texts which helps.
Exactly the type of usage I would hope to see advocated for in the future. As I wrote, when leveraged with Socratic methods and a sort of "interactive documentation", they can really boost one's understanding. Of course, this is predicated on the individual having enough information to know the right questions to ask.
> I use AI to generate a textbook style reading material for me along with "verify yourself" types of exercises along the way
This is fascinating. Any chance you have your workflow or prompts documented and would be willing to share them?
Great idea! Let me pull some stuff from my past chats, and put it in a github gist or something.
> "...if you stop training the "muscle" of logic and reasoning, it gradually atrophies, just like unused physical muscles. You become dependent on external tools that replace a capability you once had yourself."
For a bit of extra unease, consider that these people will retain their right to vote, both in making business decisions as employees, in shareholder meetings as shareholders, and in government elections as voters, despite their atrophied reasoning abilities. Who will wind up whispering what to vote for in the ears of these reverse centaurs?
theres even more recent things to learn from > https://en.wikipedia.org/wiki/Ironies_of_Automation
> Bainbridge argues that new, severe problems are caused by automating most of the work, while the human operator is responsible for tasks that can not be automated. Thus, operators will not practice skills as part of their ongoing work. Their work now also includes exhausting monitoring tasks. Thus, rather than needing less training, operators need to be trained more to be ready for the rare but crucial interventions
It believe the most likely consequence of AI is to promote generalists: people who have a domain expertise, can work cross discipline, and has enough programming knowledge to keep the LLMs on track. I don't think 'pure' software engineers will end up being as highly valued as the last decade though I also think that will be true for other disciplines as well. Just as an example, in signal processing, it's not uncommon to have a person designing the general algorithm and another person dedicated to implementing the algorithm in the embedded system. With the quality of coding agents, it's not really necessary to have both of those people anymore. A person who is moderately experienced in both can do the job now.
This seems sadly very true. Just yesterday I was implementing some slightly tricky javascript (not my main language) on hover show n neighbours to each side, and if a deficit on either side, expand to the other side. After about 20 minutes of struggling to get the offsets just right, I succumbed to just asking an agent to do it.
I'm sure I'd still be able to do it, but was saddened that I didn't get it as quickly as I think I used to be able to. Atrophy might already be in play.
TBH it was already pretty bad. There is a stark difference between the best and the average in my experience. The top, say, ten percent of coders are vastly better than anyone else when it comes to anything but boilerplate glue code(which is still needed and is better done by average coders anyway).
This is speaking from my experience as a systems/c/c++ guy. If you are a js web frontend guy, python, or whatever I have no idea if this applies to you.
Yea, it does apply. Honestly I'd argue that as you move up the abstraction layer cake, it applies even more. If a team gets a tool which 2x's their output (maybe a higher level language vs writing in C), then the difference in absolute output someone who produces 1 widget per day and 5 widgets per day becomes substantial. If you're given a tool which 10x's your output in a day, then it's even bigger. And of course you can have devs on a team who produce -.5 widgets per day.
Think about the worst colleague you've had, now give them a tool which 10x's their output :)
Yes, and it doesn't matter.
Writing code in programming languages is a skill/necessity created by us to instruct computers what we want them to do
Initially in the 60s, this was done by connecting circuits one way or another (think ENIAC). Then we devised "programmable" computers and devised a bunch of codes (computer code instructions) that abstracted away those cables.
The we created Programming Languages to further abstract away the hardware complexity, and to be able to write down our wishes in a way that is more transferable between us humans, but that is still computable by machines.
But with LLMs and neural networks, at some point these abstractions wont be necessary.
The computers will still be making computations, but the way we tell them what we want is going to evolve.
It's fascinating.
Not to detract from your point, but there were software-programmable computers in the 50s. Check out Usagi Electric's videos about the Bendix G-15, where he loads software from punched tape.
We are spending sooo much for our intellectual time and energy on discussions about alchemy-like experiments with different models, agents, harnesses, orchestrations.
And there are huge amounts of time, energy, and writing sunk into arguing about AI or trying to convince people of the danger or problems.
Sadly all of this is taking time away that could be spent on real progress and learning about coding/FP/PL/algorithms/type theory/etc.
I'm looking forward to having the same experience as COBOL engineers did after retirement: companies backing up the brinks truck to anyone with the skills to keep the lights on.
> backing up the brinks truck
now that's a phrase I have only heard in one other context: as an alleged quote from former Boston Celtics player Isaiah Thomas (who was really about 5'7 despite his official listed 5'9; yes, professional basketball player in NBA). This was around 2017 when he had just come off a career year and was soon due for a contract extension in the low 9 figures. He got injured and bounced around for a few years before eventually leaving the league, and never got that payday.
that would be nice, but unfortunately I think the supply vs demand situation is quite different this time
Yeah, every COBOL job paid pretty terrible relative to other types of work (like web dev).
Sure there might be a dozen individuals making over $300k but most COBOL devs live in the deep south making less than $100k for their entire careers. Fine salary, but that line of work is severely limited and the salaries do not reflect how valuable that work is.
If you actually look at Cobol job listings, they pay like 50k and want a master's degree. The myth of them being paid really well is not true at all.
Most computer science programs don't teach "coding" though. There is pretty heavy emphasis on datastructures, algorithms, and system design. I don't think AI atrophies those skills / knowledge as much as the coding language and I think in many cases actually improves them, especially if you are reading the code that is produced and at least understanding the flow of it.
I agree that AI reliance is hurting engineer's language understanding, but I don't know how much we should care.
I don't think that's quite correct; every CS program I've seen involves writing code, quite a lot of it. What they don't teach is /software engineering/: working large codebases, maintaining code, dealing with legacy code, long-term collaboration, working with product managed and designers, etc. That's stuff that you mostly end up having to pickup in the real world outside school.
Writing code and being formally taught a coding languages syntax and toolchain are quite different.
I did B.S. in software engineering instead computer science in college, and there were classes that indeed taught the industry software engineering process like you mentioned.
I often think back to the early GPT days before agents as maybe the last time I will deeply learn a technical subject. I think that was when I actually learned the most as I had to stay completely in the loop. I had recently started a new job and had to work with k8s for the first time. Using GPT to help me implement new services and help me diagnose and fix issues with kubectl taught me so much.
Now with agents I don't have any insights into what its doing with kubectl and would have no reason to learn how it works (other than my own curiosity).
I agree that the human no longer gains expertise, but I am not sure that will matter in the long run.
The end result is a total and complete reliance on AI providers. It's not really a secret; that's why I started the article with the direct quote from Altman. They would love to see a world where nobody is able to do any type of development work without an LLM subscription enabled...even though they also admit that these models perform best when they are steered by someone who is highly capable and knowledgeable in the first place. They're powerful, but they are also VERY fallible.
Local AI would thwart this vision (which is why they're desperately also trying to ruin the hardware market at the same time), but the effort it would take to get local models to run as well as these frontier models is an odd thing to pursue just to avoid learning the fundamentals and gaining direct experience.
yes, this seems sadly very true for me, too.
In some ways, it feel like I'd be "stuck" at my 2024 capabilities if I didn't push myself constantly, and I'm really glad for every hour I spent learning/applying new stuff before that, because now it feels way harder.
So far I think I have learned far more from LLMs than I've lost to them. I forget some syntax, definitely. But I now reach for a much wider range of tools that I have become familiar with because of LLMs.
So, partly agree, partly disagree.
Yeah, I have to say some of this deskilling argument sounds like Socrates complaining about the invention of writing ruining people's ability to truly own the text. I really don't mourn my lost assembly skills or my lost C skills. And I'm not missing the details of whatever web framework du jour my Claude is vibing for me on my extremely useful dataviz internal tools; and I have learned how to rein the agents in to not ruin my high-performance Go code and yet debug concurrency issues. Maybe coding qua coding is what some people find their summum bonum, but for me programming is instrumentive - to build real things in the world that do things to make something better. When I want pure virtuosity of intellectual construction, I read maths papers in arxiv.
(Not to say that the current state of the art with LLMs merits borrowing money for a multi-trillion spend on hardware that will be obsolete in 5 years, but for me that is a different issue).
The master decrying the invention that resulted in more wisdom for more people than anything prior:
"For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem [275b] to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise." (quoting from https://www.historyofinformation.com/detail.php?id=3439)
Yeah. Thought in Socrates' defense I think the jury is still out on writing, in the scale of human existence it's still very new. Give it a hundred thousand years or so.
I posted this elsewhere, but that story has a lot more nuance.
Plato (who actually wrote these quotes, because Socrates only spoke) wasn't against just "writing" in a general sense, or that he felt it was going to hold humanity back. He was against treatises and felt that someone could "memorize facts" without having to actually think about them on a deep level. Which I think, ironically, is not all that off the mark especially in the age of the internet and now LLMs, right? He also felt that memory could decay if we relied on written facts instead of having dialogues, which again, not all that off the mark, either. He underestimated the compounding capability of technology and our ability to record data and information, but people remember a lot less these days than they used to because we'd largely given up that ability in exchange for the instant information machines.
Of course, he's just one man (or two, if you consider he was conveying Socrates' thoughts, as well) who lived in ancient times and couldn't possibly foresee how technology would evolve...but when I went back and read his concerns, it was ironic to admit that much of what he warned about still applies to this day. People are far less informed despite having access to more information than we've ever had because we've exchanged the ability to remember for the ability to just look things up. And, we also have less meaningful dialog than ever before; people just sit on social media, copying and pasting "facts" to each other, instead of having actual productive discourse. When I look around, the world doesn't seem brimming with mindful critical thinkers and there's reasons for that (many reasons, of course).
AI tooling can be a boon to learning, but it requires us to stop using them for code generation as a primary purpose (at least for juniors) and instead advocate for Socratic workflows that still require manual coding practices. When someone decides to code something, it's not just learning syntax; it engages a variety of mental disciplines from critical thinking to planning to creative problem solving to logic and math, even.
May I ask how many years experience you have?
I fall into the category of senior engineers who benefit from LLMs for all the reasons mentioned in this post. I find it's possible to agree completely with sentiments like this and still feel as if this is all written in the sand below the high tide line, and ten years from now nobody will care about this.
Horsemanship and sailing were both specialized skills of high value to society, and now they're not. But in each case there was probably a liminal period, when being an accomplished horseman or sailor was still valuable, even as motors were taking over. Eventually that period ended, as the new generations without those skills found ways to get by with cars and motorboats.
This analogy breaks down because computers aren't going away unlike horse drawn carriages or sailing vessels. Computers aren't being replaced by something. It's that fewer people will understand how they work while the efficiency and reach of computers extends into areas of life previously unreachable due to the scale of having humans build it or the refusal of humans.
I could wax philosophical about what that leads to but enough people already have.
Boats didn't go away with the demise sailing as a commercially viable mode of transport.
If my horse is ‘a useful piece of software’, what are the cars and motorboats that AI coding is going to give me?
A faster horse.
Forcing friction the way this post describes won't work because the genie is out of the bottle.
A much easier way to force friction is to expect everyone to read the code they are producing. Yes read all of it and understand what it's doing.
A lofty bar sadly.
And not just the high level language output, or even the assembly. No no, every single piece of microcode produced.
Great read that brings some rigor (or at least cites rigor) to what has otherwise been anecdotal.
> then what is the path for one to become an expert so they can effectively use these tools?
The article briefly mentions guard rails and I think on the same note we need the AI agents we use for coding to encourage learning and comprehension in its users. Right now it’s completely voluntary and on the onus of engineers to put in the effort to learn since, as the article states, the friction that led to some degree of required learning is gone.
I don’t think there’s a big societal problem here, except that the agentic programming developers may be out of a job like the rest of us. If AI continues improving it will just keep taking over whatever skills are involved in the current AI coding meta are. If it stops improving then the meta will stabilize and after 10 years everyone will have longevity in it.
I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
Depends on what you are programming. I have been using it to build things that have never existed before, that I have wanted for over a decade. Experience gives one more imagination.
Me too. I’m failing to connect the dots from your post to mine though.
> I also see no evidence that AI programming is a difficult skill that cannot be learned by any intelligent person in much shorter amounts of time than previous professional skills required.
I do not think anyone could do the job I do with AI without my decades of experience, is my point. For now.
I’m assuming you’re a software engineer.
The AI programming meta that keeps evolving (last month it was loops) seems like a relatively trivial skill to adopt compared to learning what it’s like to do high quality work and to participate in a team, and these kinds of things. If the AI stops improving then everyone will master the AI skill set that everyone else is using, and people will acquire maturity and experience over time, like always. This is my point.
There have always been opportunities for those who seek out the less traveled paths.
If all your peers are only accepting what LLMs tell them then you should do the opposite. Spend some of your time writing code by hand, especially when working at a new layer of the system or on new things you've never looked at before.
While the agents churn out more CRUD crap in the background spend your attention learning how everything works.
I do way more experiments with way more things by taking advantage of LLMS. When I have them do research I use that as a starting point for my own reading (but don't blindly trust them).
I find I am learning more faster.
Change is an opportunity. Use it to your advantage. Let other people while and moan while you get things done.
Not just coding expertise. Wrote this a little over a year ago [1] on this exact topic.
[1] https://web.archive.org/web/20250623100834/https://ryanglove... (sorry for archive link, have a backlog to migrate old posts to my new setup).
I think we now entering this state where code is cheap but good well made programs will be rarer then ever, a lot of crap can be made and a lot of programs made without ever thinking the problem through properly will occur.
It might be like how an archeologist since the invention of plastic can date the period of the soil as post plastic, if one could cut open the software stack of the coming systems that will be built in the near future one would could data that code as post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
I am not Anti-AI but I think it's going to be interesting and I am surprised at how bad a program ends up when one tries to "vibe code", though often ends up working, though in isolation and when used with discipline (that the tools themselves psychologically make it harder to do) can produce some very good code and being able to use loops to solve difficult problems or problems that simply would have require banging ones head against the problem many times is very profitable.
How many programs can dispense with 50% of their complexity by just offering an API for LLMs to execute?
By way of example I use Kdenlive 90% less and just use Codex to issue the ffmpeg CLI commands directly when I want to crop/format video on my PC.
All those crazy complicated CLIs are now super useful for LLMs!
Yes every time I tried to understand ffmpeg I got a little of it for a little bit and then lost it by 2 days or so. Just getting LLM's to do their thing for ffmpeg is great. the other one is socat an utility I used before but LLM's can do magic with it.
> post agentic LLM as programs are going to be mostly bloated ad-hoc, poorly thought-out and patched in a way that does not concern it's self with the correctness of the algorithms or data structures chosen.
This is an accurate description of 90% of corporate software pre-LLM.
LLMs are introducing a unique kind of bloat and dysfunction but it's not like they are replacing pristine code. Most software that is subject to market pressures and is generating a profit is painful to look at and difficult to work in.
I wonder if we will eventually see the same drop in general knowledge we saw with manual driving (in some parts of the world), cursive handwriting, clothing repair, etc. Of course, coding was never as prevalent as these practices in the general population, so it might be harder to measure, but I feel we will definitely see it taught less in schools, harder to find resources, etc. in the coming decades, becoming more of an underground hobby.
It is guaranteed to happen. The key question is when is it a problem? We've lost lots of knowledge as a society but we're still going. Some skills decline in usefulness, others gain utility with the passage of time. Tale as old as time.
I would hope as long as there is a demand, there will be those willing to fill it. There never would be a problem because the skill set would stick around as long as there is the need for it. At least that has been the historical trend.
We still have horse farriers for show horses or for family pets. It is a niche there for those that need it, but nowhere near as common a job as it once was. It will likely be like this. Those that like doing it will stick to it, but we will not see Indeed listing jobs for it as much. Of course with less demand, less will have the skills, so those that do have them can definitely command quite the power when they are called on for something.
I hope that this will expose the true value of education. It's a new scale perhaps, but not a new problem. We sit in classrooms even though books and YouTube videos make available the teachings of far more illustrious teachers than the one in front of us. We watch things written out agonizingly slowly in chalk. We learned to do algebra and calculus even though we had our TI-86s and Mathematica. We learn in Haskell and Scheme even though those are rarely used later. We write toy compilers and write essays and repeat experiments with known outcomes in labs. We do it to learn how to think, how to build. We just have to be even more deliberate about it now.
I’m still resisting the AI flood. I use it but I do read code without its help. I also make sure I have the muscle memory of stuffs that will be tested in interviews — somehow big techs have forgotten that maybe they shouldn’t test people on coding problems.
So basically I’m still doing things manually for everything that preserves my edge for interviews and as a technical person. But I use AI to generate diagrams and documentation, as well as operation scripts that no one is supposed to memorize.
Does anyone else feel like the web has become so "readable" it is illegible?
I have such a hard time quickly reading pages like TFA. Headings, bullets, line-spacing, width, link hover animations, even the font. Even reader mode defaults aren't great for me.
Compared to, for example, https://www.paulgraham.com/best.html
I'm still waiting to see how LLMs have made the world a better place because it honestly feels like its done the opposite.
I think a lot of good might come from medical sciences or chemistry, where LLMs will be used in coming up with new medicines, methods or synthesis.
I can totally see where that could be the case. I'm sure there have been breakthroughs already, but it's not mainstream news. The mountain of AI slop is all over the mainstream though. You have to wonder why.
They haven't. They have made the world significantly worse.
> This applied friction is directly what builds "developer intuition" (or "taste"). The Germans have a great word for this: Fingerspitzengefühl (fingertip feeling). It’s the muscle memory that triggers when a developer looks at something and thinks, “yeah...this is probably going to cause problems.”
No, we do not have that word for that.
Yes, we have that word. No, it does not mean that.
Any LLM proofreading could've told the author that, for that matter.
__
What Fingerspitzengefühl actually means could be described as "tact", a precise approach to something and general attention to detail that leads to success. Or.. the lack of all of that leading to blunt failure.
__
I would of course comment with more Fingerspitzengefühl for the emotional needs of the author if the writing wasn't just a sales funnel for their courses and whatever else (+ posted by them themselves.)
Alternate take: How many people have known how to program in assembly for the last 30 years or so? It's been a skill on the decline. Good compilers made that happen.
My personal antidote is to write unassisted Zig in my spare time (though C would be fine as well). While I can't back with data I certainly feel sharper.
> JetBrains, a major player in developer tools, recently completed a study of junior and novice developers
The article links to this as the study: https://dl.acm.org/doi/epdf/10.1145/3632620.3671116. It doesn't appear to be related to JetBrains in any way?
I originally found the paper through this video:
https://www.youtube.com/watch?v=HTUh0OO6Kmo
The video seemed to indicate it was a study they ran (she refers to the examples as "our subjects were two of twenty four participants"). It seems they are just citing the study. While it doesn't change the study or the takeaways, I'll update the post for clarification!
Software engineering >> Coding. How many times do we have to keep repeating this. Author wrote a fucking marketing piece.
Stop the FUD for profit
This happened before with “real” engineering. Engineers a hundred years ago used to be proficient with slide rules and mental math and 3D assembly by hand. Products designed in that era often lasted a hundred years, like Singer sewing machines and Lada cars and 500 year old churches. Now things aren’t built like they used to be, and few people can rebuild civilization by hand like they did. But the world moves on fine!
It's different though because for structural engineering finite element models (set up correctly) will give you a deterministic, mathematically correct answer.
In fact FEM is arguably more deterministic than the old school pen and paper methods (yield line analysis etc.) which often rely more on correct judgement of the structural engineer doing the analysis.
So I don't think the analogy quite holds
Substantiveness of this article notwithstanding, I do want to point out that the author also runs (or is planning on running) a programming course which to me may be a slight conflict of interest. I think the author may altogether be well-intentioned, but it IS something that I'm going to keep in the back of my mind as I consume this content.
when the tool becomes a crutch you end up with a limp.
at work, i can tell that people are not thinking and losing the ability to thinking hard.
I keep coming back to what LLMs "really" are. Recursive autocomplete, it keeps choosing the best fit for the next token. This really demystifies it for me. It also reinforces that is not "reasoning", so there's no hidden mental model.
>at work, i can tell that people are not thinking and losing the ability to thinking hard.
its also wild how quickly and willingly many people handed this over. the apathy and desire for connivence runs extremely deep among my coworkers. we are at the point where people are not even running their local dev servers and clicking around to ensure their work is functionality correct, they just ingest the ticket, dump it into the model, commit the output, and say they're the most productive they've ever been.
reminds of college when everyone would use chegg, but knew it was a shortcut.
the difference now is it is shamelessly encouraged.
one interesting topic is code reviews of AI code. code reviews have always been a bottleneck on the team and there's lots of trust and rubber stamping. I don't believe people are reading the code they ship.
I will say that it is a beautiful tool if you turn it on its head. if you don't quite understand something, it will never tire of answering your questions. It is a great time to need handholding.
Writing code was never the job. It was a small part of the job that can now be delegated to AI.
Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage. You know `Engineering`. Mechanical and Electrical engineers didn't go away because of CAD.
> Eliciting the right requirements and user needs. Creating a sound architecture and design. Validating final product meets the requirements. Ensuring there is good test coverage.
Besides your first point, the rest can be delegated to AI as well. And you don't need to be a "software engineer" to spec out requirements and user needs.
This is happening at my company: all of these software phases are being delegated to AI. And the people without any understanding of software have absolutely no idea what they're doing. Features no one needs are being made in minutes. It hasn't been long enough to really tell how that's turning out. Everything seems to be in endless development.
Would you work for less money and use AI ? Or would you step out - professionally - of the industry and program for passion and fun ?
I think that is a question we are all mulling about these years.
Our expertise has been packaged into a 200$ product and the competition has become ourselves.
It's pretty simple.
You treat the AI like a junior.
You make it write pseudo code in the tickets.
Then you make it write code in the tickets for the tricky stuff.
Have it reference relevant documentation/APIs.
Then there is no lost knowledge.
(Still, sometimes it sneaks in some "helpful" belt and suspenders, but for the most part, I know everything that is going on in the code base with this method.)
The boss says we should do it this way. The boss hasn’t got a clue about how to do it. But he knows what he wants, more or less. And we can fill in the details. And we don’t make a stink about him not knowing exactly how things work. That doesn’t serve anyone. And he still thinks that he knows how to do it. Because he tells us how to do it and it gets done.
Now everyone can be the boss.
Sewing expertise is going to collapse from sewing machine reliance
At some point it won’t be realistic to ask a human to solve a technical problem any more. I’m not suggesting it will happen soon but I can at least imagine that day coming. Whereas before modern “AI”, that possibility wasn’t even on my radar.
This just feels like someone whining about how ORMs result in $#1tty data access code because lazy programmers don't take the time to learn how to use a database.
When that happens, (developer thinks using an ORM means they don't need to know how to use a database,) the project eventually fails. It's the same thing with AI; just like companies learned that they need to make sure developers actually know how to use a database, companies will learn that their engineers actually need to know how to program.
I am not personally worried on this front. It is a near-certainty that something catastrophic will happen in the not-too-distant future because of the issues Lars raises in the essay. Not until that moment occurs will there be a broader industry-wide reckoning. Most likely it will get expressed in breathless language like "you'll never believe the crazy gains we got from adding tests!" (or whatever).
I feel pretty good asserting that, as always, a few smart leaders will recognize early that quality and stability are the new game now that we have speed 'for free.' And then there will be incumbents whose eventual demise will merely be accelerated.
A list of things, that were supposed to lead to our intellectual demise:
- Writing (starting from Platos Phaedrus)
- Printed books
- Photography
- Typewriters, keyboards
- Telephones
- Recorded music
- Television
- Wikipedia
- GPS navigation
- Smartphones
- Automation in aviation, medicine, industries
If current trends are anything to go by, who's to say that these aren't stepping stones towards it?
Dear lord
Speaking on a handful of these that I'm familiar enough with to opine on:
- Photography hasn't erased other visual arts. Both require a separate skillset; both have their uses and their enthusiasts. Photography benefits from having a lower skill floor, in my experience doing it, and this in turn has led to commercial photography eclipsing commercial painting (barring things like painting buildings) as a viable business model, largely.
- Television is arguably not a net positive in society (I won't say net negative yet, but there are some reasonable arguments for that position). It enables a form of passive consumption that allows bad arguments to come through as persuasive through the gradual (on the scale of months and years) numbing of critical appraisal of the media being consumed.
- Smartphones arguably reproduce and amplify this problem, and tack on others: they make it increasingly easy to dissociate from daily life, and make it easier to continue consuming content from actors increasingly interested in steering conversations in one direction or another (on the most benign end, it's influencers trying to get you to buy things, which obviously has negative economic impacts on an individual scale; on the more overtly harmful end, they serve as a hardware channel for less-regulated or unregulated extreme and hateful content).
- In aviation, the first generation of (relatively modern) automation did lead to a high number of accidents and loss of life. American Airlines flight 965, in 1995, is a well-studied case of an over-reliance on flight automation before it was widely understood among pilots what the limits of these early-modern incarnations of autoflight were.
I think that last example (and many aviation accidents) are an interesting example to study in the context of software. Aviation's extremely and conspicuously responsive to accidents, often requiring procedural and hardware changes as a result of these. They can be as simple as a software patch, or as complex as the redesign of a component because a human factors analysis found that it could be confusing under stress.
In software, we like to think that we're a serious industry, sometimes. We spent much of the 2010s going in what I found to be a good direction: the development of IaC, the popularization and standardization of tools like Git, registries for vulnerabilities like the NVD, and even the development of languages like Rust which address historical technical shortfalls of high-performance, low-level languages like C; but we always fell short, industry-wide, on matching these toolchain upgrades with technically-rigorous procedures.
For every borrow checker, there's vibe coding.
And so what? How do you think people are going to be working on software in 10 years. They're going to use coding agents and harnesses that work much better than the current ones (although I don't think we're getting anything but incremental improvement from here on out). They will run on commodity hardware. I first got paid for writing code in 1999, I've worked at publicly traded software companies and failed startups, in every language out there. And if LLMs "atrophy" the expertise I built up over those years, then so what? What is lost, exactly? I stopped programming in C++ about 10 years ago when Rust seemed to get good. I no longer remember a lot of the specialized knowledge I had in C++. My life goes on. I am now becoming an expert at building software with agents. The problem is still the same, you still have to apply engineering principles. You just don't have to write the code.
Wasn't everyone making these arguments for the last 50 years with each wave of advances and convenience? "Real programmers use [current_tech - 1]."
I mean, they were right, then, too. My meaning is that expertise already collapsed several decades ago.
--
One of my favorite talks of all time:
Jonathan Blow, Preventing the Collapse of Civilization (2019)
https://www.youtube.com/watch?v=ZSRHeXYDLko
I was thinking about this anti AI sentiment and how it is related to American individualism. In socialist countries like China there has never been a protest related to data centres or job loss etc. unlike in the USA. Because the large scale impact of AI on the country is exclusively a net positive. The reduction of the individuals cost of labour by also reduces perceived egalitarianism which is the mental model used by ultra capitalists to allow themselves to sleep at night whilst homeless people beg for food on the streets.
The long term impact of AI on society is exclusively a net positive if utilised in the correct way. But most of the westerners lack faith in government fiscal policy and think that they are going to be left destitute when they can no longer provide for themselves. When in fact UBI is going to be the most likely outcome.
They are myopically focusing on the short term labour market effects of AI on their own profession instead of realising the long term utopian societal outcome.
"We need to ban AI because I'm going to lose my job!" - This is only said by someone who is winning in their society and wants to maintain the status quo not by those who slipped through the cracks.
[copy and pasted comment from a similar post]
It's surprisingly late for him to notice. But wait til you think through the consequences of the already collapsing human capacity for extended written composition, which is orders of magnitude more significant.
While true, I'm not sure it matters.
It kinda seems like wishful thinking on our part as software engineers that one day we'll be able to go "Hah, miss me now?" but that just isn't the writing that I see on the wall.
Over the past year, I've AI-generated two large pieces of software over 2000 commits without having much depth in either domain, and my ability to deliver value as a veteran software engineer has only shrunk over the months or it can be trivially extracted into a reusable markdown file like "design bar: ensure things 'by construction' where possible", a lesson I learned viscerally over decades.
> without having much depth in either domain
I don't mean to pick on you, but to present this in the service of this kind of fatalism... you know the inevitable follow-up question, right?
Is it: "How do you know the solution is good?"
Because I can still reason and dig into things and ask questions and have the domain explained to me out of curiosity, the same way I could build anything that was originally out of my depth.
But what I've noticed is that the need for any corrective power has gone or is going to nil and I'm mainly doing directional work. And even then, I can constantly have sota models "rank the top options and recommend one", and it's almost always the right way forward.
I used to read every line of every plan that LLMs generated, but over time I've realized I have less and less to correct or, more impressively, the LLM had foresights I never considered. The same thing happens with implementation.
Periodically I can spawn a bunch of agents to evaluate the system adversarially to find improvements, and the findings have been so good that it's evident that soon I can just automate that too.
In other words, the skills I need to excel here are more curiosity and patience than tech expertise -- the things that got me into software engineering in the first place since that was the only way to build things.
You got it. You may well be right, but from my standpoint it's hard to see how you'd know if you weren't. Yeah, the models fill in vagueness with detail sourced from "what's likely and helpful". (In one of my particular domains, that's "use SHAP values for everything everywhere".) Yeah, that detail's increasingly unlikely to be "wrong" in a way that you could explicitly argue. Is it what an expert would come up with? Is it the best it could be? Is there something else that we would have done had we been forced to think it through? Will it ever be _surprising_?
I know, all irrelevant little philosophical roadbumps. The artifact satisfies the buyer. I think I'm going to have to step away from this industry pretty soon, at least for a while.
Ever since I was a teenager, I started to develop cyclical bouts of existential crises.
While I have been feeling better for quite some time, I feel another starting to brew. While I was never one of the fabled 10x or rockstar developers, I still derived a lot of enjoyment and and sense of purpose from my work.
With the utility of LLMs becoming more ubiquitous by the day, what purpose can I provide anymore? I struggled greatly to get where I am today, and while it might not be impressive to any of my fellow programmers, I am still proud to be a programmer.
I obtained a degree in CS because I found the entire field to be fascinating. I had great passion for learning many of the subjects. What I learned in my degree changed my life forever. I seriously went from one of the, "What's the point in learning most math if you never use it?" to being absolutely infatuated with the subject. (I'm still bad at math, but that does not detract from my pleasure).
With the utmost certainty, I did not push through all my struggles just to prompt LLMs all day. Maybe's its my insecurities manifesting, but I want people to say, "You did a great job" and not "Claude did a great job." I understand the Buddhist were likely correct. My suffering really does come from my attachment, but at the same time, I do not want to let it go.
I have no opposition to using LLMs as tools. I think a fine balance can be achieved. LLMs can even be exceptional learning tools when used appropriately.
If programming is just a means to an end to you, then you will receive no ill judgements from me. We all have different motives in life. But programming has always meant something more to me.
As always, the innovators will thrive and become the new experts in the field. Everyone else will die out.
It's impossible to buy the FUD that expertise will somehow vanish because people use AI. People come in a variety of capabilities, challenges, and personal interests. Expertise varies from person to person. Those with personality traits that are more likely to take risks and persist against adversity will be the innovators and experts. We've seen this played out countless times.
You have to ignore everything you know intrinsically but can't articulate about innovation, technology, and evolution to drink this FUD Kool-Aid.
I'm seeing it happen in real time. I've seen several folks claim they "vibe-coded" something and upon review it's some seriously low quality garbage. Unfortunately it's just "functional enough" where it's encouraging the one who talked to the robot to continue thinking they can code for some reason.
It is important to ensure that any LLM code is named and shamed.
> It is important to ensure that any LLM code is named and shamed.
You need to take it further.
We need to shame and mock the people (read: delusional imbeciles) who think this is acceptable if we want it to stop.
AI users need to fear for their careers and reputations if we want to get software development back on the right track.
One interesting feature of vibe coding is that it allows you to write code in specialized areas where you have no expert knowledge. That makes the code effectively impossible to review in any useful way.
For example, I have a vibe coded Python monstrosity which I use for stitching together scans of large format negatives. The enlarging lens that I use for scanning has strong chromatic aberration in the red channel but is otherwise very sharp†. I got Codex to figure out some way of using the green and blue channels to sharpen the luminance while preserving the color information from the red channel. I sort of vaguely understand how this works, but not nearly well enough to review all the horrid numpy code that implements it.
For my use case, there is really no reason to look at the code, as I am only interested in the output. As disconcerting as it is, I feel that this is going to become more and more normalized. Looking at the code that your LLM generates will become like looking at the assembly output of your C compiler.
† This makes a lot of sense, if you think about the color of a darkroom safelight...
Coding expertise was on a downward trend way before AI arrived. Does nobody remember how most "engineers" were literally copying and pasting snippets from StackOverflow they didn't understand, and the snippets usually had tons of bugs? How many of you know Assembly well? How many of you can decompile a program, or write your own compiler or VM? How many of you could pass a computer science course (that you didn't take at Uni)?
We've been hearing about the demise of skills since before the invention of the loom. The only skills we've truly lost are for things nobody uses or makes anymore. What actually happens is, we develop a new tool that's better than the old tool, and we get good at using the new tool. We then have a class of people whose job it is to maintain the new tools. We don't need that many of them because the whole point is to eliminate that labor.
Your jobs as software engineers are going away, it's that simple. Your new job is to use the AI tools to make products. Somebody else's job will be to make the AI a better software engineer.
I hate to day this, but most coding work isn't hard at all - for a human or AI. There are a lot of important skills around coding, but for most people, much of the time, they are not doing any coding that is particularly challenging, difficult, it novel.
If you have a truly hard problem - sure write some code or take a hybrid approach. Still, let's not pretend this is what most people are tasked with.
Coding expertise is going to collapse from compiler reliance (1985 Turbo Pascal detractors)
"Assembly expertise is going to collapse from compiler reliance".
To be fair nobody codes in ASM and things are fine for decades.
ASM is still used in some areas, such as:
1. function trampolines, such as in libraries like ATL (Microsoft's COM template library);
2. hand-optimized encoders/decoders for things like ffmpeg;
3. niche applications such as Ben Eater's 6502 breadboard computer (using a modified WozMon and Microsoft BASIC);
4. reading ASM output from C/C++ code in things like the GodBolt site for performance comparisons/analysis;
5. reverse engineering old DOS games;
6. people working on things like compilers (generating optimized assembly), JIT engines, or other code generation -- though this will typically be machine code, possibly with assembly instruction comments;
and other performance critical places where assembly can speed up that part of the code.
ASM to a higher level language is a jump to a clearly defined higher level language through a deterministic compile step. The jump away to LLMs is something completely different.
Keep in mind also that LLMs are currenctly heavily subsidized. Once VCs money are gonna run out, you will see the real price, and your 200K per year dev is probably nothing.
I learned BASIC, Pascal and x86 Assembly language by the age of 14-15. None of which I remember (nor care to). The strict instructions (aka language) we gave the compiler was not the point. Those experiences taught me how to think logically, and over time develop a taste/structure for how software should be built, as well as foot guns to avoid. I don't know how you teach that without coding, but the "coding expertise" was always a means to an end. LLMs are just another abstraction layer above programming languages, machine language, etc.
> I don't know how you teach that without coding
That's the neat part: you don't!
Even Robert "Uncle Bob" Martin, who's as AI pilled as they come and doesn't read code any longer, says that junior developers shouldn't even look at AI tooling for the first three years:
https://www.youtube.com/watch?v=RxxxGkFIUJ0&t=1356s
Robert Martin has caused me so much frustration over the years, but he's spot on. "Young people with power tools tend to lose fingers"...I like that!
> LLMs are just another abstraction layer above programming languages, machine language, etc.
Programming languages are (largely) deterministic. LLMs are not. The boundary between an abstract concept and something executable sits comfortably in the mind of the programmer in non-LLM contexts. This boundary shifts when using an LLM and becomes shared between the programmer and the matrix multiplication machine.
The "layer of abstraction" analogy is therefore technically true but essentially false. "Just another" papers over how seismic of a paradigm shift all of this is.
That's the whole scam. It's a ransom with more steps.
First they fuck up your codebase with so much slop that no human does (or could) understand it.
Now if you want to keep making bug fixes, you're on the hook for whatever Anthropic wants to charge you.
The solution is to make AI developers the scapegoat. Fire the people at the top of the AI leaderboards, with malice. Otherwise, lazy AI-addict developers are going to destroy what remains of your company's IP.
"Punching card expersite is going to collapse from interactive terminals reliance"
And math skills are going to collapse from calculator reliance.
"Mental math skills" literally did collapse tho??? "Math skills" to write proofs (which a calculator cannot do) is a fundamentally different skill set than that "math skills" of mechanically doing arithmetic (which a calculator does do).
Seems like critical reading skills have already collapsed.
This is exactly the right analogy. Granted LLMs are a more sophisticated calculator and engineering is still adapting to this wonky/evolving technology. But to pretend it's a coming catastrophe is to ignore every past parallel development - where this has not happened.
Math skills are at their lowest point ever in modern society, and many are still advocating for the reduction in reliance on calculators for this reason.
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
Math skills had never anything to do with arithmetic. Maybe Ramanujan discovered a couple of theorems by dealing with numbers themselves. That is it.
Online discourse though is collapsing under dumb AI talking points.
Actually, they have for a large majority of the population. And fun fact: that debate is still going on, and the Mathematical Association of America (MAA) enacted a rule prohibiting calculator use in AMC.
https://ivyleaguecenter.org/2024/03/12/over-reliance-on-calc...
Math fosters critical thinking, and we've been in a critical thinking deficit for decades and decades. I don't think we can afford to sink even deeper...
> we've been in a critical thinking deficit for decades and decades.
Compared to when?
I have heard stories of cashiers being incapable of giving correct change. So yes it has happened.
I guarantee you that it's always been the case.
Not really the same, since your average calculator doesn't take over executive function. The user thought and decided what needs to happen, down to one extremely well-specified task, which is so unambiguous that it can be completed by rote and without context.
This is more like paying someone else to read your math homework, generate answers, and maybe even submit them on your behalf. If that had been the (average) student experience, math skills really would have collapsed.
A more apt analogy is the diminishing understanding of assembly language and machine instruction sets. I love programming and something got lost in both transitions, first to high level languages and now to coding agents. I am quite sure some contexts will require programming by hand for a long while as there are still people who are writing amd64/avx512/sass by hand. I hope in time we will find new ways to enjoy solving problems with this new programming modality. By now, I am pretty sure it is here to stay and evolve from its current crude form.