The whole actual, not necessarily the "openly admitted", utility of AI is to allow people to ship while not understanding the work, and not caring that much about it.
So such posts will make as much change as "trade, but be ethical" would make on a late 80s Wall Street trading floor:
"Before you tell the agent to start building anything, you go there and read and understand it first."
Some applications (e.g. a kernel) need to be carefully engineered with every possible edge case considered. Some apps (e.g. a web game) don't. This is a way for careful engineers who need to build dependable software to integrate AI without compromising quality.
If the software makes money because it's expected to work reliably, every edge case matters. Web games aren't exempt from engineering challenges. In fact, the more trivial the problem seems on the surface, the more likely it is to have edge cases.
Bugs are always prioritized based on how big of a deal it is for the customer. They don't like to wait and they rarely give second chances.
AI doesn't fit into any dev workflow except greenfield or analysis. Everything else needs a lot of close human attention and human discussion. AI doesn't read minds.
No that is not a correct read -it might be based on this past few months or so state of the art, while instead that state of the art is moving forward every month- and at the rate these ai coding systems are improving and our harnesses for managing them- bug handling and brownfield codebases are becoming increasingly accessible to AI coding
Ie you are making the mistake of assuming the tech is static when it most assuredly is not
I completely agree. While the best outcome may obviously involve lots of thinking, from what I've seen an experienced, the AI tools we have today put people on a slope to becoming "meat proxies". Some of the brighter, stronger engineers may have a less steep slope, but I'm convinced that with time they too will be meat proxies. AI is attacking on too many fronts for it to be otherwise:
- AI generates too much code that overwhelms humans
- AI deskills users concerning the gritty details of coding by having them not do it themselves
- Having most of one's co-workers become meat proxy will put pressure for others to convert
- Lack of effort or expertise invested into codebase leads people to care less; caring less -> not caring about optimal outcome -> not expending brain power on problems we don't care about.
I think since we as an industry are still on this slope, we're still having systems propped up by people actually using their brains. I personally think this situation is regrettable. We should as an industry consider if the direction we're going truly is something we want.
We are clearly getting to the point, or past the point, where humans will not understand the code base very well or at all. I fear we are building a tower of "it kind of works this way" (maybe it does or maybe it doesn't), in a couple years people won't know what they can change, they won't get much of the architecture.
If one can't understand the monstrosity that was built up over time, where the code base is enormous, millions and millions of lines and components, all with a few buggy corners that the tests didn't touch, and a human didn't have the insight to write a test that understands and verifies the corner cases, it will be like a magic machine that we can't change because we don't understand it and we can't afford years of effort. Some may hope their LLM will be able to understand it.
We'll have things that are buggy in ways that become very difficult to understand or fix, and no one can afford the effort. Remember when we had all those huge old programs from the 60s or early 70s, before there were tests maybe, or unit tests. Code written at a lot of human effort, but before the ideas of software engineering made it more tractable to build large programs reliably, testable.
We are creating a new problem for the people that take over our new code, just like the "vast for the time" 1960s huge cobol program can't be changed.
If we assume that AI coding is just beginning- we can assume that humans will create systems to help
Manage the increasingly complex interactions that machines will code - ie we are still early like when we first introduced the industrial revolution to the peasants (which led to decades of chaos- for sure)
But Just like a 787 is not flown by hand either - doesn't mean anyone is qualified to autopilot it
Complex engineering systems will require sme humans to help manage the underlying complex interactions in way that is comprehensible to human intelligence
No one except for a smaller group of specialists for example these days does much understanding of assembly or compilers.. but we build complex SaaS apps all the time without that knowledge
The basic concept here, that you take personal responsibility for the input and output being transmitted to/from the LLM, is an obvious but critical one, so I appreciate this terminology as a way to convey it.
I'll try using Brain Sandwich as guidance for less technical folk.
This is the same problem a lot of leaders have in all kinds of work. It is often possible to delegate a task. But if one delegates understanding, one is at a disadvantage when compared to an organization where leadership understands the task.
When I was in charge of companies, I understood the technical tasks, but delegated understanding of things like tax law. This was okay, but I was definitely at a disadvantage compared to companies that could better optimize capital allocation around understanding of tax law.
AI means more humans in every organization are having to learn how to be managers/leaders of these agents. Those who actually understand the work they delegate will maintain an advantage over those who do not.
I fully agree with the approach, but sadly that doesn't make the overwhelming amount of code reviews in the AI age much better. At best, it might slow down your colleagues' deluge of review requests a little. But also, you still have to spend time reviewing your own LLM code, of which there is more of and you're naturally less familiar with than if you'd hand-written it, so it takes longer to review.
in the post, the brain sandwich gets explained as "brain, ai, brain" but sandwich experts note that a sandwich is usually named after its contents and not the bread
> Before you tell the agent to start building anything, you go there and read and understand it first.
The sentence is maybe ambiguous, but it sounds like it's saying to not use the AI at all until you understand what's happening in the area that needs to change.
I don't agree with this as you can spend quite a bit of time digging around to first find the correct place which needs to be updated and then more time reading the code (likely in multiple files or places) to build up a mental model of what's going on.
I typically have more than one AI conversation on the go at any given time, so I'll always start by asking it to make a plan to solve a problem and while it's busy exploring to make a plan I review some other AI conversation's work in the meantime, when the AI's plan is ready I review it.
If at this point there is anything that is unclear to me about its approach to implement the plan, or I don't already have familiarity with the part of the system it's proposing to update, I start questioning the AI on how the system works. I will scrutinize the AI's plan until I'm absolutely sure that it's going to implement something that is sound from a system design perspective and as a senior software developer with architecture level experience I am capable of making such judgments.
I'm not sure what it might be like for comparitively inexperienced developers, but I would expect that AI's capacity for infinite patience allows them to ask as many questions as they need to get a clear mental model of what's happening, so if I were mentoring such a person in this age of AI I would drill into them on how to interact with AIs to ensure they have this clarity before the AI starts implementing anything.
The difference between vibe coding and highly accelerated agentic development is having the expertise and judgment to ensure you're building properly sound software.
I agree completely, I recently went thru this. Customer API session/journey security is a beast to secure, so many layers, so many moving parts, so many approaches.
> before sending your stuff to other people for review, you need to make the work reviewable.
Yes, but the amount of work needed here is overstated. With good orchestration/prompting in the task[1] (and token budget), the work on this end is significantly reduced.
If you're saying AI has _never_ found a more elegant solution than your original idea, you're not using it right or you're only using the cheapo models.
(Before someone asks: "not using it right" could mean a lot of things, but intentionally constraining its search by telling it how to do everything is one example. Or by not giving it a way to hill-climb: no tests, no iteration on a solution, etc.)
A lot of devs are micromanagers who think they’re smarter than everyone else. You’ve probably at least met a couple if you’ve had a long career. They typically leave lots of nits on code that is mostly meaningless noise or personal preference.
These types of people will get the least out of AI as their ego prevents them from letting it outshine themselves.
If you've given up on understanding your code base, AI will give you a better solution than you could come up with. Not because AI is better, but because you've become a low performer.
AI is reasonable at reviewing code, with only a 50-60% false-positive rate, and its persistence at broadening scope often finds things human reviewers would miss. It's still actually still pretty shitty at writing code, though, again, with enough ground truth and persistence, it'll often generate something that's a functional turd. It will sometimes come up with better suggestions if you prompt for it, but it needs a ton of hand-holding at the moment. And usually the suggestions are still overly complicated messes that you can trim down to 1/10th the size, while keeping the core idea.
Congrats to you if you're working in a codebase small enough to know it all. Do you know every knob in your database? Every tuning option on your webserver? Every warning and lint setting on your compiler? Wish I did! Do you read every PR that comes through the system, every day? You must keep busy.
Generally, I know a LOT about my team's code, and then in concentric circles I know less and less of the periphery. I work on <feature X> and know almost everything. That's a subsystem of <project Y> which I know well. That's part of <product Z> which I know in varying amounts.
And of course, there are things like build systems (CMake, ninja, blaze) where I know the basics, but I've never been an expert and don't care to be. Before I'd offload build features to another team; now I'd offload it to an AI. It's not something I need to be an expert in. I just want builds to work well.
That is a wonderful question. I think you might be confusing "full vibe mode" with "using AI as a tool." I don't know where some code paths lead, so I can use AI to help ("who ingests .XYZ outputs? where are the tests?"). I don't always know if policies exist so I can ask ("do we have guards around XYZ going offline? where?"). After learning the lay of the land, I decide what we need and how it fits in with the current design, and ask the agent to verify that my understanding of the flow makes sense. I'll of course review all the code that comes out.
Sometimes in review I'll be surprised that my understanding was flawed and the AI subtly misled me during the discussion, but that's super rare and that's honestly still helping me learn and understand.
Frankly it's no different than if I were pair-coding with a teammate who happened to type 25,000 wpm, and who never takes a day off.
If you had an electrician that told you "Oh, I don't learn by running wires, but if I didn't immediately see where a wire lead, I'd ask someone who knew, and then I read wiring diagrams after the other person did the work", would you hire them?
When they stop working at the site when they make the blueprints -- or worse, only look at them, you should run, or at least make sure they're only involved in management, and not any important technical decision making.
They are not going to do a good job.
Software engineers used to mock non-coding architects for good reason.
This is the same approach I use. I don't know if it actually produces good results yet as I've only been doing this for about three months. At the very least it feels like I have some agency in my work which I think is one of the things people complain they lose to AI.
I have a very simple framework I use for working with John von Neumann ... the idea is very simple, and can be summarized by: whenever you work on something with John von Neumann, you should use:
1. Your brain, and then
2. John von Neumann, and then
3. Your brain
Explaining a bit more… Before you tell John von Neumann to start working on anything, you go there and read and understand it first. And when John von Neumann brings you the solution, you will know if it’s right, or if it’s too much, or if it’s completely solving the wrong problem, etc.
This approach appears valuable for personal growth as an engineer, yet it demands discipline that may not yield returns in corporate settings. Specifically:
> The rule of thumb is: if you can’t explain what’s changing and why, it’s not ready for someone else to take a look
I have observed firsthand that the metric or "understanding" can be irrelevant to management. When delivery dominates, understanding can be deferred to future agents who will gladly consume your codebase and convince you they grasp it, and further, that you do too. If I invest substantial effort in my PRs only for them to be /review'd, does it still matter? Perhaps reviewers should practice brain sandwich as well.
The worst offenders seem to sit at either extreme: fully AI firms that treat understanding as a human weakness and prioritize outcomes alone, or large corporate teams building internal CRUD apps whose middle management merely pretends to value good engineering to satisfy finicky senior engineers.
Not a criticism on the authors work at all, I like the idea (and I don't think they were trying to solve anything to do with work culture), but ultimately if my company doesn't find my practice valuable, I can't easily justify taking it on. Especially if I have to sell to them that I'm going to be spending _more_ time delivering. Fun times!
Our CEO, COO, etc are starting to see non-technical work go to customers with huge mistakes (committing to things that don't exist for instance). I also think seeing the Claude-style of writing everywhere has become tiresome for execs. That seems to have helped force a bit of an attitude change from always wanting to know how we could use AI in every process, to insisting that certain things should never be done by AI.
This is anecdotal of course, but at least at my company it's getting a bit easier to advocate for humans to do the thinking and understanding.
The whole actual, not necessarily the "openly admitted", utility of AI is to allow people to ship while not understanding the work, and not caring that much about it.
So such posts will make as much change as "trade, but be ethical" would make on a late 80s Wall Street trading floor:
"Before you tell the agent to start building anything, you go there and read and understand it first."
Yeah, not gonna happen. 1 in 100 might do that.
Some applications (e.g. a kernel) need to be carefully engineered with every possible edge case considered. Some apps (e.g. a web game) don't. This is a way for careful engineers who need to build dependable software to integrate AI without compromising quality.
Wrong.
If the software makes money because it's expected to work reliably, every edge case matters. Web games aren't exempt from engineering challenges. In fact, the more trivial the problem seems on the surface, the more likely it is to have edge cases.
Bugs are always prioritized based on how big of a deal it is for the customer. They don't like to wait and they rarely give second chances.
AI doesn't fit into any dev workflow except greenfield or analysis. Everything else needs a lot of close human attention and human discussion. AI doesn't read minds.
No that is not a correct read -it might be based on this past few months or so state of the art, while instead that state of the art is moving forward every month- and at the rate these ai coding systems are improving and our harnesses for managing them- bug handling and brownfield codebases are becoming increasingly accessible to AI coding
Ie you are making the mistake of assuming the tech is static when it most assuredly is not
I completely agree. While the best outcome may obviously involve lots of thinking, from what I've seen an experienced, the AI tools we have today put people on a slope to becoming "meat proxies". Some of the brighter, stronger engineers may have a less steep slope, but I'm convinced that with time they too will be meat proxies. AI is attacking on too many fronts for it to be otherwise:
I think since we as an industry are still on this slope, we're still having systems propped up by people actually using their brains. I personally think this situation is regrettable. We should as an industry consider if the direction we're going truly is something we want.
We are clearly getting to the point, or past the point, where humans will not understand the code base very well or at all. I fear we are building a tower of "it kind of works this way" (maybe it does or maybe it doesn't), in a couple years people won't know what they can change, they won't get much of the architecture.
If one can't understand the monstrosity that was built up over time, where the code base is enormous, millions and millions of lines and components, all with a few buggy corners that the tests didn't touch, and a human didn't have the insight to write a test that understands and verifies the corner cases, it will be like a magic machine that we can't change because we don't understand it and we can't afford years of effort. Some may hope their LLM will be able to understand it.
We'll have things that are buggy in ways that become very difficult to understand or fix, and no one can afford the effort. Remember when we had all those huge old programs from the 60s or early 70s, before there were tests maybe, or unit tests. Code written at a lot of human effort, but before the ideas of software engineering made it more tractable to build large programs reliably, testable.
We are creating a new problem for the people that take over our new code, just like the "vast for the time" 1960s huge cobol program can't be changed.
It's leading us toward the Cult Mechanicus from Warhammer 40k.
Well to play the opposite perspective:
If we assume that AI coding is just beginning- we can assume that humans will create systems to help Manage the increasingly complex interactions that machines will code - ie we are still early like when we first introduced the industrial revolution to the peasants (which led to decades of chaos- for sure)
But Just like a 787 is not flown by hand either - doesn't mean anyone is qualified to autopilot it
Complex engineering systems will require sme humans to help manage the underlying complex interactions in way that is comprehensible to human intelligence
No one except for a smaller group of specialists for example these days does much understanding of assembly or compilers.. but we build complex SaaS apps all the time without that knowledge
The basic concept here, that you take personal responsibility for the input and output being transmitted to/from the LLM, is an obvious but critical one, so I appreciate this terminology as a way to convey it.
I'll try using Brain Sandwich as guidance for less technical folk.
Like the concept.
Pedantic addendum: this is an AI sandwich, not a brain sandwich.
Hard to truly understand the work unless if you're actually doing it.
Yes, the temptation to skip forward is going to be to great for most people.
This is the same problem a lot of leaders have in all kinds of work. It is often possible to delegate a task. But if one delegates understanding, one is at a disadvantage when compared to an organization where leadership understands the task.
When I was in charge of companies, I understood the technical tasks, but delegated understanding of things like tax law. This was okay, but I was definitely at a disadvantage compared to companies that could better optimize capital allocation around understanding of tax law.
AI means more humans in every organization are having to learn how to be managers/leaders of these agents. Those who actually understand the work they delegate will maintain an advantage over those who do not.
I fully agree with the approach, but sadly that doesn't make the overwhelming amount of code reviews in the AI age much better. At best, it might slow down your colleagues' deluge of review requests a little. But also, you still have to spend time reviewing your own LLM code, of which there is more of and you're naturally less familiar with than if you'd hand-written it, so it takes longer to review.
in the post, the brain sandwich gets explained as "brain, ai, brain" but sandwich experts note that a sandwich is usually named after its contents and not the bread
Yep, it's AI - Meat - AI
> Before you tell the agent to start building anything, you go there and read and understand it first.
The sentence is maybe ambiguous, but it sounds like it's saying to not use the AI at all until you understand what's happening in the area that needs to change.
I don't agree with this as you can spend quite a bit of time digging around to first find the correct place which needs to be updated and then more time reading the code (likely in multiple files or places) to build up a mental model of what's going on.
I typically have more than one AI conversation on the go at any given time, so I'll always start by asking it to make a plan to solve a problem and while it's busy exploring to make a plan I review some other AI conversation's work in the meantime, when the AI's plan is ready I review it.
If at this point there is anything that is unclear to me about its approach to implement the plan, or I don't already have familiarity with the part of the system it's proposing to update, I start questioning the AI on how the system works. I will scrutinize the AI's plan until I'm absolutely sure that it's going to implement something that is sound from a system design perspective and as a senior software developer with architecture level experience I am capable of making such judgments.
I'm not sure what it might be like for comparitively inexperienced developers, but I would expect that AI's capacity for infinite patience allows them to ask as many questions as they need to get a clear mental model of what's happening, so if I were mentoring such a person in this age of AI I would drill into them on how to interact with AIs to ensure they have this clarity before the AI starts implementing anything.
The difference between vibe coding and highly accelerated agentic development is having the expertise and judgment to ensure you're building properly sound software.
I agree completely, I recently went thru this. Customer API session/journey security is a beast to secure, so many layers, so many moving parts, so many approaches.
> before sending your stuff to other people for review, you need to make the work reviewable.
Yes, but the amount of work needed here is overstated. With good orchestration/prompting in the task[1] (and token budget), the work on this end is significantly reduced.
… and then the brain drain issue returns.
[1] https://claude.dev/blog/solving-code-review-at-anthropic/
This is far better than just hitting return, but there's zero chance it's as good as doing things the hard way. Tradeoffs, of course.
If you're saying AI has _never_ found a more elegant solution than your original idea, you're not using it right or you're only using the cheapo models.
(Before someone asks: "not using it right" could mean a lot of things, but intentionally constraining its search by telling it how to do everything is one example. Or by not giving it a way to hill-climb: no tests, no iteration on a solution, etc.)
A lot of devs are micromanagers who think they’re smarter than everyone else. You’ve probably at least met a couple if you’ve had a long career. They typically leave lots of nits on code that is mostly meaningless noise or personal preference.
These types of people will get the least out of AI as their ego prevents them from letting it outshine themselves.
Shoot, I'm very much the "lots of nits" reviewer but I have no inclination to be a manager and get lots of benefit from AI. What does that make me?
If you've given up on understanding your code base, AI will give you a better solution than you could come up with. Not because AI is better, but because you've become a low performer.
AI is reasonable at reviewing code, with only a 50-60% false-positive rate, and its persistence at broadening scope often finds things human reviewers would miss. It's still actually still pretty shitty at writing code, though, again, with enough ground truth and persistence, it'll often generate something that's a functional turd. It will sometimes come up with better suggestions if you prompt for it, but it needs a ton of hand-holding at the moment. And usually the suggestions are still overly complicated messes that you can trim down to 1/10th the size, while keeping the core idea.
Congrats to you if you're working in a codebase small enough to know it all. Do you know every knob in your database? Every tuning option on your webserver? Every warning and lint setting on your compiler? Wish I did! Do you read every PR that comes through the system, every day? You must keep busy.
Do you know anything about your codebase? How?
Generally, I know a LOT about my team's code, and then in concentric circles I know less and less of the periphery. I work on <feature X> and know almost everything. That's a subsystem of <project Y> which I know well. That's part of <product Z> which I know in varying amounts.
And of course, there are things like build systems (CMake, ninja, blaze) where I know the basics, but I've never been an expert and don't care to be. Before I'd offload build features to another team; now I'd offload it to an AI. It's not something I need to be an expert in. I just want builds to work well.
And, how do you learn it?
That is a wonderful question. I think you might be confusing "full vibe mode" with "using AI as a tool." I don't know where some code paths lead, so I can use AI to help ("who ingests .XYZ outputs? where are the tests?"). I don't always know if policies exist so I can ask ("do we have guards around XYZ going offline? where?"). After learning the lay of the land, I decide what we need and how it fits in with the current design, and ask the agent to verify that my understanding of the flow makes sense. I'll of course review all the code that comes out.
Sometimes in review I'll be surprised that my understanding was flawed and the AI subtly misled me during the discussion, but that's super rare and that's honestly still helping me learn and understand.
Frankly it's no different than if I were pair-coding with a teammate who happened to type 25,000 wpm, and who never takes a day off.
If you had an electrician that told you "Oh, I don't learn by running wires, but if I didn't immediately see where a wire lead, I'd ask someone who knew, and then I read wiring diagrams after the other person did the work", would you hire them?
I dunno, man. At some point an experienced electrician thinks about the blueprints and hires apprentices to run the wires.
When they stop working at the site when they make the blueprints -- or worse, only look at them, you should run, or at least make sure they're only involved in management, and not any important technical decision making.
They are not going to do a good job.
Software engineers used to mock non-coding architects for good reason.
This is the same approach I use. I don't know if it actually produces good results yet as I've only been doing this for about three months. At the very least it feels like I have some agency in my work which I think is one of the things people complain they lose to AI.
Do people not use AI this way in prod?
sounds like 'design' is back
"Other people use slop the wrong way."
https://en.wikipedia.org/wiki/Third-person_effect
I have a very simple framework I use for working with John von Neumann ... the idea is very simple, and can be summarized by: whenever you work on something with John von Neumann, you should use:
1. Your brain, and then
2. John von Neumann, and then
3. Your brain
Explaining a bit more… Before you tell John von Neumann to start working on anything, you go there and read and understand it first. And when John von Neumann brings you the solution, you will know if it’s right, or if it’s too much, or if it’s completely solving the wrong problem, etc.
This approach appears valuable for personal growth as an engineer, yet it demands discipline that may not yield returns in corporate settings. Specifically:
> The rule of thumb is: if you can’t explain what’s changing and why, it’s not ready for someone else to take a look
I have observed firsthand that the metric or "understanding" can be irrelevant to management. When delivery dominates, understanding can be deferred to future agents who will gladly consume your codebase and convince you they grasp it, and further, that you do too. If I invest substantial effort in my PRs only for them to be /review'd, does it still matter? Perhaps reviewers should practice brain sandwich as well.
The worst offenders seem to sit at either extreme: fully AI firms that treat understanding as a human weakness and prioritize outcomes alone, or large corporate teams building internal CRUD apps whose middle management merely pretends to value good engineering to satisfy finicky senior engineers.
Not a criticism on the authors work at all, I like the idea (and I don't think they were trying to solve anything to do with work culture), but ultimately if my company doesn't find my practice valuable, I can't easily justify taking it on. Especially if I have to sell to them that I'm going to be spending _more_ time delivering. Fun times!
Our CEO, COO, etc are starting to see non-technical work go to customers with huge mistakes (committing to things that don't exist for instance). I also think seeing the Claude-style of writing everywhere has become tiresome for execs. That seems to have helped force a bit of an attitude change from always wanting to know how we could use AI in every process, to insisting that certain things should never be done by AI.
This is anecdotal of course, but at least at my company it's getting a bit easier to advocate for humans to do the thinking and understanding.