One of those research areas where nothing they currently do is unethical, but also nothing they currently do is useful, and right about the point where it might ever become useful is also the point where it becomes hugely unethical. So we'd do well to outlaw it right now, but seems like we don't really have a moral framework for cases like this.
I don't see why we should outlaw this. There's so many potential insights to be gleaned from research like this that you just can't predict.
And making something like this illegal often has unintended side effects. What if brain organoid research is what ends up curing alzheimer's?
Advocating for outlawing parts of research just on the basis that in the future they could potentially turn unethical is very shortsighted in my opinion.
Sure, and you could say the same thing about unit 731. The argument is not that it is inconceivable that there are benefits, but that they should be weighed against the drawbacks, which you don't mention at all.
I want to be clear: I hate this. I hate everything about it. HOWEVER...
If you want to argue against wetware then we need a better argument than this. The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives. Unit 731 isn't an apt comparison because cells are not inherently sentient to our knowledge. If they were it would absolutely shatter the profession of medical ethics as we know it.
> The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives.
You write as if objections to (certain kinds of) stem cell research, IVF, etc. have been roundly defeated in some way simply because they've "saved and helped lives". This is naked utilitarianism and I hope I don't have to rehearse the reasons why utilitarianism fails utterly as moral theory.
I honestly don't know if better arguments are needed. I think most people simply have no idea that this research is happening right now, and if they did the vast majority would be against it. Feels fundamentally different from stem cell or IVF.
We know that human brain cells in sufficiently large and complex assemblages are sentient (I don't know if the word "inherently" has any purpose here). We know this research aims at making the assemblages larger and more complex. It's a fundamentally different kind of argument IMO compared to those against e.g. stem cells or IVF.
(Slight aside but did you know that they're also doing tests on whole human brains taken from corpses, and they give them tranquilizers "just to be sure"? https://news.ycombinator.com/item?id=48212992)
If anything, my argument would be more "just because unit 731 ended up doing live vivisections of humans, that doesn't mean we should have outlawed dissection of dead humans centuries earlier".
I don't think it's a good idea at all to try to outlaw totally reasonable research just because we can imagine a future where people end up doing a much more extreme much more problematic similar thing.
It would be the end of science. (and the end of any culture really... we're very good at imagining future dystopias and armageddons)
Certainly it wouldn't be the end of science, we could still do all the other science that doesn't involve trying to massage human neurons into doing complex computations.
It's just an extension of Dario's brand of brainrot where consciousness has an almost religious existence that just pops out if you have a large enough system.
No proof is required beyond psuedoscientific tests and the "saints" of this belief system must be given absolute authority over the system to ensure no one commits any sins.
Is it really so wacky to suggest we might want to be careful with bootstrapping complex computation from human neurons? It's quite different from an LLM and the fact that it's not made out of neurons seems to be one of the core arguments for opponents of the possibility of LLM consciousness...
>that just pops out if you have a large enough system.
certainly not a claim I would make.
>No proof is required
Isn't it legitimate to reason on a more abstract level and try to make inferences that way?
>psuedoscientific tests
no clue what you're referring to here or in the rest of your comments about saints and authorities and sins. Another parallel to religion could be drawn w.r.t. the idea that only human souls could ever feel suffering but I don't think it's such a productive level to have this conversation on.
> you must acknowledge consciousness can arise in an agglomeration of human brain cells
That doesn't follow. You could, for example, be a substance dualist. Alternatively you could suspect that there exists some special as yet undocumented property of the human CNS that is required as a prerequisite. Or etc.
So many questions! Since we still don’t understand consciousness how can we draw a line to stay on the right side of? Is consciousness correlated with intelligence or independent of it? What if we keep growing these organoids until they are near-human-brain sized. We know that such things can host a consciousness yet not always eg. “brain dead”.
What if the only reason we don’t believe they are conscious is because they have no mouth to scream with?
Also, it occurs to me that a right-to-lifer would have to be against such biotech to remain consistent.
I find it all incredibly fascinating but ethically highly fraught.
You could do that if you knew enough about how it works in detail. In practice they're just gonna make as big a blob as they can muster and apply RL on it or something, and nobody will be any the wiser as to what the thing's feeling.
"The researchers can’t yet tell why three organoids performed better than just one; it could be because individual variation between the organoids means that they complement each other."
i know ofc that for them 'knowing' means measurements and proofs etc. but i found this excerpt tickle me a bit and wanted to highlight it :').
my first thought was a bit tongue in cheek:
"who knew, working together with others that think alike is better than working alone"
Cant wait for biosofts and the ability to jack-in instead of jack-off :D
Terrifying. I just don't see how this doesn't inevitably lead to brain-scale organoids. The ethics of it seem impossible. Given how we treat non-human animals this reminds me of the torment nexus meme.
If a device made of human brain cells were designed to function like a traditional, deterministic CPU, that would raise similar questions as LLMs raise today.
I get the sense that there is a school of thought I’d call “biological exceptionalism”, a belief that somehow constructs made up of many functionally simple units, is fundamentally different and special compared to a similar construct composed of silicon units. I can’t see how someone could hold this belief and be a materialist (I’m not implying you are btw). Biological exceptionalism surely implies the existence of a “soul” or “spirit” (which raises a whole bunch more questions).
I don't think it'd have to imply a soul - one of these "functionally simple units" is much less well-understood than the other, and you can insert whatever belief you want to fill the gaps.
Maybe a materialist biological exceptionalism position could be that biological substrates are capable of sending kinds of signals that silicon substrates are not, in the sense that brain matter especially might give rise to far greater complex processing in ways that we don't know about or appreciate yet, and which we may discover can't be efficiently recreated in silicon?
I don't know if I buy that argument or even if it makes sense. But yeah this all smacks inextricably of soul talk.
I think it goes the opposite way, starting with a soul and arriving at biological exceptionalism: You start by saying "I have a soul / spirit". Then you extrapolate "things like me probably have a soul / spirit". But since there is no theory of souls, the extrapolation is not based on principles but rather on vibes and guesswork, ending up somewhere like "there must be something weird going on with neurons that somehow create a soul, connect with an existing one or something like that.. maybe.."
It would do no such thing. LLMs are neither living nor sentient nor intelligent. The fallacy of computationalism is a failure to read a coarse metaphor as merely analogical. Any correspondence is superficial and limited, and even then, merely correspondence, not identity.
So, the use of human brain cells is different territory. Bioethically speaking, it is illicit to press human body parts (which includes cells) into service for technological ends. They can only licitly enter into service of the integral human whole (such a life-saving transplants). Using them in technology also commoditizes the human body. (The reverse is not the case: it can be licit to use cybernetic devices to restore native function within a human body.)
I wondered if anyone had written about the ethics of this type of research, they have. It's super long so I've not finished it yet, but it generally seems the standard scientific assessment is that current organoids very likely lack the organization needed for "sentience", but, that methods for detecting subjective experience in organoids are not at all well established.
However, when engineers scale these chips toward cortex-level complexity, projected power climbs into the kilowatt-megawatt bracket, whereas the human brain performs comparable computation on roughly 20 watts (Mehonic and Kenyon, 2022). Biocomputing has been proposed as one route to address energy limitations in silicon-based systems by using brain organoids grown on multielectrode arrays.
All our experiments were conducted using FinalSpark's NeuroPlatform (Jordan et al., 2024), providing access to organoids over a cloud-based system, as shown in Figure 1. In this setup, forebrain organoids are cultured to approximately 500 μm in diameter (Govindan et al., 2021) and contain mature neurons, astrocytes, and oligodendrocytes displaying robust network activity. Maintained under continuous perfusion, they can remain electrophysiologically active for up to approximately 100 days under favorable conditions
This is absolutely fucked. The original argument for these human brain organoids were they're only 10k neurons. Now five minutes later they're talking about energy savings in scaling to cortex-level human brain cloud API.
One of those research areas where nothing they currently do is unethical, but also nothing they currently do is useful, and right about the point where it might ever become useful is also the point where it becomes hugely unethical. So we'd do well to outlaw it right now, but seems like we don't really have a moral framework for cases like this.
I don't see why we should outlaw this. There's so many potential insights to be gleaned from research like this that you just can't predict.
And making something like this illegal often has unintended side effects. What if brain organoid research is what ends up curing alzheimer's?
Advocating for outlawing parts of research just on the basis that in the future they could potentially turn unethical is very shortsighted in my opinion.
Sure, and you could say the same thing about unit 731. The argument is not that it is inconceivable that there are benefits, but that they should be weighed against the drawbacks, which you don't mention at all.
I want to be clear: I hate this. I hate everything about it. HOWEVER...
If you want to argue against wetware then we need a better argument than this. The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives. Unit 731 isn't an apt comparison because cells are not inherently sentient to our knowledge. If they were it would absolutely shatter the profession of medical ethics as we know it.
> The same ideas were used to argue against stem cell research, IVF, and many other treatments and ideas that have saved and helped lives.
You write as if objections to (certain kinds of) stem cell research, IVF, etc. have been roundly defeated in some way simply because they've "saved and helped lives". This is naked utilitarianism and I hope I don't have to rehearse the reasons why utilitarianism fails utterly as moral theory.
I honestly don't know if better arguments are needed. I think most people simply have no idea that this research is happening right now, and if they did the vast majority would be against it. Feels fundamentally different from stem cell or IVF.
We know that human brain cells in sufficiently large and complex assemblages are sentient (I don't know if the word "inherently" has any purpose here). We know this research aims at making the assemblages larger and more complex. It's a fundamentally different kind of argument IMO compared to those against e.g. stem cells or IVF.
(Slight aside but did you know that they're also doing tests on whole human brains taken from corpses, and they give them tranquilizers "just to be sure"? https://news.ycombinator.com/item?id=48212992)
how do u know cells are not sentient to our knowledge? (i am serious, not pulling ur strings. i want to know about proof)
How is unit 731 an apt comparison here?
Those were unethical experiments from the onset.
If anything, my argument would be more "just because unit 731 ended up doing live vivisections of humans, that doesn't mean we should have outlawed dissection of dead humans centuries earlier".
I don't think it's a good idea at all to try to outlaw totally reasonable research just because we can imagine a future where people end up doing a much more extreme much more problematic similar thing.
It would be the end of science. (and the end of any culture really... we're very good at imagining future dystopias and armageddons)
Certainly it wouldn't be the end of science, we could still do all the other science that doesn't involve trying to massage human neurons into doing complex computations.
It's just an extension of Dario's brand of brainrot where consciousness has an almost religious existence that just pops out if you have a large enough system.
No proof is required beyond psuedoscientific tests and the "saints" of this belief system must be given absolute authority over the system to ensure no one commits any sins.
Is it really so wacky to suggest we might want to be careful with bootstrapping complex computation from human neurons? It's quite different from an LLM and the fact that it's not made out of neurons seems to be one of the core arguments for opponents of the possibility of LLM consciousness...
>that just pops out if you have a large enough system.
certainly not a claim I would make.
>No proof is required
Isn't it legitimate to reason on a more abstract level and try to make inferences that way?
>psuedoscientific tests
no clue what you're referring to here or in the rest of your comments about saints and authorities and sins. Another parallel to religion could be drawn w.r.t. the idea that only human souls could ever feel suffering but I don't think it's such a productive level to have this conversation on.
If you believe you are conscious, you must acknowledge consciousness can arise in an agglomeration of human brain cells.
How sure are you that the human brain cells in the vat are not aware and suffering?
> you must acknowledge consciousness can arise in an agglomeration of human brain cells
That doesn't follow. You could, for example, be a substance dualist. Alternatively you could suspect that there exists some special as yet undocumented property of the human CNS that is required as a prerequisite. Or etc.
You can probably do a lot of interesting stuff without giving the blob any capacity of having a consciousness or higher intelligence.
So many questions! Since we still don’t understand consciousness how can we draw a line to stay on the right side of? Is consciousness correlated with intelligence or independent of it? What if we keep growing these organoids until they are near-human-brain sized. We know that such things can host a consciousness yet not always eg. “brain dead”.
What if the only reason we don’t believe they are conscious is because they have no mouth to scream with?
Also, it occurs to me that a right-to-lifer would have to be against such biotech to remain consistent.
I find it all incredibly fascinating but ethically highly fraught.
You could do that if you knew enough about how it works in detail. In practice they're just gonna make as big a blob as they can muster and apply RL on it or something, and nobody will be any the wiser as to what the thing's feeling.
Radiolab had a good episode on brain organoids - https://radiolab.org/podcast/brain-balls
Yeah folks are commenting without understanding organioids. This episode is really educational and helps explain it. Really exciting development
"The researchers can’t yet tell why three organoids performed better than just one; it could be because individual variation between the organoids means that they complement each other."
i know ofc that for them 'knowing' means measurements and proofs etc. but i found this excerpt tickle me a bit and wanted to highlight it :').
my first thought was a bit tongue in cheek: "who knew, working together with others that think alike is better than working alone"
Cant wait for biosofts and the ability to jack-in instead of jack-off :D
Terrifying. I just don't see how this doesn't inevitably lead to brain-scale organoids. The ethics of it seem impossible. Given how we treat non-human animals this reminds me of the torment nexus meme.
If a device made of human brain cells were designed to function like a traditional, deterministic CPU, that would raise similar questions as LLMs raise today.
> that would raise similar questions as LLMs raise today
I think the questions would be very much different. LLMs are not made of human brain cells.
I get the sense that there is a school of thought I’d call “biological exceptionalism”, a belief that somehow constructs made up of many functionally simple units, is fundamentally different and special compared to a similar construct composed of silicon units. I can’t see how someone could hold this belief and be a materialist (I’m not implying you are btw). Biological exceptionalism surely implies the existence of a “soul” or “spirit” (which raises a whole bunch more questions).
I don't think it'd have to imply a soul - one of these "functionally simple units" is much less well-understood than the other, and you can insert whatever belief you want to fill the gaps.
Maybe a materialist biological exceptionalism position could be that biological substrates are capable of sending kinds of signals that silicon substrates are not, in the sense that brain matter especially might give rise to far greater complex processing in ways that we don't know about or appreciate yet, and which we may discover can't be efficiently recreated in silicon?
I don't know if I buy that argument or even if it makes sense. But yeah this all smacks inextricably of soul talk.
I think it goes the opposite way, starting with a soul and arriving at biological exceptionalism: You start by saying "I have a soul / spirit". Then you extrapolate "things like me probably have a soul / spirit". But since there is no theory of souls, the extrapolation is not based on principles but rather on vibes and guesswork, ending up somewhere like "there must be something weird going on with neurons that somehow create a soul, connect with an existing one or something like that.. maybe.."
This seems reductionist with false equivalence. Wouldn't making a brain a "deterministic CPU" negate all the benefits of using organic tissue?
Not if the organic tissue is faster or more energy efficient.
Probably, but the questions I was thinking of are whether it's the design or the substrate that we consider significant.
The question comes up a lot here when there are posts about whether to consider LLMs conscious.
"We're going to condemn conscious entities to slavery"
vs
"They'll always just be machines and it's dehumanizing to consider them anything else"
It would do no such thing. LLMs are neither living nor sentient nor intelligent. The fallacy of computationalism is a failure to read a coarse metaphor as merely analogical. Any correspondence is superficial and limited, and even then, merely correspondence, not identity.
So, the use of human brain cells is different territory. Bioethically speaking, it is illicit to press human body parts (which includes cells) into service for technological ends. They can only licitly enter into service of the integral human whole (such a life-saving transplants). Using them in technology also commoditizes the human body. (The reverse is not the case: it can be licit to use cybernetic devices to restore native function within a human body.)
Maybe the real monsters were the friends we made along the way.
Ethics and Regulation of Human Brain Organoid Research: Recommendations from the Asia Pacific Neuroethics Working Group - https://link.springer.com/article/10.1007/s41649-025-00398-6
I wondered if anyone had written about the ethics of this type of research, they have. It's super long so I've not finished it yet, but it generally seems the standard scientific assessment is that current organoids very likely lack the organization needed for "sentience", but, that methods for detecting subjective experience in organoids are not at all well established.
This is terrifying. Also it isn't new: https://research.ufl.edu/publications/explore/v10n1/extract2...
From 2004.
I can't help but think of "I have no mouth, and I must scream"
Do human brain organoids provide any advantage over animal brain organoids in these types of use cases?
Someone call Stuart Hameroff and Anirban.
I have no fingers and I must read Braille. (But I really want to scream).
who is asking for biohybrid computing? this is how we get on people's bad list, can we just get quantum
However, when engineers scale these chips toward cortex-level complexity, projected power climbs into the kilowatt-megawatt bracket, whereas the human brain performs comparable computation on roughly 20 watts (Mehonic and Kenyon, 2022). Biocomputing has been proposed as one route to address energy limitations in silicon-based systems by using brain organoids grown on multielectrode arrays.
All our experiments were conducted using FinalSpark's NeuroPlatform (Jordan et al., 2024), providing access to organoids over a cloud-based system, as shown in Figure 1. In this setup, forebrain organoids are cultured to approximately 500 μm in diameter (Govindan et al., 2021) and contain mature neurons, astrocytes, and oligodendrocytes displaying robust network activity. Maintained under continuous perfusion, they can remain electrophysiologically active for up to approximately 100 days under favorable conditions
This is absolutely fucked. The original argument for these human brain organoids were they're only 10k neurons. Now five minutes later they're talking about energy savings in scaling to cortex-level human brain cloud API.