> about the fact we are dealing with a system that reasons successfully over a raw bulk of extraordinarily opaque data without breaking a sweat. That ability is alien
But it's not what's happening here at all! Read the thing in detail and you'll see that it's not doing anything alien, it does exactly what you (or at least someone with at least some knowledge abouy neural nets) would do if you were asked to perform the task. It doesn't reason about weights in a magical way, instead it reads the easy to interpret parts or use directional ablation to tweak a model's behavior, exactly like how the ERP folks have been decensoring models to produce adult content over the past two years.
It's an interesting demonstration of ML knowledge and agentic ability, but it's not alien NY any means, and it's not actually surprising given how strong today's models are with agentic work.
I think this other submission from yesterday that visualised the examples was quite helpful:
https://news.ycombinator.com/item?id=50344861
Thank you for sharing that, it's a shame that the overhyped paper received this much attention while this one ended up burrowed beyond eyesight.
Thanks for sharing. I was also onboard the hype train ready to share this widely and get others onboard, but now I'll be sharing this article instead.
I read the report; it's not complicated. I'm sure there is some basic misunderstanding here.
When I say "alien" I don't mean "magic" and I don't mean "emergent new woo-woo scary ability". I mean, not human in speed/volume. No human can process that amount of raw bulk in that amount of time successfully. And indeed, it's not magic at all. It is enormously mundane actually. It reminds me of old computers being able to do arithmetic at insane speeds compared to humans. It's not magic at all, but to be honest, I have the distinct feeling of goalposts flying all around me. Being able to handle tasks of this complexity.. I don't know about you but I still have to adjust to computers being able to do this and that it's actually "not surprising". I guess I'm getting old.
I can see how you want to calibrate the enthusiasm I see around here as it can be a bit.. well, but it's not necessary for me. I'm not overly enthused by this because I don't care about the NN adjusting itself and RSI etc. I'm sure it's all quite basic but the fact it can do it at all is reason to be.. watchful, IMO. That is to say, I can appreciate the level of capability necessary to do this work successfully, but I do take your point seriously. It made me think, thanks.
I think you are over-indexing on the word alien. What matters here is how scalable the technique is and the fact that it is autonomous. This can be a breakthrough for continual learning if it can be trained to work at scale.