Such a bullshit article, as much of Anthropic "research" is these days.
> By “agent memory,” we mean knowledge that Claude gains from experience—including interactions with users and tasks it’s been assigned—and that it retains between sessions, saved as context it can read later.
That Claude rarely, if ever, actually "remembers". Since these are just random markdown files randomly injected into the tiny context.
"I'm sorry, I'll make a note of it" and will proceed to write a 15000th note with the same content it's already written dozens of times before
> We compared the memory folders from one run each of Claude Sonnet 3.7 and Claude Fable 5, when each was about 10,000 steps into the game.
What a useless experiment
> That general guidance was enough for Opus 4.7 and Opus 5 to use memory effectively
Define "effectively"
> In this case, prompting Claude to read its memory files before answering—and then to explicitly test its theories—nearly doubled its benchmark score
Then it's not "memory". It's just telling the model to read a text/spec file before implementing something
> Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md
So, random injection of random things into context that depend on whether the short ToC entries have any relevance on what Claude is doing, and which will be ignored.
And you have to explicitly tell the agent to read specific files otherwise it will ignore them.
What is this bloated article about?
Exactly my experience with claude-code. It keeps apologising for not using the memory and keep storing the same memory again and again.
> What is this bloated article about?
Advertisement. As you correctly parsed, this article was vapid and full of fluff.
Why they write this? Probably as a vehicle to keep up word of mouth amongst software engineers; as we can see in this thread there's no shortage of people to engage with this crap as it had some merit. I can totally see this making the rounds on LinkedIn in search of engagement.
That I had to scroll to the bottom of the thread to find this is interesting, and says something about the general quality of comment section nowadays.
Reading such a passionate take as someone who has looked up is a strange experience, I must admit. Kinda like how the Bernard twist in S01 of Westworld felt. I'll try to respond as substantively as I can, regardless:
Complaining about your anecdotal experiences with Claude(Code, hopefully?) isn't a serious criticism, the markdown files are optimized to not be random, and calling 1,000,000 tokens of context "tiny" seems like a plainly unreasonable bar. How much of War and Peace can you hold in your head at once and quote flawlessly, and is it more than 133%?
The experiment is testing their fixes to the problem you lamented just one sentence prior.
They do exactly that in figure 3, as mentioned at the end of the sentence you're quoting. Opus 5 used that advice to raise its benchmark score by 46.8%, and Opus 4.7 used it for a gain of 82.7%. The specific benchmark looks far from trivial, too: https://github.com/pgasawa/continual-learning-bench/tree/mai...
It fits the colloquial definition, and there is no specific scientific definition in CogSci, CogPsych, Neuro, CogNeuro, NeuroPsych, nor philosophy. There's lots of useful ones offered in different contexts, but they differ considerably.[1] Regardless, it's well accepted industry jargon at this point: it doesn't matter in the same way that it doesn't matter than databases can never truly be "queried" because they can't engage in linguistic discourse.
Isn't this inevitably, undeniably true of any possible memory system, just from, say, a big-O perspective? If you have to persist every item in a list to decide which one is relevant enough to load into the same pool of memory, there's no point in the second step anymore.
This article is very plausibly wrong, misguided, or even philosophically bankrupt, but I don't see any support for the claim that its topic is unclear. It's discussing the efficacy of proactive state management during idle periods for longrunning LLM-backed agents.
---
[1]: For example, did you know that modern CogSci is generally convinced that memories are illusions that the brain creates through a process that merely consults ineffable underlying representations (usually), and not an object retrieved from storage? That definition would also contradict Anthropic's wording here, but only in a way that weakens your criticism rather than supports it.
> Reading such a passionate take as someone who has looked up is a strange experience, I must admit.
There's nothing passionate. Calling bullshit for what it is is not passionate, and we need more of calling out.
I would say that you are much more passionate about this and have a prior bias that makes you see something that's not there in Anthropic's marketing materials
> Complaining about your anecdotal experiences with Claude(Code, hopefully?) isn't a serious criticism
Dismissing actual real-life user experience that anyone who's ever dealt with Claude Code as "anecdotal experience" is exactly why these self-congratulatory articles exist. Even if they mention exactly these issues in passing.
> and calling 1,000,000 tokens of context "tiny" seems like a plainly unreasonable bar.
It is both tiny and easily polluted. Its training data will easily override any and all data in the context even early on in the session. And yes, that million token context is still incapable of maintaining and recalling any of the "memories" with any consistency. Which the article itself acknowledges.
> The experiment is testing their fixes to the problem you lamented just one sentence prior.
These experiments are useless because they invent a problem and valiantly find solutions for it never measuring whether it has any actual impact in real life.
So what if new way of writing summaries is marginally better than the old way? It doesn't matter if the only way to make Claude reliably use them is to explicitly tell it to read them?
> It fits the colloquial definition, and there is no specific scientific definition in CogSci, CogPsych, Neuro, CogNeuro, NeuroPsych, nor philosophy
Here are your biases showing. Anthropic calls random markdown files "memories" and you immediately take them at their word.
These are not memories. These are objectively random markdown files whose only existence is assured by random harness injections and whatever claude retains in its current context from the first 200 lines of an injected ToC.
Yes, it's an accepted industry jargon, but you don't have to behave that it does something else than what it does.
Or that explicitly telling Claude "read a specific markdown file in your memory" is in any way, shape, or form different from "read a specific markdown file in specs directory".
Of course it's the only way to make Claude actually read and act on it.
> It's discussing the efficacy of proactive state management during idle periods for longrunning LLM-backed agents.
No. No it doesn't. It considers "components of agent memory" and focuses solely on "how we can write markdown files", and spends probably less than 1% on "how we can make sure the agent actually uses them". Even the conclusion completely ignores it despite this:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence [1 000 000 tokens of context is huge lol - troupo]
...
It loads only the first 200 lines or 25 KB of MEMORY.md [but we sure love the sound of our voice deacribing how well it can write the hundreds of tiny files – troupo]
--- end quote ---
Very good reply.
... And this second reply proves my point to your original take.
The guy you are responding to engaged with that article as it had any merit. Even brings up concepts from psychology as if the misuse of the term "memory" by Anthropic was related to actual memory as would be defined in other fields.
I wonder if the excessive usage of AI damaged people's brains enough that they abandoned curiosity and the ability to read things critically, or of they already lacked those things already in the first place.
1. Please don't assume people on the internet are men.
2. "You must only think that because you've abandoned curioisity, can't think critically, and have brain damage that's your own fault" is absurdly polemic. If I took your opinion seriously at all, I'd be quite offended.
> 1. Please don't assume people on the internet are men.
I did not. I said "the guy" as a generic term. A guy can be anything.
> 2. "You must only think that because you've abandoned curioisity, can't think critically, and have brain damage that's your own fault" is absurdly polemic.
Aside the fact that you twisted my phrase to something completely different, I made no definitive assumptions.
I was wondering if excessive AI use damage people's brains. It is something that needs to be exercised. Much like sitting around all day will damage your overall health.
> I said "the guy" as a generic term. A guy can be anything.
Reminds me of "in California everything is a dude"
https://www.reddit.com/r/tumblr/comments/j0yscj/in_californi...
...you reacted to this blogpost by calling it "bullshit" and "bloated", and criticizing it with obvious hyperbole ("random markdown files randomly injectect into the tiny context", "15000th note", "dozens of times before", "useless experiment", "random injection").
You're allowed to feel emotion -- that wasn't a refutation. But denying that you indeed are typing with responding seems plainly obstinate, sorry. Again I will do my best to respond in line with the rules of this forum, even if I don't feel the same grace being given in return.
What makes you say that? Because I disagree? A comment starting with "I'll try to reply as substantively as I can" isn't exactly firey, much less one that makes pain to cede ground and offer friendly offramps to the polemics.
Well I have positive actual real-life user experience with these products. It seems we're at an impasse, there! 1-1=0, after all. This is why anecdotal evidence isn't valid, generally speaking.
This just comes down to "nuh-uh". So: yeah-huh! 133% of War and Peace is a lot of text to hold at the forefront of one's mind simultaneously.
That's the exact opposite of the findings of the experiment that you yourself quoted: achieving an 82% improvement on a challenging benchmark without detailed instructions for memory use is in no sense "incapable".
You're not taking the time to read what I'm writing :( "It fits the colloquial definition" objectively isn't taking Anthropic "at their word". On an absolute, undeniable, factual level, this attempted ad-hominem is based on a faulty premise.
What are memories, and why does text stored in a computer "objectively" not fit that definition? Even if you offered one now (which, again, would be scientifically unsound), it'd be too late; it's beyond obvious that this reasoning is a-posteriori.
How am I doing that, specifically? You're the one taking issue with the industry jargon, and I offered no description of "what it does" in the first place, much less one that conflicts with some singular objective definition of "memory" (which, again, does not exist).
This is objectively incorrect on an empirical level, as proven in the article we are commenting under.
A) The desciption I gave does not require this arbitrary focus of yours.
B) Again, every single empirical result in the entire paper relies on the mechanism that you find to be non-existent.
> Well I have positive actual real-life user experience with these products.
I never said I didn't have a positive real-life user experience with these products.
> So: yeah-huh! 133% of War and Peace is a lot of text to hold at the forefront of one's mind simultaneously.
Let me quote the article again:
--- start quote ---
Injected memories use up valuable tokens, and an incorrect note can be carried into every session. Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence
--- end quote ---
Both the article and the linked docs say that Claude will only load ~200 lines of memory index (not the entire memory), and that you should keep your Claude.md below 200 lines.
Never knew War and Peace was only 400 lines long.
> What are memories, and why does text stored in a computer "objectively" not fit that definition?
Memories is something a model wouldn't need to re-read from scratch every time. Memories isn't something you would need to be always told to reliably use.
And no, whatever human analog you come up with isn't relevant, as we're not talking about humans.
> This is objectively incorrect on an empirical level, as proven in the article we are commenting under.
Let me quote the article:
--- start quote ---
Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence.
It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
> Again, every single empirical result in the entire paper relies on the mechanism that you find to be non-existent.
1. I never said it was inexistent
2. The "empirical results" in the paper completely ignore the actual limitations of "memory". It explicitly says that "nope, Claude will not use memory because of known limitations", and immediately sweeps past it as if it wasn't important.
The entirety of "dreaming" and "memory" is completely negated by this:
--- start quote ---
Loading a note also doesn’t guarantee that Claude acts on it: our documentation on Claude Code warns that longer instruction files reduce adherence.
Claude doesn’t load all of its auto memory into every session’s context window. It loads only the first 200 lines or 25 KB of MEMORY.md
--- end quote ---
And this can be literally confirmed empirically by anyone actually working with these tools and watching what they do. E.g., Claude will completely ignore most "memories" and write the same "memory" again and again and again in potentially dozens of files when you tell it that something is wrong. Often repeating what is already there in explicit CLAUDE.md instuctions.
Oh wait. The amazing 1 000 000 token context that can seemingly hold all of "War and Peace" cannot reliably hold more than 200 lines of explicit instructions and more than 200 lines of "memory".
Irrelevant; you were the one who was raising anecdotal evidence and defending it when challenged. Also c'mon surely you realize that you're just disagreeing for the sake of it by this point?
Irrelevant; you were discussing the context window itself, not how many lines of text are loaded per note.
A) Not a definition, and, again, B) Disproven directly by the findings I jsut quoted to you.
That's how analogues work.
Irrelevant quote.
...again, surely even you realize that quibbling over 1% present and 0% present isn't productive, in this context? What's the point?
The entire article is about Claude using memory.
Is it possible that you haven't used Claude since Sonnet 3.7? Cause, again, this anecdotal experience isn't reflective of today's reality. And why would anyone care about the anecdotal experience of someone who defines themselves by hating LLMs in the first place?
All of War and Peace could fit into one line. Regardless, 1M is 1M is 1M. This isn't a thing you can have a take about, it's a plain fact. You're able to feed many thousands of sentences of instructions and memories if you'd like to, it's just inefficient.
> Irrelevant; you were the one who was raising anecdotal evidence and defending it when challenged.
That "anecdotal" evidence is literally explicitly stated in the article
> Irrelevant; you were discussing the context window itself, not how many lines of text are loaded per note.
Literally quoting the article: "longer instruction files reduce adherence"
It doesn't matter if the context is 1mln tokens if a file with more than 200 lines can lead to loss of adherence and context pollution.
It doesn't matter if it is marketed as having the ability to hold the entirety of War and Piece if in reality it cannot hold much without being polluted.
> That's how analogues work.
Analogues never work.
> Irrelevant quote.
That is basically the entirety of your entire "argument". "What I say is correct, and Anthropic's article is seminal. Whatever you say, including quoting Anthropic's own materials is irrelevant if it doesn't fit my pre-biased worldview."
> The entire article is about Claude using memory.
No, it's not really about that once you cut to the meat and heart of it.
> Is it possible that you haven't used Claude since Sonnet 3.7?
Is is possible for you to not assign your opponents thoughts and words while completely dismissing what they are actually telling you? Apparently not.
> You're able to feed many thousands of sentences of instructions and memories if you'd like to, it's just inefficient.
Lol. So the size of the context is irrelevant when it's inneficient, and the models apparently lose adherence at as much as 200 lines of instructions. But sure, calling this "tiny context" somehow insults your sensibilities.
Anyway. I've seen enough of your "arguments" to know I couldn't care less to continue this conversation.
Adieu.