About a month ago I wrote a perf tool for my application to measure its network message speed. It showed I could send 3,250,000 WebSocket messages per second in my TypeScript application. If I moved each test of an experiment into its own call stack I could send 3,250,000 messages as fast as 0.002 seconds. Claude confirmed the number.
It was about two weeks later I learned the number is wrong. The number would be correct if this were just a linear instruction processing problem. Actually the processing time, CPU and network handling, is about 0.08%. The other 99.92% of processing time is in memory. What was happening is the perf tool would push messages into a queue on the socket from a JavaScript loop, but the speed to process these messages and drain the network buffer was less than the loops execution speed.
Correcting for that dropped my transmission speed to a more realistic 100,000 messages per second. Modified queue management then allowed a rate of 800,000 messages per second. Pushing each message send into a separate call stack then allowed for a rate of 6,000,000 messages per second on a single socket but it sacrifices stability to reach that rate of transmission speed.
My wrong number was confirmed by Claude for the same reason I believed it, a confirmation bias that was well reasoned from the logic. Claude came to confirm the number for the same reason I did but came to that conclusion a completely different way.
Seems like persistent models like OpenAI's highly persistent internal model can become really effective over time. Those are the ones that drove most of the HF-OAI incident.
I have something like this very recently.
About a month ago I wrote a perf tool for my application to measure its network message speed. It showed I could send 3,250,000 WebSocket messages per second in my TypeScript application. If I moved each test of an experiment into its own call stack I could send 3,250,000 messages as fast as 0.002 seconds. Claude confirmed the number.
It was about two weeks later I learned the number is wrong. The number would be correct if this were just a linear instruction processing problem. Actually the processing time, CPU and network handling, is about 0.08%. The other 99.92% of processing time is in memory. What was happening is the perf tool would push messages into a queue on the socket from a JavaScript loop, but the speed to process these messages and drain the network buffer was less than the loops execution speed.
Correcting for that dropped my transmission speed to a more realistic 100,000 messages per second. Modified queue management then allowed a rate of 800,000 messages per second. Pushing each message send into a separate call stack then allowed for a rate of 6,000,000 messages per second on a single socket but it sacrifices stability to reach that rate of transmission speed.
My wrong number was confirmed by Claude for the same reason I believed it, a confirmation bias that was well reasoned from the logic. Claude came to confirm the number for the same reason I did but came to that conclusion a completely different way.
Why are two good quality comments on this thread dead on arrival?
I've seen this happen a lot recently, when the page still has relatively few comments. Insightful comments are dead leaving behind trivial ones.
What's going on?
Check out those accounts' other comments. They're AI spambots. This level of banal observation can now be produced automatically.
Seems like persistent models like OpenAI's highly persistent internal model can become really effective over time. Those are the ones that drove most of the HF-OAI incident.