When there's a major ongoing story, HN gets flooded with submissions about it, the majority of which just hammer on the same nail and add little that is new or interesting. During such waves, moderators downweight follow-up submissions unless they contain significant new information [1]. If we didn't, the front page would consist of stories about this one theme. Worse, that would accelerate the problem, since once they saw this, people would start submitting even more stories about that one theme. (There's a CS term for this situation—I want to say thundering herd, but is that accurate?)
We hit on this strategy after the Great Snowden Storm of 2013 [2], when many users justly complained that HN was overrun with stories that, though not strictly dupes, were filling the front page with dupiness. It has since proven itself through many epidemics, no pun intended.
You can also derive it from first principles. The thing we're optimizing for is intellectual curiosity [3]. Curiosity withers under repetition [4], so the most important thing for HN to avoid is being predictable. To a first approximation, you can say that HN is ok to the degree that it's unpredictable. But since a page of random numbers would do that even better, it's more precise to say that it needs the right balance of familiarity and unpredictability.
[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu...
[3] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
[4] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... and https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
Thanks for the explanation! I think it makes a lot of sense and I don't mind it at all. I was just surprised when I noticed the pattern at first, and was curious if HN was trying to reduce misinformation or what was going on.
I don't think I have heard "Curiosity withers under repetition" before. Is that something observed on HN over time or is it something that has been studied?
Either way I can see it working successfully on HN in comparison to other sites I visit that are just the same thing over and over.
I made that up a while ago. It seems obvious, which doesn't mean a study would be uninteresting! Unless they repeated it too often.
Tsk, tsk. Replications, Replications, Replications! ;)