It is both kind of hilarious and kind of terrifying to see this sort of anti-academic, anti-expert nonsense is bleeding in to %$&#ing software development.
All your written-in-production, battle-hardened code with no effete book-larnin' algorithms aren't going to run very well without a functional electricity grid.
What is going on? The grandparent comment is merely noting the novelty of a filesystem utilizing a recently invented data structure. The parent is weirdly mentioning how they wouldn't trust a research filesystem for real work (who would...?). Now THIS comment is claiming the parent comment is anti-academic and anti-expert when it's actually mainly raising common concerns about the disconnect between theory and practice (then this comment mentions the electricity grid, as if that's of any relevance??). Just a really strange series of disconnects between the arguments.
The grandparent comment is an example of the kind of “middlebrow dismissal” [1] that isn’t really welcome here.
It sets the tone with dismissive snark (“find and dandy”), then implicitly asserts that the project is not interesting because it’s not production-ready.
Of course it’s obvious to everyone here that version 0.2 beta software is not production-ready, so obvious that comments to that effect are at best superfluous, at worst annoying.
But its production-readiness is clearly not the focus of the discussion, rather its novelty and potential is. That’s what makes it interesting and worth discussing here.
[1]: https://news.ycombinator.com/item?id=4693920
I'm extremely pro-academic, but I think you're taking the least charitable interpretation of the parent. While I fully disagree with the parent on the value proposition here, they are quite correct that (at least most) phds aren't concerned with implementation problems like corner cases and long term maintenance. There are of course exceptions, but having worked on quite a bit of academic code, I can say that anecdatally maintainability is not a high priority. It's very much like a typical PoC is in a startup.
Why is this relevant? Just looking at the title and abstract, it is clearly among the most implementation-focused computer science papers ever written. It's the paper that accompanies BetrFS 0.2, incorporating that source code (which clearly has to handle edge cases), many measurements and discussions of tradeoffs. What more are you people asking for?
> looks at extremely practical paper, which rightly won a best paper award, probably for the very reason that it was extremely practical
> decides to have the thread descend into a dismissal of the value of best paper awards on the basis that they do not reward practicality
It's possible to regard the paper as high value, appreciate its value, and recognize that this is not a production filesystem.
To be fair, FAT32 is a production filesystem...
Why was the fact that it's not a production file system even brought up? Was it advertised as a production file system? Does the paper say that it is one? What was the rhetorical purpose of that statement?
I don’t think I dismissed the value of the paper. I pointed out that implementation may not be that good, and best paper award and quality of the implementation most of the time are not correlated.
You dismissed the value of the paper about ten times in a row, barely stopping for breath. Some of the tone is found in which thing goes on what side of the word "but", some in other words, but generally you really messed it up if you wanted to avoid insulting the authors and dismissing the value of research like this. There is an enormous gulf between a comment like yours
> That’s all fine and dandy, but [...] Winning best paper awards doesn’t say a thing about the implementation [...] the goal of PhD students is to publish papers [...] I couldn't trust [this]
and a comment like this
> This is a really impressive project. Obviously this is deeply academic, but since I am so impressed, I wonder what the plans are for this (or the same idea in a new fs) to reach the kind of commercial quality where I can use it in a production system.
If you were so aware of the general nature of academic research vs battle-tested implementations, then you would also know that filesystems are so incredibly complicated that the latter invariably takes on the order of 10+ years from a big team to create. When you forget this fact and say that a few-years-old implementation probably sucks because it's from academia, you're ignoring that NOBODY could have made it production-ready in that time, not even Microsoft or Apple or Oracle. Why would you criticise it on this basis? Choosing to do that was the biggest dismissal of the value of the work. Instead, you buried what was in effect a compliment (this would be useful for my production systems) under ten layers of insults.
The iJournaling paper was published in 2017 (and like many papers, it took multiple rounds of paper submissions before it was finally accepted; the academic procress is rigorous, and many program committees are especially picky).
The jJournaling ideas hit the upstream kernel in 2021 as ext4 fast commits, and no I don't consider it production ready yet. If the fast commits journal gets corrupted, it's possible that the file system will not be automatically recoverable, and may even lead to kernel crashes. I'd give it another year or so before it's completely ready for prime time.
But the other reason for the four year delay between 2017 and 2021 is because I had to find the business justification (and after that, the budget and head count) to invest the SWE time to actually implement the idea. A lot of people want new sexy file system features, but very few people are willing to PAY for them. So part of the job of an open source maintainer is not just to perform quality control and create a technical roadmap, but also to help the developers workin on that subsystem to make business cases to their respective employers to make a particular investment. The dirty little secret is that most people are pretty happy with the file systems as they currently exist; the bottleneck is often not the file system, and while certain file system features are _nice_, they very much aren't critical --- or at least not enough that people are willing to pay the SWE cost for them.
> anecdatally
Is this an anecdote vs. data pun? :D
Yes haha, sorry I use it way too much and it's become part of my vocabulary
I understand some of the frustration though. I was trying to do some audio processing work once. Found the paper(s), which promised code available from websites that are no longer available. Dug through the internet archive to find the zip files with the matlab code; managed to tweak it to run with the matlab version I have; found it works as described with the sample inputs, but crashes horribly on my inputs.
I've been debating where the anti-science behavior stems from. From reasonable people at least. The best i can come up with is that most reasonable people recognize how the modern age is an information war. Product sales, articles on economy, articles on politics, even some well advertised miss-steps like the sugar industry pushing/funding pro-sugar anti-fat papers way back (which may or may not be true, but it is a common trope parroted).
I assert that all this leads to people being paranoid about information of subjects well outside their expertise. Which is a really scary place to be. The answer seems non-obvious to me, but is likely nuanced.. and the public doesn't do well with propagating nuance in my experience.
I'm really interested in tooling to help disseminate information.
When I encounter it, I feel it's often a hatred of the "doers vs the thinkers"
My career path was 25 years engineering, before migrating into a hybrid EE/PM role as sort of a natural progression from being "the engineer who knew how to run the project". Once I started learning the more formal approaches to PM, it uncovered an entire world of engineers who have an incredible hatred of any sort of planning of any kind, because all planning time is wasted and we should all just be doing.
The parent comment here feels the same way. Hatred towards research because it's all theoretical (I guess?). It seems clear as day that the best approach is a marriage between the two.
I think it's summed up by the mantra "those who can, do. Those who can't, teach."
I suspect "hatred of any sort of planning of any kind" is actually hatred for planning that fails to embody any actual strategy (and is therefor a waste of time because it doesn't help solve any actual problem except maybe alleviate non-technical vips anxiety with false hope). "formal approaches to PM" evokes just that sort of thing in my mind (kpis/goals masquerading as strategy, gantt charts, etc)
When I worked in a university lab that builds stellarators I learned that misgivings towards researchers is all bullshit. There are engineers and there are pencil pushers. Pencil pushers burn money and bark loudly. Real engineers can plan and execute on time and under budget.
I think one source of anti-science behaviour might be betrayed expectations. I know that I personally have stronger expectations for academia and scientists than for other people. So when someone from academia, a scientist, an engineer betrays these expectations, I feel worse than if a regular person did it. There's a feeling of "If we can't even count on those people, what are we even supposed to do?". For example, at the beginning of the COVID pandemic (around January 2020), I read a lot about it. Lots of very smart people were saying that this could be a big pandemic. I talked about it with a doctor in a non-professional setting that told me to basically not worry about it, that it wasn't going to be anything huge. This time I was right and he was wrong. Was it because I searched more about it? Was it just luck? I don't know. But I know that it made me lose a bit of trust with that person.
I think the origin of this might be on how I (or we) see those people. You're supposed to follow what the doctor tells you, what the scientists tell you. But in a way, since you're supposed to follow what they say, they have some kind of responsibility towards you. And when they say something wrong, it's way worse than when a regular person says something wrong. It's like when you're young and your teacher or your parents are wrong, it's very frustrating.
Your example about the sugar industry is also a great one. Try to understand a bit more about nutrition, and soon you'll hear all kind of conflicting advice and explanations from very different experts.
I know that personally I have to work on myself and accept that those people are humans, and make mistakes, just like me. But just like telling people to eat less and move more didn't solve the obesity epidemic, I'm not sure that this solution will scale to a large population.
This is interesting, and I've seen a bit of this sort of behavior, too.
Some people seem to confuse expertise for a claim of infallibility, and when some expert get something wrong, the reaction is to conclude that expert advice is worth no more than the guy on the teevee hawking vitamins and anti-expert bile.
It is a sort of Leveler belief wrapped in a search of an Oracle.
Something you may be more familiar with is people's concept that someone that "knows computers" is familiar with any and every sort of task that involves a computer whereas in fact this could encompass a wide variety of different skills that require an individual investment of time.
The same can be said of medicine where encompasses a very broad set of skills. Your doctor may have been an expert in sports medicine or brain surgery but it doesn't automatically make him competent and epidemiology. It also doesn't force him to pay attention to current developments in the news which is likely what informed your opinion. Personally I found it was completely obvious in January that we would be dealing with a crisis because I followed the situation and suspect strongly that your doctor friend did not.
There is also the issue of survivor-ship bias. We worry about many things and we will absolutely recall the times our worry was justified and forget when it we are mistaken. If Yellowstone ever blows there will be many people who knew it was just around the corner and this will be true if it blows now or in a century, whether or not we have any scientific basis for the thought process.
TLDR: A singular doctor of unknown specialty getting it wrong in January isn't a flaw in science. Science isn't expected to be very good at ensuring a single expert of only tangential expertise gives you the right answer whereas it is reasonable good groups sometimes slowly arriving at increasingly correct answers. If you want a more correct answer consider consulting or reading what several people of relevant expertise who are up to the minute on current information have to say.
Readers of Nassim Taleb, doubts raised over our blind trust of institutions like their reaction to coronavirus, any mainstream nutritional advice (fat is bad and causes heart disease, “the China study” was written by a nutritional biochemist), papers that p hack to be published, replication crisis of psychology, statistics and using it to lie.
There is no paranoia, an allegory is an ivory tower of studies about language from non native speakers with a PhD, and a native speaker who gets no recognition for using it daily but doesn’t have a fancy diploma or credentials so someone who speaks “proper” Spanish from Spain who has never been to Spain is more “credible” than the Mexican speaking “improper” Spanish daily.
Academia is to be ignored unless it’s relevant, Fritz Harber didn’t need the Nobel prize to have real world effects in nitrogen fixing to help farmers grow and sustain our population, Obama wasn’t more relevant because of his Nobel prize, and Perelman’s refusal of the Fields Medal doesn’t change his contributions.
Readers of Nassim will also recognize his critiques of academia are mainly targeted at social sciences and similar fields that can only “prove” their findings statistically, where p-hacking and incorrect use of models and wrong distributions and the like result in bad findings passed off as good.
That’s not the case with computer science, at least in systems subfields like filesystems, where theories can be implemented in isolation and shown to either work or not.
Disagree. Artificial benchmarks and p hacking for showing good performance in CS are also statistically proven. The best result in geekbench doesn’t mean anything to me.
Benchmarks and geekbench are not "where theories can be implemented in isolation and shown to either work or not."
Maybe I don’t understand your point, in psychology for example sterile lab tests are isolated and can be shown to work or not, is it not the same idea here?
CompSci theories are essentially mathematical proofs. You create a proof of something, then you build it to test it, to make sure your math is actually correct, and that the theory works in implementation.
Proof of correctness doesn't rely on having a large cohort of test subjects undergoing an experimental trial of some sort, and then interpreting the results with statistical models, distributions, p-values, etc.
I don't know psychology in depth, but if there are similar kinds proofs without requiring statistical analysis of a large experimental cohort, then I don't think Taleb's criticisms are aimed at those either.
It's the fundamental problem of knowledge - can truth be known via logic and reason, or via empiricism and observation? The answer to both is, sometimes, but with caveats.
Peter Norvig also wrote a good take on all the ways studies using experimental cohorts can go wrong: https://norvig.com/experiment-design.html
Source code availaility for academic papers is important for reproducibility, so other people can run additional experiments and demonstrate that the performance numbers in the paper weren't fudged.
It's not necessarily going to be useful for production use. There are exceptions to this; for example, there are papers were the authors claim that a flash/SSD emulator is suitable for use by other academics to experiment with their FTL ideas, or to grab network traces from NFS traffic so they can be replayed to test file system performance using real world data. In those cases, the point of the paper is to share a tool that can be used by other researchers (as well as the team that created the tool in the first place), and in that case, the code had better d*mned well work. (But even then, there might be buffer overrun bugs in the SSD emulator; which is fine, since the FTL is intended to be provided by an academic researcher, and it is not expected to accept arbitrary inputs including from a malicious attacker.)
I don't know whether the papers in your case were meant to be documentation for code that was meant to be shared, or to explore a particular research idea, the code was only meant for that particular purpose. Even if it was for the former, there's an awful lot of bitrotted, unreliable "abandonware" on sourceforge or github that can be pretty skanky; that's a problem which is not restricted to academically published papers.