I know this is tinhat territory, but it's weird this happens right after the FB whistleblower interview on 60 minutes.
The outage has pretty much buried that story, and perhaps more importantly, stopped its spread on FB networks.
That said, I can't see how FB managers and engineers would actually agree to carry out something like this intentionally.
> The outage has pretty much buried that story,
Strongly disagree. The outage has millions of people entering "Facebook" into their search engines. Most engines will conveniently put related news at the top of the search results page. The most recent and widespread Facebook-related news story is about the whistleblower.
Plus everyone has a lot of spare time to read the article now that Facebook and Instagram are down.
The outage didn't bury the story. It amplified it. Any suggestions that Facebook did this on purpose don't even make sense.
> recent and widespread Facebook-related news story is about the whistleblower
With respect I am pretty sure that the most recent and widespread Facebook-related news story is this one.
Holistically I agree that this isn't the kind of distraction Facebook wants, although it tickles me to imagine Mark in the datacenter going Rambo with a pair of wire cutters.
Yeah but journalists are happy to connect the dots between the two stories and honestly my brain loves the coincidence of these two thingy being clustered: but the how is clear: Earlier this morning, something inside Facebook caused the company to revoke key digital records that tell computers and other Internet-enabled devices how to find these destinations online.
Yeah but journalists are happy to connect the dots between the two stories and honestly my brain loves the coincidence of these two thingy being clustered: https://krebsonsecurity.com/2021/10/what-happened-to-faceboo...
That is in no way gonna make people forget the whistleblower story - if anything, it's gonna increase the antipathy to having a single point of failure. Face it, everyone hates FB, even the people who spend the most time on it.
datacenterS
Anecdotal, but I just tried Google + Bing and topline Facebook-related news is all about the outage.
Also anecdotal, but I didn't know about the whistleblower until I searched Twitter for "facebook" when I learned about the outage.
I also didn't know about the whistleblower until seeing it as a top tweet, however...
The whistleblower is kinda silly
If FB could increase revenue by having a "safer" algorithm then of course they would. Every company is just trying to increase revenue..
> Any suggestions that Facebook did this on purpose don't even make sense.
Unless another disgruntled employee knew it would amplify the story.
Sample size of one but a quick google shows me zero whistleblower news and 100% outage news.
https://i.imgur.com/IaSoR0w.png
1 article about the whistleblower and 2 about the outage. Both about the outage also mention the whistleblower, so you could say that's 100% of coverage at least mentions the whistleblower.
Also 1 out of 3 tweets also mentions the whistleblower.
> Strongly disagree. The outage has millions of people entering "Facebook" into their search engines. Most engines will conveniently put related news at the top of the search results page. The most recent and widespread Facebook-related news story is about the whistleblower.
I am seeing 0 news about the whistleblower when I google Facebook. Only outage news.
Every outage piece of news I'm seeing mentions the whistleblower.
Who reads the article? If I google "Facebook" to see if there's an outage, I see the first headline that says it's an outage and leave. Maybe curious few percent will.
This wasn't the case six hours ago. I checked, there were scattered outage stories and all whistle blower
I'm one of those who had no idea about the whistleblower story, but I learned of it through reading about Facebook network outage.
Yeah but reading about it but also being able to communicate about it on the largest network (the one in question too) are 2 separate phenomena. No one can go on there right now and say I'm deleting my account, who's with me?
Not at all. I just tried searching for "Facebook" on Google. The whistleblower story is not on the first page of search results. The outage is mentioned half a dozen times on that same page.
I assume this outage is costing millions per hour. And it's not exactly great advertising for Facebook, either. I doubt very much they would do something like this on purpose.
Right, I know that, and I usually try to avoid conspiratorial thinking, but man, Zuck doesn't make it easy.
I'm just trying to process that FB is having its historic, all-networks global outage today of all days. And I bet FB would have paid double of whatever this will eventually cost them to make that story go away.
Dividing up last quarters $29B revenue leads to approximately $13.4M per hour of downtime, now past $53M after the 4 hour mark.
But I haven't paid this much attention to Facebook in over a year.
That doesn't make a lot of sense though - Facebook generates revenue primarily from ad traffic (on all sorts of sites). It needs to be up for reputation and to harvest ever more detail for 'improving' those ads, sure, but not for revenue. (Modulo blip from ads on its own site.)
So you can't just divide over time like that.
What? It absolutely needs to be up -- those ads being served are on Facebook and Insta, not display banners on random sites.
That's what I meant by 'ads on its own site' - but I was under the impression that Facebook generated most revenue from selling data/ads for display elsewhere (as well as on Facebook.com itself, and other subsidiaries). Perhaps I was wrong about that? Quick search shows up 'audience network', but I'm not sure to what extent that's what I was thinking of.
Nope, for the most part all the ads that Facebook serves are for facebook owned sites and properties. They don't sell data, or have general ad placements on 3rd party websites.
wow...
It sounds like they are not even able to serve ads, on any property. So while far from perfect, it's probably a decent estimate without doing in-depth analysis.
Sounds like a drop in the bucket for them.
Unless “they” were one or two disgruntled employees with the access, know-how, and motive to execute a “mistake”. Emphasis added.
If it was intentional, that's serious jail time territory. That's a high price to pay for such limited downtime. I'm pretty sure an intentionally malicious actor with that type of access could do much worse things.
I’m pretty sure the vast majority of entry level spy craft is about convincing people to do highly illegal and destructive things from a place of fear.
Not saying this is the work of spies, just that it’s not unimaginable to think some middle manager could convince themselves or a subordinate to do something drastically illegal out of some fear that terrible things would happen otherwise.
I'm curious as to what law, exactly, they would be breaking. Sabotage in the US code is defined mostly in terms of war material and damages done to physical "national defense" properties. Certainly an employee would be fired and sued by the company, but is deliberately changing a routing policy (and not something like a worm or virus that deletes or otherwise degrades hardware and software) a crime?
IANAL but I would assume computer fraud and abuse act:
(5)(a)knowingly causes the transmission of a program, information, code, or command, and as a result of such conduct, intentionally causes damage without authorization, to a protected computer;
That’s the one.
The recent Van Buren decision would make that unlikely.
https://news.ycombinator.com/item?id=27389500
In the cases cited under the CFAA (such as https://scholar.google.com/scholar_case?case=124545279862007...) it seems the employee deleted data and private info. In this case, no data was deleted or other computing property damaged it just became unreachable.
Proof of intent is a significant burden placed upon prosecution. If that can be overcome, there’s legal precedent for criminal conviction namely under the CFAA.
https://tadlaw.com/can-charged-crime-sabotaging-employers-co...
> stopped its spread on FB networks.
bingo. I don't care whether it's in the realm of tinfoil hat or not, this is the very real effect that this outage has had. By the time Facebook is back up, people on Facebook will be talking about the outage, not about the whistle blower report. Intentional or not, it will certainly be in Facebook's favor.
Facebook controls the algorithm, wouldn't they just be able to down amplify how much that story is spread on it's network? (Rather than resort to this?)
Just to clarify...I pretty obviously don't think that Facebook intentionally pulled the plug to suppress a critical story. But the inadvertent effect of the downtime is nonetheless the fact that the critical story will not be the center of discussion on Facebook when Facebook is back up.
A general outage is more deniable than dampening a negative story.
I love a good tinfoil hat theory, but in this case I doubt it. I have FB blocked on my network via pihole, but I don't explicitly block Instagram. Until sometime late last week (I noticed on Saturday), blocking facebook.com also blocked Instagram. As of this weekend, Instagram works just fine even with those blocks in place.
I suspect Facebook was making some change to their DNS generally, and they made some kind of mistake in deployment that blew up this morning.
I'll take the other side of that bet. Who messes with routing tables at noon on a Monday?
Someone who doesn't want to deploy on a Friday?
I was thinking more of the ticking time-bomb variety, but that seems as good a time as any?
Nah, a ticking time-bomb would "explode" on Christmas (or Aïd El Kebir, etc.), whenever most of the employees who could do something about it are absent.
Still wasn't clear enough with my analogy! I was thinking more like a dam failure due to operator/designer error, not sabotage (but who knows). The damage is really small signs initially, followed by rolling catastrophic failures.
Ah, my bad, I misread your comment!
Yeah, that could happen.
It sounds like the perfect time honestly. If you fuck something up, you have the whole week to fix it.
When I worked there a few years ago, the routing tables were being updated almost all day every day, primarily via automated processes.
They deployed this morning that doesn’t imply they implemented anything. I can’t think of a better time that way you have the whole week work on anything that uncovers. Or in the case of something this big they have the rest of the day to freak out.
> it's weird this happens right after the FB whistleblower interview on 60 minutes
Could a pang of morality have struck one of the employees with the keys to the kingdom?
Counterpoint: I had not even heard about the whistle-blower until seeing stories about the outage. One of the largest web services in the world being out of commission for multiple hours is a big deal in 2021. It's a top story on most news sites and other social media (e.g. here at HN, reddit, twitter). If you want something to pass under the radar, it's probably best to not attract global attention.
If I was so inclined to put on my conspiracy theorist robe, I’d guess more likely related to the bulk of Pandora Papers news hitting today.
Or evergrande.
I had no idea about the 60 Minutes thing until people started mentioning it in response to this outage.
Agree. I just did a quick check and 60 Minutes averages around 10 million viewers. It's not like in 1977 when something 20%+ of the US population was watching that show.
It is pretty well available — and quickly — via piracy means, which I have always thought interesting for its somewhat esoteric content.
Most people outside of the US don't even know what "60 Minutes" is. Even fewer have heard about that report. And even fewer care. But everyone has now heard about the outage. This would be the worst possible way of trying to stop the spread of the story.
The more likely scenario is that this was the final straw for some disgruntled employee who decided to pull the plug on the entire thing.
> Most people outside of the US don't even know what "60 Minutes" is.
I live in Australia. 60 Minutes exists here as well.
https://en.wikipedia.org/wiki/60_Minutes_(Australian_TV_prog...
arguing on the the basis of a strong cultural differentiation between the us and aus might not work as well as you think.
Not just that, but another story just broke about the sale of personal info on 1.5 billion FB users.
Maybe this is just to cover the fact that they leaked information about 20% of the earth's population?
> they leaked information about 20% of the earth's population
This is straight up false. It was scrapers extracting data from public profiles. They already incorporate anti-scraping techniques, so there's not much they can do other than require every one to set their profile to private.
If you don't collect the data in one place, there's no chance of leaking it.
If they want to position themselves as the global phonebook, that's fine, but they should be open about that.
Edit to add: If you aren't in the "gather and sell access to everybody's data" business, "private" is a sensible default setting for that information. On the other hand, if you're Facebook...
It's kinda in the name isn't it?
Phonebook... Facebook...
Here's the interview (which I had totally missed btw)
https://www.youtube.com/watch?v=_Lx5VmAdZSI
Hanlon's razor applies here, but it's a lot less fun. :)
If we're in "tinhat" territory: it seems extremely odd to me that this whistelblower seems to be "blowing the whistle" on the fact that facebook isn't doing enough to control what people are thinking and talking about.
Like...what? "Brave whistelblower comes out showing that facebook isn't doing enough to control what you are thinking!" is sortof arguing past the question. Should facecbook be in charge of deciding what you think?
> this whistelblower seems to be "blowing the whistle" on the fact that facebook isn't doing enough to control what people are thinking and talking about.
That is not at all what the whistleblower is alleging. Facebook already controls what content you are seeing through its news feed algorithm. The parameters to that algorithm are not a 1-dimensional "how much control", but instead uses engagement metrics for what content to show. The whistleblower claims that the engagement optimization, according to facebooks own research, prioritizes emotionally angry/hurtful/divisive content.
I didn't have the time to watch the interview yet but... wasn't it common knowledge for years?
Is there anything else in the interview the whistleblower alleges, or can prove?
We all knew Facebook is bad for society. The whistleblower showed us that Facebook has done internal studies and that these studies have shown their products are bad for society/contributed to the insurrection/promote human trafficking/damage teen mental health/etc. But even with these studies, Facebook has decided to prioritize growth and revenue, rather than fix the issues that are bad for society. What this whistleblower leaked will hopefully lead to some sort of government regulation on social media.
Without regulation, social media will always prioritize profit.
They are exercising that power already, they are just explicitly doing so in a way that tears down the trust in society because makes them money, rather than encouraging a less I divisive and more fact based conversation, because that doesn’t make them as much money.
The problem isn't that Facebook isn't do enough to control what you are thinking, it's that it's doing way too much!
That news broke ~12 hours ago right?
> I know this is tinhat territory, but it's weird this happens right after the FB whistleblower interview on 60 minutes.
It's not like this is a new thing. We've been getting [facebook does awful thing] news stories pretty consistently for years now.
I actually think most importantly it shows everyone what the world without FB is like ;)
FWIW, every article I've read has referenced the interview, and I personally find it hard to believe Facebook would be unaware of the Streisand Effect
It’s like watching a hostage over-analysing why the abductor forgot to lock the door. Just get out en enjoy your newfound, albeit temporary, freedom.
> The outage has pretty much buried that story, and perhaps more importantly, stopped its spread on FB networks.
Buried the news ... which is basically as noteworthy the news that water is still wet. What exactly did she reveal that was not known before, or is it somehow newsworthy that Facebook also knew what everyone else knew? The real news ought to be how that managed to make it to the headlines.
As much as I'd love to imagine FB rage-quitting the internet because people don't seem to appreciate them enough, I'm pretty sure it's a coincidence. Probably has more to do with it being Monday (you don't put big stories on Friday and you sure don't deploy config changes on Friday!) than anything else.
> That said, I can't see how FB managers and engineers would actually agree to carry out something like this intentionally.
They can either agree to comply with the orders from up above or they face consequences? How is that hard to comprehend?
I was thinking more along the lines of the Pandora Papers hitting the MSM.
Ah yes, the best way to bury a moral scandal of the kind that usually gets forgotten in a week is to undermine the trust of almost every single user worldwide. This is a very good conspiracy.
Did the whistleblower reveal something we didn't know already?
To me this seems like a million dollar mistake.
> Did the whistleblower reveal something we didn't know already?
A lot. The resulting Wall Street Journal series directly led to the shut down of Instagram for Kids.
I see it as similar to Snowden, in the sense that everybody kind of knew (actually guessed) but now we actually know. It doesn't come as a shock, but it's important information to have since it can be now argued with authority.
The whistleblower revealed that Facebook knows it is bad for society. The documents also show Facebook actively optimizes its algorithms for "bad for society" content because that drives engagement which makes them more money. Furthermore Facebook doesn't do as much content moderation in regions/languages with low usage numbers because it costs more than those users make them. So calls for genocide in Myanmar basically go unchallenged and unmoderated because Facebook doesn't make much money in Myanmar. Sorry genocided minority, you should have been more valuable to Facebook.
> The outage has pretty much buried that story
It hasn't on the BBC. They're airing both stories.
> would actually agree to carry out something like this intentionally.
Well, they work for Facebook. In my opinion you would have to have no morals to join that corporations in the first place, so I can imagine such ask would be just another dirty task to do. They seem to love it.
The story that a woman at Facebook doesn't think they're going far enough to control speech they hate and bad-thoughts?
I think Facebook is awful, but her primary complaint seemed to me that she lacked controls for what people like her, you know, the good people have access to prevent anyone else from seeing. That she was powerless to stop users from saying the wrong things. How was her motivation anything but a desire for more authoritarianism? She said she specifically took the job on the condition she could monitor and direct posts to prevent the wrong info from being online, that's the last type of person you want in that position, the one that wants it.
I expect that we're still pretending Facebook is "just a private business", despite it being unlike any in history and that the ties to government are completely benign.
I'm not saying she was wrong in any claim about internal discussions. But, if you can not imagine yourself being on the wrong side of someone like that, you have limited imagination.
Facebook is surprisingly tolerant of controversial subjects. YouTube has gone scorched earth on millions of channels and deleted years of work of many people. Facebook was far more lenient and you could talk about non-official covid information for example where YouTube deleted anything that wasn't official narrative with extreme prejudice. Given how much bad stuff all over the world is happening to sacrifice freedom to get everyone to tow the official line on Covid that is complete science fiction level totalitarianism, I am sure Facebook made some very powerful and determined enemies with its more lenient stance. I was downvoted earlier for saying this was an intentional takedown and deleted my comment, but now I think this could be a full blown William Gibson Neuromancer Cyberpunk level corporate takedown attempt in progress!
She said she wanted FB to do something to stop misinformation and hate speech but what we've seen from Reddit is that "are mRNA vaccines actually safe?" becomes misinformation and "we shouldn't perform elective life-altering surgery on pre-teen children" becomes hate speech. There's not much I applaud Facebook for, but not listening to this woman is one of the few I do.
It also looks like its much deeper than just people not finding the site. Employees are all locked out and there's another story on the front page on HN saying employees are locked out of the building as well.
If you wanted to scrub a lot of the data and nefarious evidence the whistle blower brought out, this would be a great way to do it, under the guise of a simple "employee screw up" cover story.
Its hard for me to think something more nefarious is afoot considering FB's track record with a myriad of other things. At this point, it seems more likely something sketchy is going on and not just some random employee who screwed up and brought down the entire network with a simple change. I would assume there are several layers of decision makers who oversee the BGP records. I have a hard time thinking one person had sole access to these and brought everything down with an innocent change.
FB has too many smart people who would allow a single point of failure for their entire network such that if it goes down, it becomes "a simple error on the part of some random employee". This is not some junior dev who broke the build, its far more serious than that.