The impact of this incident is very unevenly spread.
Over 10 years ago, Mozilla shipped OneCRL [1]. We remediated this incident in a couple of hours for Firefox users who were protected automatically without the need for updates. Chrome, Apple and Microsoft have similar tooling. Since then, we've built more advanced tooling to handle revocation at scale for leaf certs (CRLite [2]).
However, for the rest of the ecosystem, there has been little to no investment in revocation and for those clients, this is a massive problem. Hopefully, we'll see folks invest now. Efforts like upki [3] look promising.
[1] https://blog.mozilla.org/security/2015/03/03/revoking-interm...
[2] https://hacks.mozilla.org/2025/08/crlite-fast-private-and-co...
Firefox Mobile 158.0 on Android 17 (Pixel 7) connects to all the demo sites with zero warnings. Any idea why the "automatic protection without updates" doesn't seem to be working with that combination?
Pixel 10 with 158.0.1 still connects to the emSign ones, even though the patch notes for 158.0.1 mention that it updates the CRL for those without 'Remote Improvements' enabled?
edit: Disabling all the eMudhra CAs in Android settings does nothing, cool?
edit: I guess there's no way to manually ask for CRL update, cool??
edit: Gave up and reinstalled Firefox. Seems to be fine now.
I am getting the same on GrapheneOS on a P10.
I fully expected disabling them in the phone settings and rebooting would have fixed it, but no?
Don't really wanna reinstall FF and lose all my configruation.
I'm on the same Firefox version on android and they are blocked now
On my just updated Firefox 158.0.1 on Android they are not blocked.
Edit: however on Firefox Focus 157.0.1 they are blocked.
I am seeing the same thing in the beta app (159.0b4) on an identical device. The sites are blocked correctly in Chrome.
Update: properly blocked now. Seems like it takes a few hours (days?) for the CRL update to propagate.
More on upki:
https://discourse.ubuntu.com/t/an-update-on-upki/77063
My interest in building OneCRL support into upki has definitely increased this week. Issue:
https://github.com/rustls/upki/issues/229
I'm hoping some organizations will step forward to fund work on this.
(Note that curl on Ubuntu 26.10 now has CRLite support baked via upki -- unfortunately that doesn't help with this because AIUI CRLite is only for end entities, not for intermediate certificates.)
It's very inconsistent for me even on just Firefox. On multiple systems running different instances of Firefox, checking with https://emudhra.we-have-a-problem.com/:
One blocks everything; one blocks 6 out of 12; one blocks the other 6 out of 12; one blocks nothing.
You mean the response was uneven. The impact was a complete and total potential compromise of literally everyone. Every domain. Every client using one of the public root programs. Every user of those clients.
I mean, it's an emergency, but the acute severity is debatable; many times a month the major browsers quietly deal with vulnerabilities that yield attackers direct control of victim computers; here, attackers get the ability to spoof websites if they're in a position to hijack connections.
The middlebox situation, with non-updatable roots, is very annoying. But middleboxes also have unfixable RCEs, so...
It's totally fair to evaluate different projects by how well positioned they are to update their WebPKI trust anchors. I'm just saying, keep perspective.
So it other words, it allowed attackers to do the one thing that Web PKI is supposed to prevent.
Being able to quickly update trust roots is at best a hack to paper over the issue when there is no good reason why every podunk CA should have authority to generate certificates for every domain in the first place. It's an inexcusable design flaw. Or a feature that gives power to root stores at the expense of everyones security, from the perspective of the root store operators.
Ok, now do that same paragraph, but for drive-by RCE vulnerabilities.
Nobody thinks this is a good thing. It's bad. Hopefully, eMudhra gets distrusted. We've done that to bigger CAs.
Keep in mind that a _lot_ of software out there uses TLS as its only trust anchor for updates, commands or similar.
This can and will be weaponized for take-overs of systems. They will be of the consumer kind, but they will also be of the industry kind. Let's pray that they will not be of the state or military kind.
You mean you and Chrome stopped supporting normal CRLs and OCSP, replacing (for normal users) these distributed-trust technologies with central lists controlled by browsers
There is a good reason they did. CRL and especially OCSP are both terrible and not particularly effective
OCSP stapling works fine technically, which is one reason it's still available in both browsers via enterprise policy. LE dropping support for it and browsers having insufficient courage to hard-fail doesn't make the model broken. It instead makes the ecosystem broken.
Just another step towards a tiny number of browser vendors and infrastructure companies controlling the internet.
Here's a question for Mozilla et. al.:
It's cute you have a magic DIY CRL. You accepted these assholes into your root program.
They lost the KEYS.
What is the screening process for CA's that can't be bothered to use a HSM?
Even Honest Achmed wasn't this careless.
IIRC up until very recently the screening process was "they report that they're good boys."
The screening process has been more involved in that since around the late 2000's. I wouldn't call 20 years "very recently," especially in the context of the history of the web.
circa 2007 I was looking over a college textbook on computer networking that I saw at a family member's house. It still said "While networks are inherently unreliable, it's not a stretch to imagine a future where one can stream video on-demand over a WAN like the Internet"
They didn't lose the keys, a sub-CA did.
Which is to say, sub-CAs in itself are a giant loophole that needs plugging. Especially since AFAIK nobody is bothering with name constraints (RFC5280 section 4.2.1.10).
The situation is even worse than I thought.
They hold intermediate certificates issued by TWO different root CAs, neither of which they operate.
So two separate entities didn't bother to read their certificate policy or inspect their operations. Have we not learned anything from past mistakes?
Roots should be responsible for all their sub-CAs, full stop. If they don't trust them enough to accept that responsibility, then the system is broken.
If you mean Comodo and Setigo, they're the same company, pre- and post-rename.
Let's just say they've had issues with signing sub-CAs before where those sub-CAs went on to misuse them before… looks like it was around 2011?
The reason nobody bothers with NCs is because they don't work very well in practice. They also have their own baroque matching rules that are not intuitive if you're from the land of UNIX globs.
(In fairness this is a self-fulfilling prophecy: a lot of validators handle NCs poorly or not at all, so there's no impetus to rely on them. But that's not something we can really save in the Web PKI's current state.)
> But that's not something we can really save in the Web PKI's current state.
I agree with your assessment but it's not an excuse to give up without trying. I hope this is just because I'm not looking in the right places, but I don't see people trying.
What really makes me salty is that there was enough energy to mess with the rules around EKU, the primary effect of which is that non-browser software is now starting to just ignore the EKU bits. Even if it would've amounted to nothing, I'd really rather that effort had gone into NCs.
I think a lot of people have tried! The reason EKUs stuck where NCs haven’t is because EKU rules are simple (in relative terms) and OpenSSL didn’t make as much of a mess out of them.
I can't help but read your post as "OpenSSL didn't make a mess of EKUs so the CA/B had to fill that role instead" :/
(I really am salty about the EKUs. In a few years the EKU field will just be useless, due to operational annoyance driving devs to ignore it. Except in browsers of course.)
eMudhra is also a root CA operator. One root CA operator cross-signing another root CA operator is a good thing actually, and it's what allowed Let's Encrypt to work before their root certificate was widely trusted.
Issuing sub CAs to non-root operators has historically been a big loophole though it's not as bad these days since multiple root programs now require approval before CAs can do it.
Didn't Let's Encrypt have to jump through all sorts of hoops to prove they were competent and trustworthy before that happened?
Now it's coming out these guys used "4" as a private key. That's a crazy amount of incompetence and malfeasance. Did they vibe-code a CA? HOW DOES THIS HAPPEN? I am dumbfounded.
You couldn't even do this if you installed Microsoft Small Business Server 2008 today, unpatched, with the CA on default settings.
eMudhra had to jump through even more hoops.
Obviously the system failed here, but it's a failure of a root certificate operator, not a sub-CA operator as claimed by eqvinox.
>What is the screening process for CA's
it's done in the open, and you're even allowed to voice your cute opinion during the multi-week public commentary periods.
here is an overview https://wiki.mozilla.org/CA/Root_Inclusion_Overview