Since Sunday 00:18 CEST, Codeberg.org is offline. From our investigation, our primary location lost power in our racks, leaving the majority of our servers and some network switches offline. We're waiting for a fix from the datacenter operator.
" from that status page.
That, or the massive lightning that's going through the region, (due to the heatwave). Since it's quite late at night, heat wouldn't be my first guess.
Has anyone used the Repository Mirroring Feature [1] to mirror repos across self-hosted Forgejo/Codeberg/Github? How effortless is it? Ideally, I'd like to only ever push repos/branches to my self-hosted Forgejo, and have those changes automatically reflected on Codeberg/GH without thinking about it.
I am using it to backup my public and private repositories to Github and it's effortless, indeed. I am using ssh protocol and a read/write deployment key. Also, I anticipated `git push --force` could be an issue, it's not.
Depends on the nature of your startup, and if you have significant infrastructure/architrcture for your workload or not. If you have significant scale you likely have a DevOps / SysAdmin type.
Yeah, and if you're early, like a few-person startup, you hopefully have someone who can do ops things as needed.
When I was the one-person eng and ops departments for a few-person seed-stage startup, which already had a major customer in critical production... even when we bought SaaSes (e.g., GitLab), I still needed to spend significant time on the recurring manual steps of backup processes for everywhere we put data we couldn't afford to lose, even after I'd spent time to automate everything that was worthwhile to automate.
For my last side-startup, I just self-hosted Forgejo myself (Git, Issues & Boards, Wiki), and it's less work than when I was doing the same with paid GitLab SaaS.
It depends on the case-by-case circumstances, but sometimes self-hosting an open source package, on your laptop, random PC in the corner, or your existing AWS account, takes less of your valuable time than a paid SaaS does.
This seems like a weird comment since Codeberg is only for open source projects, you literally cannot use it for private code. On-prem Forgejo would be the equivalent to on-prem GitLab, both of which are unrelated to this outage.
I suppose this is a good opportunity to ask, why do people get so affected by DVCS hosts going down? You can work locally with Git without uploading every change. Despite the constant reported GitHub downtime, I have not ever been adversely affected even once, since pushing and pulling are done every few days and I can freely branch/commit/merge locally.
You could use separate tools for those tasks. JIRA/Bugzilla/etc. for issue tracking, Sublime Merge or equivalent for comparing a dev branch to a main branch, and CI/CD with Azure Pipelines or whatever the “modern” equivalent of Jenkins is.
That isn’t as convenient as an all-in-one tool, and might not be what the user you’re responding to is doing. But it’s doable.
Sure, but then you're vulnerable to that going down. The question was about why people can be substantially impacted by relatively brief forge downtime.
Nowadays, these code forges have also become a centralized place for issue tracking, kanban boards, wiki editing and, specially, as CI/CD servers, in the case of GitHub Actions, which are, sometimes, the only for you to deploy software to package repositories. The same limitations apply to GitLab CI or Codeberg's Forgejo Runners/Woodpecker.
Whenever GitLab, Codeberg, BitBucket and, mostly, GitHub goes down, a lot of the software and websites you use can't be updated, including dependencies of your software that you're pulling from npm, for instance.
Finally, companies use code forges mostly for the ease of doing code reviews through Pull Requests/Merge Requests. Developers rarely, if ever, actually merge branches locally, before having it reviewed by peers in one of these code forges.
Git is a DVCS, but many companies have a build server/cluster that depends on Github or Codeberg being available.
Teams I've worked on for the last several decades aim to push 10-20 builds per day to external alpha testers, so any downtime in Github is going to be an impediment.
I have always wondered why don't they charge at least based on storage or so? Maybe above a basic/generous tier? I mean if I'd love to pay a few USD a year for say a few tiny repos that no uses other than me, and a static landing page (like Github Pages) that no one visits except me (every few months; maybe a recruiter or two, though I seriously doubt that). I'd rather pay a place like Codeberg than Github. From latter for such basic offerings I'd expect free services, yup! (Because I already pay them in other ways, like my data and all that.. but that's really beside the point). As for funding or becoming a member, I am not there yet and also that is not regular enough for me, that is not streamlined for me personally.
How… why… what? Let's say it's like.. well… I don't know.. anyway.. people (esp. from USA), please don't be offended (discussion on this topic has felt like we are from different planets), I'd rather pay a simple restaurant bill where 2Y tip is mandatorily part of the bill than a Y tip that I would "leave at the table" or "drop in a tip jar" or "generously" "hand it" (or slip it) to the staff. I have lots of problems with the latter but let's not get into that here anyway.
I am VERY active in technology and that I haven't heard of it means something, too. I notice--far too often--that obscure things like this get posted here and elsewhere which can confuse others into thinking they have to get involved or learn about it cause they saw it on <this site> so they have to learn it, too, when it's only a niche case among few.
I see it mentioned any time GitHub is shitting the bed (frequent enough) and someone asks for self hostable alternatives or when someone ask for a European GitHub alternative. I think you just might not be paying enough attention.
Again, Codeberg isn’t obscure. Worrying about front page HN links “confusing” others because you haven’t heard of something is about the silliest thing I have read on this site.
I can't take it anymore, I'm moving all my projects to github!
Is Github free of problems? I don't know. Maybe an option would be hosting gitea and sync projects to an online account.
I believe analogpixel is being facetious
The GP is joking.
Sarcasm is a thing.
Sounds like we have the basics of an oscillating system now!
I wonder what it's resonant frequency is.
https://status.codeberg.org/status/codeberg
"Power Outage
Since Sunday 00:18 CEST, Codeberg.org is offline. From our investigation, our primary location lost power in our racks, leaving the majority of our servers and some network switches offline. We're waiting for a fix from the datacenter operator. " from that status page.
Maybe related to the heatwave? I've heard some European data centres are having trouble with their cooling systems.
I'll put my money on AI software contributions...
That, or the massive lightning that's going through the region, (due to the heatwave). Since it's quite late at night, heat wouldn't be my first guess.
Has anyone used the Repository Mirroring Feature [1] to mirror repos across self-hosted Forgejo/Codeberg/Github? How effortless is it? Ideally, I'd like to only ever push repos/branches to my self-hosted Forgejo, and have those changes automatically reflected on Codeberg/GH without thinking about it.
[1] https://forgejo.org/docs/v15.0/user/repo-mirror/
I am using it to backup my public and private repositories to Github and it's effortless, indeed. I am using ssh protocol and a read/write deployment key. Also, I anticipated `git push --force` could be an issue, it's not.
I use it to sync projects that I mainly host on Codeberg up to GitHub. It was a "set it and forget it" kind of thing.
This is exactly what i do, its very easy. I mirror across codeberg, github and a local forge instance
A large chunk of companies I've worked for or consulted for had their own on-prem Gitlab. I think they chose correctly.
On perm Gitlab has a ton of problems too.
For examples? Never ran into them myself but I don't do ops.
I don't do ops myself so I don't know the exact details but sometimes Gitlab is down or there are strange issues with CI/CD breaking.
We had zero in the last 7 years. But we are only a small team of 8.
Codeberg runs open source Forgejo, and you could on-prem that too (for no license cost), if it suits your needs.
GitLab is more powerful in some ways, but early startups might want to look at Forgejo first.
> but early startups might want to look at Forgejo first.
Sorry, but there are a million things to do. Paying someone to self-host Forgejo isn't even on that list. We'll just pay someone at the moment.
Depends on the nature of your startup, and if you have significant infrastructure/architrcture for your workload or not. If you have significant scale you likely have a DevOps / SysAdmin type.
Yeah, and if you're early, like a few-person startup, you hopefully have someone who can do ops things as needed.
When I was the one-person eng and ops departments for a few-person seed-stage startup, which already had a major customer in critical production... even when we bought SaaSes (e.g., GitLab), I still needed to spend significant time on the recurring manual steps of backup processes for everywhere we put data we couldn't afford to lose, even after I'd spent time to automate everything that was worthwhile to automate.
For my last side-startup, I just self-hosted Forgejo myself (Git, Issues & Boards, Wiki), and it's less work than when I was doing the same with paid GitLab SaaS.
It depends on the case-by-case circumstances, but sometimes self-hosting an open source package, on your laptop, random PC in the corner, or your existing AWS account, takes less of your valuable time than a paid SaaS does.
This seems like a weird comment since Codeberg is only for open source projects, you literally cannot use it for private code. On-prem Forgejo would be the equivalent to on-prem GitLab, both of which are unrelated to this outage.
I’m using it for private, not publicly visible code right now.
You may be violating Codeberg's rules. They are not a service for developmental of closed-source software.
https://docs.codeberg.org/getting-started/faq/#how-about-pri...
I suppose this is a good opportunity to ask, why do people get so affected by DVCS hosts going down? You can work locally with Git without uploading every change. Despite the constant reported GitHub downtime, I have not ever been adversely affected even once, since pushing and pulling are done every few days and I can freely branch/commit/merge locally.
Do you not spend much time writing and discussing issues or reviewing code?
You could use separate tools for those tasks. JIRA/Bugzilla/etc. for issue tracking, Sublime Merge or equivalent for comparing a dev branch to a main branch, and CI/CD with Azure Pipelines or whatever the “modern” equivalent of Jenkins is.
That isn’t as convenient as an all-in-one tool, and might not be what the user you’re responding to is doing. But it’s doable.
Mailman provides decentralized issue tracking.
Sure, but then you're vulnerable to that going down. The question was about why people can be substantially impacted by relatively brief forge downtime.
Nowadays, these code forges have also become a centralized place for issue tracking, kanban boards, wiki editing and, specially, as CI/CD servers, in the case of GitHub Actions, which are, sometimes, the only for you to deploy software to package repositories. The same limitations apply to GitLab CI or Codeberg's Forgejo Runners/Woodpecker.
Whenever GitLab, Codeberg, BitBucket and, mostly, GitHub goes down, a lot of the software and websites you use can't be updated, including dependencies of your software that you're pulling from npm, for instance.
Finally, companies use code forges mostly for the ease of doing code reviews through Pull Requests/Merge Requests. Developers rarely, if ever, actually merge branches locally, before having it reviewed by peers in one of these code forges.
Git is a DVCS, but many companies have a build server/cluster that depends on Github or Codeberg being available.
Teams I've worked on for the last several decades aim to push 10-20 builds per day to external alpha testers, so any downtime in Github is going to be an impediment.
I have always wondered why don't they charge at least based on storage or so? Maybe above a basic/generous tier? I mean if I'd love to pay a few USD a year for say a few tiny repos that no uses other than me, and a static landing page (like Github Pages) that no one visits except me (every few months; maybe a recruiter or two, though I seriously doubt that). I'd rather pay a place like Codeberg than Github. From latter for such basic offerings I'd expect free services, yup! (Because I already pay them in other ways, like my data and all that.. but that's really beside the point). As for funding or becoming a member, I am not there yet and also that is not regular enough for me, that is not streamlined for me personally.
How… why… what? Let's say it's like.. well… I don't know.. anyway.. people (esp. from USA), please don't be offended (discussion on this topic has felt like we are from different planets), I'd rather pay a simple restaurant bill where 2Y tip is mandatorily part of the bill than a Y tip that I would "leave at the table" or "drop in a tip jar" or "generously" "hand it" (or slip it) to the staff. I have lots of problems with the latter but let's not get into that here anyway.
> For the time being, it appears that all three servers are without power.
This strikes me as odd, only three servers?
3 physical servers can power a ton of requests.
From what I understand, Codeberg is a pretty small nonprofit group.
Codeberg is smaller than GitHub and, you know, Go is slightly more efficient than Ruby :)
Only well known project on Codeberg that comes to mind is Zig
That's because most of what you can see of a Codeberg is actually underwater.
Does Forgejo count?
Comaps too.
Guix?
Never heard of it. And it makes HN?
There are a couple of very vocal people who are generally liked here who wrote angry blog posts about moving from Github to Codeberg.
This is why it's getting traction.
Consider that maybe you haven’t heard of all the things that HN readers find interesting.
I am VERY active in technology and that I haven't heard of it means something, too. I notice--far too often--that obscure things like this get posted here and elsewhere which can confuse others into thinking they have to get involved or learn about it cause they saw it on <this site> so they have to learn it, too, when it's only a niche case among few.
I see it mentioned any time GitHub is shitting the bed (frequent enough) and someone asks for self hostable alternatives or when someone ask for a European GitHub alternative. I think you just might not be paying enough attention.
Again, Codeberg isn’t obscure. Worrying about front page HN links “confusing” others because you haven’t heard of something is about the silliest thing I have read on this site.