If I were to do some rough math I’d say it would cost about $300/CAD per month.
My goal is once we are approved to start accepting donations that I can purchase dedicated hardware for this instance. I’d get a used server at about $2300 which would be sufficient a good amount of extra users and through it into its own dedicated shared colo at about $100/month. Factor in about $300-400 a year for drive replacements and we are left with $2300 / 12 month= 191.66 + 100/month for the shared 1u colo + a budget of $400 for drive failures throughout the year $33/month. 191.66 + 100 + 33 = $324.66/month for the first year dropping to about $133 per month after the first 12 months. It’s worth noting that this method would give us double the amount of resources and quite a bit of extra storage.
Ideally we don’t keep this instance on a single server forever and start to think about spreading it over multiple hosts at or after around 100K users (or less if the number of active users is high).
If someone wanted to host an instance they would not need to allocate as much resources as I have to this instance and depending on how active the instance gets could run off something a lot less powerful.
I really appreciate the information, it’s very interesting to me. Given that you have a fairly specific price in mind for a server, what kind of hardware are you thinking of?
Yeah registered dimm 4x32g. I couldn’t find a way to put it in a low powered quiet box without spending a bunch of money. My home lab stuff is ddr5 now.
I love the transparency. I think we can easily reach that mark. Whenever you get approved for donations we’ll be ready. I’ve got at least tree fiddy in my account
I agree. Thanks for keeping us updated and explaining what kind of money needs to be spend.
I’ll be more than happy to chip in. edited because Thelsim doesn’t know how to proofread
Thank you for the money and time you put into making this instance work and keeping it working. I imagine the responsibility that comes from all this is both a joy and a burden.
Have you considered the implications of hardware failure on uptime? And where the cost to maintain a physical hardware will come from? What about scaling requirements?
I’m not a network engineer, but I’ve been involved in the corporate argument of Cloud vs On-Prem. hosting for years now. The costs always come out better for Cloud when factoring in other indirect costs like facilities and labor.
Granted it’s always been on the scale of hundreds of millions to billions of dollars, and I haven’t run the numbers on smaller requirements. I just wouldn’t want to expose additional points of failure in return for slightly lower monthly costs.
I think the cost always come out better for cloud for a given reliability level. But this is a volunteer run thing, so we won’t mind if there is some more important downtime than on reddit or Twitter.
I really do think that if your objective is not reaching 100% uptime but cost reduction, then on prem really becomes the cheapest option
A very good point! We don’t need constant uptime. But I worry about the hidden costs of On-Prem, and worst case scenario where TheDude is on vacation somewhere and the instance crashes, it could be down for a while. It’s also not a worry I would want to force on them either.
Yes, when I try to explain this to people, I always explain the bus factor concept: how many people could get hit by a bus until it becomes critical to run your business ! Running in the cloud allows you to avoid this problem, there will always be an oncall tech in the DC of your cloud provider, which is very hard to organize for an on prem system !
I guess The dudes can always give remote access to someone he trusts, but at the end of the day if a disk fail somebody got to go switch it
If I were to do some rough math I’d say it would cost about $300/CAD per month.
My goal is once we are approved to start accepting donations that I can purchase dedicated hardware for this instance. I’d get a used server at about $2300 which would be sufficient a good amount of extra users and through it into its own dedicated shared colo at about $100/month. Factor in about $300-400 a year for drive replacements and we are left with $2300 / 12 month= 191.66 + 100/month for the shared 1u colo + a budget of $400 for drive failures throughout the year $33/month.
191.66 + 100 + 33=$324.66/monthfor the first year dropping to about $133 per month after the first 12 months. It’s worth noting that this method would give us double the amount of resources and quite a bit of extra storage.Ideally we don’t keep this instance on a single server forever and start to think about spreading it over multiple hosts at or after around 100K users (or less if the number of active users is high).
If someone wanted to host an instance they would not need to allocate as much resources as I have to this instance and depending on how active the instance gets could run off something a lot less powerful.
I really appreciate the information, it’s very interesting to me. Given that you have a fairly specific price in mind for a server, what kind of hardware are you thinking of?
Something with Dual CPUs, at least 128GB ram, dual 750W PSUs, hardware raid (12Gbps) and 8 x 2.5" SAS/SATA slots for SSD Drives on a raid 10
Ddr4 or 5? I might have 128gb of rdimm ddr4 I’m never going to use sitting on my desk.
ECC memory though?
Yeah registered dimm 4x32g. I couldn’t find a way to put it in a low powered quiet box without spending a bunch of money. My home lab stuff is ddr5 now.
This sounds exactly like the poweredge r530 I have in my homelab. Managed to snag it on eBay with those specs, minus drives, for $350.
The 530 is 2U, the 630 is 1U size factor. Looking more at the 630
I love the transparency. I think we can easily reach that mark. Whenever you get approved for donations we’ll be ready. I’ve got at least tree fiddy in my account
I agree. Thanks for keeping us updated and explaining what kind of money needs to be spend.
I’ll be more than happy to chip in.
edited because Thelsim doesn’t know how to proofread
Thank you for the money and time you put into making this instance work and keeping it working. I imagine the responsibility that comes from all this is both a joy and a burden.
Really nice breakdown thanks for sharing! Totally reasonable goals to reach too.
This is exactly what I was after thanks for the good rundown! Also thanks for all the time money and effort spent on all of this
What are you doing for backups?
That’s the “drive replacements” part I believe
Drive replacement != backup
Drive replacement = maintenance, a subset of physical security.
Backup = logical security.
The purpose of backup is to prevent loss of data in general, not only on account of drive failure, but also other sources such as malicious activity
And what if several drives fail at once or a bad actor deletes the data? :)
Have you considered the implications of hardware failure on uptime? And where the cost to maintain a physical hardware will come from? What about scaling requirements?
I’m not a network engineer, but I’ve been involved in the corporate argument of Cloud vs On-Prem. hosting for years now. The costs always come out better for Cloud when factoring in other indirect costs like facilities and labor.
Granted it’s always been on the scale of hundreds of millions to billions of dollars, and I haven’t run the numbers on smaller requirements. I just wouldn’t want to expose additional points of failure in return for slightly lower monthly costs.
He mentioned colo, so it sounds like he’s already decided against on-prem.
I think the cost always come out better for cloud for a given reliability level. But this is a volunteer run thing, so we won’t mind if there is some more important downtime than on reddit or Twitter. I really do think that if your objective is not reaching 100% uptime but cost reduction, then on prem really becomes the cheapest option
A very good point! We don’t need constant uptime. But I worry about the hidden costs of On-Prem, and worst case scenario where TheDude is on vacation somewhere and the instance crashes, it could be down for a while. It’s also not a worry I would want to force on them either.
Yes, when I try to explain this to people, I always explain the bus factor concept: how many people could get hit by a bus until it becomes critical to run your business ! Running in the cloud allows you to avoid this problem, there will always be an oncall tech in the DC of your cloud provider, which is very hard to organize for an on prem system !
I guess The dudes can always give remote access to someone he trusts, but at the end of the day if a disk fail somebody got to go switch it