Every so often I price out RDS to replace our colocated SQL Server cluster and it's so unrealistically expensive that I just have to laugh. It's absurdly far beyond what I'd be willing to pay. The markup is enough to pay for the colocation rack, the AWS Direct Connects, the servers, the SAN, the SQL Server licenses, the maintenance contracts, and a full-time in-house DBA.

https://calculator.aws/#/estimate?id=48b0bab00fe90c5e6de68d0...

Total 12 months cost: 547,441.85 USD

Once you get past the point where the markup can pay for one or more full-time employees, I think you should consider doing that instead of blindly paying more and more to scale RDS up. You're REALLY paying for it with RDS. At least re-evaluate the choices you made as a fledgling startup once you reach the scale where you're paying AWS "full time engineer" amounts of money.

Just like any other asset.

I was once part of an acquisition from a much larger corporate entity. The new parent company was in the middle of a huge cloud migration, and as part of our integration into their org, we were required to migrate our services to the cloud.

Our calculations said it would cost 3x as much to run our infra on the cloud.

We pushed back, and were greenlit on creating a hybrid architecture that allowed us to launch machines both on-prem and in the cloud (via a direct link to the cloud datacenter). This gave us the benefit of autoscaling our volatile services, while maintaining our predictable services on the cheap.

After I left, apparently my former team was strong-armed into migrating everything to the cloud.

A few years go by, and guess who reaches out on LinkedIn?

The parent org was curious how we built the hybrid infra, and wanted us to come back to do it again.

I didn't go back.

A customer had an interest in merging the data from an older account into a new one, just to simplify matters. Enterprise data. Going back years. Not even leaving the region.

The AWS rep in the meeting kinda pauses, says: "We'll get back to you on the cost to do that."

The sticker shock was enough that the customer simply inherited the old account, rather than making things tidy.

I've heard some people here on HN say that it's slow, but I haven't noticed a difference. We're mainly dealing with multi-megabyte image files, so YMMV if you have a different workload.

I guess permissions might be more complex, as in EC2 instance profiles wouldnt grant access, etc.

It's the amount of data where it makes more sense to put hard drives on a truck and drive across the country rather than send it over a network, where this becomes an issue (actually, probably a bit before then).

> Q: Can I export data from AWS with Snowmobile? > > Snowmobile does not support data export. It is designed to let you quickly, easily, and more securely migrate exabytes of data to AWS. When you need to export data from AWS, you can use AWS Snowball Edge to quickly export up to 100TB per appliance and run multiple export jobs in parallel as necessary. Visit the Snowball Edge FAQs to learn more.

https://aws.amazon.com/snowmobile/faqs/?nc2=h_mo-lang

Why would they make it convenient to leave?

When you have enough data, that cost is quite significant.

You can peer two vpc's and as long as you are transferring within the same (real) AZ, it's free: https://aws.amazon.com/about-aws/whats-new/2021/05/amazon-vp...

Even peered VPC's only pay "normal" prices: https://aws.amazon.com/ec2/pricing/on-demand/#Data_Transfer

"Data transferred "in" to and "out" from Amazon EC2, Amazon RDS, Amazon Redshift, Amazon DynamoDB Accelerator (DAX), and Amazon ElastiCache instances, Elastic Network Interfaces or VPC Peering connections across Availability Zones in the same AWS Region is charged at $0.01/GB in each direction."

I've made a career out of inheriting other peoples whacky setups and supporting them (as well as fixing them) and almost always its documentation that has prevented the client getting anywhere.

I personally dont care if the docs are crap because usually the first thing I do is update / actually write the docs to make them usable.

For a lot of techs though crap documentation is a deal breaker.

Crap docs aren't always the fault of the guys implementing though, sometimes there are time constraints that prevent proper docs being written. Quite frequently though its outsourced development agencies that refuse to write it because its "out of scope" and a "billable extra". Which I think is an egregious stance...doxs Should be part and parcel of the project. Mandatory.

If there is only one thing that juniors should learn about writing documentation (be it comments or design documents), it is this: document why something is there. If resources are limited, you can safely skip comments that describe how something works, because that information is also available in code.

(It might help to describe what is available, especially if code is spread out over multiple repositories, libraries, teams, etc.)

(Also, I suppose the comment I'm responding to could've been slightly more forgiving to GP, but that's another story.)

It's also completely against their interest to write docs as it makes their replacement easier.

That's why you need someone competent on the buying side to insist on the docs.

A lot of companies outsource because they don't have this competency themselves. So it's inevitable that this sort of thing happens and companies get locked in and can't replace their contractors, because they don't have any docs.

Even with documentation on the hybrid setup, they'd need to get a new on-prem environment up and running (find a colo, buy machines, set up the network, blah blah).

OK so the docs are in sync for a single point of time when you finish. Plus you get to have the context in your head (bus factor of 1, job security for you, bad for the org.)

How about if we just write clean infra configs/code, stick to well known systems like docker, ansible, k8s, etc.

Then we can make this infra code available to an on prem LLM and ask it questions as needed without it drifting out of sync overtime as your docs surely will.

Wrong documentation is worse than no documentation.

I can always guarantee a stream of consciousness one note that should have most of the important data, and a few docs about the most important parts. It's up to management if they want me to spend time turning that one note into actual robust documentation that is easily read.

My nasty little secret is for single server databases I have zero fear of over provisioning disk iops and running it on SQLite or making a single RDBMS server in a container. I've never actually run into an issue with this. It surprises me the number of internal tools I see that depend on large RDS installations that have piddly requirements.

On what disk is the actual data written? How do you do backups, if you do?

Besides, running things locally can be refreshingly simple if you are just starting something and you don't need tons of extra stuff, which becomes accidental complexity between you, the problem, and a solution. This old post described that point quite well by comparing Unix to Taco Bell: http://widgetsandshit.com/teddziuba/2010/10/taco-bell-progra.... See HN discussion: https://news.ycombinator.com/item?id=10829512.

I am sure for some use-cases cloud services might be worth it, especially if you are a large organization and you get huge discounts. But I see lots of business types blindly advocating for clouds, without understanding costs and technical tradeoffs. Fortunately, the trend seems to be plateauing. I see an increasing demand for people with HPC, DB administration, and sysadmin skills.

So much this. The "keep know how" has been so greatly avoided over the past 10 years, I hope people with these skills start getting paid more as more companies realize the cost difference.

My money is on openness continuing to grow and more and more pieces of the stack being completely owned by openness (kernels anyone?) but one doesn't know.

I hear tell of a shop that was running on ephemeral instance based compute fleets (EC2 spot instances, iirc), with all their prod data in-memory. Guess what happened to their data when spot instance availability cratered due to an unusual demand spike? No more data, no more shop.

Don't even get me started on the number of privacy breaches because people don't know not to put customer information in public cloud storage buckets.

If I had a small business with very clever people I'd be very afraid of what happens if they're not available for a while.

I would have a hard time doing servers as cheap as hetzner for example including the routing and everything

I think there is an unreasonable fear of "doing the routing and everything". I run vpncloud, my server clusters are managed using ansible, and can be set up from either a list of static IPs or from a terraform-prepared configuration. The same code can be used to set up a cluster on bare-metal hetzner servers or on cloud VMs from DigitalOcean (for example).

I regularly compare this to AWS costs and it's not even close. Don't forget that the performance of those bare-metal machines is way higher than of overbooked VMs.

We are using hetzner cloud.. but we are also scaling up and down a lot right now

If you mean dealing with the physical dedicated servers that can be rented from Hetzner, that's what the person you replied to was talking about being not so difficult.

If you mean everything else at the data centre that makes having a server there worthwhile (networking, power, cooling, etc.) I don't think people were suggesting doing that themselves (unless you're a big enough company to actually be in the data centre business), but were talking about having direct control of physical servers in a data centre managed by someone like Hetzner.

(edit: and oops sorry I just realised I accidentally downvoted your comment instead of up, undone and rectified now)

Aka if I add up power (including backup) + backbone connection rental + server deprication I can not do it for the hetzner price..

That was quite imprecise, sorry about that.

But I think that people (at least jwr, and probably even nyc_data_geek saying "on prem") are talking about cloud (like AWS) vs. renting (or buying) servers that live in a data centre run by a company like Hetzner, which can be considered "on prem" if you're the kind of data centre client who has building access to send your own staff there to manage your servers (while still leaving everything else, possibly even legal ownership and therefore deprecation etc. to the data centre owner).

What you're thinking of - literally taking responsibility for running your own mini data centre - I think is hardly ever considered (at least in my experience), except by companies at the extremes of size. If you're as big as Facebook (not sure where the line is but obviously including some companies not AS big as Meta but still huge) then it makes sense to run your own data centres. If you're a tiny business getting less than thousands of website visits a day and where the website (or whatever is being hosted) isn't so important that a day of downtime every now and then isn't a big deal, then it's not uncommon to host from the company's office itself (just using a spare old PC or second hand cheap 1U server, maybe a cheap UPS, and just connected to the main internet connection that people in the office use, and probably managed by a single employee, or company owner, who happens to be geeky enough to think it's one or both of simple or fun to set up a basic LAMP server, or even a Windows server for its oh-so-lovely GUI).

If all you need are some cloud servers, or a basic load balancer, they are pretty much the same.

If you need a plethora of managed services and don't want to risk getting fired over your choice or specifics of how that service is actually rendered, they are nothing alike and you should go for AWS, or one of the other large alternatives (GCP, Azure etc.).

On the flip side, if you are using AWS or one of those large platforms as a glorified VPS host and you aren't doing this in an enterprise environment, outside of learning scenarios, you are probably doing something wrong and you should look at Hetzner, Contabo, or one of those other providers, though some can still be a bit pricey - DigitalOcean, Vultr, Scaleway etc.

Well, in my case at least, what they have in common is that I can choose to run my business on one or the other. So it's not about intuition, but rather facts in my case: I avoid spending a significant amount of money.

I (of course) do realize that if you design your software around higher-level AWS services, you can't easily switch. I avoided doing that.

Why? The only reason I'm using Hetzner and not AWS for several of my own projects (even though I know AWS much better since this is what I use at work) is an enormous price difference in each aspect (compute, storage, traffic).

If price is the only factor, your business model (or executives' decision-making) is questionable. Buy only the cheapest shit, spend your time building your own office chair rather than talking to a customer, you aren't making a premium product, and that means you're not differentiated.

Most companies don‘t need to scale up full racks in seconds. Heck, even weeks would be ok for most of them to get new hardware delivered. The cloud planted the lie into everyone‘s head that most companies dont have predictable and stable load.

Might be other alternatives than using Docker so if anyone has tips for something simpler or easier to maintain, appreciate a comment.

They sure do? BASF has 3 power plants in Hamburg, Disney operate Reedy Creek Energy with at least 1 power plant and I could list a fair bit more...

>For example, how long does it take to rent another rack that you didnt plan for?

I mean, you can also rent hardware a lot cheaper then on AWS. There certainly are providers where you can rent out a rack for a month within minutes

Sure, maybe if you are some sort of SaaS with a need for a small single DB, that also needs to be resilient, backed up, rock solid bulletproof.. it makes sense? But how many cases are there of this? If its so fundamental to your product and needs such uptime & redundancy, what are the odds its also reasonably small?

Most software startups these days? The blog post is about work done at a startup after all. By the time your db is big enough to cost an unreasonable amount on RDS, you’re likely a big enough team to have options. If you’re a small startup, saving a couple hundred bucks a month by self managing your database is rarely a good choice. There’re more valuable things to work on.

By the time your db is big enough to cost an unreasonable amount on RDS, you've likely got so much momentum that getting off is nearly impossible as you bleed cash.

You can buy a used server and find colocation space and still be pennies on the dollar for even the smallest database. If you're doing more than prototyping, you're probably wasting money.

I’ve been the guy managing a critical self-hosted database in a small team, and it’s such a distraction from focusing on the actual core product.

To me, the cost of RDS covers tons of risks and time sinks: having to document the db server setup so I’m not the only one on the team who actually knows how to operate it, setting up monitoring, foolproof backups so I don’t need to worry that they’re silently failing because a volume is full and I misconfigured the monitoring, PITR for when someone ships a bad migration, one click HA so the database itself is very unlikely to wake me at 3am, blue/green deploys to make major version upgrades totally painless, never having to think about hardware failures or borked dist-upgrades, and so on.

Each of those is ultimately either undifferentiated work to develop in-house RDS features that could have been better spent on product, or a risk of significant data loss, downtime, or firefighting. RDS looks like a pretty good deal, up to a point.

RDS is a great choice, for prototyping and only for production if you know what you're doing when setting it up.

FWIW, this is common in all cloud deployments, people assume that running something "severless" is a magical silver bullet.

Maybe it takes you a month the first time around and a week the 10th time around. First product suffers, the other products not so much. Now it just takes a week of your time and does not require you to pay large AWS fees, which means you are not bleeding money

I like to set up scrappy products that do not rack up large monthly fees. This means I can let them run unprofitable for longer and I don't have to seek an investor early, which would light up a large fire under everyone's butts and start influencing timelines because now they have the money and want a return asap.

I'll launch a week later - no biggie usually. I could have come up with the idea a month later, so I'm still 3 weeks early ;)

It doesn't work for all projects, obviously, but I've seen plenty of SaaS start out with a shopping spree, then pay monthly fees and purchase licenses for stuff that they could have set up for free if they put some (usually not a lot) effort into it. When times get rough, the shorter runway becomes a hard fact of life. Maybe they wouldn't have needed a VC and could have bootstrapped and also survived for longer.

While I generally agree as far as initial setup time goes, I favor RDS because I can forget about it, whereas the hand rolled version demands ongoing maintenance, and incurs a nonzero chance of simple mistakes that, if made, could result in a 100% dataloss unrecoverable scenario.

I’m also mostly talking about typical, funded startups here, as opposed to indie/solo devs. If you’re flying solo launching a tiny proof of concept that may only ever have a few users, by all means run it yourself if you’d like, but if you’ve raised money to grow faster and are paying employees to iterate rapidly searching for PMF…just pay for RDS and make sure as much time as possible is spent on product features that provide actual business value. It starts at like $15/month. The cost of simply not being laser-focused on product is far greater.

Databases are not particularly difficult to migrate between machines. Of all the cloud services to migrate, they might actually be the easiest, since the databases don't have different API's that need to be rewritten for, and database replication is a well-established thing.

Getting off is quite the opposite of nearly impossible.

Optionality and flexibility are extremely valuable, and that is why cloud compute continues to be popular, especially for rapidly/burstily growing businesses like startups.

If you want a different opportunity cost, get people with different experience. If RDS is objectively expensive, objectively slow, but subjectively easy, change the subject.

I don't think that's accurate. I've self-managed databases, and I still think that RDS is compelling for small engineering teams.

There's a lot to get right when managing a database, and it's easy to screw something up. Perhaps none of the individual parts are super-complicated, but the cost of failure is high. Outsourcing that cost to AWS is pretty compelling.

At a certain team size, you'll end up with a section of the team that's dedicated to these sorts of careful processes. But the first place these issues come up is with the database, and if you can put off that bit of organizational scaling until later, then that's a great path to choose.

If the cost of a hosted DB is going to sink the company, then of course, I will figure it out and run it myself. But it’s not, for most startups. And therefore that knowledge isn’t providing much leverage.

Starting an AI company with deep expertise in training models - that is an example of knowledge providing huge leverage. DB tech is not in this bucket for most businesses.

Its about tradeoffs, and some tradeoffs are often more applicable than others - getting a ping at 7am on a Sunday because your ec2 instance filled it's drive up with logs and your log rotation script failed because it didn't have a long enough retey is a problem I'm happy to outsource when I should be focusing on the actual app.

It would have cost a negligible amount. But the sheer amount of time I wasted before I gave up was honestly quite surprising. Let’s see:

- I wanted one simple extension. I could have compromised on this, but getting it to work on RDS was a nonstarter.

- I wanted RDS to _import the data_. Nope, RDS isn’t “SUPER,” so it rejects a bunch of stuff that mysqldump emits. Hacking around it with sed was not confidence-inspiring.

- The database uses GTIDs and needed to maintain replication to a non-AWS system. RDS nominally supports GTID, but the documented way to enable it at import time strongly suggests that whoever wrote the docs doesn’t actually understand the purpose of GTID, and it wasn’t clear that RDS could do it right. At least Azure’s docs suggested that I could have written code to target some strange APIs to program the thing correctly.

Time wasted: a surprising number of hours. I’d rather give someone a bit of money to manage the thing, but it’s still on a combination of plain cloud servers and bare metal. Oh well.

Sounds like you are walking massive edge

Very small businesses with phone apps or web apps are often using it. There are cheaper options of course, but when there is no "prem" and there are 1-5 employees then it doesn't make much sense to hire for infra. You outsource all digital work to an agency who sets you up a cloud account so you have ownership, but they do all software dev and infra work.

> If its so fundamental to your product and needs such uptime & redundancy, what are the odds its also reasonably small?

Small businesses again, some of my clients could probably run off a Pentium 4 from 2008, but due to nature of the org and agency engagement it often needs to live in the cloud somewhere.

I am constantly beating the drum to reduce costs and use as little infra as needed though, so in a sense I agree, but the engagement is what it is.

Additionally, everyone wants to believe they will need to hyperscale, so even medium scale businesses over-provision and some agencies are happen to do that for them as they profit off the margin.

AWS and the like are rarely a cost effective option, but it is something a lot of agencies like, largely because they are not paying the bills. The clients do not usually care because they are comfortable with a known brand and the costs are a small proportion of the overall costs.

A real small business will be fine just using a VPS provider or a rented server. This solves the problem of not having on premise hardware. They can then run everything on a single server, which is a lot simpler to set up, and a lot simpler to secure. That means the cost of paying someone to run it is a lot lower too as they are needed only occasionally.

They rarely need very resilient systems as they amount of money lost to downtime is relatively small - so even on AWS they are not going to be running in multiple availability zones etc.

These questions always sound flawed to me. It's like asking won't I regret moving to California and paying high taxes once I start making millions of dollars? Maybe? But that's an amazing problem to have and one that I may be much better equipped to solve.

If you are small, RDS is much cheaper, and many company killing events, such as not testing your backups are solved. If you are big and you can afford a 60K/yr RDS bill than you can make changes to move on-prem. Or you can open up excel and do the math if your margins are meaningfully affected by moving on-prem.

On some level, AWS/GCP/California relies on you doing this calculation for the things that you can do it on easily (the savings of moving away), while not doing this calculation on things where it's hard to do (new development). That way, you can pretend that your new features are a lot more valuable than the $Xk/year you will save by moving your infra.

Yes, I've done the math. The piece you are missing is, saving money on infra will bring in $0 new dollars. There is a floor to how much money I can save. There is no ceiling to how much money the right feature can bring in. Penny pinching on infra, especially when the amount of money is saved is less than the cost of an engineer is almost always a waste of time while you are growing a company. If you are at the point where you are wasting 1x,2x,3x of an engineers salary of superflous infrastructure - then congratulations you have survived the great filter for 99% of startups.

>That way, you can pretend that your new features are a lot more valuable than the $Xk/year you will save by moving your infra.

Finding product market fit is 1000x harder than moving from RDS to On-prem. If you haven't solved PMF, then no amount of $Xk/year in savings will save you from having to shut down your company.

The thing is, most features, particularly later in the life of a company, don't have an easy-to-measure revenue impact, and I suspect that many features are actually worth $0 of revenue. However, they cost money to implement (both in engineering time and infra), making them very much net negative value propositions. This is why Facebook and Google can cut tons of staff and lose nothing off their revenue number.

Also, there's a bit of a gambling mentality here which is that a feature could be worth effectively infinite revenue (ie it could be the thing that gives you PMF), so it's always worth doing over things with known, bounded impact on your bottom line. However, improving your efficiency gives you more cracks at finding good features before you run out of money.

So moving to/from Aurora/RDS/own EC2/on-prem should be a matter of networking and changing connection strings in the clients.

Your operational requirements and processes (backup/restore, failover, DR etc) will change, but that's because you're making a deliberate decision weighing up those costs vs benefits.

You can use DNS to mitigate the pain of changing those connection strings, decoupling client change management from backend change process, or if you had foresight, not having to change client connection strings at all.

I mean, DNS change can work, but when you're doing that one-in-years change, why risk the extra failure modes.

No one runs their own electricity supply (well until recently with renewables/storage), they buy it as a service, up to a pretty high scale before it becomes more economic to invest the capex and opex to run your own.

I mean, I'm looking after a 4 instance Aurora cluster which is great feature wise, is slightly overprovisioned for special events, and is more likely to shrink than grow 2x in the next decade. If we start experiencing any issues, there's lots of optimisations that can be still gained from better caching and that work will be cheaper than the instance size upgrade.

There’s still a defined cost to swapping your DB code over to a different backend. At the point where it becomes uneconomical, you’re also at a scale you can afford rewriting a module.

That’s why we have things like “hexagonal architecture”, which focus on isolating the storage protocol from the code. There’s an art to designing such that your prototype can scale with only minor rework — but that’s why we have senior engineers.

People pay for RDS because they want to believe in a fairy tale that it will keep potential problems away and that it worked well for other customers. But those mythical other customers also paid based on such belief. Plus, no one wants to admit that they pay money in such irrational way. It's a bubble

Come time for force major upgrade shoved down our throat? Downtime, surprise, surprise

One of these is not like the others (DBAs are not capex.)

Have you ever considered that if a company can get the same result for the same price ($100K opex for RDS vs same for human DBA), it actually makes much more sense to go the route that takes the human out of the loop?

The human shows up hungover, goes crazy, gropes Stacy from HR, etc.

RDS just hums along without all the liabilities.

Hard disagree. An r6i.12xl Multi-AZ with 7500 IOPS / 500 GiB io1 books at $10K/month on its own. Add a read replica, even Single-AZ at a smaller size, and you’re half that again. And this is without the infra required to run a load balancer / connection pooler.

I don’t know what your definition of “large” is, but the described would be adequate at best at the ~100K QPS level.

RDS is expensive as hell, because they know most people don’t want to take the time to read docs and understand how to implement a solid backup strategy. That, and they’ve somehow convinced everyone that you don’t have to tune RDS.

If you don't have a reserved instance, then you're giving up potentially a 50% discount on on-demand pricing.

An r6i.12xl is a huge instance.

There are other equivalents in the range of instances available (and you can change them as required, with downtime).

For MySQL and Postgres, RDS stripes across four volumes once you hit 400 GiB. Doesn't matter the type.

The latency variation on gp3 is abysmal [0], and the average [1] isn't great either. It's probably fine if you have low demands, or if your working set fits into memory and you can risk the performance hit when you get an uncached query.

12K IOPS sounds nice until you add latency into it. If you have 2 msec latency, then (ignoring various other overheads, and kernel or EBS command merging) the maximum a single thread can accomplish in one second is (1000 msec / 1 sec / 2 msec) = 500 I/O. Depending on your needs that may be fine, of course.

> If you don't have a reserved instance, then you're giving up potentially a 50% discount on on-demand pricing.

True, of course. Large customers also don't pay retail.

> An r6i.12xl is a huge instance.

I mean, it goes well past that to .32xl, so I wouldn't say it's huge. I work with DBs with 1 TiB of RAM, and I'm positive there are people here who think those are toys. The original comment I replied to said, "large SaaS," and a .12xl, as I said, would be roughly adequate for ~100K QPS, assuming no absurdly bad queries.

[0]: https://www.percona.com/blog/performance-of-various-ebs-stor...

[1]: https://silashansen.medium.com/looking-into-the-new-ebs-gp3-...

Assuming you have any. You might not, because of AWS.

After initial setup, managing equivalent of $5k/m RDS is not full time job. If you add to this, that wages differ a lot around the world, $5k can take you very, very far in terms of paying someone.

Frankly, this is anti-competitive, and the FTC should look into it, however, Microsoft has been anti-competitive and customer hostile for decades, so if you're still using their products, you must have accepted the abuse already.

We technically aren't supposed to talk about pricing publically, but I'm just going to say that we run a few 8XL and 12Xl RDS instances and we pay ~40% off the sticker price.

If you switch to Aurora engine the pricing is absurdly complex (its basically impossible to determine without a simulation calculator) but AWS is even more aggressive with discounting on Aurora, not to mention there are some legit amazing feature benefits by switching.

I'm still in agreeance that you could do it cheaper yourself at a Data Center. But there are some serious tradeoffs made by doing it that way. One is complexity and it certainly requires several new hiring decisions. Those have their own tangible costs, but there are a huge amount of intangible costs as well like pure inconvenience, more people management, more hiring, split expertise, complexity to network systems, reduce elasticity of decisions, longer commitments, etc.. It's harder to put a price on that.

When you account for the discounts at this scale, I think the cost gap between the two solutions is much smaller and these inconveniences and complexities by rolling it yourself are sometimes worth bridging that smaller gap in cost in order to gain those efficiencies.

Genuinely curious, how do you that?

We pay a couple of million dollars per year and the biggest spend is RDS. The bulk of those are 8xl and 12xl as you mention and we have a lot of these. We do have savings plans, but those are nowhere near 40%.

It looks like a reserved instance is 35% off sticker price? Add probably a discount and you'd be around 40% off.

That: and trust is hard earned over a long tail which is harder if you are trying to compete on price.

But you are totally right it can be expensive. I worked with a startup that had some inefficient queries, normally it would matter, but with RDS it cost $3,000 a month for a tiny user base and not that much data (millions of rows at most).

I haven't run a Postgres instance with proper backup and restore, but it doesn't seem like rocket science using barman or pgbackrest.

Small databases or test environment databases you can also leverage kubernetes to host an operator for that tiny DB. When it comes to serious data and it needs a beeline recovery strategy RDS.

Really it should be a mix self hosted for things you aren't afraid to break. Hosted for the things you put at high risk.

1. Hiring someone full time to work on the database means migrating off RDS

2. Database work is only about spend reduction

DynamoDB as a replacement, pay per request, was essentially free.

I found Dynamo foreign and rather ugly to code for initially, but am happy with the performance and especially price at the end.

RDS make perfect sense for them

They need to replicate everything in multiple availability zones, which is going to be more expensive than replicating data centres.

They still need to test their cloud infrastracuture works.

Just to pay someone else enough money to provide the same service and make a profit while do it

It's like accounting and finance. Yeah a lot of companies use tax firms, but they all have finance and accounting in-house.

Seems like a solid cost effective approach for when a company reaches a certain scale.

OTOH, as noted, EBS does not perform as well as native NVMe and is hilariously expensive if you try. And quite a few use cases are just fine on plain old NVMe.

The query times are incredible.

I know this is an unpopular opinion but I think google cloud is amazing compared to AWS. I use google cloud run and it works like a dream. I have never found an easier way to get a docker container running in the cloud. The services all have sensible names, there are fewer more important services compared to the mess of AWS services, and the UI is more intuitive. The only downside I have found is the lack of community resulting in fewer tutorials, difficulty finding experienced hires, and fewer third party tools. I recommend trying it. I'd love to get the user base to an even dozen.

The reasoning the author cites is that AWS has more responsive customer service and maybe I am missing out but it would never even occur to me to speak to someone from a cloud provider. They mention having "regular cadence meetings with our AWS account manager" and I am not sure what could be discussed. I must be doing simper stuff.

As being on a number of those calls, its just a bunch of crap where they talk like a scripted bot reading from corporate buzzword bingo card over a slideshow. Their real intention is two fold. To sell you even more AWS complexity/services, and to provide "value" to their person of contact (which is person working in your company).

We're paying north of 500K per year in AWS support (which is a highway robbery), and in return you get a "team" of people supposedly dedicated to you, which sounds good in theory but you get a labirinth of irresponsiblity, stalling and frustration in reality.

So even when you want to reach out to that team you have to first to through L1 support which I'm sure will be replaced by bots soon (and no value will be lost) which is useful in 1 out of 10 cases. Then if you're not satisfied with L1's answer(s), then you try to escalate to your "dedicated" support team, then they schedule a call in three days time, or if that is around Friday, that means Monday etc.

Their goal is to stall so you figure and fix stuff on your own so they shield their own better quality teams. No wonder our top engineers just left all AWS communication and in cases where unavoidable they delegate this to junior people who still think they are getting something in return.

I’ve found a lot of the time the issues we run into are self-inflicted. When we call support for these, they have to reverse-engineer everything which takes time.

However when we can pinpoint the issue to AWS services, it has been really helpful to have them on the horn to confirm & help us come up with a fix/workaround. These issues come up more rarely, but are extremely frustrating. Support is almost mandated in these cases.

It’s worth mentioning that we operate at a scale where the support cost is a non-issue compared to overall engineering costs. There’s a balance, and we have an internal structure that catches most of the first type of issue nowadays.

I am so tired of the support team having all the real metrics, especially in io and throttling, and not surfacing it to us somehow.

And cadence is really an opportunity for them to sell to you, the parent is completely right.

They often provide that consulting for free, and we know their biases. There's nothing hidden about the fact that they will push us to use AWS services.

On the other hand, they will also help us optimize those services and save money that is directly measurable.

GCP might have a better API and better "naming" of their services, but the breadth of AWS services, the incorporation of IAM across their services, governance and automation all makes it worth while.

Cloud has come a long way from "it's so easy to spin up a VM/container/lambda".

Our account team don't even do that. We use a lot of AWS anyway and they know it, so they're happy to help with competitor offerings and integrating with our existing stack. Their main push on us has been to not waste money.

AWS wants happy customers to stick around for a long time, not one month of goosed income

If you still decided to move away, and want to take data with you, yeah... there is a cost. Heck there is a cost to delete the data you have with them (like S3 content).

Its a good way to do business.

I suspect the team that manages it was OKR-ed into using AJAX but come from a classic ASP background, so don't understand what all this "single page app" fad is all about and hope it blows over one day

It amazes me a company that makes that much money has such a crappy client.

You'd think a company with 1.5m employees could find half a dozen decent front end developers, but apparently not

Pub/Sub - Kinesis

Cloud CDN - CloudFront

Cloud Domains - Route 53

...

Google Cloud Run - Lambda

Sure I get the reference to the underlying algebraic representation of coding but come on, Lambda tells us nothing of what it does.

Products (not brands, products) should be named in a way that means something to the customer afaic.

> Google Cloud Run - Lambda

ECS is the AWS equivalent of Cloud Run. GCP Cloud Functions are the equivalent of AWS Lambda.

ECS / Cloud Run = managed container service that autoscales

Lambda / Cloud Functions = serverless functions as a service

One, and you often times only need one.

Also, I don't think trying to emulate AWS's support and consistent API makes sense as a strategy for other cloud providers. They will never beat AWS at their own game, it is light years ahead. If cloud providers want to survive they need to fill a different niche and try different things.

Idk why you don't see AWS as a utility providing building blocks.

> I don't think trying to emulate AWS's support and consistent API makes sense as a strategy for other cloud providers.

Those are such essential things. It's very hard to imagine prioritising something else and succeeding in anything.

How do they do this jedi mind trick?

GCP SDK docs must be mentioned separately as it's a bizarre auto-generated nonsense. Have you seen them? How can you even say that GCP docs are good after that?

wdym? As far as I see, it's either CLI or Terraform. GCP SDK is complete garbage, at least for Python compared to AWS boto3. I have personally made web UI for AWS CLI man pages as a fun project and can index everything myself if needed. Googling works fine. If you are not happy with it then ChatGPT is to the rescue. I honestly do not see any problem at all.

I don't have a ton of Azure or cloud experience but I run an Unraid server locally which has a decent Docker gui.

Getting a docker container running in Azure is so complicated. I gave up after an hour of poking around.

AWS has "fun" features like the ability to just lose track of some resource and still be billed for it. It's in here... somewhere. Not sure which region or account. I'll find it one day.

GCP is made by Google, also known as children that forgot to take their ADHD medication. Any minute now they'll just casually announce that they're cancelling the cloud because they're bored of it.

Azure is the only one I've seen with a sane management interface, where you can actually see everything everywhere all at once. Search, filter, query-across-resources, etc... all work reasonably well.

How can you even compare it to AWS is a mystery to me. There are pages showing all your resources, not sure why you think it's a problem. Could be a problem from long time ago?

[0] https://azure.microsoft.com/en-gb/products/container-apps

https://learn.microsoft.com/en-us/azure/container-instances/... is the one I was going to plug, because coming from a kubernetes background it seems to damn near be the PodSpec and thus both expresses a lot of my needs and also is very familiar https://learn.microsoft.com/en-us/azure/templates/microsoft....

Your link does seem to be a lot more "container, plus all the surrounding stuff" in line with the "apps" part, whereas mine more closely matches my actual experience of what you said: container, go

The "what the fucking hell is wrong with you people?" part is that their naming is just all over the place, and changes constantly, and is almost designed to be misleading in any sane conversation. I quite literally couldn't have guessed whether Container Apps was a prior name of Container Instances, a super set of it, subset, other? And one will observe that while I said Container Instances, and the URL says Container Instances, the ARM is Container Groups. Are they the same? different? old? who fucking knows. It's horrific

I also worked at a smaller company using GCP. GCP refused to do a small quota increase (which AWS just does via a web form) unless I got on a call with my sales representative and listened to a 30 minute upsell pitch.

I’ve had technicians at both GCP and Azure debug code and spend hours on developing services.

Almost every time Google pulled in a specialist engineer working on a service/product we had issues with it was very very clear the engineer had no desire to be on that call or to help us. In other words they'd get no benefit from helping us and it was taking away from things that would help their career at Google. Sometimes they didn't even show up to the first call and only did to the second after an escalation up the management chain.

The only weird part I experienced with AWS is their SNS API. Maybe due to legacy reasons, but what a bizarre mess when you try doing it cross-account. This one is odd.

I have been trying GCP for a while and DevX was horrible. The only part that more-or-less works is CLI but the naming there is inconsistent and not as well-done as in AWS. But it's relative and subjective, so I guess someone likes it. I have experienced GCP official guides that broken, untested or utterly braindead hello-world-useless. And also they are numerous and spread so it takes time to find anything decent.

No dark mode is an extra punch. Seriously. Tried to make it myself with an extension but their page is Angular hell of millions embedded divs. No thank you.

And since you mentioned Cloud Run -- it takes 3 seconds to deploy a Lambda version in AWS and a minute or more for GCP Could Function.

We started using Azure Container Apps (ACA) and it seems simple enough.

Create ACA, point to GitHub repo, it runs.

Push an update to GitHub and it redeploys.